Reputation: 99
from concurrent.futures import ThreadPoolExecutor
import time
import io
import pandas as pd
from image_exporter import export
import numpy as np
df1 = pd.DataFrame(np.random.rand (6, 4))
df2 = pd.DataFrame(np.random.rand (6, 4))
df3 = pd.DataFrame(np.random.rand (6, 4))
df4 = pd.DataFrame(np.random.rand (6, 4))
def get_exported(df):
s = time.time()
buffer = io.BytesIO()
export(df.style.background_gradient(), buffer)
buffer.seek(0)
print('get_exported', time.time() - s)
def test_parallel():
tasks = [df1, df2, df3, df4]
with ThreadPoolExecutor() as executor:
executor.map(get_exported, tasks)
def test_seq():
tasks = [df1, df2, df3, df4]
for t in tasks:
get_exported(t)
s = time.time()
test_parallel()
print('Total time parallel', time.time() - s)
s = time.time()
test_seq()
print('Total time sequential', time.time() - s)
image_exporter
file that contains the export
method
https://gist.github.com/vukrado/bd14192984c51569f6ed94bcd49f4b24
The test_seq
on my machine takes ~0.035 seconds per call to get_exported
for a total run time of ~.14 seconds
The test_parallel
on my machine takes ~0.18 seconds per call to get_exported
for a total run time of ~.19 seconds
Upvotes: 0
Views: 45