Filling pandas Data Frame

Question

I have a list of ticker values

ticker = ["AAPL","MSFT","GOOG"]

and I want to create a DF with "high" values of prices for all the stocks in the ticker list.

Creating an empty DF:

high_df = pd.DataFrame(columns = ticker)

Filling the DF:

import pandas_datareader as web
import datetime

start = datetime.datetime(2010,1,1)
end = datetime.datetime(2010,2,1)

for each_column in high_df.columns:
   high_df[each_column] = web.DataReader(each_column, "yahoo",start,end)["High"]

This works but takes a long time if the ticker list is huge. Any other suggestions for approaches to speed up? Speed up with the way the DF is filled.

@WeNYoBen. Yes, but my question was more like can we speed up with parallelizing for loop with apply or something. — data_person, May 08 '19 at 22:33
https://stackoverflow.com/questions/9786102/how-do-i-parallelize-a-simple-python-loop — BENY, May 08 '19 at 22:34

score 0 · Accepted Answer · answered May 08 '19 at 22:44

Seems like just need parallel computing.

from joblib import Parallel, delayed

def yourfunc(tic):
    start = datetime.datetime(2010, 1, 1)
    end = datetime.datetime(2010, 2, 1)
    result=web.DataReader(tic, "yahoo", start, end)["High"]
    return result

results = Parallel(n_jobs=-1, verbose=verbosity_level, backend="threading")(
             map(delayed(yourfunc), ticker ))

About the conversion , you can using pd.DataFrame(results,columns=ticker)

Cool. I will test it now. – data_person May 08 '19 at 22:48 — data_person, May 08 '19 at 22:48

Filling pandas Data Frame

1 Answers1