CODEWITHSUNDEEP
pythonpandasfinance
Madhu Sareen
Madhu Sareen

Reputation: 549

Unable to get help on retrieve data from Nasdaq in Python

I am planning to do some financial research and learning using data from the NASDAQ.

I want to retrieve data from Nasdaq such that the header has the following:

Stock Symbol
Company Name
Last Sale
Market Capitalization
IPO Year
Sector
Industry
Last Update

And I used Python code to get the "list of companies and ticker names" using:

import pandas as pd
import json

PACKAGE_NAME = 'nasdaq-listings'
PACKAGE_TITLE = 'Nasdaq Listings'

nasdaq_listing = 'ftp://ftp.nasdaqtrader.com/symboldirectory/nasdaqlisted.txt'# Nasdaq only


def process():
    nasdaq = pd.read_csv(nasdaq_listing,sep='|')

    nasdaq = _clean_data(nasdaq)

    # Create a few other data sets
    nasdaq_symbols = nasdaq[['Symbol','Company Name']] # Nasdaq  w/ 2 columns

    # (dataframe, filename) datasets we will put in schema & create csv
    datasets = [(nasdaq,'nasdaq-listed'), (nasdaq_symbols,'nasdaq-listed-symbols')]

    for df, filename in datasets:
        df.to_csv('data/' + filename + '.csv', index=False)

    with open("datapackage.json", "w") as outfile:
        json.dump(_create_datapackage(datasets), outfile, indent=4, sort_keys=True)


def _clean_data(df):
    # TODO: do I want to save the file creation time (last row)
    df = df.copy()
    # Remove test listings
    df = df[df['Test Issue'] == 'N']

    # Create New Column w/ Just Company Name
    df['Company Name'] = df['Security Name'].apply(lambda x: x.split('-')[0]) #nasdaq file uses - to separate stock type
    #df['Company Name'] = TODO, remove stock type for otherlisted file (no separator)

    # Move Company Name to 2nd Col
    cols = list(df.columns)
    cols.insert(1, cols.pop(-1))
    df = df.loc[:, cols]

    return df


def _create_file_schema(df, filename):
    fields = []
    for name, dtype in zip(df.columns,df.dtypes):
        if str(dtype) == 'object' or str(dtype) == 'boolean': # does datapackage.json use boolean type?
            dtype = 'string'
        else:
            dtype = 'number'

        fields.append({'name':name, 'description':'', 'type':dtype})

    return {
            'name': filename,
            'path': 'data/' + filename + '.csv',
            'format':'csv',
            'mediatype': 'text/csv',
            'schema':{'fields':fields}
            }


def _create_datapackage(datasets):
    resources = []
    for df, filename in datasets:
        resources.append(_create_file_schema(df,filename))

    return {
            'name': PACKAGE_NAME,
            'title': PACKAGE_TITLE,
            'license': '',
            'resources': resources,
            }


process()

Now for each of these symbols, I want to get the other data (as in above).

Is there anyway I could do this?

Upvotes: 0

Views: 510

Answers (1)

Lingster
Lingster

Reputation: 1087

Have you taken a look at pandas-datareader? You maybe able to get the other data from there. It has multiple data sources, such as Google, Yahoo Finance,

http://pandas-datareader.readthedocs.io/en/latest/remote_data.html#remote-data-google

Upvotes: 1

Related Questions