CODEWITHSUNDEEP
pythonpython-requests
user14145822
user14145822

Reputation: 21

How do I fix this Python request 500 error?

I'm using the following Python code to scrape data using a form request. But I received a 500 error. Can some some help find the fix? Thanks!

import requests

tickers = ['MSFT']

myurl = "https://www.theocc.com/Market-Data/Market-Data-Reports/Series-and-Trading-Data/Series-Search"
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}

for ticker in tickers:
    
    session = requests.session()
    # construct the POST request
    form_data = {
        'symbolType' : 'U',
        'symbolText' : ticker
    }
    r = requests.post(myurl, headers=headers, data=form_data)
    print(r)
    print(r.text)

Upvotes: 0

Views: 94

Answers (2)

Dorin Botan
Dorin Botan

Reputation: 1388

B exploring the API via Chrome DevTools I can see 3 things:

  1. It should be a GET request instead of POST.
  2. Request parameters should be sent as URL parameters instead of form-data
  3. The URL should be https://www.theocc.com/mdapi/series-search

The form data should be:

import requests

tickers = ['MSFT']
myurl = "https://www.theocc.com/mdapi/series-search"

for ticker in tickers:
    session = requests.session()
    # construct the POST request
    payload = {
        'symbol_type' : 'U',
        'symbol' : ticker,
        'exchange': '',
    }
    r = requests.get(myurl, params=payload),
    print(r[0])
    print(r[0].text)

Upvotes: 1

BrokenBenchmark
BrokenBenchmark

Reputation: 19252

You should be using a GET request rather than a POST request, and the form data should be query parameters:

query_params = {
    'symbolType' : 'U',
    'symbolText' : ticker
}
r = requests.get(myurl, headers=headers, params=query_params)

Upvotes: 1

Related Questions