CODEWITHSUNDEEP
pythonjsonurl
bikoman57
bikoman57

Reputation: 126

Python parsing JSON from url incomplete

I am trying to get a JSON from a URL I have succeeded in doing that but when I print the result it seems like I get only half of the content.

this is what I wrote:

with urllib.request.urlopen("https://financialmodelingprep.com/api/v3/company/stock/list") as url:
    data = json.loads(url.read().decode())
    print(data)

I won't print what I am getting because it's very long(if you look at the source and ctrl+f look up the sentence you will find that its in the middle of the page) but it starts with ties Municipal Fund', 'price': 9.83,

thanks for the help.

Upvotes: 0

Views: 313

Answers (2)

Gabio
Gabio

Reputation: 9484

I think that your output has been cut by your IDE.

When I run the same request and I write it into a file, you can see the that the data is fully written:

import requests
with requests.get("https://financialmodelingprep.com/api/v3/company/stock/list") as url:
    data = url.content.decode()
    with open("file.txt","w") as f:
        f.write(data)

Upvotes: 2

Serge Ballesta
Serge Ballesta

Reputation: 148870

When the response is large, the first read may only return what is available at that time. If you want to be sure to have the full data, you should loop reading until you get nothing (an empty byte string here).

Here, it would probably be more robust to pass the response stream to json.load and let it read until the json string is completed:

with urllib.request.urlopen(
        "https://financialmodelingprep.com/api/v3/company/stock/list") as url:
    data = json.load(url)
    print(data)

But I think that if the data was partial, json.loads should have choked on it. So the problem is likely elsewhere...

Upvotes: 1

Related Questions