CODEWITHSUNDEEP
pythonpython-requestsmediawiki-api
Jeremy
Jeremy

Reputation: 2000

Why is this python request returning the same value each time?

I'm using they python requests library for the first time, and I am confused. When I run the function below in a for loop, with different base URLs, it appears to receive a response, but the content that is returned is the same for both URLs.

If I look at the API URL in my browser, then I see that it's the content for the first URL that's being returned both times. What am I missing?

base_urls = ['http://kaweki.wikia.com/','http://solarmovie.wikia.com/']


def getEdits(wikiObj, limit=500):     
    payload = {'limit': limit}                             
    r = requests.get('{}api/v1/Activity/LatestActivity'.format(wikiObj),
                     params=payload)
    edits = r.json()
    return edits['items']

for url in base_urls:
    print getEdits(url)

Upvotes: 5

Views: 3303

Answers (3)

Nir Alfasi
Nir Alfasi

Reputation: 53535

There is a bug on the server side which ignores cache-control headers and such for a period of time.

Introducing sleep of 5 secs (maybe even shorter periods) works around the bug. I've marked the lines that were added below:

import requests
import json
from time import sleep #ADDED

base_urls = ['http://kaweki.wikia.com/', 'http://solarmovie.wikia.com/']


def getEdits(wikiObj, limit=500):       
    payload = {'limit': limit}   
    url = '{}api/v1/Activity/LatestActivity'.format(wikiObj)
    r = requests.get(url, params=payload) 
    edits = json.loads(r.content)
    return edits['items']

for url in base_urls:    
    print getEdits(url)  
    sleep(5) # ADDED

OUTPUT

[{u'article': 1461, u'revisionId': 14, u'user': 26127114, u'timestamp': 1424389645}, {u'article': 1461, u'revisionId': 13, u'user': 26127114, u'timestamp': 1424389322}, {u'article': 1461, u'revisionId': 12, u'user': 26127114, u'timestamp': 1424389172}, {u'article': 1461, u'revisionId': 5, u'user': 26127114, u'timestamp': 1424388924}]
[{u'article': 1461, u'revisionId': 14, u'user': 26127165, u'timestamp': 1424389107}, {u'article': 1461, u'revisionId': 7, u'user': 26127165, u'timestamp': 1424388706}]

Upvotes: 5

abraham
abraham

Reputation: 47853

The API endpoints are "broken". Refreshing the two endpoints in a browser repeatedly has them switching back and forth between two responses. You can replicate it by making refreshing one request half a dozen times, and then refreshing the other request half a dozen times and switching back and forth every half a dozen requests.

Request A:

http://solarmovie.wikia.com/api/v1/Activity/LatestActivity

Request B:

http://kaweki.wikia.com/api/v1/Activity/LatestActivity

Response 1:

{
    items: [
        {
            article: 1461,
            user: 26127114,
            revisionId: 14,
            timestamp: 1424389645
        },
        {
            article: 1461,
            user: 26127114,
            revisionId: 13,
            timestamp: 1424389322
        },
        {
            article: 1461,
            user: 26127114,
            revisionId: 12,
            timestamp: 1424389172
        },
        {
            article: 1461,
            user: 26127114,
            revisionId: 5,
            timestamp: 1424388924
        }
    ],
    basepath: "http://kaweki.wikia.com"
}

Response 2:

{
    items: [
        {
            article: 1461,
            user: 26127165,
            revisionId: 14,
            timestamp: 1424389107
        },
        {
            article: 1461,
            user: 26127165,
            revisionId: 7,
            timestamp: 1424388706
        }
    ],
    basepath: "http://solarmovie.wikia.com"
}

Upvotes: 3

Andrew Gorcester
Andrew Gorcester

Reputation: 19983

I downloaded and ran the script and got apparently identical output. There doesn't seem to be anything wrong with the script, though! I think the output is simply identical, for some reason. Try to change return edits['items'] to just return edits and you'll see that the output is different in that case. If there really is a bug in the code, that should help you isolate it; if not, then maybe you can figure out why the real output is like that.

Upvotes: 0

Related Questions