CODEWITHSUNDEEP
pythonpython-3.xhttp-redirect
Brent
Brent

Reputation: 778

How to get the link I get redirected to

I have a program with a config file in my program that you can add company stock symbols to then it takes those stock symbols that are in that config file and searches for news articles which is all information getting pulled from an API. One of the categories that I am printing out is a URL field so the URL that I'm printing out is a URL that redirects you to another URL (the real URL). Now what I'm trying to do is get the URL I get redirected to print out

I have a global companyurl list that I am appending all the URLs I pull to so all the generic redirect URLs are going in there. I am getting the redirected URL, the only problem is I am only getting 1 redirected URL and in turn it is printing that URL for every news article I am printing. It's a little hard to explain, so if you need further clarification just ask.

Here is my code, and what I have commented out is what I was trying.

For testing purposes here are 2 stock symbols you can add into a config file. If you wish to test something: aapl and yelp, just put them on separate lines in the config file.

import sys
import json
import urllib.request
import time
import datetime
import requests

def main():
    openconfigfile()
    searchfornews()

def openconfigfile():
    mylist = []
    with open('config.txt') as myfile:
        for company in myfile:
            mylist.append(company.strip())
    return mylist

companyurl = []
def searchfornews():
    myurl = []
    global companyurl
    url = 'https://api.iextrading.com/1.0/stock/'
    companies = openconfigfile()
    for company in companies:
        stockinput = company + '/news/last/2'
        createdurl = url + stockinput
        myurl.append(createdurl)
    while True:
        try:
            for url in myurl:
                fob = urllib.request.urlopen(url)
                data = fob.read().decode('utf-8')
                companydata = json.loads(data)
                for company in companydata:
                    company['datetime'] = reformatdate()
                    companyurl.append(company['url'])
                    # r = getredirectedlink()
                    # company['url'] = r.url

                    print('''======== [%s] ========
%s:   "%s"
%s
tags: %s''' % (company['datetime'], company['source'], company['headline'], company['url'], company['related']))

            time.sleep(30)
        except Exception as e:
            print()
            print('''ERROR: news not found for 1 or more stock symbols
You have a stock symbol in the config file that doesnt match any known stock symbol''', e)
            time.sleep(30)

def reformatdate():
    time = datetime.datetime.today()
    newtime = time.strftime('%B %d %Y, %I:%M %p')
    return newtime

# def getredirectedlink():
#     global companyurl
#     for x in companyurl:
#         r = requests.get(x)
#         return r

if __name__ == '__main__':
    sys.exit(main())

Upvotes: 1

Views: 50

Answers (1)

MegaIng
MegaIng

Reputation: 7886

You are nearly finished. You just have to change two things:

  1. inside searchfornews:

    company['datetime'] = reformatdate()
companyurl.append(company['url'])
# r = getredirectedlink()
# company['url'] = r.url

    change to

    company['datetime'] = reformatdate()
company['url'] = getredirectedlink(company['url'])
companyurl.append(company['url'])

    And change getredirectedlink to the following:

    def getredirectedlink(companyurl):
    r = requests.get(companyurl)
    return r.url

Upvotes: 1

Related Questions