CODEWITHSUNDEEP
pythonrequestsession-cookies
BedSon
BedSon

Reputation: 57

requests.Session() creating different session every time. How to reuse it?

I am trying to initialize a requests.Session, to keep a connection with a webpage. However, I read that each time the session class is called, a new session is created.

How is it possible to keep the connection alive? Because with my current code, it's giving me the webpage content after I call the login method (that's OK, it shows that I logged into the page and gives me the content I want), but when I call the update method, it gives me the content from the login page again, not from the page I actually want after login.

import requests

class LoginLogout:

    # creating session
    def __init__(self):
        self.s = requests.Session()

    # login method
    def login(self, user, password, server):
        payload_data = {'user': user, 'pass': password, 'server': server}
        print(self.s.post(LOGIN_LINK, payload_data))

    # update method
    def update(self, updt_link):
        print(self.s.get(updt_link))

    def logout(self):
        response = self.s.get('...some webpage/user/logout...')
        self.s.close()
        print(response)

Here I am calling the objects:

if switch_parameter == "login":
    login_var = LoginLogout()
    login_var.login(USER, PASSWORD, SERVER)
    print('IS IT OK ?', login_var.s.get('.../login...'))  # <-OK it shows 200 result (but should I use there "s" too ?)
elif switch_parameter == "start":
    start()
elif switch_parameter == "stop":
    stop()
elif switch_parameter == "update":
    update_prem = LoginLogout()
    update_prem.update('...different/page...')
    # (am I missing here "s" ?, or I shouldnt be using it here anyway)
elif switch_parameter == "logout":
    logout()
else:
    pass

What am I doing wrong here? I just want to use login to log into the website and keep the session active, while calling update every time I need to get another page. Am I even on the right track or completely wrong?

Upvotes: 2

Views: 3072

Answers (1)

Daniil Fajnberg
Daniil Fajnberg

Reputation: 18488

The whole point of requests.Session is to persist ephemeral constants (like cookies) between requests. In your code you initialize a new session object, when you initialize a LoginLogout object.

You do that here:

if switch_parameter == "login":        
    login_var = LoginLogout()
...

And you do that here:

elif switch_parameter == "update":
    update_prem = LoginLogout()
...

Now login_var and update_prem are obviously different objects and both have the s attribute, each holding a different requests.Session object. How do you expect the attributes of one session to be magically available to the other?

If you want to use an existing session, use it. Don't create a new one.

I don't know about your actual use case of course, but from what you have presented here, it seems you need to do something like this:

scraper_obj = LoginLogout()
scraper_obj.login(USER, PASSWORD, SERVER)
...
scraper_obj.update('...')
...
scraper_obj.logout()

Since your created a wrapper around the actual requests.Session instance with LoginLogout, you should not ever need to deal with its s attribute directly, assuming you have methods on LoginLogout for every kind of request you want to make. You initialize it once and then use its methods to perform requests via its internal session object.

PS

You casually mentioned in a follow-up comment that you set this up as a script to be called repeatedly from the outside and depending on the parameter passed to the script, you want to either log into the site or scrape a specific page.

This shows that you either don't understand how "logging in" even works or that you don't understand how processes work. Typically some session attribute (e.g. cookie) is created on the client so that it can present it to the server to show that it is already authenticated. When using requests as an HTTP client library, this data is stored inside a requests.Session object.

When you call a Python script, you create a new process. Just because you start the same script twice in a row does not mean that one of those processes has any connection to the other. Calling the script to login once has absolutely no effect on what happens the next time you call that script to do something else. None of those earlier session attributes will be present in the second process. I hope this is clear now.

Upvotes: 3

Related Questions