CODEWITHSUNDEEP
pythonweb-scrapingpython-requests
Eduardo
Eduardo

Reputation: 121

Python HTTP Request without success

I am moving from VBA to Python and trying to migrate some code. I have so me good results so far but i get stuck every now and then.

This is the case. 

from datetime import datetime
import requests
import re
from dateutil.relativedelta import relativedelta

date_from = datetime.now() - relativedelta(months=36)
date_to = datetime.now()


def login_tokyo(ism_imo):
    r_tokyo_entry = requests.get('https://apcis.tmou.org/public/')
    string_with_number = re.search("<span.*?>(.*?)</span>", r_tokyo_entry.text)
    numbers = re.findall('[0-9]+', str(string_with_number.group(1)))
    captcha = int(numbers[0]) + int(numbers[1])

    payload = {'captcha': captcha}

    r_tokyo_main = requests.post('https://apcis.tmou.org/public/?action=login',
                                 data=payload, cookies=r_tokyo_entry.cookies)

    payload1 = {'Param': '0', 'Value': ism_imo, 'imo': '', 'callsign': '', 'name': '', 'compimo': ism_imo,
                'compname': '', 'From': date_from.strftime('%d.%m.%Y'), 'Till': date_to.strftime('%d.%m.%Y'),
                'authority': '0', 'flag': '0', 'class': '0', 'ro': '0', 'type': "0", 'result': '0', 'insptype': '-1',
                'sort1': '0', 'sort2': 'DESC', 'sort3': '0', 'sort4': 'DESC'}

    r_tokyo_performance = requests.post('https://apcis.tmou.org/public/?action=getcompanies',
                                        data=payload1, cookies=r_tokyo_entry.cookies)
    print(r_tokyo_performance.text)


ism_imo = 3028090
login_tokyo(ism_imo)

When trying to print the print(r_tokyo_main.text) I get nothing when I was suposed to get some xml response.

Any suggestions? Edited to have the full code.

Upvotes: 0

Views: 119

Answers (2)

ajay gandhi
ajay gandhi

Reputation: 563

In This case you need to send back the cookies as well when sending post request.

r_tokyo_main = requests.post('https://apcis.tmou.org/public/?action=login', data=payload, cookies = r_tokyo_entry.cookies)

so, whatever cookies you received in get response, send them back

Upvotes: 2

SIM
SIM

Reputation: 22440

The following should work. When you run the script, you will get this text SEARCH IN DATABASE from the target page:

import re
import requests

def login_tokyo(s):
    r = s.get('https://apcis.tmou.org/public/')
    str_number = re.findall("<span[^>]+(.*?)</span>", r.text)[0]
    numbers = re.findall('[0-9]+', str_number)
    captcha = int(numbers[0]) + int(numbers[1])
    payload = {'captcha': captcha}
    r = s.post('https://apcis.tmou.org/public/?action=login', data=payload)
    check_text = re.findall('<b>(.*?)</b>', r.text)[0]
    print(check_text)

if __name__ == '__main__':
    with requests.Session() as s:
        login_tokyo(s)

Upvotes: 1

Related Questions