CODEWITHSUNDEEP
pythonpython-3.xpython-requests
Gabriel
Gabriel

Reputation: 42459

Submit form using requests package

I know that there are many similar questions in SO, but I've tried every combination of get, post, data, params, etc in the requests package and I can not get the form to submit. I either get back an empty string, or the same page (ie: not the page I'm expecting to get after hitting Submit)

import requests
webserver = 'http://stev.oapd.inaf.it/cgi-bin/cmd'

# Returns the landing site
r = requests.get(webserver, data={'submit_form': 'Submit'}).text
# Returns an empty string
r = requests.get(webserver, params={'submit_form': 'Submit'}).text
# Returns an empty string
r = requests.post(webserver, data={'submit_form': 'Submit'}).text
# Returns the landing site
r = requests.post(webserver, params={'submit_form': 'Submit'}).text

The site changed recently, and I remember that one of these commands definitely used to work. Why is neither working now?

Upvotes: 2

Views: 80

Answers (1)

Nazim Kerimbekov
Nazim Kerimbekov

Reputation: 4783

As you mentioned the website has changed recently. It appears that it is now using form-data in order to receive the POST requests and also requires the default values to be sent.

All in all, your code should look like this:

import requests

webserver = 'http://stev.oapd.inaf.it/cgi-bin/cmd'

headers = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:66.0) Gecko/20100101 Firefox/66.0", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Language": "en-US,en;q=0.5"}

files = {
    'submit_form': (None,"Submit"),
    'cmd_version': (None,"3.2"),
    'track_parse': (None,"parsec_CAF09_v1.2S"),
    'track_colibri': (None,"parsec_CAF09_v1.2S_S35"),
    'track_postagb': (None,"no"),
    "n_inTPC": (None,"10"),
    "eta_reimers": (None,"0.2"),
    "kind_interp": (None,"1"),
    "kind_postagb": (None,"-1"),
    "photsys_file": (None,"tab_mag_odfnew/tab_mag_ubvrijhk.dat"),
    "photsys_version": (None,"YBC"),
    "dust_sourceM": (None,"dpmod60alox40"),
    "dust_sourceC": (None,"AMCSIC15"),
    "kind_mag": (None,"2"),
    "kind_dust": (None,"0"),
    "extinction_av": (None,"0.0"),
    "extinction_coeff": (None,"constant"),
    "extinction_curve": (None,"cardelli"),
    "imf_file": (None,"tab_imf/imf_kroupa_orig.dat"),
    "isoc_isagelog": (None,"0"),
    "isoc_agelow": (None,"1.0e9"),
    "isoc_ageupp": (None,"1.0e10"),
    "isoc_dage": (None,"0.0"),
    "isoc_lagelow": (None,"6.6"),
    "isoc_lageupp": (None,"10.13"),
    "isoc_dlage": (None,"0.0"),
    "isoc_ismetlog": (None,"0"),
    "isoc_zlow": (None,"0.0152"),
    "isoc_zupp": (None,"0.03"),
    "isoc_dz": (None,"0.0"),
    "isoc_metlow": (None,"-2"),
    "isoc_metupp": (None,"0.3"),
    "isoc_dmet": (None,"0.0"),
    "output_kind": (None,"0"),
    "output_kind": (None,"0"),
    "output_evstage": (None,"1"),
    "lf_maginf": (None,"-15"),
    "lf_magsup": (None,"20"),
    "lf_deltamag": (None,"0.5"),
    "sim_mtot": (None,"1.0e4"),
    ".cgifields": (None,"track_parsec"),
    ".cgifields": (None,"extinction_coeff"),
    ".cgifields": (None,"dust_sourceC"),
    ".cgifields": (None,"output_kind"),
    ".cgifields": (None,"extinction_curve"),
    ".cgifields": (None,"dust_sourceM"),
    ".cgifields": (None,"isoc_ismetlog"),
    ".cgifields": (None,"output_gzip"),
    ".cgifields": (None,"photsys_version"),
    ".cgifields": (None,"track_colibri"),   
    }

r = requests.post(webserver, headers=headers, files = files)

Hope this helps!

Upvotes: 2

Related Questions