Sudip Adhikari
Reputation: 202
Getting 500 Response on Python Requests
import requests
url = "https://uk.eu-supply.com/ctm/supplier/publictenders?B=BLUELIGHT"
payload={'Branding': 'BLUELIGHT',
'SearchFilter.BrandingCode': 'BLUELIGHT',
'CpvContainer.CpvIds': '',
'CpvContainer.CpvIds': '',
'SearchFilter.PagingInfo.PageNumber': '2',
'SearchFilter.PagingInfo.PageSize': '25'}
files=[
]
headers = {
'Cookie': 'EUSSESSION=e3cc7bc4-ea51-4c4b-99f7-2ebde589c8e0'
}
response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.status_code)
eu-supply.com
https://uk.eu-supply.com
I am trying to hit request to this site, the same payload and url request works with postman. Notice (two same fields on payload "CpvContainer.CpvIds", this is must. In order to get 200 response , we need to pass this field twice. But as dict can only set one key unique. I am getting 500.
Upvotes: 1
Views: 134
Answers (3)
Anand Tripathi
Reputation: 16126
In the above API use the payload as raw text. I cross-checked the below code is successfully fetching the data
import requests
url = "https://uk.eu-supply.com/ctm/Supplier/publictenders/PublicTenders"
payload="Branding=BLUELIGHT&SearchFilter.BrandingCode=BLUELIGHT&CpvContainer.CpvIds=&CpvContainer.CpvIds=&SearchFilter.PagingInfo.PageNumber=2&SearchFilter.PagingInfo.PageSize=25"
headers = {
'Connection': 'keep-alive',
'sec-ch-ua': '" Not;A Brand";v="99", "Google Chrome";v="91", "Chromium";v="91"',
'Accept': '*/*',
'X-Requested-With': 'XMLHttpRequest',
'sec-ch-ua-mobile': '?0',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36',
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
'Origin': 'https://uk.eu-supply.com',
'Sec-Fetch-Site': 'same-origin',
'Sec-Fetch-Mode': 'cors',
'Sec-Fetch-Dest': 'empty',
'Referer': 'https://uk.eu-supply.com/ctm/supplier/publictenders?B=BLUELIGHT',
'Accept-Language': 'en-US,en;q=0.9',
'Cookie': 'EUSSESSION=b18ce90d-32b9-429c-8d74-7151a997e8cd'
}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)
Upvotes: 1
Meysam Kermani
Reputation: 41
Make the repeated key's values as a list:
payload={
...
'CpvContainer.CpvIds': ['',''],
...
}
Upvotes: 1
Andrej Kesely
Reputation: 195408
To read a table from the page, simple pd.read_html is enought:
import pandas as pd
url = "https://uk.eu-supply.com/ctm/supplier/publictenders?B=BLUELIGHT"
df = pd.read_html(url)[0]
print(df)
Prints:
Quote/tender Id Reference Name Date of publication Response deadline (UK time) Process Buyers Countries
0 45024 7F- 2020-0278 Kennelling of Seized & Dangerous Dogs (Kent Police and Essex Police) 25/06/2021 26/07/2021 12:00 Above Threshold OPEN Procedure 7 Forces Procurement United Kingdom
1 45074 CONT0056 Leaflets for Crime Commissioner 25/06/2021 05/07/2021 12:00 Tender/Quotation (below Threshold value) Leicestershire Police United Kingdom
2 45072 7F-2021-P038 Bury St Edmunds Police Station - Car Park Extension 25/06/2021 26/07/2021 12:00 Formal Tender/Quotation (Advertised) (over £50,000) 7 Forces Procurement United Kingdom
3 44234 7F - 2020 - 0387 Fire Doors Inspection and Maintenance PMA 24/06/2021 26/07/2021 10:00 Above Threshold OPEN Procedure 7 Forces Procurement United Kingdom
4 45068 7F-2021-P040 Armoury Improvements- Bury St Edmunds Police Station 24/06/2021 26/07/2021 12:00 Formal Tender/Quotation (Advertised) (over £50,000) 7 Forces Procurement United Kingdom
5 45042 BLC-Aviation-003 Provision of Police Aviation Services including a Fleet Replacement Programme 23/06/2021 06/07/2021 12:00 Prior Information Notice (PIN) (Standalone) BlueLight Commercial United Kingdom
6 42896 DP0570 Reactive Mechanical Services and Small Works Projects 23/06/2021 03/08/2021 12:00 Above Threshold OPEN Procedure The Police and Crime Commissioner for Derbyshire United Kingdom
7 45043 BLC-Aviation-003 Market Engagement Event for the Provision of Police Aviation Services including a Fleet Replacement Programme 23/06/2021 06/07/2021 12:00 Request for Information /Market Consultation BlueLight Commercial United Kingdom
8 44914 20-21 EM15 Asbestos Management Services Contract 22/06/2021 20/07/2021 12:00 Formal Tender/Quotation (Advertised) ( up to £50,000) Tyne and Wear Fire and Rescue Service (TWFRS) United Kingdom
9 44793 1363-2016 Recovery of loose, stray and abandoned horses 21/06/2021 21/07/2021 14:00 Prior Information Notice (PIN) (Standalone) West Yorkshire Combined Authority United Kingdom
10 44734 7F 2020 0339 Bodyshop Replacement and Repairs 21/06/2021 23/07/2021 12:00 Above Threshold OPEN Procedure 7 Forces Procurement United Kingdom
11 44984 NaN Integrated Communications Control System (ICCS) for Northamptonshire Police 18/06/2021 30/06/2021 17:00 Prior Information Notice (PIN) (Standalone) MINT Commercial Services LLP United Kingdom
12 44983 WMP0126 Garage Equipment - Maintenance, Calibration and Repair 18/06/2021 19/07/2021 12:00 Tender/Quotation (below Threshold value) West Midlands Police United Kingdom
13 44962 DHR Chair & Author Domestic Homicide Review Chair & Author 17/06/2021 05/07/2021 12:00 Prior Information Notice (PIN) (Standalone) 7 Forces Procurement United Kingdom
14 44950 1197 Access Control/Automated Security Gate Maintenance and Related Services 17/06/2021 23/07/2021 14:00 Formal Tender/Quotation (Advertised) (over £50,000) Royal Berkshire Fire Authority United Kingdom
15 44959 627_RFI Evidence Based Practice Partnership Board 17/06/2021 16/07/2021 17:00 Request for Information /Market Consultation Chief Constable for Devon and Cornwall Police United Kingdom
16 44944 W344 Repair & Maintenance of UPS Systems 16/06/2021 13/07/2021 12:00 Tender/Quotation (below Threshold value) Northumbria Police United Kingdom
17 44942 7F-2021-C058 Specialist Accredited Fire Investigation 16/06/2021 30/06/2021 12:00 Request for Information /Market Consultation 7 Forces Procurement United Kingdom
18 44916 WYP/CS/019/JB Fire Risk Assessments 16/06/2021 30/06/2021 14:00 Formal Tender/Quotation (Advertised) ( up to £50,000) West Yorkshire Combined Authority United Kingdom
19 44594 7F-2020-0273 Norfolk Integrated Domestic Abuse Service 15/06/2021 20/07/2021 12:00 Above Threshold OPEN Procedure 7 Forces Procurement United Kingdom
20 44911 SPCC Sussex PCC RfQ - An evaluation for the domestic abuse perpetrator programme 15/06/2021 30/06/2021 12:00 Formal Tender/Quotation (Advertised) ( up to £50,000) Office of the Sussex Police & Crime Commissioner United Kingdom
21 44543 2420-2021 Force Medical Advisor (FMA) / Selected Medical Practitioner (SMP) and other Occupational Health Services 15/06/2021 14/07/2021 13:00 Above Threshold OPEN Procedure West Yorkshire Combined Authority United Kingdom
22 44845 C003196 THERAPEUTIC INTERVENTIONS AND TRAINING PROVISION FOR WMFRA 15/06/2021 06/07/2021 16:00 Formal Tender/Quotation (Advertised) (over £30,000) West Midlands Fire and Rescue Authority United Kingdom
23 44544 ESFA0199 CMI Leadership and Management Online Training Courses 14/06/2021 09/07/2021 12:00 Formal Tender/Quotation (Advertised) (over £25,000) East Sussex Fire Authority United Kingdom
24 44714 590 Regional Grounds Maintenance and Winter Salting Service 11/06/2021 15/07/2021 12:00 Above Threshold OPEN Procedure Chief Constable for Devon and Cornwall Police United Kingdom
Upvotes: 1
Related Questions
- Why don't I get a response from my request?
- Python Web Scrape Request resulting in a 406 Error
- Issue using python request module
- How do I fix this Python request 500 error?
- Python Requests Response 404
- Why I am getting response 400 in Python requests?
- Python Requests POST - 400 - Bad Request with Python requests
- Python HTTP Request without success
- Web Scraping Error: Request and No Connection
- python requests POST 400 error