CODEWITHSUNDEEP
pythonbeautifulsoup
J. Smith
J. Smith

Reputation: 33

Web scraping issue

I am trying to web scrape a page but I keep getting an error message. "raise HTTPError(req.full_url, code, msg, hdrs, fp) HTTPError: Not Found".

Can anyone see why I am getting this issue and how I can fix it?

Here is my code:

import bs4
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
U="https://llis.nasa.gov/search?organization=arc&page=1"


uClient=uReq(U)
page_html=uClient.read()
uClient.close()
page_soup=soup(page_html,"html.parser")
page_soup.h2

Upvotes: 0

Views: 85

Answers (1)

RomanPerekhrest
RomanPerekhrest

Reputation: 92854

Use requests module as the more powerful and flexible for dealing with HTTP requests:

import bs4, requests

url = "https://llis.nasa.gov/search?organization=arc&page=1"
data = requests.get(url)
soup = bs4.BeautifulSoup(data.content, 'html.parser')

print(soup.body)

The output:

<body>
<script src="assets/vendor.js"></script>
<script src="assets/js/bootstrap.min.js"></script>
<script src="assets/llis.js"></script>
</body>

Upvotes: 1

Related Questions