CODEWITHSUNDEEP
pythonhtmlweb-scrapingbeautifulsoup
ttammountain
ttammountain

Reputation: 17

Extract list from a Python script in web scraped html page

I'm new to web scraping and ran into a small road block with the following code:

import requests
from bs4 import BeautifulSoup
url = "www.website.com"
page = requests.get(url)
soup = BeautifulSoup(page.content, "html.parser")
price_scripts = soup.find_all('script')[23]
print(price_scripts)

The scripts that are pulled all appear to be Python scripts. Here's what's printed from the above code: 

<script>
        p.a = [0,"6.93","9.34","3.42","7.88"];
        p.output();
</script>

What I'm trying to do is pull the list from this script, but when I attempt it just returns "None".

Upvotes: 1

Views: 82

Answers (1)

Jack Fleeting
Jack Fleeting

Reputation: 24930

You should be able to extract the data this way:

target = price_scripts.text

which outputs:

p.a = [0,"6.93","9.34","3.42","7.88"];
    p.output();

At this point you need to resort to string manipulation, by stripping out everything between the brackets, like so:

print(target.text.split('[')[1].split(']')[0])

Note that each use of the split() method creates a list, so you have to choose the correct element from the list. output:

0,"6.93","9.34","3.42","7.88"

Upvotes: 1

Related Questions