CODEWITHSUNDEEP
pythonselenium
Reezal 2040
Reezal 2040

Reputation: 55

PYTHON : Finding hidden element in html source code

I'm trying get the url below which is hidden inside the page source code at https://www.aliexpress.com/item/32212764152.html but it's hidden inside a script tag.

<script>
                    window.runParams = {"descriptionModule":{"descriptionUrl":"https://aeproductsourcesite.alicdn.com/product/description/pc/v2/en_US/desc.htm?productId=32212764152&key=HTB1GwO_aVY7gK0jSZKzM7OikpXac.zip&token=f32528ddd34e37aecddda1c7778d5f0c"} .... </script>

I've manage to get the source code but not sure how to extract the url as an object.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
import re

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
CHROMEDRIVER_PATH = '/Users/reezalaq/PycharmProjects/wholesale/driver/chromedriver'
options = Options()
options.headless = False
driver = webdriver.Chrome(CHROMEDRIVER_PATH, options=options)
driver.get('https://www.aliexpress.com/item/32212764152.html')

html = driver.page_source

def run_script():
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    body = driver.find_element_by_css_selector('body')
    body.send_keys(Keys.PAGE_UP)

count = 0
while count < 3: #13
     run_script()
     count+=1
     time.sleep(5)

x = html.startswith('https://aeproductsourcesite.alicdn.com')
print(x)

How do I filter everything else in the source code and have an object ?

x = "https://aeproductsourcesite.alicdn.com/product/description/pc/v2/en_US/desc.htm?productId=32212764152&key=HTB1GwO_aVY7gK0jSZKzM7OikpXac.zip&token=f32528ddd34e37aecddda1c7778d5f0c"

Upvotes: 3

Views: 370

Answers (1)

Sers
Sers

Reputation: 12255

You can use Regular Expression to extract the value:

import re
#..
url = re.compile(r'"descriptionUrl":"([^"]*)"').search(html).group(1)

Upvotes: 2

Related Questions