CODEWITHSUNDEEP
pythonpython-3.xweb-scrapingbeautifulsoup
Rafiul Islam
Rafiul Islam

Reputation: 319

Why didn't I get any content between HTML tag?

I am trying to extract data from this website "https://www.physiotherapyexercises.com/". But when i run the following code i get only the html body , didn't get the content between the HTMl tags.

import requests
import bs4

url = "https://www.physiotherapyexercises.com/"
res = requests.get(url)
soup = bs4.BeautifulSoup(res.text, 'lxml')

print(soup)

Upvotes: 1

Views: 54

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195418

The data you are searching for is loaded from other location using Javascript. You can use requests/re/json modules to obtain the information.

For example:

import re
import json
import requests

url = 'https://www.physiotherapyexercises.com/Js/Data/ExerciseData_English_2019_12_06_00_49_56.js'

data = json.loads(re.search(r'exerciseRecords=(.*?]);', requests.get(url).text).group(1))

# uncomment this to print all data:
# print(json.dumps(data, indent=4))

for exercise in data:
    print(*exercise['Texts'], sep='\n')
    print('-' * 80)

Prints:

...

Strength - knee - flexors - wall mounted pulleys - prone.
Knee flexor strengthening in prone using pulleys
To strengthen the knee flexors.
To strengthen the muscles at the back of your thigh.
Position the patient in prone facing away from the pulleys. Adjust the pulley system so that the direction of pull opposes knee flexion. Instruct the patient to flex their knee.
Position yourself lying on your stomach. Adjust the pulley system so that the direction of pull is towards the top of the bed. Start with your knee straight and leg resting on the bed. Finish with your knee bent.
More advanced: 1. Progress using strength training principles.

--------------------------------------------------------------------------------
Strength - knee - flexors - theraband - prone.
Knee flexor strengthening in prone using theraband
To strengthen the knee flexors.
To strengthen the muscles at the back of your thigh.
Position the patient in prone with their knee extended. Adjust the theraband so that the direction of pull opposes knee flexion. Instruct the patient to flex their knee.
Position yourself lying on your stomach. Adjust the theraband so that the direction of pull is towards the top of the bed. Start with your knee straight and leg resting on the bed. Finish with your knee bent.
Less advanced: 1. Downgrade the colour of the theraband. More advanced: 1. Upgrade the colour of the theraband.

... and so on.

Upvotes: 2

Related Questions