CODEWITHSUNDEEP
pythonparsingweb-scrapingbeautifulsoupscreen-scraping
user10445685
user10445685

Reputation:

<table> becomes empty, when I'm trying to get it via BeautifulSoup

I'm trying to parse a table from website https://www.kp.ru/best/kazan/abiturient_2018/ivmit/. DevTools by Chrome shows me that table is:

<div class="t431__table-wapper" data-auto-correct-mobile-width="false"> 
<table class="t431__table " style="">
...
</table>
</div>

But when I do this:

url = r"https://www.kp.ru/best/kazan/abiturient_2018/ivmit/"
r = requests.get(url)
soup = BeautifulSoup(r.text, 'html.parser')
tag = soup.find_all('div', {'class':r't431__table-wapper'})
print(tag)

It returns me like <table> is empty:

[<div class="t431__table-wapper" data-auto-correct-mobile-width="false">
<table class="t431__table" style=""></table></div>, 
<div class="t431__table-wapper" data-auto-correct-mobile-width="false">
<table class="t431__table" style=""></table></div>,
<div class="t431__table-wapper" data-auto-correct-mobile-width="false">
<table class="t431__table" style=""></table></div>,
<div class="t431__table-wapper" data-auto-correct-mobile-width="false">
<table class="t431__table" style=""></table></div>]

Is it JavaScript or something? How to fix this?

Upvotes: 1

Views: 52

Answers (1)

QHarr
QHarr

Reputation: 84465

You can get that info from another tag

import requests
from bs4 import BeautifulSoup as bs

url = 'https://www.kp.ru/best/kazan/abiturient_2018/ivmit/'
soup = bs(requests.get(url).content, 'lxml')
print(soup.select_one('.t431__data-part2').text)

Output:

enter image description here

Upvotes: 1

Related Questions