CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Hadeess
Hadeess

Reputation: 29

How to scrape a livetable with BS4 Python

I'm trying to scrape https://www.scorespro.com/soccer/england/premier-league/standings/ to return the table, but i can't seem to find the row class name to return the search.

This is the row i want to scrape:

<tr class="sp-livetable__tableRow spm-firstRow spm-is-uppercase" data-sp-team-id data-sp-unique-team-id>

And my current code:

    table = soup.find(True, {'id':'standings_1a'})
    table_body = table.find('tbody')

    teams = table_body.find_all('tr')

It returns an empty list.

Upvotes: 1

Views: 50

Answers (1)

Jack Fleeting
Jack Fleeting

Reputation: 24940

In this particular case, you are better off using pandas:

import pandas as pd
url = 'https://www.scorespro.com/soccer/england/premier-league/standings/'
teams = pd.read_html('https://www.scorespro.com/soccer/england/premier-league/standings/')

df = teams[0].iloc[: ,1:-1] #remove unnecessary columns #extract the relevant dataframe

#convert the first row to a column header
nh = df.iloc[0] #select the first row
df = df.iloc[1:]#grab the df without the first row
df.columns = nh #add the new header

print(df)

Output:

0  Pos                   Team   P   W   D   L    GF:GA DIFF pts   Form
1    1    Manchester City MNC  35  25   5   5  72 : 26   46  80  LWWLW
2    2  Manchester United MNU  34  20  10   4  67 : 36   31  70  WDWWW
3    3            Chelsea CHE  35  18  10   7  55 : 32   23  64  WWWDW
4    4          Leicester LEC  35  19   6  10  63 : 43   20  63  LDWWL
5    5           West Ham WHU  35  17   7  11  55 : 45   10  58  LWLLW
6    6          Liverpool LFC  34  16   9   9  57 : 39   18  57  WDDWW
7    7          Tottenham TOT  35  16   8  11  61 : 41   20  56  LWWDL

etc.

Upvotes: 1

Related Questions