Read html table correctly

Question

Here is a HTML table:

The last row here has 2x more columns than others. When I'm trying to read it into the Pandas dataframe I get this result:

table = pd.read_html('1111.html')
table[0]

   0   1  2   3  4   5  6   7
0  A   A  B   B  C   C  D   D
1  E   E  F   F  G   G  H   H
2  I   I  J   J  K   K  L   L
3  M  M2  N  N2  O  O2  P  P2

How to read it correctly, without dubbing? I don't need the last row.

Ajax1234 · Accepted Answer

You can use BeautifulSoup to parse the table and then convert the results to a dataframe:

import pandas as pd
from bs4 import BeautifulSoup as soup
df = pd.DataFrame([[k[1:-1] for i in b.find_all('td') if (k:=i.text) is not None] for b in soup(html, 'html.parser').table.find_all('tr')])

Output:

   0   1  2   3     4     5     6     7
0  A   B  C   D  None  None  None  None
1  E   F  G   H  None  None  None  None
2  I   J  K   L  None  None  None  None
3  M  M2  N  N2     O    O2     P    P2

Edit: solution without assignment expression:

df = pd.DataFrame([[i.text[1:-1] if i else i for i in b.find_all('td')] for b in soup(html, 'html.parser').table.find_all('tr')])

Output:

   0   1  2   3     4     5     6     7
0  A   B  C   D  None  None  None  None
1  E   F  G   H  None  None  None  None
2  I   J  K   L  None  None  None  None
3  M  M2  N  N2     O    O2     P    P2

Read html table correctly

Answers (1)

Related Questions