CODEWITHSUNDEEP
htmlpython-3.xbeautifulsoup
Lucas Mengual
Lucas Mengual

Reputation: 415

Extracting HTML from outside the <table> tag

I´m trying to extract the HTML part that is located above and below a <table> tag, so for example from the example html below:

sample_html = """
<html>
<title><b>Main Title</b></Title>
<b>more</b>
<b>stuff</b>
<b>in here!</b>
<table class="softwares" border="1" cellpadding="0" width="99%">
    <thead style="background-color: #ededed">
        <tr>
            <td colspan="5"><b>Windows</b></td>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td><b>Type</b></td>
            <td><b>Issue</b></td>
            <td><b>Restart</b></td>
            <td><b>Severity</b></td>  
            <td><b>Impact</b></td>  
        </tr>
        <tr>
            <td>some item</td>
            <td><a href="some website">some website</a><br></td>
            <td>Yes<br></td>
            <td>Critical<br></td>
            <td>stuff<br></td>
        </tr>    
        <tr>
            <td>some item</td>
            <td><a href="some website">some website</a><br></td>
            <td>Yes<br></td>
            <td>Important<br></td>
            <td>stuff<br></td>    
        </tr>
    </tbody>
</table>
<b>AGAIN</b>
<b>more</b>
<b>stuff</b>
<b>down here!</b>
</html>
"""

I would like to obtain something like.

top_html = """
<html>
<title><b>Main Title</b></Title>
<b>more</b>
<b>stuff</b>
<b>in here!</b>
</html>
"""

bottom_html = """
<html>
<b>AGAIN</b>
<b>more</b>
<b>stuff</b>
<b>down here!</b>
</html>
"""

Or already in text format, like:

top_html = 'Main Title more stuff down here!'

bottom_html = 'AGAIN more stuff down here!'

So I´ve been able to extract the <table> part of from the whole HTML and do my processing (I separate the rows <tr> and columns <td> so I can extract the values I need), with the following code:

soup = BeautifulSoup(input_html, "html.parser")
table = soup.find('table')

Upvotes: 0

Views: 65

Answers (2)

QHarr
QHarr

Reputation: 84475

Split the html on the table html then extract text

from  bs4 import BeautifulSoup as bs

sample_html = """
<html>
<title><b>Main Title</b></Title>
<b>more</b>
<b>stuff</b>
<b>in here!</b>
<table class="softwares" border="1" cellpadding="0" width="99%">
    <thead style="background-color: #ededed">
        <tr>
            <td colspan="5"><b>Windows</b></td>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td><b>Type</b></td>
            <td><b>Issue</b></td>
            <td><b>Restart</b></td>
            <td><b>Severity</b></td>  
            <td><b>Impact</b></td>  
        </tr>
        <tr>
            <td>some item</td>
            <td><a href="some website">some website</a><br></td>
            <td>Yes<br></td>
            <td>Critical<br></td>
            <td>stuff<br></td>
        </tr>    
        <tr>
            <td>some item</td>
            <td><a href="some website">some website</a><br></td>
            <td>Yes<br></td>
            <td>Important<br></td>
            <td>stuff<br></td>    
        </tr>
    </tbody>
</table>
<b>AGAIN</b>
<b>more</b>
<b>stuff</b>
<b>down here!</b>
</html>
"""
soup = bs(sample_html, 'lxml')
results = str(soup).split(str(soup.select_one('table.softwares')))
top_text = bs(results[0], 'lxml').get_text().replace('\n',' ')
bottom_text = bs(results[1], 'lxml').get_text().replace('\n',' ')
print(top_text)
print(bottom_text)

Upvotes: 1

Adeyinka Badmus
Adeyinka Badmus

Reputation: 56

This solution doesn't extensively use BeautifulSoup but works. Get index of opening and closing table tags, extract strings before and after.

soup = BeautifulSoup(sample_html, "html.parser")

def extract_top_and_bottom(soup):
    index_of_opening_tag = soup.index("<table")
    index_of_closing_tag = soup.index("</table>")

    top_html = soup[:index_of_opening_tag]
    bottom_html = soup[index_of_closing_tag::].replace("</table>", '')

    print(top_html)
    print(bottom_html)

extract_top_and_bottom(str(soup))

Upvotes: 1

Related Questions