Print heading and corresponding paragraph in beautiful soup

Question

Here is my code

soup = BeautifulSoup(open('MUFC.html'))

a_tag = soup.find_all()
f=open('out.txt','w')
i=0
while(i<len(a_tag)):
    if(a_tag[i].find('<h1>') or a_tag[i].find('<h2>') or a_tag[i].find('<h3>')):
        f.write(a_tag[i].get_text().encode('utf-8'))
        f.write('\n')
        i=i+1
        f.write(a_tag[i].get_text().encode('utf-8'))
        f.write('\n')
    elif(a_tag[i].find('<p>')):
        f.write(a_tag[i].get_text().encode('utf-8'))
        f.write('\n')
        i=i+1
    i=i+1

What I want to do is take heading and corresponding paragraph(s), but this takes a lot of time.

My sample Input is

<h1>Manchester United F.C.</h1>
<p>
Manchester United Football Club is an English professional football club, based in Old Trafford, Greater Manchester that plays in the Premier League. Founded as Newton Heath LYR Football Club in 1878, the club changed its name to Manchester United in 1902 and moved to Old Trafford in 1910 and is one of the most successful clubs in English football.
</p>

<p>
Manchester United have won the most League titles (20) of any English club,[3] a joint record 11 FA Cups, four League Cups, and a record twenty FA Community Shields. The club has also won three European Cups, one UEFA Cup Winners' Cup, one UEFA Super Cup, one Intercontinental Cup, and one FIFA Club World Cup. In 1998–99, the club won a continental treble of the Premier League, the FA Cup and the UEFA Champions League.
</p>

<h3>Early years (1878–1945)</h3>
<p>
Manchester United was formed in 1878 as Newton Heath LYR Football Club by the Carriage and Wagon department of the Lancashire and Yorkshire Railway (LYR) depot at Newton Heath.[15] The team initially played games against other departments and rail companies, but on 20 November 1880, they competed in their first recorded match; wearing the colours of the railway company – green and gold – they were defeated 6–0 by Bolton Wanderers' reserve team.[16] By 1888, the club had become a founding member of The Combination, a regional football league. Following the league's dissolution after only one season, Newton Heath joined the newly formed Football Alliance, which ran for three seasons before being merged with the Football League. This resulted in the club starting the 1892–93 season in the First Division, by which time it had become independent of the rail company and dropped the "LYR" from its name.[15] After two seasons, the club was relegated to the Second Division.[15]
</p>

What I want as output is something like

 Manchester United F.C.

    Manchester United Football Club is an English professional football club, based in Old Trafford, Greater Manchester that plays in the Premier League. Founded as Newton Heath LYR Football Club in 1878, the club changed its name to Manchester United in 1902 and moved to Old Trafford in 1910 and is one of the most successful clubs in English football.


    Manchester United have won the most League titles (20) of any English club,[3] a joint record 11 FA Cups, four League Cups, and a record twenty FA Community Shields. The club has also won three European Cups, one UEFA Cup Winners' Cup, one UEFA Super Cup, one Intercontinental Cup, and one FIFA Club World Cup. In 1998–99, the club won a continental treble of the Premier League, the FA Cup and the UEFA Champions League.


    Early years (1878–1945)

    Manchester United was formed in 1878 as Newton Heath LYR Football Club by the Carriage and Wagon department of the Lancashire and Yorkshire Railway (LYR) depot at Newton Heath.[15] The team initially played games against other departments and rail companies, but on 20 November 1880, they competed in their first recorded match; wearing the colours of the railway company – green and gold – they were defeated 6–0 by Bolton Wanderers' reserve team.[16] By 1888, the club had become a founding member of The Combination, a regional football league. Following the league's dissolution after only one season, Newton Heath joined the newly formed Football Alliance, which ran for three seasons before being merged with the Football League. This resulted in the club starting the 1892–93 season in the First Division, by which time it had become independent of the rail company and dropped the "LYR" from its name.[15] After two seasons, the club was relegated to the Second Division.[15]

Can we improvise this?

Answer 1

Find all headers, and from there loop over all next siblings with the element.next_siblings generator:

import io

with io.open('output.txt', 'w', encoding='utf8') as f:
    for header in soup.find_all(['h1', 'h2', 'h3']):
        f.write(header.get_text() + u'\n')
        for elem in header.next_siblings:
            if elem.name and elem.name.startswith('h'):
                # stop at next header
                break
            if elem.name == 'p':
                f.write(elem.get_text() + u'\n')

io.open() here makes it easier to write Unicode out as UTF-8.

Demo:

>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup('''\
... <h1>Manchester United F.C.</h1>
... <p>
... Manchester United Football Club is an English professional football club, based in Old Trafford, Greater Manchester that plays in the Premier League. Founded as Newton Heath LYR Football Club in 1878, the club changed its name to Manchester United in 1902 and moved to Old Trafford in 1910 and is one of the most successful clubs in English football.
... </p>
... 
... <p>
... Manchester United have won the most League titles (20) of any English club,[3] a joint record 11 FA Cups, four League Cups, and a record twenty FA Community Shields. The club has also won three European Cups, one UEFA Cup Winners' Cup, one UEFA Super Cup, one Intercontinental Cup, and one FIFA Club World Cup. In 1998–99, the club won a continental treble of the Premier League, the FA Cup and the UEFA Champions League.
... </p>
... 
... <h3>Early years (1878–1945)</h3>
... <p>
... Manchester United was formed in 1878 as Newton Heath LYR Football Club by the Carriage and Wagon department of the Lancashire and Yorkshire Railway (LYR) depot at Newton Heath.[15] The team initially played games against other departments and rail companies, but on 20 November 1880, they competed in their first recorded match; wearing the colours of the railway company – green and gold – they were defeated 6–0 by Bolton Wanderers' reserve team.[16] By 1888, the club had become a founding member of The Combination, a regional football league. Following the league's dissolution after only one season, Newton Heath joined the newly formed Football Alliance, which ran for three seasons before being merged with the Football League. This resulted in the club starting the 1892–93 season in the First Division, by which time it had become independent of the rail company and dropped the "LYR" from its name.[15] After two seasons, the club was relegated to the Second Division.[15]
... </p>
... ''')
>>> for header in soup.find_all(['h1', 'h2', 'h3']):
...     print header.get_text()
...     for elem in header.next_siblings:
...         if elem.name and elem.name.startswith('h'):
...             break
...         if elem.name == 'p':
...             print elem.get_text()
... 
Manchester United F.C.

Manchester United Football Club is an English professional football club, based in Old Trafford, Greater Manchester that plays in the Premier League. Founded as Newton Heath LYR Football Club in 1878, the club changed its name to Manchester United in 1902 and moved to Old Trafford in 1910 and is one of the most successful clubs in English football.


Manchester United have won the most League titles (20) of any English club,[3] a joint record 11 FA Cups, four League Cups, and a record twenty FA Community Shields. The club has also won three European Cups, one UEFA Cup Winners' Cup, one UEFA Super Cup, one Intercontinental Cup, and one FIFA Club World Cup. In 1998–99, the club won a continental treble of the Premier League, the FA Cup and the UEFA Champions League.

Early years (1878–1945)

Manchester United was formed in 1878 as Newton Heath LYR Football Club by the Carriage and Wagon department of the Lancashire and Yorkshire Railway (LYR) depot at Newton Heath.[15] The team initially played games against other departments and rail companies, but on 20 November 1880, they competed in their first recorded match; wearing the colours of the railway company – green and gold – they were defeated 6–0 by Bolton Wanderers' reserve team.[16] By 1888, the club had become a founding member of The Combination, a regional football league. Following the league's dissolution after only one season, Newton Heath joined the newly formed Football Alliance, which ran for three seasons before being merged with the Football League. This resulted in the club starting the 1892–93 season in the First Division, by which time it had become independent of the rail company and dropped the "LYR" from its name.[15] After two seasons, the club was relegated to the Second Division.[15]