Creating a html table from a list in Python BeautifulSoup

Question

I am using bs4 in Python and I want to take contents from a list in python and enter it into an html code using bs4 such that the html table can be posted onto a website link using requests.put() method. The html code is such that each row consists of the tag:

<tr></tr>

Each cell i.e one data element in each column corresponding to one row is denoted by tag:

<td></td>

And so each data element would go inside the td tag surrounded my a p tag like:

<tr><td><p>data 1 in cell 1</p></td><td><p>data 2 in cell 2</p></td></tr>

The data that should go inside the html table is in the form of a list and looks like:

rows = ["1" + "````" + "Mon, 22 Feb 2021 13:44:27 -0800" + "````" + "Jam" + "````" + "IAP-5998" + "````" + "10004" + "````" + "Model Observing a ModelIPCException" + "````" + "1ba4416fdd7", "2" + "````" + "Mon, 30 Feb 2021 13:44:27 -0800" + "````" + "Rizwan" + "````" + "IAP-6998" + "````" + "10014" + "````" + "Model Observing." + "````" + "3ba4416fdd7", "3" + "````" + "Fri, 20 Mar 2021 13:44:27 -0800" + "````" + "John" + "````" + "ATL-5998" + "````" + "10456" + "````" + "Exception during JumpToROM function call." + "````" + "8ca4416fdd7", "4" + "````" + "Mon, 14 Feb 2021 13:44:27 -0800" + "````" + "Brock Lesnar" + "````" + "IAP-6005" + "````" + "10009" + "````" + "RAM flushing JumpToROM function call." + "````" + "1ba4416fd10"]

So in the list each element correponds to a single row and each cell is being split in terms of the "````" so 1 goes in first cell and Jam goes into 3rd cell in the first row. The html table string should be preceded by a table header and should be concluded with a table footer like this:

html_table_header = "<p><br /></p><table><colgroup><col style=\"width: 115.0px;\" /><col style=\"width: 95.0px;\" /><col style=\"width: 58.0px;\" /><col style=\"width: 105.0px;\" /><col style=\"width: 110.0px;\" /><col style=\"width: 215.0px;\" /><col style=\"width: 215.0px;\" /></colgroup><tbody><tr><th><p>No.</p></th><th><p>Date and Time</p></th><th><p>Author</p></th><th><p>Jira</p></th><th><p>PR</p></th><th><p>Title</p></th><th><p>Commit ID</p></th></tr>"

html_table_footer = "</tbody></table><p class=\"auto-cursor-target\"><br /></p>"

So the overall html code that constitutes the data for creating a table should look like:

<p><br /></p><table><colgroup><col style=\"width: 115.0px;\" /><col style=\"width: 95.0px;\" /><col style=\"width: 58.0px;\" /><col style=\"width: 105.0px;\" /><col style=\"width: 110.0px;\" /><col style=\"width: 215.0px;\" /><col style=\"width: 215.0px;\" /></colgroup><tbody><tr><th><p>No.</p></th><th><p>Date and Time</p></th><th><p>Author</p></th><th><p>Jira</p></th><th><p>PR</p></th><th><p>Title</p></th><th><p>Commit ID</p></th></tr><tr><td><p>1</p></td><td><p>Mon, 22 Feb 2021 13:44:27 -0800</p></td><td><p>Jam</p></td><td><p>IAP-5998</p></td><td><p>10004</p></td><td><p>Model Observing a ModelIPCException</p></td><td><p>1ba4416fdd7</p></td></tr><tr><td><p>2</p></td><td><p>Mon, 30 Feb 2021 13:44:27 -0800</p></td><td><p>Rizwan</p></td><td><p>IAP-6998</p></td><td><p>10014</p></td><td><p>Model Observing</p></td><td><p>1ba4416fdd7</p></td></tr>....................................Other elements in list according to rows go here.............</tbody></table><p class=\"auto-cursor-target\"><br /></p>

Here is my code that I used:

import re
import sys
import requests
import json
from requests.auth import HTTPBasicAuth
from bs4 import BeautifulSoup

html_table_header = "<p><br /></p><table><colgroup><col style=\"width: 115.0px;\" /><col style=\"width: 95.0px;\" /><col style=\"width: 58.0px;\" /><col style=\"width: 105.0px;\" /><col style=\"width: 110.0px;\" /><col style=\"width: 215.0px;\" /><col style=\"width: 215.0px;\" /></colgroup><tbody><tr><th><p>No.</p></th><th><p>Date and Time</p></th><th><p>Author</p></th><th><p>Jira</p></th><th><p>PR</p></th><th><p>Title</p></th><th><p>Commit ID</p></th></tr>"

html_table_footer = "</tbody></table><p class=\"auto-cursor-target\"><br /></p>"

rows = ["1" + "````" + "Mon, 22 Feb 2021 13:44:27 -0800" + "````" + "Jam" + "````" + "IAP-5998" + "````" + "10004" + "````" + "Model Observing a ModelIPCException" + "````" + "1ba4416fdd7", "2" + "````" + "Mon, 30 Feb 2021 13:44:27 -0800" + "````" + "Rizwan" + "````" + "IAP-6998" + "````" + "10014" + "````" + "Model Observing." + "````" + "3ba4416fdd7", "3" + "````" + "Fri, 20 Mar 2021 13:44:27 -0800" + "````" + "John" + "````" + "ATL-5998" + "````" + "10456" + "````" + "Exception during JumpToROM function call." + "````" + "8ca4416fdd7", "4" + "````" + "Mon, 14 Feb 2021 13:44:27 -0800" + "````" + "Brock Lesnar" + "````" + "IAP-6005" + "````" + "10009" + "````" + "RAM flushing JumpToROM function call." + "````" + "1ba4416fd10"]

row_string = ""
for idx in range(0, len(rows)):
    soup = BeautifulSoup("<tr></tr>", 'html.parser')
    for cell_id in range(0, 7):
        original_tag = soup.tr
        new_tag = soup.new_tag("td")
        original_tag.append(new_tag)
        p_tag = soup.new_tag("p")
        original_tag.td.next_sibling.append(p_tag)
        original_tag.p.string = rows[idx].split("````")[cell_id]
        row_string += str(original_tag)

pass_str = html_table_header + row_string + html_table_footer
pass_string = str(pass_str).replace('\"', '\\"')

headers = {
    'Content-Type': 'application/json',
}

data = '{"id":"534756378","type":"page", "title":"GL_Engine Output","space":{"key":"CSSAI"},"body":{"storage":{"value":"' + pass_string + '","representation":"storage"}}, "version":{"number":2}}'

response = requests.put('https://confluence.ai.com/rest/api/content/534756378', headers=headers, data=data,
                        auth=HTTPBasicAuth('svc-Automation@ai.com', 'AIengineering1@ai'))

But in my code only the first element in list i.e the number 1, 2, 3 etc. goes into the right cell but the other elements are still being inserted into the first column so the table does not look right when it gets posted into the website as only the header of the table is right but the other elements are all crunched together in the first column itself. I looked at the rest/api html code that gets posted onto my website and it does not look right as shown by this screenshot:

Answer 1

I think you can use pandas to view the table and a list comprehension and split, in a loop over rows, to create the table html

from pandas import read_html as rh

pd.set_option('display.expand_frame_repr', False)

html_table_header = "<p><br /></p><table><colgroup><col style=\"width: 115.0px;\" /><col style=\"width: 95.0px;\" /><col style=\"width: 58.0px;\" /><col style=\"width: 105.0px;\" /><col style=\"width: 110.0px;\" /><col style=\"width: 215.0px;\" /><col style=\"width: 215.0px;\" /></colgroup><tbody><tr><th><p>No.</p></th><th><p>Date and Time</p></th><th><p>Author</p></th><th><p>Jira</p></th><th><p>PR</p></th><th><p>Title</p></th><th><p>Commit ID</p></th></tr>"

html_table_footer = "</tbody></table><p class=\"auto-cursor-target\"><br /></p>"

rows = ["1" + "````" + "Mon, 22 Feb 2021 13:44:27 -0800" + "````" + "Jam" + "````" + "IAP-5998" + "````" + "10004" + "````" + "Model Observing a ModelIPCException" + "````" + "1ba4416fdd7", "2" + "````" + "Mon, 30 Feb 2021 13:44:27 -0800" + "````" + "Rizwan" + "````" + "IAP-6998" + "````" + "10014" + "````" + "Model Observing." + "````" + "3ba4416fdd7", "3" + "````" + "Fri, 20 Mar 2021 13:44:27 -0800" + "````" + "John" + "````" + "ATL-5998" + "````" + "10456" + "````" + "Exception during JumpToROM function call." + "````" + "8ca4416fdd7", "4" + "````" + "Mon, 14 Feb 2021 13:44:27 -0800" + "````" + "Brock Lesnar" + "````" + "IAP-6005" + "````" + "10009" + "````" + "RAM flushing JumpToROM function call." + "````" + "1ba4416fd10"]
body = ''

for row in rows:
    body+= '<tr>' + ''.join([f'<td><p>{i}</p></td>' for i in row.split('````')]) + '</tr>'
    
html = html_table_header + body + html_table_footer 
print(rh(html)[0])

---

---

With bs4 included (seems a bit redundant):

from bs4 import BeautifulSoup as bs

soup = bs(html, 'lxml')
print(html)
print(rh(str(soup))[0])