CODEWITHSUNDEEP
pythoncsv
Neverend
Neverend

Reputation: 35

Writer outputs strange format to CSV

I have been trying to solve this issue for a few days now and can't seem to figure out how to get it done properly. Originally had a problem with printing entries to csv but thanks to SO that was solved quickly.

The problem is that when I print my xpath entries to csv the format becomes really weird and thus giving a lot of formatting issues when importing it to Excel or web. Printed format in CSV looks like this when opening or previewing it:

['something'] ['something\nblabla'] ['som&asddng\n2blas']

I purposely used writerow(entries) as I want the list to be in one row and each entry as a column. I have tried to use:

  1. writerows(zip(entries) but the formatting of CSV is same.
  2. writerows(entries) which gave me a correct formatting for each entry but obviously writes each entry as a new row.
  3. using re but this only destroys the formatting when many rows are appended to the csv file (i.e. when the code is used in a for loop for multiple pages).

Thus I have two questions:

1. Why is this formatting happening? 2. How can the formatting problem be solved!?

I used the following code:

import csv
def get_elements_by_xpath(driver, xpath):
        return [entry.text for entry in driver.find_elements_by_xpath(xpath)]

File = open('list.csv', 'w')
writer = csv.writer(File)
facts = [("//div[@class='left-col']/h6[2]"),
    ("//div[@class='left-col']"),
    ("//div[@class='left-col']/h6[1]"),
    ("//div[@class='left-col']/strong[1]"),
    ("//div[@class='left-col']/strong[2]"),
    ("//div[@class='left-col']//a[@rel='nofollow']")]
entries = []
for xpath in facts:
    entries.append(get_elements_by_xpath(driver, xpath))
print(entries)
writer.writerow(entries)
File.close()

Upvotes: 0

Views: 81

Answers (1)

Neverend
Neverend

Reputation: 35

As user @furas suggested I tried applying entries.extend() instead of entries.append() and this turned out to work in terms of arranging the list of xpath entries into one row with each xpath entry being a column as well as a clean csv output.

See solution applied to code below:

import csv
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get('https://www.example.com/list/')

File = open('list.csv', 'w')
writer = csv.writer(File)

def get_elements_by_xpath(driver, xpath):
    return [entry.text for entry in driver.find_elements_by_xpath(xpath)]

facts = [("//div[@class='left-col']/h6[2]"),
    ("//div[@class='left-col']"),
    ("//div[@class='left-col']/h6[1]"),
    ("//div[@class='left-col']/strong[1]"),
    ("//div[@class='left-col']/strong[2]"),
    ("//div[@class='left-col']//a[@rel='nofollow']")]

entries = []
for xpath in facts:
    entries.extend(get_elements_by_xpath(driver, xpath)) # .extend instead of .append
writer.writerow(entries)
File.close()

Upvotes: 1

Related Questions