Reputation: 35
I have been trying to solve this issue for a few days now and can't seem to figure out how to get it done properly. Originally had a problem with printing entries to csv but thanks to SO that was solved quickly.
The problem is that when I print my xpath entries to csv the format becomes really weird and thus giving a lot of formatting issues when importing it to Excel or web. Printed format in CSV looks like this when opening or previewing it:
['something'] ['something\nblabla'] ['som&asddng\n2blas']
I purposely used writerow(entries)
as I want the list to be in one row and each entry as a column. I have tried to use:
writerows(zip(entries)
but the formatting of CSV is same. writerows(entries)
which gave me a correct formatting for each entry but obviously writes each entry as a new row.re
but this only destroys the formatting when many rows are appended to the csv file (i.e. when the code is used in a for loop for multiple pages).Thus I have two questions:
1. Why is this formatting happening? 2. How can the formatting problem be solved!?
I used the following code:
import csv
def get_elements_by_xpath(driver, xpath):
return [entry.text for entry in driver.find_elements_by_xpath(xpath)]
File = open('list.csv', 'w')
writer = csv.writer(File)
facts = [("//div[@class='left-col']/h6[2]"),
("//div[@class='left-col']"),
("//div[@class='left-col']/h6[1]"),
("//div[@class='left-col']/strong[1]"),
("//div[@class='left-col']/strong[2]"),
("//div[@class='left-col']//a[@rel='nofollow']")]
entries = []
for xpath in facts:
entries.append(get_elements_by_xpath(driver, xpath))
print(entries)
writer.writerow(entries)
File.close()
Upvotes: 0
Views: 81
Reputation: 35
As user @furas suggested I tried applying entries.extend()
instead of entries.append()
and this turned out to work in terms of arranging the list of xpath entries into one row with each xpath entry being a column as well as a clean csv output.
See solution applied to code below:
import csv
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get('https://www.example.com/list/')
File = open('list.csv', 'w')
writer = csv.writer(File)
def get_elements_by_xpath(driver, xpath):
return [entry.text for entry in driver.find_elements_by_xpath(xpath)]
facts = [("//div[@class='left-col']/h6[2]"),
("//div[@class='left-col']"),
("//div[@class='left-col']/h6[1]"),
("//div[@class='left-col']/strong[1]"),
("//div[@class='left-col']/strong[2]"),
("//div[@class='left-col']//a[@rel='nofollow']")]
entries = []
for xpath in facts:
entries.extend(get_elements_by_xpath(driver, xpath)) # .extend instead of .append
writer.writerow(entries)
File.close()
Upvotes: 1