CODEWITHSUNDEEP
pythonpandasseleniumcsvdataframe
Vinicius Gomes
Vinicius Gomes

Reputation: 23

Problems with CSV output using Pandas for webscraping

Early today with the help of an user i could get this nobbie webscraping project work. But the final CSV has all the information in just one column (Photo Attached). How can i put each class in one particular column with the respective row?

Thanks guys in advance.

Print of the final result

CODE:

from bs4 import BeautifulSoup
import pandas as pd
from selenium import webdriver
import pandas as pd

products=[] #List to store name of the product
prices=[] #List to store price of the product
ratings=[] #List to store rating of the product
driver = webdriver.Chrome(executable_path = r'C:\Users\gomes\Desktop\chromedriver.exe')
driver.get("https://www.flipkart.com/laptops/~buyback-guarantee-on-laptops-/pr?sid=6bo%2Cb5g&uniq")
content = driver.page_source
soup = BeautifulSoup(content, 'html.parser')
for a in soup.findAll('a',href=True, attrs={'class':'_31qSD5'}):
    name=a.find('div', attrs={'class':'_3wU53n'})
    price=a.find('div', attrs={'class':'_1vC4OE _2rQ-NK'})
    rating=a.find('div', attrs={'class':'hGSR34'})
    products.append(name.text)
    prices.append(price.text)
    ratings.append(rating.text)
    data = dict({'Product Name': products,
                 'Price': prices,
                 'Rating': ratings
                 })
    # create dataframe
    products_df = pd.DataFrame(
        dict([(k, pd.Series(v)) for k, v in data.items()])
        )
    products_df.to_csv("C:\\Users\\gomes\\Desktop\\preços.csv")

Upvotes: 0

Views: 48

Answers (2)

bckmnn
bckmnn

Reputation: 64

If you would like to use your data in Excel anyways, pandas also has the dataframe.to_excel() method. You might need to install a excel writer with pip install openpyxl and then export directly to excel and skipping the csv altogether.

products_df.to_excel("C:\\Users\\gomes\\Desktop\\preços.xlsx")

Upvotes: 1

SUDHEER TALLURI
SUDHEER TALLURI

Reputation: 168

I'm getting the information in different columns only, there is no mistake in your code. The issue is in excel.

Go to Data, then select the column, then In data tools, click on Text to Columns and click on delimited and use comma as delimiter

Upvotes: 2

Related Questions