CODEWITHSUNDEEP
pythonpandasxpathwebdriver
Mary
Mary

Reputation: 797

cannot print df.shape, and I read it in dataframe format from pandas

cannot print df.shape, and it read in dataframe format already from pandas, thanks !

# -*- coding:UTF-8 -*-

from pyvirtualdisplay import Display
from selenium import webdriver
import pandas as pd

display = Display(visible=0, size=(1024, 768))
display.start()

driver = webdriver.Firefox()
driver.get("http://www.fdmbenzinpriser.dk/searchprices/5/")

lines = [event.get_attribute('outerHTML') for event in driver.find_elements_by_xpath('//table[@id="sortabletable"]')]

df = pd.read_html(lines[0])
print df.shape

driver.close()
display.stop()

Output:

AttributeError: 'list' object has no attribute 'shape'

lines[0] return:

[                                           Unnamed: 0   Pris                        Adresse      Tidspunkt
0                                                 NaN   8.99       Odinsvej 2 4100 Ringsted  11 timer 55 m
1                                                 NaN   9.09   Sdr.Havnegade 3 6000 Kolding  14 timer 48 m
2                                                 NaN   9.09    Vestermarksvej 2 6600 Vejen  16 timer 35 m
3                                                 NaN  10.99  Bøsbrovej 92B 8940 Randers SV   21 timer 1 m

Upvotes: 1

Views: 1182

Answers (2)

jezrael
jezrael

Reputation: 863166

I think need change:

df = pd.read_html(lines[0])

to:

df = pd.read_html(lines[0])[0]

And for all data:

df = pd.concat([pd.read_html(line)[0] for line in lines], ignore_index=True)

Upvotes: 2

Neil
Neil

Reputation: 3291

Pandas read html method doesn't return a dataframe, it returns a list of dataframes: http://pandas.pydata.org/pandas-docs/version/0.22/generated/pandas.read_html.html You can always check the type of an object in python by using:

print(type(obj))

Upvotes: 1

Related Questions