Reputation: 539
the code like this, I can get one csv file ,but I get some error:
other answer can't solve my problem.
'DataFrame' object is not callable
Dataframe Object is not callable
import requests,datetime,re,time,pandas
from bs4 import BeautifulSoup
session=requests.Session()
headers={
'User-Agent':'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.101 Safari/537.36',
'Accept-Language':'zh-CN,zh;q=0.8,en-US;q=0.6,en;q=0.4',
'Accept-Encoding':'gzip, deflate, sdch'
}
today=datetime.date.today()
yesterday = today - datetime.timedelta(days=1)
url_dict={'SafariPark':'http://ticket.lvmama.com/scenic-100417',
'Paradise':'http://ticket.lvmama.com/scenic-120604',
'WaterPark':'http://ticket.lvmama.com/scenic-123120',
'BirdsPark':'http://ticket.lvmama.com/scenic-104830',
'InternationalCircus':'http://ticket.lvmama.com/scenic-120603',
'OceanKingdom':'http://ticket.lvmama.com/scenic-159567',
'HengqinTheatre':'http://ticket.lvmama.com/scenic-159568',
'SZHappyValley':'http://ticket.lvmama.com/scenic-104960',
'SHdisneyrestort':'http://ticket.lvmama.com/scenic-175649',
'HKdisneyland':'http://ticket.lvmama.com/scenic-105907',
'Octeast':'http://ticket.lvmama.com/scenic-122400',
'HKOceanPark':'http://ticket.lvmama.com/scenic-103686'
}
def get_id(url):
place_id = url.split('-')[-1]
return place_id
def get_review(url):
place_id=get_id(url)
urls=['http://ticket.lvmama.com/vst_front/comment/newPaginationOfComments?type=all¤tPage={}&totalCount=1&placeId={}&productId=&placeIdType=PLACE&isPicture=&isBest=&isPOI=Y&isELong=N'.format(i,str(place_id)) for i in range(1,6)]
review_list = []
for u in urls:
res=requests.get(u)
soup = BeautifulSoup(res.text,'lxml')
all_div = soup.find_all('div', {'class', 'comment-li'})
for t in all_div:
data = {
'name': t.select(' div.com-userinfo > p > a')[0].text,
'date': t.select('div.com-userinfo > p > em')[0].text,
'review': re.sub('( |\n+|\r+)','',t.find('div', {'class', 'ufeed-content'}).text),
'tourist_type': t.find('a', {'class', 'com-proTit'}).text,
}
review_list.append(data)
return review_list
def final_data(url):
review_list=get_review(url)
df=pandas.DataFrame(review_list)
data = df[df.date == str(yesterday)]
if data.empty:
return 'No review'
else:
final_data=data.drop_duplicates()
return final_data
for name,url in url_dict.items():
final_data=final_data(url)
if type(final_data)==str:
print name+'empty'
else:
final_data.to_csv(name+'_{}.csv'.format(yesterday),encoding='utf-8')
time.sleep(8)
the error:
Traceback (most recent call last):
File "D:/project/������ȡ/lvmama.py", line 61, in <module>
final_data=final_data(url)
TypeError: 'DataFrame' object is not callable
Upvotes: 0
Views: 3149
Reputation: 965
On the second iteration, final_data is not considered as a function. You should change name of returning object.
if __name__ == '__main__':
for name, url in url_dict.items():
final = final_data(url)
if type(final) == str:
print name + 'empty'
else:
final.to_csv(name+'_{}.csv'.format(yesterday),encoding='utf-8')
time.sleep(8)
Upvotes: 4