CODEWITHSUNDEEP
pythonpandasdataframeweb-scraping
Brennan Mosher
Brennan Mosher

Reputation: 17

How to append multiple columns at the same time using panda for python

I am currently using python to web scrape the three-point statistics for every NBA player and am trying to put this data in a data frame. The code below is my attempt at adding the values to the data frame. The variables players,teams,threePointAttempts, and threePointPercentage are all lists containing 50 values. These are refilled after every iteration of the while loop because the script moves through each page of the NBA site.

while i<10:
soup = BeautifulSoup(d.page_source, 'html.parser').find('table')
headers, [_, *data] = [i.text for i in soup.find_all('th')], [[i.text for i in b.find_all('td')] for b in soup.find_all('tr')]
final_data = [i for i in data if len(i) > 1]

data_attrs = [dict(zip(headers, i)) for i in final_data]
print(data_attrs)

players = [i['PLAYER'] for i in data_attrs]
teams = [i['TEAM'] for i in data_attrs]
threePointAttempts = [i['3PA'] for i in data_attrs]
threePointPercentage = [i['3P%'] for i in data_attrs]


data_df = data_df.append(pd.DataFrame(players, columns=['Player']),ignore_index=True)
data_df = data_df.append(pd.DataFrame(teams, columns=['Team']),ignore_index=True)
data_df = data_df.append(pd.DataFrame(threePointAttempts, columns=['3PA']),ignore_index=True)
data_df = data_df.append(pd.DataFrame(threePointPercentage, columns=['3P%']),ignore_index=True)
data_df = data_df[['Player','Team','3PA','3P%']]

The issue I am having is the data frame fills like this:

First columnSecond columnThird column

Upvotes: 0

Views: 7110

Answers (1)

Luke Ning
Luke Ning

Reputation: 162

Try:

temp_df = pd.DataFrame({'Player': players,
                        'Team': teams,
                        '3PA': threePointAttempts,
                        '3P%': threePointPercentage})

data_df = data_df.append(temp_df, ignore_index=True)

Upvotes: 1

Related Questions