CODEWITHSUNDEEP
pythonbeautifulsoup
Kevin
Kevin

Reputation: 33

Web scrape attributes that are not always included in the tag Python Beautifulsoup

I am trying to scrape a URL 'https://www.pro-football-reference.com/teams/nwe/2013_injuries.htm' using Beautiful Soup. I want to scrape the players name, their injury and the week of their injury

The players name is straight forward to scrape as it is text in a certain tag <th> and is always included in the tag. The week is an attribute ["data-stat"] of the tag <td> and is also always included in the tag. The injury is also an attribute ["data-tip"] of the same tag week is <td>, but it is only included in the tag when the player has an injury.

I tried using an if else statement for the injury status, so if the <td> tag contained an injury it would print the injury ["data-tip"] and if not it would simply print "NA". From the code I wrote, it prints the first two players' names, injury and the week of the injury but the third player does not contain the injury attribute ["data-tip"] in the <td> tag and the code would break and just print the first two players:

[['Danny Amendola'], 'Questionable: hamstring', 'week_1']
[['Armond Armstead'], 'Out: infection', 'week_1']

Outcome of my code! Experiencing a KeyError.

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = 'https://www.pro-football-reference.com/teams/nwe/2013_injuries.htm'

# opening up connection, grabbing the page
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

# html parsing
page_soup = soup(page_html, "html.parser")

containers = page_soup.find("tbody")

player = containers.find_all("tr")
for tr in player:
    th = tr.find_all("th")
    name = [i.text for i in th]

    week = tr.td["data-stat"]

    injury = tr.td["data-tip"]
    if injury is None:
        injury = "NA"
        print([name, injury, week])
    else:
        print([name, injury, week])

The outcome I am looking for is for the code to print the player names, injury (if no injury print "NA") and the week of injury for all the players in the table. For example, the third player in the table does not have an injury for week 1, therefore his injury should print "NA":

[['Danny Amendola'], 'Questionable: hamstring', 'week_1']
[['Armond Armstead'], 'Out: infection', 'week_1']
[['Kyle Arrington'], 'NA', 'week_1']

The list should continue on for the rest of the players like this.

Upvotes: 2

Views: 116

Answers (3)

chitown88
chitown88

Reputation: 28630

I'm piggy backing on Jack Moody's solution (just adding the additional weeks), but here's the additional data/columns:

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = 'https://www.pro-football-reference.com/teams/nwe/2013_injuries.htm'

# opening up connection, grabbing the page
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

# html parsing
page_soup = soup(page_html, "html.parser")

containers = page_soup.find("tbody")
head = page_soup.find("thead")


player = containers.find_all("tr")

weeks = head.find_all('th')
week_list = [i['data-stat'] for i in weeks][1:]

for week in week_list:
    for tr in player:
        th = tr.find_all("th")
        name = [i.text for i in th]
        
        td = tr.find('td', {'data-stat':week})
        week = td["data-stat"]
    
        try:
            injury = td["data-tip"]
            print([name, injury, week])
        except KeyError:
            injury = "NA"
            print([name, injury, week])

Upvotes: 1

Ajax1234
Ajax1234

Reputation: 71461

You can scrape the table, and then use zip to transpose the rows for each week:

import requests
from bs4 import BeautifulSoup as soup
d = soup(requests.get('https://www.pro-football-reference.com/teams/nwe/2013_injuries.htm').text, 'html.parser')
_header, *data = d.find('table', {'id':'team_injuries'}).find_all('tr')
_all_data = [(lambda x:[c.find('th').text, *[i.attrs.get('data-tip', 'N/A') for i in x]])(c.find_all('td')) for c in data]
new_data = [dict(zip([i.text.rstrip() for i in _header.find_all('th')], c)) for c in _all_data]

Output:

[{'Player': 'Danny Amendola', '09/08vs. BUF': 'Questionable: hamstring', '09/12vs. NYJ': 'Questionable: groin', '09/22vs. TAM': 'Doubtful: groin', '09/29vs. ATL': 'Questionable: groin', '10/06vs. CIN': 'Questionable: groin', '10/13vs. NOR': 'Probable: groin', '10/20vs. NYJ': 'Out: concussion', '10/27vs. MIA': 'Questionable: concussion', '11/03vs. PIT': 'Questionable: groin', '11/18vs. CAR': 'Probable: groin', '11/24vs. DEN': 'Probable: groin', '12/01vs. HOU': 'Probable: groin', '12/08vs. CLE': 'Probable: groin', '12/15vs. MIA': 'Probable: groin', '12/22vs. BAL': 'Questionable: groin', '12/29vs. BUF': 'Probable: groin', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Armond Armstead', '09/08vs. BUF': 'Out: infection', '09/12vs. NYJ': 'Out: infection', '09/22vs. TAM': 'Out: infection', '09/29vs. ATL': 'Out: infection', '10/06vs. CIN': 'Out: infection', '10/13vs. NOR': 'Out: infection', '10/20vs. NYJ': 'Out: infection', '10/27vs. MIA': 'Out: infection', '11/03vs. PIT': 'Out: infection', '11/18vs. CAR': 'Out: infection', '11/24vs. DEN': 'Out: infection', '12/01vs. HOU': 'Out: infection', '12/08vs. CLE': 'Out: infection', '12/15vs. MIA': 'Out: infection', '12/22vs. BAL': 'Out: infection', '12/29vs. BUF': 'Out: infection', '01/11vs. IND': 'Out: infection', '01/19vs. DEN': 'Out: infection'}, {'Player': 'Kyle Arrington', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'Questionable: groin', '10/06vs. CIN': 'Questionable: groin', '10/13vs. NOR': 'Probable: groin', '10/20vs. NYJ': 'Probable: groin', '10/27vs. MIA': 'Probable: groin', '11/03vs. PIT': 'Questionable: groin', '11/18vs. CAR': 'Questionable: groin', '11/24vs. DEN': 'Probable: groin', '12/01vs. HOU': 'Questionable: groin', '12/08vs. CLE': 'Questionable: groin', '12/15vs. MIA': 'Questionable: groin', '12/22vs. BAL': 'Questionable: groin', '12/29vs. BUF': 'Questionable: groin', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Brandon Bolden', '09/08vs. BUF': 'Questionable: knee', '09/12vs. NYJ': 'Questionable: knee', '09/22vs. TAM': 'Questionable: knee', '09/29vs. ATL': 'Questionable: knee', '10/06vs. CIN': 'Questionable: knee', '10/13vs. NOR': 'Probable: knee', '10/20vs. NYJ': 'Questionable: knee', '10/27vs. MIA': 'Questionable: knee', '11/03vs. PIT': 'Questionable: knee', '11/18vs. CAR': 'Questionable: knee', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Josh Boyce', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'Doubtful: hip', '12/29vs. BUF': 'Questionable: hip', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Tom Brady', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'Probable: right shoulder', '11/03vs. PIT': 'Probable: right shoulder', '11/18vs. CAR': 'Probable: right shoulder', '11/24vs. DEN': 'Probable: right shoulder', '12/01vs. HOU': 'Probable: shoulder', '12/08vs. CLE': 'Probable: right shoulder', '12/15vs. MIA': 'Questionable: shoulder', '12/22vs. BAL': 'Probable: right shoulder', '12/29vs. BUF': 'Probable: right shoulder', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Marcus Cannon', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'Questionable: shoulder', '10/27vs. MIA': 'Questionable: shoulder', '11/03vs. PIT': 'Questionable: shoulder', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'Questionable: ankle', '12/08vs. CLE': 'Questionable: ankle', '12/15vs. MIA': 'Questionable: ankle', '12/22vs. BAL': 'Questionable: ankle', '12/29vs. BUF': 'Questionable: ankle', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Marquice Cole', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'Probable: hamstring', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'Questionable: hamstring', '10/06vs. CIN': 'Questionable: hamstring', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'Questionable: leg', '12/08vs. CLE': 'Questionable: shin', '12/15vs. MIA': 'Questionable: shin', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Austin Collie', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Dan Connolly', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'Questionable: finger', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'Questionable: head', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Alfonzo Dennard', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'Probable: ankle', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'Questionable: leg', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'Questionable: knee', '12/08vs. CLE': 'Questionable: knee', '12/15vs. MIA': 'Questionable: knee/shoulder', '12/22vs. BAL': 'Questionable: knee/shoulder', '12/29vs. BUF': 'Questionable: knee/shoulder', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Aaron Dobson', '09/08vs. BUF': 'Questionable: hamstring', '09/12vs. NYJ': 'Questionable: hamstring', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'Doubtful: shoulder', '10/06vs. CIN': 'Questionable: neck', '10/13vs. NOR': 'Questionable: neck', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'Questionable: undisclosed', '12/08vs. CLE': 'Questionable: foot', '12/15vs. MIA': 'Questionable: foot', '12/22vs. BAL': 'Questionable: foot', '12/29vs. BUF': 'Questionable: foot', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Nate Ebner', '09/08vs. BUF': 'Questionable: ankle', '09/12vs. NYJ': 'Questionable: ankle', '09/22vs. TAM': 'Questionable: ankle', '09/29vs. ATL': 'Questionable: ankle', '10/06vs. CIN': 'Questionable: ankle', '10/13vs. NOR': 'Probable: ankle', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Julian Edelman', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'Questionable: thigh', '10/27vs. MIA': 'Questionable: thigh', '11/03vs. PIT': 'Probable: thigh', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Dane Fletcher', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'Questionable: groin', '12/29vs. BUF': 'Questionable: groin', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Tyronne Green', '09/08vs. BUF': 'Injured Reserve: undisclosed', '09/12vs. NYJ': 'Injured Reserve: undisclosed', '09/22vs. TAM': 'Injured Reserve: undisclosed', '09/29vs. ATL': 'Injured Reserve: undisclosed', '10/06vs. CIN': 'Injured Reserve: undisclosed', '10/13vs. NOR': 'Injured Reserve: undisclosed', '10/20vs. NYJ': 'Injured Reserve: undisclosed', '10/27vs. MIA': 'Injured Reserve: undisclosed', '11/03vs. PIT': 'Injured Reserve: undisclosed', '11/18vs. CAR': 'Injured Reserve: undisclosed', '11/24vs. DEN': 'Injured Reserve: undisclosed', '12/01vs. HOU': 'Injured Reserve: undisclosed', '12/08vs. CLE': 'Injured Reserve: undisclosed', '12/15vs. MIA': 'Injured Reserve: undisclosed', '12/22vs. BAL': 'Injured Reserve: undisclosed', '12/29vs. BUF': 'Injured Reserve: undisclosed', '01/11vs. IND': 'Injured Reserve: undisclosed', '01/19vs. DEN': 'Injured Reserve: undisclosed'}, {'Player': 'Steve Gregory', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'Out: thumb', '11/24vs. DEN': 'Questionable: finger', '12/01vs. HOU': 'Questionable: finger', '12/08vs. CLE': 'Questionable: finger', '12/15vs. MIA': 'Questionable: finger', '12/22vs. BAL': 'Questionable: finger', '12/29vs. BUF': 'Questionable: knee/finger', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Cory Grissom', '09/08vs. BUF': 'Injured Reserve: knee', '09/12vs. NYJ': 'Injured Reserve: knee', '09/22vs. TAM': 'Injured Reserve: knee', '09/29vs. ATL': 'Injured Reserve: knee', '10/06vs. CIN': 'Injured Reserve: knee', '10/13vs. NOR': 'Injured Reserve: knee', '10/20vs. NYJ': 'Injured Reserve: knee', '10/27vs. MIA': 'Injured Reserve: knee', '11/03vs. PIT': 'Injured Reserve: knee', '11/18vs. CAR': 'Injured Reserve: knee', '11/24vs. DEN': 'Injured Reserve: knee', '12/01vs. HOU': 'Injured Reserve: knee', '12/08vs. CLE': 'Injured Reserve: knee', '12/15vs. MIA': 'Injured Reserve: knee', '12/22vs. BAL': 'Injured Reserve: knee', '12/29vs. BUF': 'Injured Reserve: knee', '01/11vs. IND': 'Injured Reserve: knee', '01/19vs. DEN': 'Injured Reserve: knee'}, {'Player': 'Rob Gronkowski', '09/08vs. BUF': 'Doubtful: arm/back', '09/12vs. NYJ': 'Questionable: arm/back', '09/22vs. TAM': 'Doubtful: arm/back', '09/29vs. ATL': 'Questionable: arm/back', '10/06vs. CIN': 'Doubtful: arm/back', '10/13vs. NOR': 'Probable: arm/back', '10/20vs. NYJ': 'Questionable: arm/back', '10/27vs. MIA': 'Probable: back/forearm', '11/03vs. PIT': 'Probable: back/forearm/hamstring', '11/18vs. CAR': 'Probable: back/forearm/hamstring', '11/24vs. DEN': 'Probable: back/forearm/hamstring', '12/01vs. HOU': 'Probable: hamstring', '12/08vs. CLE': 'Questionable: ankle', '12/15vs. MIA': 'Injured Reserve: torn right ACL/MCL', '12/22vs. BAL': 'Injured Reserve: torn right ACL/MCL', '12/29vs. BUF': 'Injured Reserve: torn right ACL/MCL', '01/11vs. IND': 'Injured Reserve: torn right ACL/MCL', '01/19vs. DEN': 'Injured Reserve: torn right ACL/MCL'}, {'Player': 'Duron Harmon', '09/08vs. BUF': 'Questionable: hamstring', '09/12vs. NYJ': 'Questionable: hamstring', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Mark Harrison', '09/08vs. BUF': 'Out: foot', '09/12vs. NYJ': 'Out: foot', '09/22vs. TAM': 'Out: foot', '09/29vs. ATL': 'Out: foot', '10/06vs. CIN': 'Out: foot', '10/13vs. NOR': 'Out: foot', '10/20vs. NYJ': 'Out: foot', '10/27vs. MIA': 'Out: foot', '11/03vs. PIT': 'Out: foot', '11/18vs. CAR': 'Out: foot', '11/24vs. DEN': 'Out: foot', '12/01vs. HOU': 'Out: foot', '12/08vs. CLE': 'Out: foot', '12/15vs. MIA': 'Out: foot', '12/22vs. BAL': 'Out: foot', '12/29vs. BUF': 'Out: foot', '01/11vs. IND': 'Out: foot', '01/19vs. DEN': 'Out: foot'}, {'Player': "Dont'a Hightower", '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'Questionable: knee', '10/13vs. NOR': 'Probable: knee', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Michael Hoomanawanui', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'Questionable: knee', '10/27vs. MIA': 'Questionable: knee', '11/03vs. PIT': 'Questionable: knee', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'Questionable: knee', '12/01vs. HOU': 'Questionable: knee', '12/08vs. CLE': 'Probable: knee', '12/15vs. MIA': 'Questionable: knee', '12/22vs. BAL': 'Questionable: knee', '12/29vs. BUF': 'Probable: knee', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Tommy Kelly', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'Questionable: knee', '10/20vs. NYJ': 'Questionable: knee', '10/27vs. MIA': 'Questionable: knee', '11/03vs. PIT': 'Questionable: knee', '11/18vs. CAR': 'Injured Reserve: knee', '11/24vs. DEN': 'Injured Reserve: knee', '12/01vs. HOU': 'Injured Reserve: knee', '12/08vs. CLE': 'Injured Reserve: knee', '12/15vs. MIA': 'Injured Reserve: knee', '12/22vs. BAL': 'Injured Reserve: knee', '12/29vs. BUF': 'Injured Reserve: knee', '01/11vs. IND': 'Injured Reserve: knee', '01/19vs. DEN': 'Injured Reserve: knee'}, {'Player': 'Jerod Mayo', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'Questionable: ankle', '10/06vs. CIN': 'Questionable: ankle', '10/13vs. NOR': 'Probable: ankle', '10/20vs. NYJ': 'Injured Reserve: shoulder', '10/27vs. MIA': 'Injured Reserve: shoulder', '11/03vs. PIT': 'Injured Reserve: shoulder', '11/18vs. CAR': 'Injured Reserve: shoulder', '11/24vs. DEN': 'Injured Reserve: shoulder', '12/01vs. HOU': 'Injured Reserve: shoulder', '12/08vs. CLE': 'Injured Reserve: shoulder', '12/15vs. MIA': 'Injured Reserve: shoulder', '12/22vs. BAL': 'Injured Reserve: shoulder', '12/29vs. BUF': 'Injured Reserve: shoulder', '01/11vs. IND': 'Injured Reserve: shoulder', '01/19vs. DEN': 'Injured Reserve: shoulder'}, {'Player': 'Devin McCourty', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'Questionable: shoulder', '10/27vs. MIA': 'Probable: shoulder', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'Questionable: head', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'T.J. Moe', '09/08vs. BUF': 'Injured Reserve: Achilles', '09/12vs. NYJ': 'Injured Reserve: Achilles', '09/22vs. TAM': 'Injured Reserve: Achilles', '09/29vs. ATL': 'Injured Reserve: Achilles', '10/06vs. CIN': 'Injured Reserve: Achilles', '10/13vs. NOR': 'Injured Reserve: Achilles', '10/20vs. NYJ': 'Injured Reserve: Achilles', '10/27vs. MIA': 'Injured Reserve: Achilles', '11/03vs. PIT': 'Injured Reserve: Achilles', '11/18vs. CAR': 'Injured Reserve: Achilles', '11/24vs. DEN': 'Injured Reserve: Achilles', '12/01vs. HOU': 'Injured Reserve: Achilles', '12/08vs. CLE': 'Injured Reserve: Achilles', '12/15vs. MIA': 'Injured Reserve: Achilles', '12/22vs. BAL': 'Injured Reserve: Achilles', '12/29vs. BUF': 'Injured Reserve: Achilles', '01/11vs. IND': 'Injured Reserve: Achilles', '01/19vs. DEN': 'Injured Reserve: Achilles'}, {'Player': 'Rob Ninkovich', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'Probable: groin', '10/20vs. NYJ': 'Probable: groin', '10/27vs. MIA': 'Probable: groin', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'Questionable: foot', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'Questionable: ankle', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Stevan Ridley', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'Probable: shoulder', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'Questionable: knee', '10/13vs. NOR': 'Questionable: knee', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Matt Slater', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'Questionable: knee', '09/22vs. TAM': 'Out: wrist', '09/29vs. ATL': 'Out: wrist', '10/06vs. CIN': 'Out: wrist', '10/13vs. NOR': 'Out: wrist', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'Questionable: wrist', '11/03vs. PIT': 'Probable: wrist', '11/18vs. CAR': 'Probable: wrist', '11/24vs. DEN': 'Probable: wrist', '12/01vs. HOU': 'Probable: wrist', '12/08vs. CLE': 'Probable: right shoulder', '12/15vs. MIA': 'Probable: wrist', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Nate Solder', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'Probable: back', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'Questionable: concussion', '12/22vs. BAL': 'Questionable: concussion', '12/29vs. BUF': 'Questionable: concussion', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Brandon Spikes', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'Questionable: knee', '12/01vs. HOU': 'Probable: knee', '12/08vs. CLE': 'Questionable: knee', '12/15vs. MIA': 'Questionable: knee', '12/22vs. BAL': 'Questionable: knee', '12/29vs. BUF': 'Questionable: knee', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Zach Sudfeld', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'Questionable: hamstring', '09/22vs. TAM': 'Probable: hamstring', '09/29vs. ATL': 'Probable: hamstring', '10/06vs. CIN': 'Questionable: hamstring', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Will Svitek', '09/08vs. BUF': 'Questionable: knee', '09/12vs. NYJ': 'Questionable: knee', '09/22vs. TAM': 'Questionable: knee', '09/29vs. ATL': 'Questionable: knee', '10/06vs. CIN': 'Questionable: knee', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'Questionable: ankle', '12/15vs. MIA': 'Questionable: ankle', '12/22vs. BAL': 'Questionable: ankle', '12/29vs. BUF': 'Questionable: ankle', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Aqib Talib', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'Questionable: hip', '10/20vs. NYJ': 'Questionable: hip', '10/27vs. MIA': 'Questionable: hip', '11/03vs. PIT': 'Questionable: hip', '11/18vs. CAR': 'Questionable: hip', '11/24vs. DEN': 'Questionable: hip', '12/01vs. HOU': 'Questionable: hip', '12/08vs. CLE': 'Questionable: hip', '12/15vs. MIA': 'Questionable: hip', '12/22vs. BAL': 'Questionable: hip', '12/29vs. BUF': 'Probable: hip', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Kenbrell Thompkins', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'Questionable: shoulder', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'Questionable: hip', '12/15vs. MIA': 'Questionable: hip', '12/22vs. BAL': 'Questionable: hip', '12/29vs. BUF': 'Questionable: hip', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Shane Vereen', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'Out: wrist', '09/22vs. TAM': 'Injured Reserve: wrist', '09/29vs. ATL': 'Injured Reserve: wrist', '10/06vs. CIN': 'Injured Reserve: wrist', '10/13vs. NOR': 'Injured Reserve: wrist', '10/20vs. NYJ': 'Injured Reserve: wrist', '10/27vs. MIA': 'Injured Reserve: wrist', '11/03vs. PIT': 'Injured Reserve: wrist', '11/18vs. CAR': 'Injured Reserve: wrist', '11/24vs. DEN': 'Probable: wrist', '12/01vs. HOU': 'Probable: wrist', '12/08vs. CLE': 'Probable: wrist', '12/15vs. MIA': 'Probable: wrist', '12/22vs. BAL': 'Questionable: groin', '12/29vs. BUF': 'Probable: groin', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Sebastian Vollmer', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'Questionable: foot', '10/06vs. CIN': 'Questionable: foot', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'Injured Reserve: leg', '11/18vs. CAR': 'Injured Reserve: leg', '11/24vs. DEN': 'Injured Reserve: leg', '12/01vs. HOU': 'Injured Reserve: leg', '12/08vs. CLE': 'Injured Reserve: leg', '12/15vs. MIA': 'Injured Reserve: leg', '12/22vs. BAL': 'Injured Reserve: leg', '12/29vs. BUF': 'Injured Reserve: leg', '01/11vs. IND': 'Injured Reserve: leg', '01/19vs. DEN': 'Injured Reserve: leg'}, {'Player': 'Leon Washington', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'Questionable: thigh', '09/22vs. TAM': 'Questionable: thigh', '09/29vs. ATL': 'Questionable: thigh', '10/06vs. CIN': 'Questionable: thigh', '10/13vs. NOR': 'Questionable: ankle', '10/20vs. NYJ': 'Questionable: ankle', '10/27vs. MIA': 'Questionable: ankle', '11/03vs. PIT': 'Questionable: ankle', '11/18vs. CAR': 'Questionable: ankle', '11/24vs. DEN': 'Questionable: ankle', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Ryan Wendell', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'Questionable: concussion', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Chris White', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'N/A', '10/13vs. NOR': 'N/A', '10/20vs. NYJ': 'N/A', '10/27vs. MIA': 'N/A', '11/03vs. PIT': 'N/A', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'Questionable: back', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Vince Wilfork', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'Probable: foot', '10/06vs. CIN': 'Out: Achilles', '10/13vs. NOR': 'Injured Reserve: Achilles', '10/20vs. NYJ': 'Injured Reserve: Achilles', '10/27vs. MIA': 'Injured Reserve: Achilles', '11/03vs. PIT': 'Injured Reserve: Achilles', '11/18vs. CAR': 'Injured Reserve: Achilles', '11/24vs. DEN': 'Injured Reserve: Achilles', '12/01vs. HOU': 'Injured Reserve: Achilles', '12/08vs. CLE': 'Injured Reserve: Achilles', '12/15vs. MIA': 'Injured Reserve: Achilles', '12/22vs. BAL': 'Injured Reserve: Achilles', '12/29vs. BUF': 'Injured Reserve: Achilles', '01/11vs. IND': 'Injured Reserve: Achilles', '01/19vs. DEN': 'Injured Reserve: Achilles'}, {'Player': 'Adrian Wilson', '09/08vs. BUF': 'Injured Reserve: hamstring', '09/12vs. NYJ': 'Injured Reserve: hamstring', '09/22vs. TAM': 'Injured Reserve: hamstring', '09/29vs. ATL': 'Injured Reserve: hamstring', '10/06vs. CIN': 'Injured Reserve: hamstring', '10/13vs. NOR': 'Injured Reserve: hamstring', '10/20vs. NYJ': 'Injured Reserve: hamstring', '10/27vs. MIA': 'Injured Reserve: hamstring', '11/03vs. PIT': 'Injured Reserve: hamstring', '11/18vs. CAR': 'Injured Reserve: hamstring', '11/24vs. DEN': 'Injured Reserve: hamstring', '12/01vs. HOU': 'Injured Reserve: hamstring', '12/08vs. CLE': 'Injured Reserve: hamstring', '12/15vs. MIA': 'Injured Reserve: hamstring', '12/22vs. BAL': 'Injured Reserve: hamstring', '12/29vs. BUF': 'Injured Reserve: hamstring', '01/11vs. IND': 'Injured Reserve: hamstring', '01/19vs. DEN': 'Injured Reserve: hamstring'}, {'Player': 'Tavon Wilson', '09/08vs. BUF': 'N/A', '09/12vs. NYJ': 'N/A', '09/22vs. TAM': 'N/A', '09/29vs. ATL': 'N/A', '10/06vs. CIN': 'Questionable: hamstring', '10/13vs. NOR': 'Questionable: hamstring', '10/20vs. NYJ': 'Questionable: hamstring', '10/27vs. MIA': 'Questionable: hamstring', '11/03vs. PIT': 'Questionable: hamstring', '11/18vs. CAR': 'N/A', '11/24vs. DEN': 'N/A', '12/01vs. HOU': 'N/A', '12/08vs. CLE': 'N/A', '12/15vs. MIA': 'N/A', '12/22vs. BAL': 'N/A', '12/29vs. BUF': 'N/A', '01/11vs. IND': 'N/A', '01/19vs. DEN': 'N/A'}, {'Player': 'Markus Zusevics', '09/08vs. BUF': 'Injured Reserve: undisclosed', '09/12vs. NYJ': 'Injured Reserve: undisclosed', '09/22vs. TAM': 'Injured Reserve: undisclosed', '09/29vs. ATL': 'Injured Reserve: undisclosed', '10/06vs. CIN': 'Injured Reserve: undisclosed', '10/13vs. NOR': 'Injured Reserve: undisclosed', '10/20vs. NYJ': 'Injured Reserve: undisclosed', '10/27vs. MIA': 'Injured Reserve: undisclosed', '11/03vs. PIT': 'Injured Reserve: undisclosed', '11/18vs. CAR': 'Injured Reserve: undisclosed', '11/24vs. DEN': 'Injured Reserve: undisclosed', '12/01vs. HOU': 'Injured Reserve: undisclosed', '12/08vs. CLE': 'Injured Reserve: undisclosed', '12/15vs. MIA': 'Injured Reserve: undisclosed', '12/22vs. BAL': 'Injured Reserve: undisclosed', '12/29vs. BUF': 'Injured Reserve: undisclosed', '01/11vs. IND': 'Injured Reserve: undisclosed', '01/19vs. DEN': 'Injured Reserve: undisclosed'}]

Upvotes: 0

Jack Moody
Jack Moody

Reputation: 1771

You probably want to use a try and except statement. You are running into a KeyError because if a player doesn't have an injury, then there won't be a data-tip attribute. After the line of code week = # ...rest of code, do the following:

try:
    injury = tr.td["data-tip"]
    print([name, injury, week])
except KeyError:
    injury = "NA"
    print([name, injury, week])

This will try to get the injury name and if it doesn't exist then it will set it to "NA". With that said, your fixed code shoud look like this:

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = 'https://www.pro-football-reference.com/teams/nwe/2013_injuries.htm'

# opening up connection, grabbing the page
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

# html parsing
page_soup = soup(page_html, "html.parser")

containers = page_soup.find("tbody")

player = containers.find_all("tr")
for tr in player:
    th = tr.find_all("th")
    name = [i.text for i in th]

    week = tr.td["data-stat"]

    try:
        injury = tr.td["data-tip"]
        print([name, injury, week])
    except KeyError:
        injury = "NA"
        print([name, injury, week])

Upvotes: 1

Related Questions