Beautifulsoup parsing the right value

Question

https://www.transfermarkt.us/manchester-city/kader/verein/281/saison_id/2020/plus/1

From the website above, I am trying to parse the number, player name, and the position using Beautifulsoup.

For example, I want to print

1. Ederson Goalkeeper
2. Arijanet Muric Goalkeeper ...

I tried something like

page = requests.get(url, headers={'User-Agent':'Mozilla/5.0'})
soup = bs(page.content, 'html.parser')
rows = soup.find("table", class_="items").find('tbody').find_all('a')
for row in rows:
    if row.find('img') is None:
        continue
    print(row.find('img')['title'])
    print('\n')

First to print the name, but it doesn't necessarily indicate the player name and sometimes the value is empty. Also, getting the number and the position data seem impossible in this branch. How can I access other branches at the same time to get the number and the position data as well?

Answer 1

To get player numbers, names and positions, you can use this example:

import requests
from bs4 import BeautifulSoup

url = 'https://www.transfermarkt.us/manchester-city/kader/verein/281/saison_id/2020/plus/1'

headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0'}
soup = BeautifulSoup(requests.get(url, headers=headers).content, 'html.parser')

for row in soup.select('table.items > tbody > tr:has(td)'):
    data = row.get_text(strip=True, separator='|').split('|')
    print('{:<5} {:<30} {}'.format(data[0], data[1], data[3] if len(data) == 10 else data[4]))

Prints:

31    Ederson                        Goalkeeper
-     Zack Steffen                   Goalkeeper
49    Arijanet Muric                 Goalkeeper
33    Scott Carson                   Goalkeeper
14    Aymeric Laporte                Centre-Back
5     John Stones                    Centre-Back
6     Nathan Aké                     Centre-Back
50    Eric García                    Centre-Back
30    Nicolás Otamendi               Centre-Back
25    Fernandinho                    Centre-Back
34    Philippe Sandler               Centre-Back
24    Tosin Adarabioyo               Centre-Back
78    Taylor Harwood-Bellis          Centre-Back
22    Benjamin Mendy                 Left-Back
11    Oleksandr Zinchenko            Left-Back
12    Angeliño                       Left-Back
2     Kyle Walker                    Right-Back
27    João Cancelo                   Right-Back
-     Yan Couto                      Right-Back
16    Rodri                          Defensive Midfield
8     Ilkay Gündogan                 Central Midfield
47    Phil Foden                     Central Midfield
17    Kevin De Bruyne                Attacking Midfield
-     Luka Ilic                      Attacking Midfield
7     Raheem Sterling                Left Winger
-     Marlos Moreno                  Left Winger
20    Bernardo Silva                 Right Winger
26    Riyad Mahrez                   Right Winger
21    Ferran Torres                  Right Winger
-     Patrick Roberts                Right Winger
9     Gabriel Jesus                  Centre-Forward
10    Sergio Agüero                  Centre-Forward