BeautifulSoup returns empty list, while code works perfectly on others computer

Question

I am coding a small/simple scraper.

My problem is that I get an empty list but others get a full list.

import requests
from bs4 import BeautifulSoup
url_nba = 'https://stats.nba.com/players/list/'

response = requests.get(url_nba)
soup = BeautifulSoup(response.content, 'html5lib')

# Prints perfectly the HTML
print(soup.prettify())

# Should return a list of HTML tags containing the players
# But returns an empty list (code works fine on other people laptop)
print(soup.select(".players-list__name"))

I have seen Here, that it could be because the API that fill the website does it after the scraper runs. But could not find any answer...

1- I tryed to use time.sleep(10) but did not work

2- Using Jupyter it do not return [] but

[<li class="players-list__name" ng-repeat="player in letter.items"><a href="/player/{{ ::player.id }}/">{{ ::player.name }}</a></li>]

EDIT

Code using selenium

from bs4 import BeautifulSoup
from time import sleep
url="https://stats.nba.com/players/list/"
from selenium import webdriver
driver = webdriver.Chrome(executable_path="C:\\Users\\benoi\\Downloads\\chromedriver_win32\\chromedriver.exe")
driver.get(url)
sleep(5)
soup = BeautifulSoup(driver.page_source,"html5lib")
print(soup.prettify())

sleep(5)
print(soup.select(".players-list__name"))

Answer 1

The issue is with the parser. Change "html5lib" to either "lxml" or "html.parser".

Check out this table to see Pros/Cons of the different parser options

Secondly, unless you need Selenium to wait for a specific element (like a button) to be present, you don't need to wait. It won't move on the the next bit of your code until the initial rendering (although I guess it doesn't hurt to wait). So I removed those.

from bs4 import BeautifulSoup
from selenium import webdriver

url="https://stats.nba.com/players/list/"

driver = webdriver.Chrome(executable_path="C:\\Users\\benoi\\Downloads\\chromedriver_win32\\chromedriver.exe")
driver.get(url)
soup = BeautifulSoup(driver.page_source,"lxml") # <----- here was your issue

print(soup.select(".players-list__name"))

driver.close()

Answer 2

Try selenium and chrome webdriver.It will work.

from selenium import webdriver
from bs4 import BeautifulSoup
import time
url_nba = 'https://stats.nba.com/players/list/'
br=webdriver.Chrome()
br.get(url_nba)
time.sleep(5)
soup = BeautifulSoup(br.page_source, 'html.parser')

for player in soup.select('li.players-list__name'):
   print(player.select_one('a').text)

Output:

Abrines, Alex
Acy, Quincy
Adams, Jaylen
Adams, Steven
Adebayo, Bam
Adel, Deng
Akoon-Purcell, DeVaughn
Aldridge, LaMarcus
Alexander, Kyle
Alexander-Walker, Nickeil
Alkins, Rawle
Allen, Grayson
Allen, Jarrett
Allen, Kadeem
Aminu, Al-Farouq
Anderson, Justin
Anderson, Kyle
Anderson, Ryan
Anigbogu, Ike
Antetokounmpo, Giannis
Antetokounmpo, Kostas
Antetokounmpo, Thanasis
Anthony, Carmelo
Anunoby, OG
Arcidiacono, Ryan
Ariza, Trevor
Augustin, D.J.
Ayton, Deandre
Bacon, Dwayne
Bagley III, Marvin
Baker, Ron
Baldwin IV, Wade
Ball, Lonzo
Bamba, Mo
Barea, J.J.
Barnes, Harrison
Barrett, RJ
Barton, Will
Bates-Diop, Keita
Batum, Nicolas
Bayless, Jerryd
Baynes, Aron
Bazemore, Kent
Bazley, Darius
Beal, Bradley
Beasley, Malik
Beasley, Michael
Belinelli, Marco
Bell, Jordan
Bembry, DeAndre'
Bender, Dragan
Bennett, Anthony
Bertans, Dairis
Bertans, Davis
Beverley, Patrick
Birch, Khem
Bitadze, Goga
Biyombo, Bismack
Bjelica, Nemanja
Blakeney, Antonio
Bledsoe, Eric
Blossomgame, Jaron
Bluiett, Trevon
Bogdanovic, Bogdan
Bogdanovic, Bojan
Bogut, Andrew
Bol, Bol
Bolden, Jonah
Bone, Jordan
Bonga, Isaac
Booker, Devin
Boucher, Chris
Bowen II, Brian
Bowman, Ky
Bradley, Avery
Bradley, Tony
Brantley, Jarrell
Brazdeikis, Ignas
Brewer, Corey
Bridges, Mikal
Bridges, Miles
Brimah, Amida
Briscoe, Isaiah
Brissett, Oshae
Broekhoff, Ryan
Brogdon, Malcolm
Brooks, Armoni
Brooks, Dillon
Brooks, MarShon
Brown, Bruce
Brown, Charles
Brown, Jaylen
Brown, Lorenzo
Brown, Moses
Brown, Sterling
Brown Jr., Troy
Brunson, Jalen
Bryant, Thomas
Bullock, Reggie
Burke, Trey
Burks, Alec
Burton, Deonte
Butler, Jimmy
Caboclo, Bruno
Cacok, Devontae
Calderon, Jose
Caldwell-Pope, Kentavious
Canaan, Isaiah
Cancar, Vlatko
Capela, Clint
Caroline, Jordan
Carroll, DeMarre
Carter, Jevon
Carter, Vince
Carter Jr., Wendell
Carter-Williams, Michael
Caruso, Alex
Casspi, Omri
Cauley-Stein, Willie
Caupain, Troy
Cavanaugh, Tyler
Chandler, Tyson
Chandler, Wilson
Chealey, Joe
Cheatham, Zylan
Chiozza, Chris
Chriss, Marquese
Clark, Gary
Clark, Ian
Clarke, Brandon
Clarkson, Jordan
Claxton, Nicolas
Clemons, Chris
Cleveland, Antonius
Coffey, Amir
Collins, John
Collins, Zach
Collison, Darren
Colson, Bonzie
Conley, Mike
Connaughton, Pat
Cook, Quinn
Cook, Tyler
Cousins, DeMarcus
Covington, Robert
Crabbe, Allen
Craig, Torrey
Crawford, Jamal
Creek, Mitchell
Crowder, Jae
Culver, Jarrett
Cunningham, Dante
Curry, Seth
Curry, Stephen
Daniels, Troy
Davis, Anthony
Davis, Deyonta
Davis, Ed
Davis, Terence
Davis, Tyler
DeRozan, DeMar
Dedmon, Dewayne
Dekker, Sam
Delgado, Angel
Dellavedova, Matthew
Deng, Luol
Derrickson, Marcus
DiVincenzo, Donte
Diallo, Cheick
Diallo, Hamidou
Dieng, Gorgui
Dinwiddie, Spencer
Doncic, Luka
Dorsey, Tyler
Dort, Luguentz
Dotson, Damyean
Doumbouya, Sekou
Dozier, PJ
Dragic, Goran
Drummond, Andre
Dudley, Jared
Dunn, Kris
Durant, Kevin
Duval, Trevon
Edwards, Carsen
Edwards, Vincent
Ellenson, Henry
Ellington, Wayne
Embiid, Joel
Ennis III, James
Eubanks, Drew
Evans, Jacob
Evans, Jawun
Evans, Tyreke
Exum, Dante
Fall, Tacko
Faried, Kenneth
Favors, Derrick
Felicio, Cristiano
Felton, Raymond
Ferguson, Terrance
Fernando, Bruno
Ferrell, Yogi
Finney-Smith, Dorian
Forbes, Bryn
Fournier, Evan
Fox, De'Aaron
Franks, Robert
Frazier, Michael
Frazier, Tim
Frazier Jr., Melvin
Fredette, Jimmer
Frye, Channing
Fultz, Markelle
Gabriel, Wenyen
Gafford, Daniel
Gallinari, Danilo
Galloway, Langston
Gant, JaKeenan
Garland, Darius
Garrett, Billy
Gasol, Marc
Gasol, Pau
Gay, Rudy
George, Paul
Gibson, Jonathan
Gibson, Taj
Giles III, Harry
Gilgeous-Alexander, Shai
Gobert, Rudy
Goodwin, Brandon
Gordon, Aaron
Gordon, Eric
Gortat, Marcin
Graham, Devonte'
Graham, Treveon
Grant, Jerami
Grant, Jerian
Grantham, Donte
Gravett, Hassani
Gray, Josh
Green, Danny
Green, Draymond
Green, Gerald
Green, JaMychal
Green, Javonte
Green, Jeff
Griffin, Blake
Guduric, Marko
Guy, Kyle
Hachimura, Rui
Hall, Devon
Hamilton, Daniel
Hands, Jaylen
Hannahs, Dusty
Hardaway Jr., Tim
Harden, James
Harkless, Maurice
Harrell, Montrezl
Harris, Devin
Harris, Gary
Harris, Joe
Harris, Tobias
Harrison, Andrew
Harrison, Shaquille
Hart, Josh
Hartenstein, Isaiah
Haslem, Udonis
Hayes, Jaxson
Hayward, Gordon
Henson, John
Hernandez, Dewan
Hernangomez, Juancho
Hernangomez, Willy
Herro, Tyler
Hezonja, Mario
Hicks, Isaiah
Hield, Buddy
Highsmith, Haywood
Hill, Ahmed
Hill, George
Hill, Solomon
Hoard, Jaylen
Holiday, Aaron
Holiday, Jrue
Holiday, Justin
Holland, John
Hollis-Jefferson, Rondae
Holmes, Richaun
Hood, Rodney
Horford, Al
Horton-Tucker, Talen
House Jr., Danuel
Howard, Dwight
Howard, William
Huerter, Kevin
Humphries, Isaac
Hunter, De'Andre
Hunter, RJ
Hutchison, Chandler
Ibaka, Serge
Iguodala, Andre
Ilyasova, Ersan
Ingles, Joe
Ingram, Andre
Ingram, Brandon
Irving, Kyrie
Isaac, Jonathan
Iwundu, Wes
Jackson, Demetrius
Jackson, Frank
Jackson, Josh
Jackson, Justin
Jackson, Reggie
Jackson Jr., Jaren
James, Justin
James, LeBron
Jefferson, Amile
Jeffries, DaQuan
Jenkins, John
Jerebko, Jonas
Jerome, Ty
Johnson, Alize
Johnson, Amir
Johnson, BJ
Johnson, Cameron
Johnson, James
Johnson, Keldon
Johnson, Stanley
Johnson, Tyler
Johnson, Wesley
Jokic, Nikola
Jones, Damian
Jones, Jalen
Jones, Jemerrio
Jones, Terrence
Jones, Tyus
Jones Jr., Derrick
Jordan, DeAndre
Joseph, Cory
Kabengele, Mfiondu
Kaminsky, Frank
Kanter, Enes
Kennard, Luke
Kidd, Stanton
Kidd-Gilchrist, Michael
King, George
Kleber, Maxi
Knight, Brandon
Knox, Kevin
Konate, Sagaba
Konchar, John
Korkmaz, Furkan
Kornet, Luke
Korver, Kyle
Koufos, Kosta
Koumadje, Christ
Kurucs, Rodions
Kuzma, Kyle
LaVine, Zach
Labissiere, Skal
Lamb, Jeremy
Langford, Romeo
Law, Vic
Layman, Jake
LeVert, Caris
Leaf, TJ
Lecque, Jalen
Lee, Courtney
Lee, Damion
Lemon Jr., Walter
Len, Alex
Leonard, Kawhi
Leonard, Meyers
Lessort, Mathias
Leuer, Jon
Lillard, Damian
Lin, Jeremy
Little, Nassir
Livingston, Shaun
Lofton, Zach
Looney, Kevon
Lopez, Brook
Lopez, Robin
Love, Kevin
Lowry, Kyle
Loyd, Jordan
Lucas, Kalin
Luwawu-Cabarrot, Timothe
Lydon, Tyler
Lyles, Trey
Machado, Scott
Mack, Shelvin
Macon, Daryl
Macura, J.P.
Magette, Josh
Mahinmi, Ian
Maker, Thon
Mann, Terance
Marinkovic, Vanja
Marjanovic, Boban
Markkanen, Lauri
Martin, Caleb
Martin, Cody
Martin, Jarell
Martin, Jeremiah
Martin, Kelan
Mason, Frank
Maten, Yante
Mathews, Garrison
Mathias, Dakota
Matthews, Wesley
Mbah a Moute, Luc
McCall, Tahjere
McCaw, Patrick
McCollum, CJ
McConnell, T.J.
McDaniels, Jalen
McDermott, Doug
McDowell-White, William
McGee, JaVale
McGruder, Rodney
McKinnie, Alfonzo
McLaughlin, Jordan
McLemore, Ben
McRae, Jordan
Meeks, Jodie
Mejri, Salah
Melli, Nicolo
Melton, De'Anthony
Metu, Chimezie
Middleton, Khris
Mika, Eric
Miles, CJ
Miles, Isaiah
Miller, Darius
Miller, Malcolm
Mills, Patty
Millsap, Paul
Milton, Shake
Mirotic, Nikola
Mitchell, Donovan
Mitrou-Long, Naz
Mokoka, Adam
Monk, Malik
Monroe, Greg
Mooney, Matt
Moore, Ben
Moore, E'Twaun
Morant, Ja
Moreland, Eric
Morris, Jaylen
Morris, Marcus
Morris, Markieff
Morris, Monte
Motiejunas, Donatas
Motley, Johnathan
Mozgov, Timofey
Mudiay, Emmanuel
Mulder, Mychal
Murphy, Jordan
Murray, Dejounte
Murray, Jamal
Musa, Dzanan
Muscala, Mike
Mykhailiuk, Svi
Nene
Nader, Abdel
Nance Jr., Larry
Napier, Shabazz
Neto, Raul
Niang, Georges
Noah, Joakim
Noel, Nerlens
Norvell Jr., Zach
Nowell, Jaylen
Nowitzki, Dirk
Ntilikina, Frank
Nunn, Kendrick
Nunnally, James
Nurkic, Jusuf
Nwaba, David
O'Neale, Royce
O'Quinn, Kyle
Ojeleye, Semi
Okafor, Jahlil
Okeke, Chuma
Okobo, Elie
Okogie, Josh
Okpala, KZ
Oladipo, Victor
Olynyk, Kelly
Oni, Miye
Osman, Cedi
Oubre Jr., Kelly
Pachulia, Zaza
Parker, Jabari
Parker, Tony
Parsons, Chandler
Paschall, Eric
Patterson, Patrick
Patton, Justin
Paul, Chris
Payne, Cameron
Payton, Elfrid
Payton II, Gary
Pelle, Norvel
Perkins, Josh
Perrantes, London
Phillip, Tarik
Pineiro, Isaiah
Pinson, Theo
Plumlee, Mason
Plumlee, Miles
Poeltl, Jakob
Poirier, Vincent
Pondexter, Quincy
Ponds, Shamorie
Poole, Jordan
Porter Jr., Kevin
Porter Jr., Michael
Porter Jr., Otto
Portis, Bobby
Porzingis, Kristaps
Powell, Dwight
Powell, Norman
Poythress, Alex
Prince, Taurean
Rabb, Ivan
Randle, Chasson
Randle, Julius
Reaves, Josh
Reddish, Cameron
Redick, JJ
Reed, Davon
Reynolds, Cameron
Richardson, Josh
Richardson, Malachi
Rivers, Austin
Roberson, Andre
Robinson, Devin
Robinson, Duncan
Robinson, Jerome
Robinson, Justin
Robinson, Mitchell
Robinson III, Glenn
Roby, Isaiah
Rondo, Rajon
Rose, Derrick
Ross, Terrence
Rozier, Terry
Rubio, Ricky
Russell, D'Angelo
Sabonis, Domantas
Samanic, Luka
Sampson, Brandon
Sampson, JaKarr
Saric, Dario
Satoransky, Tomas
Schofield, Admiral
Schroder, Dennis
Scott, Mike
Sefolosha, Thabo
Selden, Wayne
Sexton, Collin
Shamet, Landry
Shayok, Marial
Shumpert, Iman
Siakam, Pascal
Sibert, Jordan
Silva, Chris
Simmons, Ben
Simmons, Jonathon
Simmons, Kobi
Simons, Anfernee
Sirvydis, Deividas
Smailagic, Alen
Smart, Marcus
Smith, Ish
Smith, JR
Smith, Jason
Smith, Zhaire
Smith Jr., Dennis
Snell, Tony
Spalding, Ray
Spellman, Omari
Stauskas, Nik
Stephens, DJ
Stephenson, Lance
Strus, Max
Sumner, Edmond
Swanigan, Caleb
Tatum, Jayson
Teague, Jeff
Temple, Garrett
Teodosic, Milos
Terrell, Jared
Terry, Emanuel
Theis, Daniel
Thomas, Isaiah
Thomas, Khyri
Thomas, Lance
Thomas, Matt
Thompson, Klay
Thompson, Tristan
Thornwell, Sindarius
Thybulle, Matisse
Tolliver, Anthony
Towns, Karl-Anthony
Trent Jr., Gary
Trier, Allonzo
Tucker, PJ
Turner, Evan
Turner, Myles
Udoh, Ekpe
Ulis, Tyler
Valanciunas, Jonas
Valentine, Denzel
VanVleet, Fred
Vanderbilt, Jarred
Vonleh, Noah
Vucevic, Nikola
Wade, Dean
Wade, Dwyane
Wagner, Moritz
Waiters, Dion
Walker, Kemba
Walker IV, Lonnie
Wall, John
Wallace, Tyrone
Wanamaker, Brad
Warren, T.J.
Washburn, Julian
Washington, P.J.
Watanabe, Yuta
Waters, Tremont
Weatherspoon, Quinndary
Welsh, Thomas
Westbrook, Russell
White, Coby
White, Derrick
White, Okaro
Whitehead, Isaiah
Whiteside, Hassan
Wiggins, Andrew
Wigginton, Lindell
Wilcox, CJ
Williams, Alan
Williams, C.J.
Williams, Grant
Williams, Johnathan
Williams, Kenrich
Williams, Lou
Williams, Marvin
Williams, Troy
Williams III, Robert
Williams-Goss, Nigel
Williamson, Zion
Wilson, D.J.
Windler, Dylan
Winslow, Justise
Wood, Christian
Wright, Delon
Wright-Foreman, Justin
Yabusele, Guerschon
Young, Nick
Young, Thaddeus
Young, Trae
Zhou Qi
Zeller, Cody
Zeller, Tyler
Zizic, Ante
Zubac, Ivica

You can download latest Chrome driver from here Download