Nothing returning in beautifulsoup

Question

I'm trying to pull the book titles off the respective url in the r variable but can't get anything to return. I'm not sure which tag I should be using but neither the "a" tag or "span" tag have worked. Here's the code

from bs4 import BeautifulSoup
import requests
#https://www.datacamp.com/community/tutorials/amazon-web-scraping-using-beautifulsoup

no_pages = 2

def get_data(pageNo):  
    headers = {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0", "Accept-Encoding":"gzip, deflate", "Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "DNT":"1","Connection":"close", "Upgrade-Insecure-Requests":"1"}

    r = requests.get('https://www.amazon.com/s?k=science+fiction+adventure&i=digital-text&ref=nb_sb_noss_1'+str(pageNo)+'?ie=UTF8&pg='+str(pageNo), headers=headers)#, proxies=proxies)
    content = r.content
    soup = BeautifulSoup(content)
    #print(soup)
    
    
    for a in soup.select("a.a-link-normal.a-text-normal"):
        print(a.get_text())

Answer 1

Actually, it seems to be the exception was incorrect element selection. Here is the working example:

Code:

from bs4 import BeautifulSoup
import requests
import pandas as pd

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "DNT": "1",
    "Connection": "close",
    "Upgrade-Insecure-Requests": "1",
}
for page in range(1, 5):
    cookies = {'session': '134-8225175-0355220'}
    r = requests.get("https://www.amazon.com/s?k=science+fiction+adventure&i=digital-text&page={page}&qid=1632193003&ref=sr_pg_2".format(
            page=page
        ),
        headers=headers,
        cookies=cookies
    )
    soup = BeautifulSoup(r.content, "lxml")
    for d in soup.select("span.a-size-medium.a-color-base.a-text-normal"):
        
        print(d.text)

Output:

Backyard Starship
Valkyrie's Daughter (The Lunar Free State Book 3)
Earthrise - Super Box Set (Book 1-6): An Epic Sci-Fi Adventure
Earthrise - Super Box Set (Book 1-6): An Epic Sci-Fi Adventure
Renaissance and Reboot in Flyover Country
The Best Science Fiction of the Year Volume 5: Volume Five
Red Bounty (Backyard Starship Book 2)
Shaman's Call- Frostburn: A Litrpg Adventure
The Raged
Zero Day Threat: A space opera adventure (The Ungovernable Book 1)        
The Bad Company: A Military Space Opera Adventure
Futures (Death's Handmaiden Book 4)
Judged: A Dark Sci-Fi Romance
Mage-Commander (Starship's Mage Book 11)
The Power of Your Subconscious Mind
Fortitude (Scattered Stars: Conviction Book 4)
Magnitude: A Military Science Fiction Thriller (Multiverse Space Book One)
Black Sand Baron (The Ripple System Book 2)
The Vacuum of Space: A Funny Sci Fi Mystery (Space Janitor Book 1) 
A New Home (Osprey Chronicles Book 3)
Fierce: A Sci-Fi Alien Romance (Rogue Alien Warriors Book 1)       
Nancy and the Naga: A SciFi Alien Romance (Alien Abduction Book 14)
The Dragon's Mate: A SciFi Romance
Dune: The Lady of Caladan (The Caladan Trilogy Book 2)
Vishkanya: Steven Gordon Series Book 11
Zombie Apocalypse: I Can Upgrade: System of Post-Apocalyptic LitRPG Epic Saga Fantasy Book 1
Blood, Oil and Love (Combined Operations Book 2)
Alien Hunter's Captive: A Sci-fi Alien Romance (Scovein Mates Book 3)
The Dark Prince's Prize (Curizan Warrior Book 2)
Purgatory's Shore (Artillerymen Book 1)
Fuzz: When Nature Breaks the Law
Orc Conqueror: A Litrpg Adventure
When Sorrows Come: An October Daye Novel
Earthrise - Super Box Set (Book 1-6): An Epic Sci-Fi Adventure
Shadow's Fall: New White Sands City Cyberpunk Book 1
The Amur River: Between Russia and China
Running Black (The Black Chronicles Book 3)
Lady Liege (The Serving Magic Series Book 3)
The Actual Star: A Novel
A Song of Flight (Warrior Bards Book 3)
Enter Prehistoric (West Of Prehistoric Book 2)
Spark of Imagination: Source of Power Book 1
Farseeker: An Epic Fantasy Sci-Fi Adventure (A Farseeker Novel)
Cascadia Fallen: The Complete Trilogy
Snake Keeper: Alien First Contact Romance (The Federation's Keepers Book 1)
Monster Girl Defense Force: Skulls and Stones
The Last Graduate: A Novel (The Scholomance Book 2)
Winterlight (Green Rider Book 7)
Rogue Ops (Rogue Agents of Magic Book 1)
Salvage Mother (The Coalition Book 10)
Earthrise - Super Box Set (Book 1-6): An Epic Sci-Fi Adventure
The Forgotten Sky: A Fantasy Space Opera
The Invasion (Tower of Power Book 6)
Father Brown Complete Murder Mysteries: The Innocence of Father Brown, The Wisdom of Father Brown, The Donnington Affair…
Deadly Rescue: A Post-Apocalyptic Pandemic Survival Thriller (Escaping the Virus Book 3)
Hell Divers VIII: King of the Wastes (The Hell Divers Series Book 8)
Thrown Away - Book One - "In a Fallen World"
Mary and the Minotaur: A SciFi Alien Romance (Alien Abduction Book 13)
Trouble with the Cursed (Hollows Book 16)
Drone Rider: Cyborg AI Science Fiction
Reborn: Apocalypse (Volume 3)(Re-Release): (A LitRPG/Wuxia Story)
Princess for the Alien Commander: A Scifi Alien Romance (Alien Recruitment Agency Book 3)
Shadowcroft Academy For Dungeons: Year Two
He Who Fights with Monsters 3: A LitRPG Adventure
H. P. Lovecraft: The Complete Fiction
The Great Troll War (The Last Dragonslayer Chronicles)
Chaos Quarter: Syndicate Ways (The Chaos Quarter, Book 5)
Earthrise - Super Box Set (Book 1-6): An Epic Sci-Fi Adventure