CODEWITHSUNDEEP
pythonfor-loopbeautifulsoup
darylvickerman
darylvickerman

Reputation: 612

How do I link a title with a url in Python and BeautifulSoup?



So im working on a script to download videos for me automatically, however it would seem i have stumbled upon a problem(insufficient experience).

How do i link a category title with a bunch of url's?

Expected output:
->CATEGORY 1
->https://example.com/part/of/link/x
->https://example.com/part/of/link/x
->https://example.com/part/of/link/x
.....
->CATEGORY 2
->https://example.com/part/of/link/x

for category_title in soup.findAll(class_='section-title'):
    title = category_title.get_text().lstrip()
    print(title)

for onelink in soup.findAll('a'):
    link = onelink.get('href')
    print(f'https://example.com{link}')

What this does is:
Lists all the titles(10 titles)
Lists all the links(100 links)

<div class="section-title">
    CATEGORY 1
    
    <li class="section-item next-random">
        <a class="item" href="/part/of/link/x">

            <div class="title-container">
                <div class="btn-primary btn-sm pull-right">
                    BUTTON
                </div>

                <span class="random-name">
                    URL TITLE
                </span>
            </div>
        </a>
    </li>
</ul>

Upvotes: 0

Views: 180

Answers (1)

Benliam12
Benliam12

Reputation: 81

Depending on how the HTML structure of your page is made, you could check the "section-title" parent object, and then, list all the link of that particular section. Giving you all the link for category #

Here some help

Upvotes: 1

Related Questions