CODEWITHSUNDEEP
python
Eniola
Eniola

Reputation: 133

How to successfully scrape webpage for headline links using Newspaper

I am trying to scrape headline links on finviz.com using python library called newspaper. I've successfully scrape for links on the actual website, but when I search for specific company it brings empty.

Here is the sample of my code.

    import newspaper
    news_site = newspaper.build("https://finviz.com/")
    news_site.article_urls()

The above code works pretty well and gave me what I want, but when I use the below code, it brings an empty list instead of list of all the headlines link.

    import newspaper
    news_site = newspaper.build("https://finviz.com/quote.ashx?t=GOOGL")
    news_site.article_urls()

The only difference here is that I'm searching for GOOGL stock related news and trying to get all those links.

What can I do differently so that I can get the list of headline links on this link.

    https://finviz.com/quote.ashx?t=GOOGL

A sample code will be helpful too, thanks

Upvotes: 0

Views: 143

Answers (1)

Krishna Singhal
Krishna Singhal

Reputation: 661

Use this Snippet, it works in my machine

import newspaper
news_site = newspaper.build("https://finviz.com/quote.ashx?t=GOOGL",memoize_articles=False, fetch_images=False)
news_site.article_urls()

Upvotes: 1

Related Questions