CODEWITHSUNDEEP
pythonyahooyql
Jason Costabile
Jason Costabile

Reputation: 149

Doing a Simple Yahoo Search in Python

I need to write a Python script which, at one point, does a Yahoo web search to find and download a bunch of C source files. I'm very new to this and I can't figure out how to just get started with doing a simple web search... I've seen a lot of stuff about BOSS but, from my understanding, this is something you need to pay to use? I am not willing to pay for this.

I've used Python YQL to get some RSS results as follows:

import yql
y = yql.Public()
result = y.execute('select * from rss where url="http://www.un.org/apps/news/rss/rss_top.asp"');

for row in result.rows:
   print row.get('title')

And this seems to work, but I can't figure out how to just do a normal web search (since the search.web table is apparently gone). A basic working example would be much appreciated.

Upvotes: 1

Views: 967

Answers (2)

salathe
salathe

Reputation: 51970

I can't figure out how to just do a normal web search (since the search.web table is apparently gone). A basic working example would be much appreciated.

You can use Bing as your search provider and use the microsoft.bing.web data table to perform a web search.

A basic example in Python, which prints the titles of the first 10 results for cake, might look like:

import yql
y = yql.Public()
env = "http://datatables.org/alltables.env"
query = "select * from microsoft.bing.web where query=@query"

results = y.execute(query, {"query": "cake"}, env=env)

for row in results.rows:
    print row.get("Title")

Upvotes: 1

Tom
Tom

Reputation: 22841

You might want to try it using mechanize, which simulates a browser. If you need to clean out some of the crud in the resulting files, use Beautiful Soup.

Upvotes: 2

Related Questions