CODEWITHSUNDEEP
pythonbrowsergoogle-search
R.A.Y
R.A.Y

Reputation: 125

Open Browser with python along with a search command

My exact problem statement is:

I have:

  1. sim_ques(a Text) & 2. options(list)

and I want to do this:

  1. Open web browser like chrome and search with string search_query
  2. press ctrl + F programatically and search for an element of list options

if step 2 is possible I want to search multiple strings in the same browser.

NOTE

I have already tried:

  1. Using Google-search-api (https://github.com/abenassi/Google-Search-API) to get results and then retrieved link descriptions from the list. Then I searched the string in the that description.

Code is shown here:

print('Googling')
num_pages = 1
points = list()
content = ""

search_res = google.search(sim_ques, num_pages)
print('\nsearch results achieved\n')

page = ""

for re in search_res:
    page = page+re.description

page = page.lower()

# link = search_res[0].link
# print('\nlink obtained\n')
#
# content = get_page(link)
# print('\ncontent recieved\n')
#
# soup = BeautifulSoup(content, "lxml")
# print('\nsoup initialized\n')
#
# # kill all script and style elements
# for script in soup(["script", "style"]):
#     script.decompose()  # rip it out
#
# # get text
# text = soup.get_text().lower()
#
# # break into lines and remove leading and trailing space on each
# lines = (line.strip() for line in text.splitlines())
# # break multi-headlines into a line each
# chunks = (phrase.strip() for line in lines for phrase in line.split("  "))
# # drop blank lines
# page = '\n'.join(chunk for chunk in chunks if chunk)

print('page retrieved' + page)

for o in options:
    points.append(page.count(o.lower()))

return points

But I want the results in new browser and not inside python ide. Also results of google-search-api is very slow. Is there any any to make it fast.

Upvotes: 0

Views: 968

Answers (1)

Grismar
Grismar

Reputation: 31354

Although you tried using Google-search-api https://github.com/abenassi/Google-Search-API, this is really just a wrapper around some scraping code and the poor performance you are experiencing is a result of the specific implementation.

To get performance similar to the performance you experience when using Google Search in a browser, you can set up a custom search on the actual Google Custom Search JSON API, more here https://developers.google.com/custom-search/v1/introduction

This requires that you get an API key and limits you to 10,000 searches daily, but the control panel does allow you to include web search and you can tweak the actual search in many ways. Many people ignore this option because they believe it is restricted to a single site.

Once you obtain an API key and set up your custom search engine, using it and getting good performance is as simple as basic http calls using the standard Python 3 package urllib.

Upvotes: 0

Related Questions