CODEWITHSUNDEEP
pythonsearch
Uncle_Dick
Uncle_Dick

Reputation: 79

Using Python to ask a web page to run a search

I have a list of protein names in the "Uniprot" format, and I'd like to convert them all to the MGI format. If you go to www.uniprot.org and type the uniprot protein name into the "Query" bar, it will generate a page with a bunch of information about that protein, including its MGI name (albeit much further down the page).

For example, one Uniprot name is "Q9D880", and by scrolling down, you can see that its corresponding MGI name is "1913775".

I already know how to use Python's urllib to extract the MGI name from a page once I get to that page. What I don't know how to do is write Python code to get the main page to run a query of "Q9D880". My list contains 270 protein names, so it would be nice to avoid copying&pasting each protein name into the Query bar.

I saw the "Google Search from a Python App" post, and I have a firmer understanding of this concept, but I suspect that running a google search is different from running the search function on some other website, like uniprot.org.

I'm running Python 2.7.2, but I'm open to implementing solutions that use other versions of Python. Thanks for the help!

Upvotes: 6

Views: 41247

Answers (5)

brksfrb
brksfrb

Reputation: 1

You can use webbrowser library, code like this:

import webbrowser

webbrowser.open(URL)

Upvotes: 0

jdotjdot
jdotjdot

Reputation: 17082

Easier way to do this is with the requests library. My solution for you also grabs the information itself from the page using BeautifulSoup4.

All you'd have to do, given a dictionary of your query parameters, is:

from bs4 import BeautifulSoup as BS
for protein in my_protein_list:
    text = requests.get('http://www.uniprot.org/uniprot/' + protein).text
    soup = BS(text)
    MGI = soup.find(name='a', onclick="UniProt.analytics('DR-lines', 'click', 'DR-MGI');").text
    MGI = MGI[4:]
    print protein +' - ' + MGI

Upvotes: 7

Bryan
Bryan

Reputation: 17703

You can also do this with PyQuery:

>>> from pyquery import PyQuery as pq    
>>> url = "http://www.uniprot.org/uniprot/{name}"
>>> name = "Q9D880"
>>> response = pq(url=url.format(name=name))
>>> print html("a").filter(lambda e: pq(this).text().startswith("MGI:")).text()
MGI:1913775

Upvotes: 3

Anonymous
Anonymous

Reputation: 11

The query is in the URL, you can call:
http://www.uniprot.org/uniprot/?query=1913775&sort=score

I didn't have time to test this script since I don't have 2.x installed anymore butthe code in 2.x should be something like this:

import urllib
MGIName = "1913775"
print urllib.urlopen(
    "http://www.uniprot.org/uniprot/?query="+ MGIName +"&sort=score").read()

The code in 3.2 I ran was this and it worked fine:

>>> import urllib.request
>>> MGIName = "1913775"
>>> print(urllib.request.urlopen("http://www.uniprot.org/uniprot/?query="+ MGIName +"&sort=score").read())

Just loop the MGIname over the list of names

Upvotes: 1

Silas Ray
Silas Ray

Reputation: 26160

Running the search appears to do a GET on

http://www.uniprot.org/?dataset=uniprot&query=Q9D880&sort=score&url=&lucky=no&random=no

Which eventually redirects you to

http://www.uniprot.org/uniprot/Q9D880

So you should be able to use urllib or an http library (I use httplib2) to do a GET on that address, parameterizing the protein name in the URL so you can search for whichever protein name you want.

Upvotes: 4

Related Questions