CODEWITHSUNDEEP
pythontextauthorpubmed
randomal
randomal

Reputation: 6592

Obtaining author affiliations from PubMed via Python

I am working on a Python script (modified from here and reported below) to search on PubMed the number of papers from a certain university, and download the affiliation of the collaborators. If I run the code, instead of the affiliations I get <Element 'Affiliation' at 0x106ea7e50>. Do you know how to fix this? And what should I do to download the affiliation for all the authors? Thanks!

import urllib, urllib2, sys
import xml.etree.ElementTree as ET

def chunker(seq, size):
    return (seq[pos:pos + size] for pos in xrange(0, len(seq), size))

query = '(("University of Copenhagen"[Affiliation]))# AND ("1920"[Publication Date] : "1930"[Publication Date]))'

esearch = 'http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&mindate=2001&maxdate=2010&retmode=xml&retmax=10000000&term=%s' % (query)
handle = urllib.urlopen(esearch)
data = handle.read()

root = ET.fromstring(data)
ids = [x.text for x in root.findall("IdList/Id")]
print 'Got %d articles' % (len(ids))

for group in chunker(ids, 100):
    efetch = "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?&db=pubmed&retmode=xml&id=%s" % (','.join(group))
    handle = urllib.urlopen(efetch)
    data = handle.read()

    root = ET.fromstring(data)
    for article in root.findall("PubmedArticle"):
        pmid = article.find("MedlineCitation/PMID").text
        year = article.find("MedlineCitation/Article/Journal/JournalIssue/PubDate/Year")
        if year is None: year = 'NA'
        else: year = year.text
        aulist = article.findall("MedlineCitation/Article/AuthorList/Author")
        affiliation = article.find("MedlineCitation/Article/AuthorList/Author/Affiliation")
        print pmid, year, len(aulist), affiliation

Upvotes: 1

Views: 2007

Answers (2)

Franck Dernoncourt
Franck Dernoncourt

Reputation: 83177

This answer updates the code to Python 3, and fixes the affiliation location in the XML (I see it in MedlineCitation/Article/AuthorList/Author/AffiliationInfo, not "MedlineCitation/Article/AuthorList/Author/Affiliation, maybe it has changed location over time?). In this example, we'll retrieve the author affiliations for only 1 paper, https://pubmed.ncbi.nlm.nih.gov/31888621/, based on its PMID (31888621):

import xml.etree.ElementTree as ET
from urllib.request import urlopen

def chunker(seq, size):
    return (seq[pos:pos + size] for pos in xrange(0, len(seq), size))

efetch = "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?&db=pubmed&retmode=xml&id=%s" % ('31888621')
handle = urlopen(efetch)
data = handle.read()

root = ET.fromstring(data)
for article in root.findall("PubmedArticle"):
    pmid = article.find("MedlineCitation/PMID").text
    year = article.find("MedlineCitation/Article/Journal/JournalIssue/PubDate/Year")
    if year is None: year = 'NA'
    else: year = year.text
    aulist = article.findall("MedlineCitation/Article/AuthorList/Author")
    affiliation = article.find("MedlineCitation/Article/AuthorList/Author/AffiliationInfo")
    #print(pmid, year, len(aulist), affiliation, aulist, ET.dump(root))
    for author in aulist:    
        print(ET.dump(author))

outputs:

<Author ValidYN="Y">
                    <LastName>Tang</LastName>
                    <ForeName>Lingkai</ForeName>
                    <Initials>L</Initials>
                    <AffiliationInfo>
                        <Affiliation>Department of Mechanical Engineering, University of Saskatchewan, Saskatoon, S7N 5A9, Canada.</Affiliation>
                    </AffiliationInfo>
                </Author>

None
<Author ValidYN="Y">
                    <LastName>Mostafa</LastName>
                    <ForeName>Sakib</ForeName>
                    <Initials>S</Initials>
                    <AffiliationInfo>
                        <Affiliation>Division of Biomedical Engineering, University of Saskatchewan, Saskatoon, S7N 5A9, Canada.</Affiliation>
                    </AffiliationInfo>
                </Author>

None
<Author ValidYN="Y">
                    <LastName>Liao</LastName>
                    <ForeName>Bo</ForeName>
                    <Initials>B</Initials>
                    <AffiliationInfo>
                        <Affiliation>School of Mathematics and Statistics, Hainan Normal University, Haikou, 571158, China.</Affiliation>
                    </AffiliationInfo>
                </Author>

None
<Author ValidYN="Y">
                    <LastName>Wu</LastName>
                    <ForeName>Fang-Xiang</ForeName>
                    <Initials>FX</Initials>
                    <Identifier Source="ORCID">0000-0002-4593-9332</Identifier>
                    <AffiliationInfo>
                        <Affiliation>Department of Mechanical Engineering, University of Saskatchewan, Saskatoon, S7N 5A9, Canada. [email protected].</Affiliation>
                    </AffiliationInfo>
                    <AffiliationInfo>
                        <Affiliation>Division of Biomedical Engineering, University of Saskatchewan, Saskatoon, S7N 5A9, Canada. [email protected].</Affiliation>
                    </AffiliationInfo>
                </Author>

None

Upvotes: 1

furkle
furkle

Reputation: 5059

The reason this is occurring is that the affiliation object refers to an XML element, not a piece of text. If the string you want is contained within the value, like so:

<affiliation>
    your_affiliation_text
</affiliation>

you'd want to print affiliation.text.

If the string you wanted was contained within an attribute, like so:

 <affiliation your_attribute_name="your_affiliation">

you'd want to use affiliation.attrib[name].

Upvotes: 2

Related Questions