CODEWITHSUNDEEP
pythonerror-handlingscopuspybliometrics
Caio Henrique Pinheiro
Caio Henrique Pinheiro

Reputation: 1

Handling Scopus400Error in pybliometrics: Exceeding Maximum Query Limit

I'm using the pybliometrics library to conduct a search on the Scopus database for a range of publication years, using a complex query. However, when I run my script, I encounter a Scopus400Error stating that I have exceeded the maximum number allowed for the service level. Here is the part of the code that seems to be causing the problem:

x = ScopusSearch(
    f'PUBYEAR IS {year} AND TITLE ( "ethanol upgrading" OR "Guerbet" OR "Lebedev" OR "ethanol to" ) AND NOT TITLE ( "reforming" OR "electrocatalysis" OR "photocatalysis" OR "homogeneous") AND TITLE-ABS-KEY ( "1-butanol" OR "1,3-butadiene" OR "n-butanol" OR "butadiene" OR "higher alcohols" OR "Guerbet" OR "Lebedev" OR "diethyl ether" OR "BTX" OR "buetenes" OR "butene" OR "Isobutene" ) AND SUBJAREA ( ceng OR chem OR ener OR engi OR envi OR mate ) AND NOT SUBJAREA ( medi OR nurs OR vete OR dent OR heal OR mult ) AND NOT SUBJAREA ( agri OR bioc OR immu OR neur OR phar ) AND NOT SUBJAREA ( arts OR busi OR deci OR econ OR psyc OR soci ) AND NOT SUBJAREA ( phys OR math ) AND DOCTYPE ( "AR" ) AND SRCTYPE ( j ) ',
    view="STANDARD")

Here's the error message:

pybliometrics.scopus.exception.Scopus400Error: Exceeds the maximum number allowed for the service level

I suspect the issue might be related to the complexity or length of the query. I tried running the script exactly as it was provided by the author, using the same query, but still encountered the issue. It's important to note that earlier today the script did seem to run a few times, as it progressed and then requested the API. How can I modify my query or manage the search to avoid hitting this limit? Is there a way to break the query into smaller parts or adjust the service level?

Here's a link to the related GitHub repo and a YouTube video explaining the project setup and goals in detail.

Upvotes: 0

Views: 180

Answers (2)

MERose
MERose

Reputation: 4421

Try this:

q = f'PUBYEAR IS {year} AND TITLE ( "ethanol upgrading" OR "Guerbet" OR "Lebedev" OR "ethanol to" ) AND '\
    'NOT TITLE ( "reforming" OR "electrocatalysis" OR "photocatalysis" OR "homogeneous") AND '\
    'TITLE-ABS-KEY ( "1-butanol" OR "1,3-butadiene" OR "n-butanol" OR "butadiene" OR "higher alcohols" OR "Guerbet" OR "Lebedev" OR "diethyl ether" OR "BTX" OR "buetenes" OR "butene" OR "Isobutene" ) AND '\
    'SUBJAREA ( ceng OR chem OR ener OR engi OR envi OR mate ) AND '\
    'NOT SUBJAREA ( medi OR nurs OR vete OR dent OR heal OR mult ) AND '\
    'NOT SUBJAREA ( agri OR bioc OR immu OR neur OR phar ) AND '\
    'NOT SUBJAREA ( arts OR busi OR deci OR econ OR psyc OR soci ) AND '\
    'NOT SUBJAREA ( phys OR math ) AND '\
    'DOCTYPE ( "AR" ) AND SRCTYPE ( "j" )'
x = ScopusSearch(q, view="STANDARD")

If this doesn't work, than your institution doesn't pay for the API access.

Upvotes: 0

miah
miah

Reputation: 10433

I did a quick google and it appears that the error means that your query is messed up.

pybliometrics.scopus.exception.Scopus400Error: BAD REQUEST: Usually an invalid search query, such as a missing parenthesis. Verify that your query works in Advanced Search.

I suspect that you are missing quotes around parts of the query. Without the quotes, the sql engine is most likely trying to map them to field names.

AND SUBJAREA ( ceng OR chem OR ener OR engi OR envi OR mate )
AND NOT SUBJAREA ( medi OR nurs OR vete OR dent OR heal OR mult )
AND NOT SUBJAREA ( agri OR bioc OR immu OR neur OR phar )
AND NOT SUBJAREA ( arts OR busi OR deci OR econ OR psyc OR soci )
AND NOT SUBJAREA ( phys OR math )

should be:

AND SUBJAREA ( "ceng" OR "chem" OR "ener" OR "engi" OR "envi" OR "mate" )
AND NOT SUBJAREA ( "medi" OR "nurs" OR "vete" OR "dent" OR "heal" OR "mult" )
AND NOT SUBJAREA ( "agri" OR "bioc" OR "immu" OR "neur" OR "phar" )
AND NOT SUBJAREA ( "arts" OR "busi" OR "deci" OR "econ" OR "psyc" OR "soci" )
AND NOT SUBJAREA ( "phys" OR "math" )

and AND SRCTYPE ( j ) should be AND SRCTYPE ( "j" )

Upvotes: 0

Related Questions