Reputation: 26600
I have a script named requests.py
that needs to use the third-party requests
package. The script either can't import the package, or can't access its functionality.
Why isn't this working, and how do I fix it?
Trying a plain import and then using the functionality results in an AttributeError
:
import requests
res = requests.get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "/Users/me/dev/rough/requests.py", line 1, in <module>
import requests
File "/Users/me/dev/rough/requests.py", line 3, in <module>
requests.get('http://www.google.ca')
AttributeError: module 'requests' has no attribute 'get'
In more recent versions of Python, the error message instead reads AttributeError: partially initialized module 'requests' has no attribute 'get' (most likely due to a circular import)
.
Using from-import of a specific name results in an ImportError
:
from requests import get
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import get
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests import get
ImportError: cannot import name 'get'
In more recent versions of Python, the error message instead reads ImportError: cannot import name 'get' from partially initialized module 'requests' (most likely due to a circular import) (/Users/me/dev/rough/requests.py)
.
Using from-import for a module inside the package results in a different ImportError
:
from requests.auth import AuthBase
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests.auth import AuthBase
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests.auth import AuthBase
ImportError: No module named 'requests.auth'; 'requests' is not a package
Using a star-import and then using the functionality raises a NameError
:
from requests import *
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import *
File "/Users/me/dev/rough/requests.py", line 3, in <module>
res = get('http://www.google.ca')
NameError: name 'get' is not defined
From Python 3.13 onwards the error message becomes very clear, from the documentation:
A common mistake is to write a script with the same name as a standard library module. When this results in errors, we now display a more helpful error message:
$ python random.py Traceback (most recent call last): File "/home/me/random.py", line 1, in <module> import random File "/home/me/random.py", line 3, in <module> print(random.randint(5)) ^^^^^^^^^^^^^^ AttributeError: module 'random' has no attribute 'randint' (consider renaming '/home/me/random.py' since it has the same name as the standard library module named 'random' and prevents importing that standard library module)
Similarly, if a script has the same name as a third-party module that it attempts to import and this results in errors, we also display a more helpful error message:
$ python numpy.py Traceback (most recent call last): File "/home/me/numpy.py", line 1, in <module> import numpy as np File "/home/me/numpy.py", line 3, in <module> np.array([1, 2, 3]) ^^^^^^^^ AttributeError: module 'numpy' has no attribute 'array' (consider renaming '/home/me/numpy.py' if it has the same name as a library you intended to import)
For cases where you name your module the same as an existing one on purpose and want to handle that situation, see How can I import from the standard library, when my project has a module with the same name? (How can I control where Python looks for modules?)
Upvotes: 103
Views: 56637
Reputation: 8123
The reason for the circular dependency error mentioned in the question is explained in the accepted answer:
This happens because your local module named
requests.py
shadows the installedrequests
module you are trying to use.
In my case, I encountered a circular dependency error in a module but had no file or directory with that name.
My guess is that the module became corrupted somehow because after I uninstalled it and reinstalled the same version, it worked:
pip uninstall <module>==<version>
pip install <module>==<version>
You can identify the module causing the error in the error log stack trace.
This might be helpful for those receiving this error when it is not related to the file name.
Upvotes: 0
Reputation: 1104
Circular dependency : It occurs when two or more modules depend on each other. This is due to the fact that each module is defined in terms of the other.
If you are getting circular import error similar to below request module error.
AttributeError: partially initialized module 'requests' has no attribute 'post' (most likely due to a circular import)
Please try to rename the file.
The error for this error is usually due to conflict with the file name where you are trying to import the requests
module.
I was also having same problem, where my file name was email.py
and I was trying to import the requests module. So, it was having some conflict with email.parser
. So, I changed the file name from email.py
to email1.py
and it worked.
For more info on circular dependency: https://stackabuse.com/python-circular-imports/
Upvotes: 1
Reputation: 26600
This happens because your local module named requests.py
shadows the installed requests
module you are trying to use. The current directory is prepended to sys.path
, so the local name takes precedence over the installed name.
An extra debugging tip when this comes up is to look at the Traceback carefully, and realize that the name of your script in question is matching the module you are trying to import:
Notice the name you used in your script:
File "/Users/me/dev/rough/requests.py", line 1, in <module>
The module you are trying to import: requests
Rename your module to something else to avoid the name collision.
Python may generate a requests.pyc
file next to your requests.py
file (in the __pycache__
directory in Python 3). Remove that as well after your rename, as the interpreter will still reference that file, re-producing the error. However, the pyc
file in __pycache__
should not affect your code if the py
file has been removed.
In the example, renaming the file to my_requests.py
, removing requests.pyc
, and running again successfully prints <Response [200]>
.
Note: This doesn't only happen when naming your file as the module you are trying to import. This can also happen if you name your file the same as a module imported by a module you import directly. For example, having a file called copy.py
and trying to import pandas
from there, will give
ImportError: cannot import name 'copy' from 'copy'
That is because pandas
imports copy
. There is no magic solution here as you can't know all the modules' names in the world, but a rule of thumb is to try to make names of modules as unique as possible and try to change the name whenever you get such error.
Upvotes: 94
Reputation: 91
just put down the import which causing error to the last
ex:
import requests
import flask
import numpy
if import flask
is causing error then move that to bottom of the imports
solution:-
import requests
import numpy
import flask
Upvotes: 1
Reputation: 61635
Problems such as this occur when a Python source file in the project has the same name as some external library module (either in the standard library or an installed third-party package). When attempting to import
from the external library (which requires using an absolute import), the project's own module is found instead, causing various errors depending on the exact details.
The easiest way to fix the problem, in ordinary cases, is to rename the affected file. It may also be necessary to locate and delete any corresponding .pyc
files. (This is perfectly safe; the files are just a cache of the work that was previously done to translate Python source code into bytecode for the Python virtual machine, analogous to .class
files in Java.)
When the project's own module is preferred, this is because of how Python searches for module source code for absolute imports. Depending on exactly how Python is started, the module search path defined in sys.path
will usually start with a path that is within the current project. Python will look there first, before looking in any standard library or third-party library folders. This search process does not care about the folder where the importing module is located; any relative paths are relative to the process' current working directory, not the importing module. (However, the standard library paths will normally be absolute paths, and near the end of sys.path
, anyway.)
AttributeError
An AttributeError
occurs because the project's own module simply doesn't define the function, class etc. that the calling code wants to use from the external library module. As a result, the 'module' object
that represents the project's module has no attribute
with the specified name, so that's exactly what the error message claims.
In rare, unfortunate cases, the project's own module might happen to define something with the same name such that it does something different. This can cause any kind of exception or other logical error. For example, if the first example from the question is modified:
import requests
def get():
pass
res = requests.get('http://www.google.ca')
print(res)
now a TypeError
will be raised instead, because the code will attempt to call the locally-defined get
function with the wrong number of arguments.
ImportError
Using a specific from-import causes the error to be reported differently because the problem is now detected during the importing process itself. In the example in the question, from requests import get
means, instead of creating a new global name requests
which names the module, there should be a new global name get
which names the function from that module (the one which would be referred to as requests.get
after a plain import). This requires looking up the get
attribute from that module; but since the wrong module was loaded, that attribute lookup fails. More recent versions of Python report this as a probable circular import.
Other import attempts can cause different ImportError
s which complain that the imported module "is not a package". This is self-explanatory: in the example in the question, requests
is an ordinary module (because it is defined by a source code file, requests.py
), but the desired requests
- defined by the third-party library - is a package (defined by several files inside a requests
folder elsewhere, including an __init__.py
which defines some top-level package contents that are not modules, like the get
function).
NameError
Using a star-import, like from requests import *
, will not fail directly - it creates global names for all the module contents. However, since the module in the question example doesn't contain anything named get
, its attempt to star-import itself will not define that name either. Thus, a NameError
occurs - the normal error for trying to use a global name that doesn't exist.
Python's default import
system is based on names, not file paths, and Python does not distinguish between "(driver) script files" and "library (module) files". Any Python source code can be import
ed. Although Python caches module imports, the main script will not generally be in this cache, which means it is perfectly capable of attempting to import
itself. This is what happens in the example in the question: since requests
was not already import
ed, there is nothing in the cache, so the attempt in the driver script (named requests.py
) to import requests
will search for something to import, and find the driver script (i.e., its own source file).
This only causes a problem at import time, if the module tries to use that import in its own initialization, e.g. by doing a from-import. Otherwise, the problem will be deferred, resulting in (typically) AttributeError
or NameError
when trying to use the functionality. See also:
Whenever a module is loaded (i.e., imported from its source, rather than using the cache), its own top-level import
statements run, causing more imports, recursively. Any of these indirect imports could potentially find the local code. The Python standard library is not a package, and mostly uses absolute import to refer to its other components. For example, as pointed out in Dave Rove's answer, attempting to import the standard library decimal
module could fail when tried from a source file named numbers.py
, or within a project that has such a source file.
In one especially pernicious case, having a file named token.py
in a project (or the current working directory, when starting up Python in interactive mode) causes the interactive help to break:
$ touch token.py
$ python
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/_sitebuiltins.py", line 102, in __call__
import pydoc
File "/usr/lib/python3.8/pydoc.py", line 66, in <module>
import inspect
File "/usr/lib/python3.8/inspect.py", line 40, in <module>
import linecache
File "/usr/lib/python3.8/linecache.py", line 11, in <module>
import tokenize
File "/usr/lib/python3.8/tokenize.py", line 35, in <module>
from token import EXACT_TOKEN_TYPES
ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/current/working directory/token.py)
The traceback tells us all we need to know: calling help
triggers a deferred import of the standard library pydoc
, which indirectly attempts to import the standard library token
, but finds our token.py
which doesn't contain the appropriate name. In older versions of Python, it was even worse: tokenize
would do a star-import from token
, and then its top-level code would try to use a name defined there, resulting in NameError
- and a stack trace not mentioning the file name token.py
.
Python can import from cached bytecode files (.pyc
) even if the corresponding source file is renamed or deleted. This is intended to speed up imports, but it creates an extra step to keep code directories "clean". After renaming any .py
file to resolve a problem like this, make sure to check the bytecode cache as well.
Of course, the standard advice is simple and straightforward: just rename the .py
files that were mistakenly imported, and delete any cached bytecode .pyc
files for those modules. By default, in Python 3.x, the .pyc
files for any .py
files within a folder will be put into a subfolder with the special name __pycache__
; in 2.x, they simply appeared alongside the corresponding .py
files.
However, while this approach quickly solves the immediate problem for most people encountering it, the advice does not scale very well. There are a lot of potentially problematic names; while most projects will not be interested in including a source file named e.g. os.py
, some other names might be more desirable, or harder to work around. So, here are some other useful techniques:
sys.path
Of course, since the problem is caused by sys.path
specifying that absolute imports should look in the current project first, it can be avoided by just changing the sys.path
.
The problematic path is described in the documentation as a "potentially unsafe path". When using the interactive Python prompt, this will be an empty string (a relative path equivalent to '.'
) - i.e., the current working directory for the Python process, reflecting any changes made by using e.g. os.chdir
. For driver scripts started normally (python driver.py
), it will be the directory where the script is located, as an absolute path (not necessarily the current working directory, since a path could be specified on the command line, like python path/to/driver.py
). For modules run using the -m
command-line flag, it will be the initial current working directory (not necessarily where the module is located, but also an absolute path that will not be affected by os.chdir
).
To avoid having this path in sys.path
, do one of the following:
In Python 3.11 and up, set the PYTHONSAFEPATH
environment variable, or use the -P
command-line option to Python.
In Python 3.4 and up, use the -I
command-line option to start in isolated mode. This, however, has several other effects: it will ignore environment variables like PYTHONPATH
and PYTHONHOME
, and it will also skip adding the user-specific site-packages directory to the path (therefore the code will not have access to third-party libraries that were installed using the --user
option for Pip).
If all else fails, consider manually manipulating sys.path
. This is messy and error-prone, but sys.path
is just an ordinary list of strings with file paths, and modifying its contents will affect future import
s. (Do not try to replace the list object; this will not work, because Python does not look for it by the name sys.path
, but uses a hard-coded internal reference. It is not possible to destroy or replace that object from Python; sys.path
is just a name that is initialized to refer to it.)
Keep in mind that removing the "unsafe" path will prevent intentional absolute imports from working within the package. This is inconvenient for some small projects, but also a good reason to learn proper package organization. Speaking of which:
A well-organized Python project will typically consist of one or more (usually just one) packages, stored in subfolders of the main project folder, plus one or more driver scripts placed outside of the package subfolders. The driver scripts will typically use a single absolute import to access the package functionality, while implementing some simple wrapper logic (e.g. to parse command-line arguments, format and report otherwise-uncaught exceptions, etc.). The package, meanwhile, will use relative imports throughout for its own content, and absolute import only where necessary to access other packages (the standard library and third-party dependencies).
For example, a project might be organized like:
project
├── src
│ └── my_package
│ └── x.py
│ └── y.py
│ └── z.py
└── driver.py
Code in driver.py
will use absolute import into the package, like:
from my_package.x import entry_point
if __name__ == '__main__':
entry_point()
(If logic is needed to parse the command-line arguments, it should normally go in the driver rather than in the package's code.)
Then, code inside the package will use relative imports - so x.py
might contain something like:
from .y import first_thing
from .z import second_thing
def entry_point():
first_thing()
second_thing()
This gets the best of both worlds: the initial absolute import sets up the top-level package so that relative imports will work, and the relative imports will avoid depending on the sys.path
configuration. Even without taking steps to configure sys.path
, it will typically include the folder with the driver scripts, but not any package folders; thus, this also automatically avoids import path conflicts. (An absolute import won't find the package contents unless it specifies the corresponding package path; but typically when it does, importing from the current package was intentional.)
This also avoids setting traps for the next project which has this one as a dependency. Say for example that we implement and publish an API
package, and someone else writes Client
which has API
as a dependency. Since API
code will use relative import for other API
functionality (say, from . import functionality
), the Client
project's own functionality.py
won't cause a problem.
For the initial absolute import to work, of course, the top-level package folder needs to be mentioned in sys.path
. However, this is normally accomplished by installing the package, so it does not cause a problem.
If a file needs to be renamed, avoiding problems with .pyc
files will be easier if they simply don't exist in the first place. They are not necessary, after all; they are, again, simply intended to speed up imports on subsequent runs of the program.
To suppress .pyc
file generation, use the -B
command-line option or set the PYTHONDONTWRITEBYTECODE
environment variable. (There is no built-in solution to delete all existing .pyc
files, but this is easy enough to implement by hand using e.g. the shutil
standard library module.)
Consider using third-party tools (such as IDE plugins) to warn about filenames used by the standard library or by third-party libraries in the project. In Python 3.10 and up, the full list of standard library module names is also available as sys.stdlib_module_names
. This includes names that might not be present in the current Python installation (e.g. OS-specific components or things that are sometimes disabled or omitted, such as Tkinter).
Upvotes: 3
Reputation: 953
The error occurs because a user-created script has a name-clash with a library filename. Note, however, that the problem can be caused indirectly. It might take a little detective work to figure out which file is causing the problem.
For example: suppose that you have a script mydecimal.py
that includes import decimal
, intending to use the standard library decimal
library for accurate floating-point calculations with decimal numbers. That doesn't cause a problem, because there is no standard library mydecimal
. However, it so happens that decimal
imports numbers
(another standard library module) for internal use, so a script called numbers.py
in your project would cause the problem.
If you still encounter problems like this after tracking own and renaming or removing the appropriate .py
files in your project, also check for .pyc
files that Python uses to cache bytecode compilation when importing modules. In 3.x, these will be stored in folders with the special name __pycache__
; it is safe to delete such folders and files, and possible to suppress them (but you normally won't want to).
Upvotes: 28