Reputation: 1733
I am relatively new to python and even newer to Lambda. I have created a lambda function that requires some external dependencies (elasticsearch
and elasticsearch-curator
).
My root folder is called index_curator
, in which I have one single python file main.py
. I installed the dependencies via pip
as per Amazon's instructions e.g.
pip install elasticsearch elasticsearch-curator -t /path/to/index_curator
There are now many other files in this root directory and many child directories, which is not a surprise as these dependencies are quite large. For someone else looking at this package it would be difficult to differentiate between files I wrote and external dependencies. For example:
index_curator/
dateutil/
click/
idna/
main.py <-- the only file I wrote
README
LICENSE
six.py
...
Is there any way of shifting all these external dependencies into a sub-folder, e.g.
index_curator/
/external/
dateutil/
click/
idna/
README
LICENSE
six.py
main.py <-- the only file I wrote
For completeness the imports in main.py
are:
from elasticsearch import Elasticsearch, RequestsHttpConnection
import curator
Any pointers would be appreciated.
Upvotes: 1
Views: 800
Reputation: 2269
Separating external dependencies from your code is definitely best practice. There are a number of ways you can achieve this in python.
By default, python searches for modules in the locations specified here. To specify an additional location (i.e your external
dependencies folder), this new location must be added to the python path. You can do this in your main.py
as follows:
import sys, os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), 'external')))
Then you will be able to import all your dependencies as usual, since the interpreter will check the additional folder during runtime:
from elasticsearch import Elasticsearch, RequestsHttpConnection
import curator
For more detail, check the answers here
Since your code will be a lambda function, you will always have one handler. But for a more generalised case, or if you start to write multiple files, and need to manage external dependencies in those as well - you can choose to maintain a context.py
file, which sets the dependency path and all imports like so:
import sys, os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), 'external')))
import elasticsearch
import context
Then in your files, you can call them with:
from context.elasticsearch import Elasticsearch, RequestsHttpConnection
import context.curator
Upvotes: 2