Reputation: 1223
I am using Airflow 2.0 and have installed the slack module through requirements.txt in MWAA. I have installed all the below packages, but still, it says package not found
ModuleNotFoundError: No module named 'airflow.providers.slack'.
But it still gives me an error saying module not found. What should do ?
Please find the Requirements here :
alembic
anyio
apache-airflow-backport-providers-amazon
apache-airflow-providers-amazon
apache-airflow-providers-ftp
apache-airflow-providers-http
apache-airflow-providers-imap
apache-airflow-providers-slack
apache-airflow-providers-sqlite
apispec
argcomplete
asn1crypto
attrs
Babel
beautifulsoup4
blinker
boto3
botocore
cached-property
cachelib
cattrs
certifi
cffi
chardet
charset-normalizer
click
clickclick
colorama
colorlog
commonmark
configparser
connexion
croniter
cryptography
decorator
defusedxml
Deprecated
dill
dnspython
docutils
email-validator
Flask
Flask-Admin
Flask-AppBuilder
Flask-Babel
Flask-Caching
Flask-JWT-Extended
Flask-Login
Flask-OpenID
Flask-Session
Flask-SQLAlchemy
flask-swagger
Flask-WTF
funcsigs
future
graphviz
gunicorn
h11
httpcore
httpx
idna
importlib-metadata
importlib-resources
inflection
install
iso8601
itsdangerous
Jinja2
jmespath
json-merge-patch
jsonpath-ng
jsonschema
lazy-object-proxy
lockfile
lxml
Mako
Markdown
MarkupSafe
marshmallow
marshmallow-enum
marshmallow-oneofschema
marshmallow-sqlalchemy
mypy-boto3-rds
mypy-boto3-redshift-data
natsort
packaging
pandas
pendulum
ply
prison
psutil
pycparser
Pygments
PyJWT
pyparsing
pyrsistent
python-daemon
python-dateutil
python-nvd3
python-slugify
python3-openid
pytz
pytzdata
PyYAML
redshift-connector
requests
rfc3986
rich
s3transfer
scramp
setproctitle
six
slack-sdk
slackclient
sniffio
soupsieve
SQLAlchemy
SQLAlchemy-JSONField
sqlalchemy-redshift
SQLAlchemy-Utils
swagger-ui-bundle
tabulate
tenacity
termcolor
text-unidecode
thrift
typing_extensions
tzlocal
unicodecsv
urllib3
watchtower
websocket-client
Werkzeug
wrapt
WTForms
zipp
zope.Deprecation
Thanks for your help.
Thanks, Xi
Upvotes: 2
Views: 10360
Reputation: 26
Solution I found was using constraints at the top of requirements.txt
as answered by Andrew Nguonly (https://raw.githubusercontent.com/apache/airflow/constraints-2.2.2/constraints-3.7.txt) and confirming that all dependencies were properly installed. A depreciated package name blocked the installation of the airflow.providers.slack
package for me. You can check for any failures via this guide and rename your packages correctly: https://docs.aws.amazon.com/mwaa/latest/userguide/working-dags-dependencies.html#working-dags-dependencies-logs.
Upvotes: 0
Reputation: 2621
By default, MWAA is constrained to using version 3.0.0
for the package apache-airflow-providers-slack
. If you specify version 4.2.3
in requirements.txt
, it will not be installed (error logs should be available in CloudWatch). You'll have to downgrade to version 3.0.0
.
apache-airflow-providers-slack
(constraints.txt)
OR
Add constraints file to the top of requirements.txt
to use version 4.2.3
of apache-airflow-providers-slack
.
Add the constraints file for your Apache Airflow v2 environment to the top of your requirements.txt file.
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-3.7.txt"
Installing Python dependencies using PyPi.org Requirements File Format (MWAA)
Constraints files (Airflow)
Upvotes: 2