CODEWITHSUNDEEP
pythonamazon-web-servicesaws-gluedatadog
Josh Jacobson
Josh Jacobson

Reputation: 101

How to import 3rd party python libraries for use with glue python shell script

I'm trying to import a 3rd party library (datadog) for use with a glue shell script and I'm running into issues. I've packaged the file as a .egg and given the path to it in the glue job, as instructed here. This ends up throwing an error saying zipimport.ZipImportError: not a Zip file: '/tmp/glue-python-libs/datadog.egg'. When I try using a zip file instead, it throws ModuleNotFoundError: No module named 'datadog'. How do I go about importing the library?

Upvotes: 1

Views: 575

Answers (1)

andyb
andyb

Reputation: 80

As shown in the documentation in your link, WHL files are also supported. It says:

You might already have one or more Python libraries packaged as an .egg or a .whl file.

There is a .whl file available for the DataDog python library here: https://pypi.org/project/datadog/#files. You might try downloading that file, uploading it to your S3 bucket, and using that as your Python library for your Glue job. You might be more successful using that than trying to build your own .egg file.

Upvotes: 1

Related Questions