CODEWITHSUNDEEP
bashpandasamazon-web-servicesamazon-emr
RAHUL VISHWAKARMA
RAHUL VISHWAKARMA

Reputation: 99

Bash Script to install libraries in EMR

I would like to know the procedure how do I install libraries in EMR,I am starting an EMR through a python script using boto3

but the steps that I want to run is failing in EMR because it is dependent on third party libraries and it is not installed in EMR how do I get my third party libraries to be installed in the python.

Upvotes: 0

Views: 335

Answers (1)

gorros
gorros

Reputation: 1461

You can use bootstrap script. Here is an example:

#!/bin/bash

sudo easy_install-3.6 pip
sudo /usr/local/bin/pip3 install scipy scikit-learn pandas
exit 0

You need to add corresponfing section to cluster config:

 "BootstrapActions": [
        {
            "Name": "install-libs",
            "ScriptBootstrapAction": {
                "Path": "s3://path/to/script/install-libs.sh"
            }
        }
    ]

Upvotes: 1

Related Questions