CODEWITHSUNDEEP
linuxbashamazon-web-servicesamazon-ec2
Gleeb
Gleeb

Reputation: 11299

How to set Environment Variables on EC2 instance via User Data

I am trying to set environment variables with EC2s user data, but nothing i do seems to work

here are the User data scripts i tried

#!/bin/bash
echo "export HOST_URL=checkEmai-LoadBala-ICHJ82KG5C7P-23235232.us-east-1.elb.amazonaws.com" >> /env.sh 
source /env.sh

And another:

#!/bin/bash
echo "#!/bin/bash" >> /env.sh
echo "export HOST_URL=checkEmai-LoadBala-ICHJ82KG5C7P-67323523.us-east-1.elb.amazonaws.com" >> /env.sh 
chmod +x /env.sh
/env.sh

They both do absolutly nothing, and if i log in and issue the command source /env.sh or /env.sh it works. so this must be something forbidden that i am trying to do.

Here is the output from /var/log/cloud-init-output.log using -e -x

+ echo 'export HOST_URL=checkEmai-LoadBala-ICHJ82KG5C7P-2141709021.us-east-1.elb.amazonaws.com'
+ source /env.sh
++ export HOST_URL=checkEmai-LoadBala-ICHJ82KG5C7P-2141709022.us-east-1.elb.amazonaws.com
++ HOST_URL=checkEmai-LoadBala-ICHJ82KG5C7P-2141709022.us-east-1.elb.amazonaws.com

Still, echo $HOST_URL is empty

As requested, the full UserData script

#!/bin/bash
set -e -x 
echo "export HOST_URL=checkEmai-LoadBala-ICHJ82KG5C7P-2141709021.us-east-1.elb.amazonaws.com" >> /env.sh 
source /env.sh
/startup.sh staging 2649

Upvotes: 53

Views: 92335

Answers (12)

David K
David K

Reputation: 1

This is for Ubuntu 22.04 - it's similar to hardik-khandelwal's answer above. Put the following script into your user-data, start the instance, SSH into it, run env, and DEMOVAR will be there. You can combine the echo commands if you want - they're only separated for clarity.

#!/bin/bash
echo '#!/bin/bash' > ~/envconfig.sh
echo "export DEMOVAR=1234" >> ~/envconfig.sh
chmod +x ~/envconfig.sh
cp ~/myconfig.sh /etc/profile.d/envconfig.sh

Upvotes: 0

Mehul Prajapati
Mehul Prajapati

Reputation: 71

Here is what working for me

   UserData:
    Fn::Base64:
      !Sub |
        #!/bin/bash
        echo "TEST=THISISTEST" >> /etc/environment

Upvotes: 1

Filippo Secchi
Filippo Secchi

Reputation: 51

The easiest way is definitely to use AWS Elastic Beanstalk, it creates for you everything you need with very small effort and have the easiest way in the entire AWS eco-system to set your environment variables.

check it out, there are also some exhaustive tutorials based on different languages https://docs.aws.amazon.com/elastic-beanstalk/index.html

Upvotes: -4

Nate
Nate

Reputation: 13252

From this Medium.com article, you can put a script in UserData that writes to a file in /etc/profile.d that will get run automatically when a new shell is run.

Here is an example cloudformation.yaml

Parameters:
  SomeOtherResourceData:
    default: Fn::ImportValue: !Sub "SomeExportName"
Resources:
  WebApi:
    Type: AWS::EC2::Instance
    Properties:
      # ...
      UserData:
        Fn::Base64: !Sub
          - |
            #!/bin/bash

            cat > /etc/profile.d/load_env.sh << 'EOF'

            export ACCOUNT_ID=${AWS::AccountId}
            export REGION=${AWS::Region}
            export SOME_OTHER_RESOURCE_DATA=${SomeOtherResourceData}

            EOF
            chmod a+x /etc/profile.d/load_env.sh

And a YAML that exports something

# ...
Outputs:
  SomeExportName:
    Value: !Sub "${WebDb.Endpoint.Address}"
    Export:
      Name: SomeExportName

Upvotes: 2

Ziffusion
Ziffusion

Reputation: 8933

The user data script on EC2 executes at after boot in its own process. The environment variables get set in that process and disappear when the process exits. You will not see the environment variables in other processes, i.e., login shell or other programs for that matter.

You will have to devise a way to get these environment variables into whatever program needs to see them.

Where do you need these variables to be available? In /startup.sh staging 2649?

EDIT

Try this:

#!/bin/bash
set -e -x 
export HOST_URL="checkEmai-LoadBala-ICHJ82KG5C7P-2141709021.us-east-1.elb.amazonaws.com"
/startup.sh staging 2649

Then edit /startup.sh, and put the following line on the top:

echo $HOST_URL > /tmp/var

Boot the instance, and then paste /tmp/var here.

Upvotes: 20

Ashan
Ashan

Reputation: 19758

One of the more configurable approach to define environment variables for EC2 instances, is to use Systems Manager Parameter Store. This approach will make it easier to manage different parameters for large number of EC2 instances, both encrypted using AWS KMS as well as in plain text. It will also allows to change the parameter values with minimal changes in EC2 instance level. The steps are as follows.

  • Define string parameters (Encrypted with KMS or Unencrypted) in EC2 Systems Manager Parameter Store.
  • In the IAM role EC2 assumes, give required permission to access the parameter store.
  • Using the AWS CLI commands for EC2 System Manager, read the parameters and export to environment variables in User Data section using Get-Parameter or Get-Parameters AWS CLI commands and controlling command output as required.

e.g Using Get-Parameter command to retrieve db_connection_string parameter(Unencrypted).

export DB_CONNECTION=$(aws --region=us-east-2 ssm get-parameter --name 'db_connection' --query 'Value')

Note: For more details in setting up AWS KMS Keys, defining encrypted strings, managing IAM policies & etc., refer the following articles.

Upvotes: 41

Parag Tyagi
Parag Tyagi

Reputation: 8970

This maynot be the exact answer to the OP's question but similar. I've thought of sharing this as I've wasted enough time searching for the answer and finally figured it out.

Example assuming - AWS EC2 running ubuntu.

If there is a scenario where you need to define the environment variables as well use it in the same bash session (same user-data process), then either you can add the variables to /etc/profile, /etc/environment or /home/ubuntu/.zshrc file. I have not tried /home/ubuntu/.profile file BTW.

Assuming adding to .zshrc file,

sudo su ubuntu -c "$(cat << EOF 
    echo 'export PATH="/tmp:\$PATH"' >> /home/ubuntu/.zshrc
    echo 'export ABC="XYZ"' >> /home/ubuntu/.zshrc
    echo 'export PY_VERSION=3.8.1' >> /home/ubuntu/.zshrc
    source /home/ubuntu/.zshrc
    echo printenv > /tmp/envvars  # To test
EOF
)"

Once the user data is finished running, you can see the environment variables which you have added in the script are echoed to the envvars file. Reloading the bash with source /home/ubuntu/.zshrc made the newly added variables available in the bash session.

(additional info) How to install zsh and oh-my-zsh?

sudo apt-get install -y zsh
sudo su ubuntu -c "$(cat << EOF 
    ssh-keyscan -H github.com >> /home/ubuntu/.ssh/known_hosts
    git clone https://github.com/robbyrussell/oh-my-zsh.git /home/ubuntu/.oh-my-zsh
    cp /home/ubuntu/.oh-my-zsh/templates/zshrc.zsh-template /home/ubuntu/.zshrc
    echo DISABLE_AUTO_UPDATE="true" >> /home/ubuntu/.zshrc
    cp /home/ubuntu/.oh-my-zsh/themes/robbyrussell.zsh-theme /home/ubuntu/.oh-my-zsh/custom
EOF
)"
sudo chsh -s /bin/zsh ubuntu

Wondering why I didn't added the environment variable in .bashrc? The scenario which I mentioned above (using the environment variables in the same user-data session) adding to .bashrc won't work. .bashrc is only sourced for interactive Bash shells so there should be no need for .bashrc to check if it is running in an interactive shell. So just like above,

source /home/ubuntu/.bashrc

won't reload the bash. You can check this out written right in the beginning of the .bashrc file,

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *) return;;
esac

Upvotes: 1

mRyan
mRyan

Reputation: 400

Adding this to the init script of the node will add environment variables on launch. They won't show up in the node configuration page but they will be able to use in any job. 

#!/bin/bash
echo 'JAVA_HOME="/usr/lib/jvm/java-8-oracle/"' | sudo tee -a /etc/profile#

This answer is similar to what hamx0r proposed however, jenkins doesn't have permission to echo to /etc/profiles with or without sudo.

Upvotes: 1

Avinash Sidhwani
Avinash Sidhwani

Reputation: 51

You can use this script:

#!/bin/bash
echo HOST_URL=\"checkEmai-LoadBala-ICHJ82KG5C7P-23235232.us-east-1.elb.amazonaws.com\" >> /etc/environment

I created an EC2 instance with Amazon Linux AMI 2018.03.0 and added this user data to it and it works fine.

Refer to this answer for more details.

Upvotes: 4

hardik khandelwal
hardik khandelwal

Reputation: 104

You can add another shell script in /etc/profile.d/yourscript.sh which will contain the set of environment variables you want to add.

This script will run at every bootup and your variable will be available to all users.

#!/bin/sh
echo 'export AWS_DEFAULT_REGION=ap-southeast-2' > ~/myconfiguration.sh
chmod +x ~/myconfiguration.sh
sudo cp ~/myconfiguration.sh /etc/profile.d/myconfiguration.sh

The above code creates a shell script to set environment variable for aws default region and copies it to profile.d .

 

Upvotes: 7

hamx0r
hamx0r

Reputation: 4278

I find this to be a pretty easy way to set environment variables for all users using User Data. It allows me to configure applications so the same AMI can work with multiple scenarios:

#!/bin/bash
echo export DB_CONNECTION="some DB connection" >> /etc/profile
echo export DB_USERNAME="my user" >> /etc/profile
echo export DB_PASSWORD="my password" >> /etc/profile

Now, all users will have DB_CONNECTION, DB_USERNAME and DB_PASSWORD set as environment variables.

Upvotes: 26

harsh tibrewal
harsh tibrewal

Reputation: 835

After doing the stuffs in the user data script, the process exits. So, whatever environment variable you export will not be there in the next process. One way is to to put exports in the .bashrc file so that it gets available in the next session also.

    echo "export HOST_URL=checkEmai-LoadBala-ICHJ82KG5C7P-23235232.us-east-1.elb.amazonaws.com" >> ~/.bashrc

Upvotes: 2

Related Questions