CODEWITHSUNDEEP
pythondockeramazon-ec2aws-batch
Ajit KV
Ajit KV

Reputation: 1

Issue to hit the endpoint of AWS docker build/push/run

As a part of an in-house project, we are working with a few AWS services, and AWS Batch is one of them. The objective of our exercise was to hit the end-point of an AWS EC2 instance, through an API interface, from a docker running on the same EC2 instance. This docker has a front end tool up and running all the time and when a "Run" button is triggered from the tool, the API should hit our code base in the EC2 instance to run a python pipeline which will generate results in the tool.

In the process of doing the above we have tried hitting the codes directly by specifying the paths to the code base, but we faced issues related to the environment as the docker environment from which we are trying to hit the codebase and the EC2 environment had conflicts. So as another attempt, we tried using the DIND framework, which is a docker inside a docker setup to hit the codebase, but were faced with other issues as mentioned in the “aws_error” file attached.

File "/usr/local/lib/python3.8/site-packages/botocore/httpsession.py", line 477, in send
raise EndpointConnectionError(endpoint_url=request.url, error=e)botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://batch.ap-south-1-MCBC.amazonaws.com/v1/describejobqueues"

We have forecasting module which runs inside a docker, this module have below functionalities:

  1. Building a docker image for AWS batch(docker image inside forecasting docker)
  2. AWS CLI login inside the forecast docker
  3. Creating a docker or creating layer for AWS docker
  4. Pushing docker image of AWS to AWS batch
  5. Running final forecasting model inside the AWS batch

When we run the main file in docker we are able to do step 1-4 but we are facing issue in step 5.

docker run -v $HOME/.aws:/root/.aws -e ENCODED_BATCH_CONFIG_JSON=$ENCODED_BATCH_CONFIG_JSON -e ENCODED_DATA_CONFIG_JSON=$ENCODED_DATA_CONFIG_JSON $IMG_NAME_TAG python launch_batch_jobs.py #/bin/bash

I would appreciate any kind of help in the Issue I'm currently facing. Please feel free to reach out in case of more queries about the issue.

Upvotes: 0

Views: 28

Answers (0)

Related Questions