Reputation: 719
I am configuring a workflow where messages from an SQS queue trigger AWS Batch jobs, and the content of these messages is passed as environment variables to the container executing the job. I am using AWS EventBridge Pipes to connect the SQS queue to AWS Batch.
# 📌 EventBridge Pipes - Ligação SQS -> AWS Batch
resource "aws_pipes_pipe" "pipe" {
name = "sqs-to-batch-pipe"
role_arn = aws_iam_role.pipes.arn
source = aws_sqs_queue.queue.arn
target = aws_batch_job_queue.queue.arn
source_parameters {
sqs_queue_parameters {
batch_size = 1
}
}
target_parameters {
batch_job_parameters {
job_definition = aws_batch_job_definition.job.arn
job_name = "my-batch-job"
parameters = {
MESSAGE = "$.body" <<<<< I can't send the message to the container
}
}
}
}
Here is the configuration I am using:
In my Node.js application, I am attempting to access the MESSAGE environment variable as follows:
console.log("Received message:", process.env.MESSAGE);
However, the displayed value is always 'default_value', indicating that the SQS message body is not being correctly passed to the container.
Question:
How can I configure EventBridge Pipes to correctly pass the content of SQS messages as environment variables to the AWS Batch container? Is there an appropriate way to override the MESSAGE variable defined in the aws_batch_job_definition with the SQS message content when the job is triggered by EventBridge Pipes?
Additional Details:
I understand that AWS Batch allows defining environment variables in the job_definition, but I am unable to override the MESSAGE variable with the SQS message content at runtime.
I have attempted to use the parameters attribute in the aws_pipes_pipe, but have not been successful in passing the SQS message content to the container.
Any guidance or configuration examples would be greatly appreciated.
Upvotes: -1
Views: 42