Reputation: 121
I am trying to create an S3 Batch (not AWS Batch, this is S3 Batch operation) job via boto3 using S3Control, but I get an "invalid request" response. I tried it through AWS S3 batch operation through the console which worked but now I am trying to do it through boto3 to create the batch job. Attached is the code and error message.
import json
import boto3
import time
import datetime
import os
s3ControlClient = boto3.client('s3control')
def lambda_handler(event, context):
accountId = boto3.client('sts').get_caller_identity().get('Account')
print(accountId)
response = s3ControlClient.create_job(
AccountId = accountId,
ConfirmationRequired = False,
Operation = {
'S3PutObjectRetention': {
'BypassGovernanceRetention': False,
'Retention': {
'RetainUntilDate' : datetime.datetime(2050, 06, 03),
'Mode' : 'GOVERNANCE'
}
}
},
Report={
'Bucket' : 'arn:aws:s3:::test-s3-inventory',
'Format' : 'Report_CSV_20180820',
'Enabled' : True,
#'Prefix' : time.strftime('%Y%m%d'),
'Prefix' : 'report',
'ReportScope' : 'AllTasks'
},
Manifest={
'Spec': {
'Format': 'Report_CSV_20180820',
'Fields': [
'Bucket', 'Key', 'VersionId', 'TaskStatus', 'ErrorCode', 'HTTPStatusCode', 'ResultMessage'
]
},
'Location': {
'ObjectArn' : 'https://test-s3-inventory.s3.amazonaws.com/job-34455-4eb5-829d-7eedrrr8564/manifest.json',
'ETag' : 'f4a7c0618aaed7777a5be40c266abe1f'
}
},
Description = time.strftime('%Y-%m-%d')+' - Apply Retention and Legal Hold',
Priority = 10,
RoleArn = 'arn:aws:iam::277696433388194:role/s3BatchRole',
Tags = [
{'Key': LOB', 'Value': 'Test'},
{'Key': 'project', 'Value': ‘test project’}
]
)
Error:
Response:
{
"errorMessage": "An error occurred (InvalidRequest) when calling the CreateJob operation: Request invalid",
"errorType": "ClientError",
"stackTrace": [
" File \"/var/task/lambda_function.py\", line 13, in lambda_handler\n response = s3ControlClient.create_job(\n",
" File \"/opt/python/botocore/client.py\", line 316, in _api_call\n return self._make_api_call(operation_name, kwargs)\n",
" File \"/opt/python/botocore/client.py\", line 635, in _make_api_call\n raise error_class(parsed_response, operation_name)\n"
]
}
Upvotes: 5
Views: 4791
Reputation: 101
Additionally, ClientRequestToken is missing. According to docs,
ClientRequestToken (string) -- [REQUIRED]
An idempotency token to ensure that you don't accidentally submit the same request twice. You can use any string up to the maximum length.
This field is autopopulated if not provided.
ClientRequestToken can be added in the following way -
import uuid
...
def lambda_handler(event, context):
...
RequestToken = str(uuid.uuid4())
...
response = clientControl.create_job(
...
ClientRequestToken=RequestToken,
...
)
....
Also you need to specify the region when you are creating the boto3 client as best practice.
s3ControlClient=boto3.client('s3control', region_name=[Region_Name])
I have documented my syntax of using create job in S3 Control Create Job Invalid Request Error
Upvotes: 0
Reputation: 51
I think you have two small errors in your code:
arn:aws:s3:::test-s3-inventory/job-34455-4eb5-829d-7eedrrr8564/manifest.json
Report_CSV_20180820
) is wrong; instead you want:
S3BatchOperations_CSV_20180820
Upvotes: 5