CODEWITHSUNDEEP
amazon-web-servicesamazon-sagemaker
tlanigan
tlanigan

Reputation: 959

Using the AWS SageMaker create-training-job command: type Error

I'm trying to use the AWS sagemaker cli to run the create-training-job command. Here is my command:

aws sagemaker create-training-job \
--training-job-name $(DEPLOYMENT_NAME)-$(BUILD_ID) \
--hyper-parameters file://sagemaker/hyperparameters.json \
--algorithm-specification TrainingImage=$(IMAGE_NAME),\
TrainingInputMode="File" \
--role-arn $(ROLE) \
--input-data-config ChannelName=training,DataSource={S3DataSource={S3DataType=S3Prefix,S3Uri=$(S3_INPUT),S3DataDistributionType=FullyReplicated}},ContentType=string,CompressionType=None,RecordWrapperType=None \
--output-data-config S3OutputPath=$(S3_OUTPUT) \
--resource-config file://sagemaker/train-resource-config.json \
--stopping-condition file://sagemaker/stopping-conditions.json

and here is the error:

Parameter validation failed:
Invalid type for parameter InputDataConfig[0].DataSource.S3DataSource, value: S3DataType=S3Prefix, type: <type 'unicode'>, valid types: <type 'dict'>
Invalid type for parameter InputDataConfig[1].DataSource.S3DataSource, value: S3Uri=s3://hs-machine-learning-processed-production/inbound-autotag/data, type: <type 'unicode'>, valid types: <type 'dict'>
Invalid type for parameter InputDataConfig[2].DataSource.S3DataSource, value: S3DataDistributionType=FullyReplicated, type: <type 'unicode'>, valid types: <type 'dict'>
make: *** [train] Error 255

The error is happening with the --input-data-config flag. I'm trying to use the Shorthand Syntax so I can inject some variables (the capitalized words). Haalp!

Upvotes: 5

Views: 2179

Answers (1)

Ankita Mehta
Ankita Mehta

Reputation: 600

So, your input config is not correctly formatted. Checkout the sample json here: https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html

# look at the format of input-data-config, it is a dictionary
  "InputDataConfig": [ 
      { 
         "ChannelName": "string",
         "CompressionType": "string",
         "ContentType": "string",
         "DataSource": { 
            "FileSystemDataSource": { 
               "DirectoryPath": "string",
               "FileSystemAccessMode": "string",
               "FileSystemId": "string",
               "FileSystemType": "string"
            },
            "S3DataSource": { 
               "AttributeNames": [ "string" ],
               "S3DataDistributionType": "string",
               "S3DataType": "string",
               "S3Uri": "string"
            }
         },
         "InputMode": "string",
         "RecordWrapperType": "string",
         "ShuffleConfig": { 
            "Seed": number
         }
      }
   ]

Upvotes: 1

Related Questions