Reputation: 91
I'm to trying to automated appflow creation via cloud formation. I have multiple fields (130+) in source & I'm unable to map it manually. From UI I can directly choose multiple columns but from cloudformation I have to specify, therefore looking if any option which will automatically fetch all columns from source.
AWSTemplateFormatVersion: "2010-09-09"
Metadata:
Generator: "Automation"
Description: ""
Parameters:
snowusername:
Type: String
MaxLength: 128
snowpassword:
Type: String
MaxLength: 128
ServicenowUrl:
Type: String
MaxLength: 180
Resources:
AppFlowFlowTest:
Type: "AWS::AppFlow::Flow"
Properties:
FlowName: "IMGroupTest"
Description: "Servicenow"
SourceFlowConfig:
ConnectorType: "Servicenow"
ConnectorProfileName: !Ref AppFlowConnectorProfile
SourceConnectorProperties:
ServiceNow:
Object: "business_service_group"
DestinationFlowConfigList:
-
ConnectorType: "S3"
DestinationConnectorProperties:
S3:
BucketName: "raw-data
BucketPrefix: "servicenow/appflow"
S3OutputFormatConfig:
FileType: "PARQUET"
PrefixConfig:
PrefixType: "FILENAME"
AggregationConfig:
AggregationType: "None"
TriggerConfig:
TriggerType: "OnDemand"
Tasks:
-
SourceFields:
- "id"
- "site_name"
.
.
.
.
Upvotes: 9
Views: 5021
Reputation: 131
The AWS documentation on using Map_all is a bit lacking. The Map_all task, at least when using YAML, errors out when no TaskProperties are passed. To get around this, I added a property of EXCLUDE_SOURCE_FIELDS_LIST and pass an empty list, since I don't want any fields excluded.
Here is the relevant section of the template:
Tasks:
- TaskType: Map_all
SourceFields: []
TaskProperties:
- Key: EXCLUDE_SOURCE_FIELDS_LIST
Value: '[]'
ConnectorOperator:
Salesforce: NO_OP
Here is my full template
AWSTemplateFormatVersion: 2010-09-09
Description: CloudFormation Template for a MAP_ALL Salesforce AppFlow
Parameters:
Connection:
Type: String
S3Bucket:
Type: String
BucketPrefix:
Type: String
ObjectName:
Type: String
FlowName:
Type: String
Resources:
GenericFlow:
Type: 'AWS::AppFlow::Flow'
Properties:
Description: !Join [ "", [ 'App Flow for ', !Ref ObjectName, ' object' ] ]
DestinationFlowConfigList:
- ConnectorType: S3
DestinationConnectorProperties:
S3:
BucketName: !Ref S3Bucket
BucketPrefix: !Ref BucketPrefix
S3OutputFormatConfig:
AggregationConfig:
AggregationType: None
FileType: PARQUET
FlowName: !Ref FlowName
SourceFlowConfig:
ConnectorProfileName: !Ref Connection
ConnectorType: Salesforce
SourceConnectorProperties:
Salesforce:
EnableDynamicFieldUpdate: true
IncludeDeletedRecords: false
Object: !Ref ObjectName
Tasks:
- TaskType: Map_all
SourceFields: []
TaskProperties:
- Key: EXCLUDE_SOURCE_FIELDS_LIST
Value: '[]'
ConnectorOperator:
Salesforce: NO_OP
TriggerConfig:
TriggerType: OnDemand
Upvotes: 4
Reputation: 2431
To map all field, you have to use taskType with Map_all and SourceFields and task properties will be empty.
tasks=[
{
'sourceFields': [],
'taskType':'Map_all',
'taskProperties: '{}'
},
]
Upvotes: 1