suda ks
suda ks

Reputation: 91

How to map fields automatically in appflow?

I'm to trying to automated appflow creation via cloud formation. I have multiple fields (130+) in source & I'm unable to map it manually. From UI I can directly choose multiple columns but from cloudformation I have to specify, therefore looking if any option which will automatically fetch all columns from source.

AWSTemplateFormatVersion: "2010-09-09"
Metadata:
    Generator: "Automation"
Description: ""
Parameters:
  snowusername:
    Type: String
    MaxLength: 128
  snowpassword:
    Type: String
    MaxLength: 128
  ServicenowUrl:
    Type: String
    MaxLength: 180
Resources:
    AppFlowFlowTest:
        Type: "AWS::AppFlow::Flow"
        Properties:
            FlowName: "IMGroupTest"
            Description: "Servicenow"
            SourceFlowConfig:
                ConnectorType: "Servicenow"
                ConnectorProfileName: !Ref AppFlowConnectorProfile
                SourceConnectorProperties:
                    ServiceNow:
                        Object: "business_service_group"
            DestinationFlowConfigList:
              -
                ConnectorType: "S3"
                DestinationConnectorProperties:
                    S3:
                        BucketName: "raw-data
                        BucketPrefix: "servicenow/appflow"
                        S3OutputFormatConfig:
                            FileType: "PARQUET"
                            PrefixConfig:
                                PrefixType: "FILENAME"
                            AggregationConfig:
                                AggregationType: "None"
            TriggerConfig:
                TriggerType: "OnDemand"
            Tasks:
              -
                SourceFields:
                  - "id"
                  - "site_name"
                  .
                  .
                  .
                  .

Upvotes: 9

Views: 5021

Answers (2)

CarstenOSU
CarstenOSU

Reputation: 131

The AWS documentation on using Map_all is a bit lacking. The Map_all task, at least when using YAML, errors out when no TaskProperties are passed. To get around this, I added a property of EXCLUDE_SOURCE_FIELDS_LIST and pass an empty list, since I don't want any fields excluded.

Here is the relevant section of the template:

Tasks:
  - TaskType: Map_all
    SourceFields: []
    TaskProperties: 
    - Key: EXCLUDE_SOURCE_FIELDS_LIST
      Value: '[]'
    ConnectorOperator:
      Salesforce: NO_OP

Here is my full template

    AWSTemplateFormatVersion: 2010-09-09
Description: CloudFormation Template for a MAP_ALL Salesforce AppFlow
Parameters:
  Connection:
    Type: String
  S3Bucket:
    Type: String
  BucketPrefix:
    Type: String
  ObjectName:
    Type: String
  FlowName:
    Type: String
Resources:

  GenericFlow:
    Type: 'AWS::AppFlow::Flow'
    Properties:
      Description: !Join [ "", [ 'App Flow for ', !Ref ObjectName, ' object' ] ]
      DestinationFlowConfigList:
        - ConnectorType: S3
          DestinationConnectorProperties:
            S3:
              BucketName: !Ref S3Bucket
              BucketPrefix: !Ref BucketPrefix
              S3OutputFormatConfig:
                AggregationConfig:
                  AggregationType: None
                FileType: PARQUET
      FlowName: !Ref FlowName
      SourceFlowConfig:
        ConnectorProfileName: !Ref Connection
        ConnectorType: Salesforce
        SourceConnectorProperties:
          Salesforce:
            EnableDynamicFieldUpdate: true
            IncludeDeletedRecords: false
            Object: !Ref ObjectName
      Tasks:
        - TaskType: Map_all
          SourceFields: []
          TaskProperties: 
          - Key: EXCLUDE_SOURCE_FIELDS_LIST
            Value: '[]'
          ConnectorOperator:
            Salesforce: NO_OP
      TriggerConfig:
        TriggerType: OnDemand

Upvotes: 4

Nikhil Suthar
Nikhil Suthar

Reputation: 2431

To map all field, you have to use taskType with Map_all and SourceFields and task properties will be empty.

    tasks=[
        {
            'sourceFields': [],
            'taskType':'Map_all',
            'taskProperties: '{}'
        },
    ]

Upvotes: 1

Related Questions