CODEWITHSUNDEEP
azure-data-factory
jeancallisti
jeancallisti

Reputation: 1673

Error "DSL compilation failed", "extraneous input ','"

I'm having a classic issue in Data Factory where some forbidden character causes a failure when parsing the script file and/or the Json code representation.

However in this situation the "extraneous character" (according to the message: an unexpected comma) turns out to be elusive. The files I'm looking at don't seem to be the ones that DataFactory is complaining about (there's no position 47 on line 2!).

For clarity's sake, I'm only showing my one ultrabasic flowlet. Could it be that the error is somewhere else in my many flows? I don't see how, as I click on "Data preview" specifically on this flow. Also, it passes validation.

The CSV file:

enter image description here

enter image description here

enter image description here

Spark job failed: { "text/plain": "{"runId":"REDACTED","sessionId":"REDACTED","status":"Failed","payload":{"statusCode":400,"shortMessage":"com.microsoft.dataflow.broker.InvalidOperationException: DSL compilation failed: DF-DSL-001 - DSL stream has parsing errors\nLine 2 Position 47: extraneous input ',' expecting ...

Showing the files in question :

source(useSchema: false,
    allowSchemaDrift: true,
    validateSchema: false,
    ignoreNoFilesFound: false,
    format: 'delimited',
    container: 'newmessagingtemp',
    folderPath: 'testinputs',
    fileName: 'conversation_top_3.csv',
    columnDelimiter: ',',
    escapeChar: '\\',
    quoteChar: '\"',
    columnNamesAsHeader: true) ~> conversationsTop3
conversationsTop3 output() ~> conversationsTop3Output

and

{
    "name": "conversations_csv",
    "properties": {
        "description": "Use a CSV file in place of the source SQL Table for Conversations ",
        "folder": {
            "name": "Tests (can be deleted)"
        },
        "type": "Flowlet",
        "typeProperties": {
            "sources": [
                {
                    "linkedService": {
                        "referenceName": "AzureBlobStorage1",
                        "type": "LinkedServiceReference"
                    },
                    "name": "conversationsTop3",
                    "description": ""
                }
            ],
            "sinks": [],
            "transformations": [
                {
                    "name": "conversationsTop3Output"
                }
            ],
            "scriptLines": [
                "source(useSchema: false,",
                "     allowSchemaDrift: true,",
                "     validateSchema: false,",
                "     ignoreNoFilesFound: false,",
                "     format: 'delimited',",
                "     container: 'newmessagingtemp',",
                "     folderPath: 'testinputs',",
                "     fileName: 'conversation_top_3.csv',",
                "     columnDelimiter: ',',",
                "     escapeChar: '\\\\',",
                "     quoteChar: '\\\"',",
                "     columnNamesAsHeader: true) ~> conversationsTop3",
                "conversationsTop3 output() ~> conversationsTop3Output"
            ]
        }
    }
}

Upvotes: 0

Views: 978

Answers (1)

Pratik Lad
Pratik Lad

Reputation: 8382

Error "DSL compilation failed", "extraneous input ','" Error "DSL compilation failed", "extraneous input ','"

Error "DSL compilation failed", "extraneous input ','"

I tried to reproduce the issue and got similar error.

enter image description here

The issue is because we are not using the schema of file.

To resolve this, you need to import the schema for the file. you can import schema by go to Projection >> Import Schema >> import

enter image description here

Output:

enter image description here

Upvotes: 1

Related Questions