Azure Logic for getting data from SQL to FTP

Question

I've a task of taking data from SQL and uploading the data as a CSV file up to an FTP server.

Now I've done this for a single SQL row just fine. The problem I'm having is looping over all rows (foreach loop) and inserting these rows as the content of the CSV file. I've tried a FTP Create File Task inside a foreach loop, but I can only access a single row at a time to set as the file's content - I need all the rows!

Also to keep in mind is that these files will have 200k+ rows.

I could of course just write a C# console app for this but the ease at which I got this far without writing any code makes it seem like it will be a worthwhile endeavor.

Answer 1

So just following up to show how Derek's answer helped me with my problem to get a large number of rows to up to a file on an FTP server. I ended up using the output body of the Execute Stored Procedure action as the GetRows action was limited to 512 rows.

NOTE: As the Table action is not available in the designer, yet, do everything in the code viewer, opening the designer caused issues and deleted all my code at one point.

"actions": {
        "Create_file": {
            "inputs": {
                "body": "@body('tableCsv0')",
                "host": {
                    "api": {
                        "runtimeUrl": "https://logic-apis-northeurope.azure-apim.net/apim/ftp"
                    },
                    "connection": {
                        "name": "@parameters('$connections')['ftp']['connectionId']"
                    }
                },
                "method": "post",
                "path": "/datasets/default/files",
                "queries": {
                    "folderPath": "transactions/ready/ecommerce/tickets_test/",
                    "name": "grma_tickets_@{formatDateTime(utcNow(),'yyyyMMdd_hhmmss')}.csv"
                }
            },
            "runAfter": {
                "tableCsv0": [
                    "Succeeded"
                ]
            },
            "type": "ApiConnection"
        },
        "Execute_stored_procedure": {
            "inputs": {
                "host": {
                    "api": {
                        "runtimeUrl": "https://logic-apis-northeurope.azure-apim.net/apim/sql"
                    },
                    "connection": {
                        "name": "@parameters('$connections')['sql']['connectionId']"
                    }
                },
                "method": "post",
                "path": "/datasets/default/procedures/@{encodeURIComponent(encodeURIComponent('[Scheduledjob].[GetBArcodesForGRMA]'))}"
            },
            "runAfter": {},
            "type": "ApiConnection"
        },
        "tableCsv0": {
            "inputs": {
                "columns": [
                    {
                        "header": "EventDateTime",
                        "value": "@item()?['EventDateTime']"
                    },
                    {
                        "header": "EventName",
                        "value": "@item()?['EventName']"
                    }
                ],
                "format": "csv",
                "from": "@body('Execute_stored_procedure')['ResultSets']['Table1']"
            },
            "runAfter": {
                "Execute_stored_procedure": [
                    "Succeeded"
                ]
            },
            "type": "Table"
        }

Answer 2

We recently added "Table" primitive for this scenario, support in designer is still work in progress, but you can use it in code view.

In below scenario, I'm getting rows from a table in SQL Azure, producing an CSV with two columns using data from the SQL query (First Name, Last Name), then send it via e-mail.

"Get_rows": {
    "inputs": {
        "host": {
            "api": {
                "runtimeUrl": "https://logic-apis-southcentralus.azure-apim.net/apim/sql"
            },
            "connection": {
                "name": "@parameters('$connections')['sql']['connectionId']"
            }
        },
        "method": "get",
        "path": "/datasets/default/tables/@{encodeURIComponent(encodeURIComponent('[SalesLT].[Customer]'))}/items",
        "queries": {
            "$top": 10
        }
    },
    "runAfter": {},
    "type": "ApiConnection"
},
"tableCsv0": {
    "inputs": {
        "columns": [
            {
                "header": "First Name",
                "value": "@item()?['FirstName']"
            },
            {
                "header": "Last Name",
                "value": "@item()?['LastName']"
            }
        ],
        "format": "csv",
        "from": "@body('Get_rows')?['value']"
    },
    "runAfter": {
        "Get_rows": [
            "Succeeded"
        ]
    },
    "type": "Table"
},
"Send_an_email": {
    "inputs": {
        "body": {
            "Body": "@body('tableCsv0')",
            "Subject": "Subject",
            "To": "deli@microsoft.com"
        },
        "host": {
            "api": {
                "runtimeUrl": "https://logic-apis-southcentralus.azure-apim.net/apim/office365"
            },
            "connection": {
                "name": "@parameters('$connections')['office365']['connectionId']"
            }
        },
        "method": "post",
        "path": "/Mail"
    },
    "runAfter": {
        "tableCsv0": [
            "Succeeded"
        ]
    },
    "type": "ApiConnection"
}