CODEWITHSUNDEEP
azure-functionsazure-logic-apps
Marknl
Marknl

Reputation: 182

Azure Logic Apps FTP concurrent triggers

I am working on a Azure Logic app that reads added files from an FTP and sends them as POST to an Azure function. After this the file is removed from the FTP server. The total execution time is about 1 second. The problem that i am facing is that the Logic app only seems to process one file at a time, because of this the logic app can only handle max 60 new files a minute. I have set the concurrent runs to 42, but this does not seem to change anything.

How can i make this Logic app process these files quicker?

  "triggers": {
            "When_a_file_is_added_or_modified": {
                "inputs": {
                    "host": {
                        "connection": {
                            "name": "@parameters('$connections')['ftp']['connectionId']"
                        }
                    },
                    "method": "get",
                    "path": "/datasets/default/triggers/onupdatedfile",
                    "queries": {
                        "folderId": "L2Z0cC9maWxlcw==",
                        "includeFileContent": true
                    }
                },
                "metadata": {
                    "L2Z0cC9maWxlcw==": "/ftp/files"
                },
                "recurrence": {
                    "frequency": "Second",
                    "interval": 15
                },
                "runtimeConfiguration": {
                    "concurrency": {
                        "runs": 42
                    }
                },
                "type": "ApiConnection"
            }
        }

Upvotes: 0

Views: 939

Answers (2)

George Chen
George Chen

Reputation: 14334

This is not the FTP connector problem, you could check the trigger description, it never say it could response multiple files.

If you modify multiple files same time, it will trigger the logic app multiple times. And if concurrent setting is off it means unlimited, also no max 60 files limit.

I test with 100 files, it all could trigger. The result will be like the below picture, they will run at the same time.

enter image description here

Upvotes: 1

jeffhollan
jeffhollan

Reputation: 3227

I believe the behavior of the FTP trigger is that it only returns one file at a time (especially in this case when file content is included in the body of the trigger), but gets a signal if more files ready to be triggered and will immediately fetch the next. Whether it will go fetch the next is where your concurrency setting is helpful. So behavior:

  1. At polling interval, check for new files
  2. If a new file, trigger. Trigger will also include a “is there more” or “is that all?” Flag (the retry-after header)
  3. If there is more, immediately go back and fetch next file.
  4. If there is not more, wait for next polling interval
  5. Repeat

Upvotes: 0

Related Questions