CODEWITHSUNDEEP
azureazure-data-factoryazure-eventgrid
E. Erfan
E. Erfan

Reputation: 1411

How to setup Azure Event Grid for Azure Data Factory triggers?

I am checking how Azure Data Factory (ADF) can be triggered by Event Grid. I have created an Event Grid in the same resource group of my data factory. From ADF it is easy to connect it to the Event Grid topic. However, from Event Grid, I don't know which "Endpoint type" I need to choose. ADF is not one of the available options in "event type" as shown below:

Event Grid event type

when I choose web hook, it requires an endpoint, If I use the ADF URL it will fail to create the event subscription with error: "Deployment has failed with the following error: {"code":"Url validation","message":"Webhook validation handshake failed for https://adf.azure.com/en/authoring/pipeline/pipeline1.". This is kind of expected. But still I am confused how I need to set up my Event Grid subscription. which of the above options should I choose?

From the ADF side, I can choose the Event Grid for custom trigger. (I also created one Event Grid Topic from the portal parallel to the Event Grid service, however I am not sure these two are different services!) ADF trigger is shown below:

Define custom trigger on ADF

As you see I can make a custom trigger, but the problem is from Event Grid side, how to create a subscription that sends events to ADF. Also in the trigger at ADF, what should be the "Event type"? Just a name is enough?

One other thing, after I create the trigger at ADF side, when I open it again, it goes back to "enter manually" option and the event grid disappears, I am not sure why.

Event grid sets back to enter manually

Upvotes: 0

Views: 6086

Answers (1)

Abhishek Khandave
Abhishek Khandave

Reputation: 3230

You must be able to do the Microsoft.EventGrid/eventSubscriptions/ action. This action is part of the EventGrid EventSubscription Contributor built-in role.

Prerequisite -

Data Factory expects events to follow the Event Grid event schema. Make sure event payloads have the following fields:

[
  {
    "topic": string,
    "subject": string,
    "id": string,
    "eventType": string,
    "eventTime": string,
    "data":{
      object-unique-to-each-publisher
    },
    "dataVersion": string,
    "metadataVersion": string
  }
]

Follow below Steps:

  1. Go to Azure Data Factory and sign in.
  2. Switch to the Edit tab. Look for the pencil icon.
  3. Select Trigger on the menu and then select New/Edit.
  4. On the Add Triggers page, select Choose trigger, and then select +New.
  5. Select Custom events for Type.
  6. Select your custom topic from the Azure subscription dropdown or manually enter the event topic scope.
  7. The Subject begins with and Subject ends with properties allow you to filter for trigger events. Both properties are optional.
  8. Use + New to add Event Types to filter on. The list of custom event triggers uses an OR relationship. When a custom event with an eventType property that matches one on the list, a pipeline run is triggered. The event type is case insensitive. For example, in the following screenshot, the trigger matches all copycompleted or copysucceeded events that have a subject that begins with factories.
  9. A custom event trigger can parse and send a custom data payload to your pipeline. You create the pipeline parameters, and then fill in the values on the Parameters page. Use the format @triggerBody().event.data.keyName to parse the data payload and pass values to the pipeline parameters.
  10. After you've entered the parameters, select OK.

For more information refer to this official document

Upvotes: 1

Related Questions