CODEWITHSUNDEEP
azureazure-data-factory
flyingDutchman
flyingDutchman

Reputation: 16

Azure datafactory: My REST api call works in copy data but not in data flow, why?

I am trying to set up a data pipeline in ADF. I can't seem to figure out why an API call works as a source in the "copy data" block in the pipeline, but when I try to use it as a source in the data flow it throws an error.

This is the request in the copy data block: https://i.sstatic.net/9wzSZ.png'

This is the request in the data flow tab: : https://i.sstatic.net/Onpuk.png

And this is the error I get: https://i.sstatic.net/f9zdb.png

The request body should be the same. Still I get a good response when I preview the data in the copy block: https://i.sstatic.net/cKZsb.png

Is there some setting that is different that I am missing?

EDIT: the url for the request is https://www.daggegevens.knmi.nl/klimatologie/uurgegevens with the request body start=2022010101&end=2022090724&stns=330&vars=DD:FH:FF:FX&fmt=json

Upvotes: 0

Views: 914

Answers (1)

Aswin
Aswin

Reputation: 7156

I have reproduced same error in Dataflow.

error

  • I tried to give Request body in query parameters as in below image

Query params

  • Response of this API had header and body in each line item. response
  • Thus, select transformation is used to select only the body response and copied it to the sink select transformation

Outputs that I received from both copy activity and data flow activity are same.

copy activity output copy activity output

data flow activity outputdata flow activity output

Upvotes: 0

Related Questions