CODEWITHSUNDEEP
azureazure-data-factory
Subha
Subha

Reputation: 5

How to exclude ADF auto generated properties from Web activity

I have a web activity in ADF pipeline. This web activity calls an API and get data from that. The output Web activity is in below format,

{
  "fdgdhgfh": {
    "so2_production": 7hjhgj953,
    "battery_charge": jkjlkj,
    "battery_discharge": kjlklj,
    "critical_load_energy": 4ljljh4
},
  "9fsdsfb": {   
    "so2_production": asdasd,
    "battery_charge": sdaasf,
    "battery_discharge": ewewrwer,
    "critical_load_energy": bmvkbjk
  },
"ADFWebActivityResponseHeaders":{
"Content-Length":14268,
"Content-Type":"application/json"
},
"effectiveIntegrationRunTime":"Azure IR",
"billingReference":{
"activityType":"ExternalActivity"
},
}

From above output I only want API response without ADF auto generated properties like,

{
  "fdgdhgfh": {
    "so2_production": 7hjhgj953,
    "battery_charge": jkjlkj,
    "battery_discharge": kjlklj,
    "critical_load_energy": 4ljljh4
  },
  "9fsdsfb": {
   
    "so2_production": asdasd,
    "battery_charge": sdaasf,
    "battery_discharge": ewewrwer,
    "critical_load_energy": bmvkbjk
  }
}

Let me know the process or expression to achieve in ADF pipeline.

Upvotes: 0

Views: 455

Answers (1)

Saideep Arikontham
Saideep Arikontham

Reputation: 6114

  • Since you are not getting the response attribute in the set variable activity, you can use the following procedure instead.
  • Store the output of web activity output as a string in a variable using set variable activity. The output would look like something shown below:

enter image description here

  • Now, use another set variable activity to split the above string using ADFWebActivityResponseHeaders as delimiter.
@split(variables('tp2'),'ADFWebActivityResponseHeaders')[0]

enter image description here

  • Finally, using the following dynamic content in another set variable activity, I was able to extract required part:
@array(json(substring(variables('tp1'),0,sub(length(variables('tp1')),3))))

enter image description here

  • I have taken it as an array for demonstration. You can access the required part from this using the dynamic content @variables('tp3')[0] where tp3 is the variable with value in the above image.

Upvotes: 0

Related Questions