Mule 4 - Can't combine results from parallel for_each loop into a single json

Question

In my flow in Mule 4, I iterate over multiple files to get data from them and transform them to json. I want to merge the results of each iteration into a single json output. I am using a variable to store the results, but it remains empty. Here's are the outputs I want to combine

result 1:

{
"plant": "CD909837289",
"serial": "SRF",
"product": "CMNPSD"
},
{
 "plant": "CD909837290",
 "serial": "SFG",
 "product": "CMNHSA"
}

result 2:

{
"plant": "CD909837296",
"serial": "SFG",
"product": "ERTYUI"
},
{
 "plant": "CD909837297",
 "serial": "SVH",
 "product": "SDFGHJ"
}

I want to combine them together. I set a variable like this:

<set-variable doc:name="Set Variable" doc:id="d2bd8c47-da05-4847-b5d5-f83331a1d011" variableName="STORED_DATA" value="#[[]]"/>

Once the payload is transformed I add another transform message component where I add the transformed payload to the variable

%dw 2.0
output application/json
---
vars.STORED_DATA ++ payload

When I do this the variable remains empty. In the logger component I the value of the variable is still []. How do I make sure the results are stored in the variable?

I appreciate all help.

Answer 1

As the documentation mentions the changes to variables inside the parallel foreach scope are not accessible outside it. This makes sense because we are dealing with parallelism and would cause all kinds of concurrency issues.

The parallel foreach returns a list of all the messages returned by each execution. Each message contains it's payload. You already have all the needed data in the output. To transform it to JSON you just need a transformation similar to this after the foreach:

output application/JSON
---
payload map $.payload

I can't test it right now so it may need some tweaking.

Answer 2

Please check the below code which will help you. I am setting the null variable and concatenating the httpRequest response.

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core" xmlns:http="http://www.mulesoft.org/schema/mule/http"
    xmlns="http://www.mulesoft.org/schema/mule/core"
    xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
    <flow name="Testing_foreach_stackoverflowFlow" doc:id="95fe6a04-4aca-4106-a0b3-b53afb7be95a" >
        <scheduler doc:name="Scheduler" doc:id="e6a81b1c-43f9-486a-a451-b31bcadbe23f" >
            <scheduling-strategy >
                <fixed-frequency frequency="5" timeUnit="MINUTES"/>
            </scheduling-strategy>
        </scheduler>
        <set-variable value="#[%dw 2.0
&#10;
&#10;output application/json
&#10;
&#10;---
&#10;
&#10;[]]" doc:name="Set Variable" doc:id="7b1bbba6-d63d-4f89-905b-8b9c7d6a7555" variableName="result"/>
        <set-payload value='#[["test1","test2"]]' doc:name="Set Payload" doc:id="1b7ff757-1a42-4291-b759-50632db842d3" />
        <foreach doc:name="For Each" doc:id="2010c075-b487-4977-8a34-0f58adb23a7d" >
            <http:request method="GET" doc:name="Request" doc:id="a0524fe5-2344-46c2-bb03-90660fb8fc04" url='#["http://localhost:8091/" ++ payload]'/>
            <ee:transform doc:name="Transform Message" doc:id="0aca9608-f7a1-4e1a-a246-11830b6f245e" >
                <ee:message >
                </ee:message>
                <ee:variables >
                    <ee:set-variable variableName="result" ><![CDATA[%dw 2.0
output application/json
---
vars.result + payload 
                ]]></ee:set-variable>
                </ee:variables>
            </ee:transform>
            <logger level="INFO" doc:name="payload" doc:id="22bc3348-90f9-4290-bb53-ee4c19af1f18" message="#[payload]"/>
        </foreach>
        <logger level="INFO" doc:name="payload" doc:id="00e70062-6898-42d7-bc50-974f23a4cd93" message="#[payload]"/>
    </flow>
    <flow name="Testing_foreach_stackoverflowFlow1" doc:id="fc055a5c-09a0-4dce-99be-70e863ffb0ff" >
        <http:listener doc:name="Listener" doc:id="298d5fff-c712-4307-895a-7d4faac42eb1" config-ref="api-httpListenerConfig" path="/test1"/>
        <set-payload value='"grapes"' doc:name="Set Payload" doc:id="bfce761e-c63a-4a06-ab9b-f17360d9a7d2" />
    </flow>
    <flow name="Testing_foreach_stackoverflowFlow2" doc:id="12d67768-525e-46b0-91b4-ce9377b0b7d1" >
        <http:listener doc:name="Listener" doc:id="24fc80d9-0f93-4217-a1a2-3802b57ce3cb" config-ref="api-httpListenerConfig" path="/test2"/>
        <set-payload value='"banana"' doc:name="Set Payload" doc:id="01ad0e82-5f50-49a3-9804-10190953b1c0" />
    </flow>
</mule>