97008 48403
+91 40-27177600


Removing duplicate record sets from Dataweave in Mule ESB


From the technical center of excellence of Massil Technologies, our technology leaders make it easy to understand the complex situations you come across in Mule ESB from the experience of Massil Technologies to have countered them in their experience of working on client projects in real time. This blog talks about removing duplicate record sets from Dataweave in Mule ESB. This blog is a part of series of blogs being authored and published by Massil Technologies for the benefit of the IT community globally.

During one of the projects on Mule ESB, we had come across this unique requirement which mostly involves removing duplicate record set on one record set fields value. In the next part of this article, we cover in steps the best solution that worked for our technology team with the client on that day.

Step 1:

In Anypoint Studio click File > New > Mule Project to create a new project, give it whatever name you wish and click Finish.

Step 2:

Drag an HTTP Connector from the palette to your empty canvas, then a Transform Message     Component


Step 3:

Click on the HTTP Connector to open its properties editor, then click the green arrow icon to create a new configuration for it. Leave all the fields in default and click OK.

Step 4:

Note that if you open the Transform Message component, the input section should show the fields that will be present in the incoming payload

Step 5:

You write the output payload in

%dw 1.0

%output application/xml inlineCloseOn = “empty” ,skipNullOn=”attributes”

%namespace ns0 http://www.example.org/IUGetMasterOrganizationRecordByIdOrName

%namespace ns01 urn:siperian.api

{CustomerMasterDataDSUltimateResponse :   {((payload.ns01#searchQueryReturn.*ns01#record  map

DSUltimate:   {

(  $.*field map {

(ElectronicAddressTypeCd: $.ns01#stringValue ) when $.ns01#name == “EADDR_TYPE_CD” and  $.ns01#stringValue !=  null and $.ns01#stringValue != “”,

(ElectronicAddressTxt: $.ns01#stringValue ) when $.ns01#name == “EADDR_TXT” and   $.ns01#stringValue != null and $.ns01#stringValue != “”,

(DSUltimateId: trim $.ns01#stringValue) when $.ns01#name == “IU_ROWID”,

(DSUltimateNm : $.ns01#stringValue) when $.ns01#name == “IU_NM”,

(DSUltimateClassificationCd : $.ns01#stringValue) when $.ns01#name == “IU_CLASS_CODE”,

(ApprovalStatusDSUltimateRequesterId : $.ns01#stringValue) when $.ns01#name == “APRV_STS_IU_RQSTR_ID”,

(DSUltimateCreateReasonCd : $.ns01#stringValue) when $.ns01#name == “IU_CRE_RSN_CD”,

(DSUltimateApprovalStatusCd : $.ns01#stringValue) when $.ns01#name == “IU_APRV_STS_CD”,

(DSUltimateEffectiveStartDtm : $.ns01#dateValue) when $.ns01#name == “EFF_STRT_DTM”,

(DSUltimateEffectiveEndDtm : $.ns01#dateValue) when $.ns01#name == “EFF_END_DTM”  and  $.ns01#stringValue !=  null and $.ns01#stringValue != “”




) distinctBy  $.DSUltimate.DSUltimateId )}}




Step 6:

In logger write  #[payload]


Step 7:

Save & Run your Mule project.

Step 8:

Send xml file using postman to the project


Step 9:

The output is


Hence, we have gone through the step by step process to depict the way Massil Technologies team has addressed. Hope this article from Massil Technologies was useful for you. If you have any further queries on this topic, please reach out to info@massil.tech.

Leave a Reply

Your email address will not be published. Required fields are marked *

Contact Us

Get in touch with us

Scroll to top