How to read the result from fileconnector.search (v2) in WSO2 ESB

Question

My goal here is to find a list of files matching a given pattern (D[MMDDYYYY]to[MMDDYYYY].zip in my case), and then iterate thru the returned list of file names and copy them to a destination. I am not sure how to correctly parse the result out of fileconnector.search operation. What is the correct XPath expression for parsing the file names returned by fileconnector.search? My codes and output shown below.

Thank you in advance.

<sequence name="TestFTPDownloadSeq" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
  <fileconnector.search>
    <source>ftp://username:pwd@ftphost/data/</source>
    <filePattern>D08082016/.+/.csv</filePattern>
    <recursiveSearch>false</recursiveSearch>
  </fileconnector.search>
  <iterate expression="//file" xmlns:ns2="http://org.apache.synapse/xsd" xmlns:sec="http://secservice.samples.esb.wso2.org">
  <target>
    <sequence>
      <property expression="//file" name="fileName"
      scope="default" type="STRING"/>
      <fileconnector.copy>
        <source>ftp://username:pwd@ftphost/data/</destination>
        <destination>C:/MyApps/data/in</destination>
        <filePattern>{$ctx:fileName}</filePattern>
      </fileconnector.copy>
      <drop/>
    </sequence>
  </target>
  </iterate>
</sequence>

Log file

[2016-08-10 13:00:08,519] DEBUG - TemplateMediator Start : EIP Sequence paramNames : [source, filePattern, recursiveSearch, setTimeout, setPassiveMode, setSoTimeout, setStrictHostKeyChecking, setUserDirIsRoot]
[2016-08-10 13:00:08,519] DEBUG - TemplateMediator Sequence <TemplateMediator> :: mediate()
[2016-08-10 13:00:08,519] DEBUG - TemplateMediator Mediation started from mediator position : 0
[2016-08-10 13:00:08,519] DEBUG - TemplateMediator Building message. Sequence <TemplateMediator> is content aware
[2016-08-10 13:00:08,519] DEBUG - PropertyMediator Start : Property mediator
[2016-08-10 13:00:08,519] DEBUG - PropertyMediator Setting property : source at scope : default to : ftp://username:pwd@ftphost/data/ (i.e. result of expression : $func:source)
[2016-08-10 13:00:08,519] DEBUG - PropertyMediator End : Property mediator
[2016-08-10 13:00:08,519] DEBUG - PropertyMediator Start : Property mediator
[2016-08-10 13:00:08,519] DEBUG - PropertyMediator Setting property : filePattern at scope : default to : D08082016/.+/.csv (i.e. result of expression : $func:filePattern)
[2016-08-10 13:00:08,519] DEBUG - PropertyMediator End : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Start : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Setting property : recursiveSearch at scope : default to : false (i.e. result of expression : $func:recursiveSearch)
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator End : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Start : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Setting property : setTimeout at scope : default to : null (i.e. result of expression : $func:setTimeout)
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator End : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Start : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Setting property : setPassiveMode at scope : default to : null (i.e. result of expression : $func:setPassiveMode)
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator End : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Start : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Setting property : setSoTimeout at scope : default to : null (i.e. result of expression : $func:setSoTimeout)
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator End : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Start : Property mediator
[2016-08-10 13:00:08,520] DEBUG - PropertyMediator Setting property : setStrictHostKeyChecking at scope : default to : null (i.e. result of expression : $func:setStrictHostKeyChecking)
[2016-08-10 13:00:08,521] DEBUG - PropertyMediator End : Property mediator
[2016-08-10 13:00:08,521] DEBUG - PropertyMediator Start : Property mediator
[2016-08-10 13:00:08,521] DEBUG - PropertyMediator Setting property : setUserDirIsRoot at scope : default to : null (i.e. result of expression : $func:setUserDirIsRoot)
[2016-08-10 13:00:08,521] DEBUG - PropertyMediator End : Property mediator
[2016-08-10 13:00:08,521] DEBUG - ClassMediator Start : Class mediator
[2016-08-10 13:00:08,521] DEBUG - ClassMediator invoking : class org.wso2.carbon.connector.FileSearch.mediate()
[2016-08-10 13:00:08,547] DEBUG - ClassMediator End : Class mediator
[2016-08-10 13:00:08,547] DEBUG - IterateMediator Start : Iterate mediator
[2016-08-10 13:00:08,550] DEBUG - IterateMediator Splitting with XPath : //file resulted in 0 elements

Answer 1

Sample Proxy to search by pattern and move to the location. for more here

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
   name="SampleProxy"
   transports="https,http"
   statistics="disable"
   trace="disable"
   startOnLoad="true">
<target>
<inSequence>
    <property name="fileLocation" expression="//fileLocation"/>
    <property name="fileName" expression="//fileName"/>
    <property name="newfilelocation" expression="//newfilelocation"/>
    <property name="filebeforeprocess" expression="//filebeforeprocess"/>
    <property name="fileafterprocess" expression="//fileafterprocess"/>
    <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
    <fileconnector.search>
        <filelocation>{$ctx:fileLocation}</filelocation>
        <filepattern>([^\s]+(\.(?i)(txt|png|gif|bmp))$)</filepattern>
        <searchinlocal>false</searchinlocal>
    </fileconnector.search>
    <property name="sequence" value="after-smooks"/>
</log>
<iterate xmlns:ns2="http://org.apache.synapse/xsd"
         xmlns:sec="http://secservice.samples.esb.wso2.org"
         expression="//file">
    <target>
        <sequence>
            <log level="full"/>
            <fileconnector.move>
                <filelocation>{$ctx:fileLocation}</filelocation>
                <file>fileSample.txt</file>
                <newfilelocation>{$ctx:newfilelocation}</newfilelocation>
                <filebeforeprocess>{$ctx:filebeforeprocess}</filebeforeprocess>
                <fileafterprocess>{$ctx:fileafterprocess}</fileafterprocess>
            </fileconnector.move>
            <drop/>
        </sequence>
    </target>
</iterate>
</inSequence>
</target>
<description/>
    </proxy>

Answer 2

Please follow this blog post https://medium.com/@lmanohara99/wso2-esb-file-connector-version-2-f789ad389d7d#.2ivi97d9x

Also, Check whether your regex pattern is getting the results. You can validate your regex pattern from here https://regex101.com/