CODEWITHSUNDEEP
springspring-batch
Gopi
Gopi

Reputation: 669

How to process files of different formats using spring batch?

My requirement is to poll two different directories for different format files (PSV and CSV) and to process it using spring batch.

I used inbound channel adapter to poll the directory . But , i couldnt able to find a way to call corresponding reader and its tokenizer and field mapper based on the type of file .

e.g. if it is psv - call PSV reader , PSV line mapper if it is csv - call CSVReader , CSV line mapper

Any help would be appreciated .

Thanks

Upvotes: 1

Views: 2163

Answers (3)

Gopi
Gopi

Reputation: 669

Thank you for all your suggestions.

I followed a different approach. I have setted the item reader and writer as a job parameter. So, based on file type , respective item reader and writer will be invoked

<chunk reader= "#{JobParameter[bean.reader]}" writer="#{JobParameter[bean.writer]}"/>

Thanks

Upvotes: 1

Rafik BELDI
Rafik BELDI

Reputation: 4158

You can create your Own Reader : CustomFilesReader : 

public class CustomFilesReader implements InitializingBean{

    private List<File> yourFiles= null;

    public File read() {
        if ((yourFiles!= null) && (yourFiles.size() != 0)) {  
          return yourFiles.remove(0);
        }
        return null;
    }

    //Reading Items from your dir
    private void loadItems() {
        this.yourFiles= new ArrayList<File>();
       // populate the items
     }


    @Override
    public void afterPropertiesSet() throws Exception {
        loadItems();
    }
}

Register the bean : 

<bean id="customFilesReader " class="mypackage.CustomFilesReader "/>


<bean id="myReader"
    class="org.springframework.batch.item.adapter.ItemReaderAdapter">
    <property name="targetObject" ref="customFilesReader " />
    <property name="targetMethod" value="read" />
</bean>

The Same thing can be Done for a custom ItemProcessor and a custom ItemWriter.

Upvotes: 1

Thrax
Thrax

Reputation: 1964

An alternative would be to create a class similar to ClassifierCompositeItemProcessor or ClassifierCompositeItemWriter. That is to say, it would take a Classifier, a list of delegated ItemReader and choose the right ItemReader depending of the result of the Classifier.

Problem is, you don't have an input to classify, since the read isn't done yet. That's why I thought about using the resource property as the element to classify. Here's the result :

public class ResourceClassifierCompositeItemReader<T> implements ItemReader<T> {

    private Classifier<String, ItemReader<? extends T>> classifier = new ClassifierSupport<String, ItemReader<? extends T>> (null);
    private Resource resource;

    public void setClassifier(Classifier<String, ItemReader<? extends T>> classifier) {
        this.classifier = classifier;
    }

    @Override
    public T read() throws Exception, UnexpectedInputException, ParseException {
        return readItem(classifier.classify(resource.getFilename()));
    }

    private T readItem(ItemReader<? extends T> reader) throws Exception {
        return reader.read();
    }

    public void setResource(Resource resource) {
        this.resource = resource;
    }

}

Now for how to use it. First you'll need a MultiResourceItemReader to read both .PSV and .CSV at the same time. Then you'll have to delegate the read to the ResourceClassifierCompositeItemReader. You'll have to add a BackToBackPatternClassifier to classify the String resource.getFilename() (i.e. the file name) and call an ItemReader accordingly (via a MatcherMap). To do that, you'll need to write your own RouterDelegate (taking the filename, split it to get the extension, and return it as a String to be matched).

Upvotes: 0

Related Questions