CODEWITHSUNDEEP
spring-batch
jyn
jyn

Reputation: 73

Spring Batch Footer Validation

I am using Spring batch for processing a file with a header, detail and footer records. The footer contains the total number of records in the file. If the detail record count dosent match the count in the footer, the file should not be processed.

I am using a Custom Line Tokenizer that processes the header, detail and footer record. When the footer record is encountered, if the count dosent match the detail record count, I am throwing an exception.

But the problem I am facing is if the chunk size is set to small numbers like 10 and the file has 20 records, the first 10 detail records are being persisted into the DB, even though the footer count dosent match the total number of records.

Is there a way to validate the footer count with the number of records in the file before the call to the Writer?

Thanks.

Upvotes: 0

Views: 4382

Answers (2)

Baji Shaik
Baji Shaik

Reputation: 1122

By writing our own Item Reader as well as Item classes to handle Header,Footer,Data records and finding the counts of Header,Footer,Data records

ItemReader Class

public class AggregateItemReader<T> implements ItemStreamReader<ResultHolder> {


private ItemStreamReader<AggregateItem<T>> itemReader;

@Override
public ResultHolder read() throws Exception {
    ResultHolder holder = new ResultHolder();

    while (process(itemReader.read(), holder)) {
        continue;
    }

    if (!holder.isExhausted()) {
        return holder;
    }
    else {
        return null;
    }
}

private boolean process(AggregateItem<T> value, ResultHolder holder) {
    // finish processing if we hit the end of file
    if (value == null) {
        LOG.debug("Exhausted ItemReader");
        holder.setExhausted(true);
        return false;
    }

    // start a new collection
    if (value.isHeader()) {
        LOG.debug("Header Record detected");
        holder.addHeaderRecordCount();
        return true;
    }

    // mark we are finished with current collection
    if (value.isFooter()) {
        LOG.debug("Tailer Record detected");
        holder.addTailerRecordCount();
        holder.setFiledRecordCount(value.getFieldSet().readInt(3));
        System.out.println("###########################################"+holder.getDataRecordCount()+"############################################");
        return false;
    }

    // add a simple record to the current collection

    holder.addDataRecordCount();
    return true;
}

And Item Class is

public class AggregateItem<T> {

@SuppressWarnings("unchecked")
public static <T> AggregateItem<T> getData(FieldSet fs) {
    return new AggregateItem(fs, false, false, true);
}

@SuppressWarnings("unchecked")
public static <T> AggregateItem<T> getFooter(FieldSet fs) {
    return new AggregateItem(fs, false, true, false);
}


@SuppressWarnings("unchecked")
public static <T> AggregateItem<T> getHeader(FieldSet fs) {
    return new AggregateItem(fs, true, false, false);
}

private boolean data = false;
private FieldSet fieldSet;

private boolean footer = false;

private boolean header = false;

private T item;

public AggregateItem(FieldSet fs, boolean header, boolean footer, boolean data) {
    this(null);
    this.header = header;
    this.footer = footer;
    this.data = data;
    this.fieldSet = fs;
}


public AggregateItem(T item) {
    super();
    this.item = item;
}

public FieldSet getFieldSet() {
    return fieldSet;
}


public T getItem() {
    return item;
}

public boolean isData() {
    return data;
}


public boolean isFooter() {
    return footer;
}


public boolean isHeader() {
    return header;
}

}

And ResultHolder class is

public class ResultHolder implements {
private Integer headerRecordCount = 0;
private Integer dataRecordCount = 0;
private Integer tailerRecordCount = 0;
private Integer filedRecordCount;//this is to save record count given in source File
private boolean exhausted = false;//setters & getters

}

If any doubts feel free to mail at [email protected]

Upvotes: 0

zeus1208
zeus1208

Reputation: 78

What you need is a reader with a footer callback handler defined. I had faced a similar problem and this link helped me a lot! See the last post by Atefeh Zareh. He has also included the xml configuration.

And regarding the first ten being persisted, you can have another validation step before the main processing step which will just check the header and trailer counts. Do not write any persisting logic in the writer. If the count fails, stop the job so that it does not go into the processing step.

Upvotes: 0

Related Questions