CODEWITHSUNDEEP
javamultithreading
keepmoving
keepmoving

Reputation: 2043

Read from multiple files and write them into file file sequenction using multithread

I want to read from multiple files and write them into single file using multithread. I have done some code which is reading content of files perfectly fine and writing them into another file. But my problem is data is not coming sequential. Like reading f1 and write into output file, then read f2 and write into out file and so on. Have a look on my code 

public MultiFileReadingThread(BufferedWriter writer, BufferedReader reader, String name, java.util.List<File> files){
        this.writer = writer;
        this.reader = reader;
        this.name = name;
        this.files = files;
    }

    @Override
    public void run() {
        SimpleDateFormat formatter = new SimpleDateFormat("dd/mm/yyyy hh:mm:ss:ms");
        System.out.println("Start Time : " + name + formatter.format(System.currentTimeMillis()));
//      readFile();
        if(i == 1){

        }
        writeIntoSharedBufferedWriter();
        System.out.println("End Time : " + name +  formatter.format(System.currentTimeMillis()));
    }

public void writeIntoBufferedWriter(){

        for(File f : files){
            try{
            BufferedReader r = new BufferedReader(new FileReader(f));
            String line = null;
            while((line = r.readLine()) != null){
                writer.write(line);
                writer.write("\n");
            }
            } catch(Exception ex){
                ex.printStackTrace();
            }
        }
    }

Code to run the threads. There are five different task and worker for them.Again this code giving me the out but not correct. like f1, f2, f3

BufferedReader r1 = new BufferedReader(new FileReader(new File("/home/naveen/f1")));
        BufferedReader r2 = new BufferedReader(new FileReader(new File("/home/naveen/f2")));
        BufferedReader r3 = new BufferedReader(new FileReader(new File("/home/naveen/f3")));
        BufferedReader r4 = new BufferedReader(new FileReader(new File("/home/naveen/f4")));
        BufferedReader r5 = new BufferedReader(new FileReader(new File("/home/naveen/f5")));

        BufferedWriter writer = new BufferedWriter(new FileWriter(new File("/home/naveen/write")));
        Runnable task1 = new MultiFileReadingThread(null, writer, r1, "f");
        Runnable task2 = new MultiFileReadingThread(null, writer, r2, "s");
        Runnable task3 = new MultiFileReadingThread(null, writer, r3, "t");
        Runnable task4 = new MultiFileReadingThread(null, writer, r4, "fu");
        Runnable task5 = new MultiFileReadingThread(null, writer, r5, "fi");

        Thread wroker1 = new Thread(task1, "w1");
        Thread wroker2 = new Thread(task2, "w2");
        Thread wroker3 = new Thread(task3, "w3");
        Thread wroker4 = new Thread(task4, "w4");
        Thread wroker5 = new Thread(task5, "w5");

Can somebody help to achieve the taks. Thanks in advance.

Upvotes: 1

Views: 1971

Answers (2)

Justin
Justin

Reputation: 2031

Well, there's a lot of unnecessary code duplication here.

Each instance of MultiFileReadingThread should have it's own reader and writer which is never accessed from outside the thread. It seems like you have multiple threads using the same writer, in this case that's not going to work out very well.

Try this:

public MultiFileReadingThread(String name, List<File> inputFiles, File outputFile) {
    this.writer = new BufferedWriter(new FileWriter(outputFile));
    this.name = name;
    this.files = inputFiles;
}

@Override
public void run() {
    SimpleDateFormat formatter = new SimpleDateFormat("dd/mm/yyyy hh:mm:ss:ms");
    System.out.println("Start Time : " + name + formatter.format(System.currentTimeMillis()));
    //readFile();
    if(i == 1){

    }
    writeIntoSharedBufferedWriter();
    System.out.println("End Time : " + name +  formatter.format(System.currentTimeMillis()));
}

public void writeIntoBufferedWriter(){
    for(File f : files){
        try{
        BufferedReader r = new BufferedReader(new FileReader(f));
        String line = null;
        while((line = r.readLine()) != null){
            writer.write(line);
            writer.write("\n");
        }
        } catch(Exception ex){
            ex.printStackTrace();
        }
    }
}

Don't forget that you'll also need to close your BufferedReader and BufferedWriter.

Threading can be difficult and unfortunately there's no single tutorial or resource to provide all the information you need. I'd suggest reading every one you can find, multiple times. Trying to implement multithreading before you have a strong grasp on the subject can lead to lots of problems. Sometimes threading problems don't become apparent until late in an applications life cycle when major changes are required to correct them.

Upvotes: 0

VGaur
VGaur

Reputation: 715

The better way is to use producer consumer pattern. The producer threads will read the files and send the content to consumer by wrapping it in some class. Which will write the combined file into the disk.

On consumer side you can use priority queue and let producer set the priority.

Upvotes: 1

Related Questions