Java threading queue with thread pool

Question

I have made a program which uses a single thread to read data from a file into a linked list lets call it LL1. From this i have made a thread pool which allocates each thread with a processing task which reads data from LL1 and outputs its calculations to a new linked-list. From this i need to output each threads new linked-list into a SINGLE file. I'm trying to output each linked list in sequential blocks so that threads don't mix the data so i have used a synchronization point as follows:

public synchronized void appendContents(List<Vector2> output1) {
    try {
        sFileName = outFilePath + "\\file" +fileCount+ ".cntr";
        File oFile = new File(sFileName);
        if (!oFile.exists()) {
            oFile.createNewFile();
        }
        if (oFile.canWrite()) {
            //BufferedWriter oWriter = new BufferedWriter(new FileWriter(sFileName, true));
             FileWriter wstream = new FileWriter(oFile, true);
             BufferedWriter outWriter = new BufferedWriter(wstream);
             for(int i = 0; i < output1.size(); i++)
             {
                //replace the space marker values with a newline
                if(output1.get(i).y == -200.0){
                outWriter.newLine();
                }else{
                outWriter.write(String.valueOf(output1.get(i).x) + " " + String.valueOf(output1.get(i).y) + " " + String.valueOf(interval));
                outWriter.newLine();    
                }
             }           
             outWriter.close();
        }
    }
    catch (IOException oException) {
        throw new IllegalArgumentException("Error appending/File cannot be written: \n" + sFileName);
    }

The problem i am facing is that the data isn't coming out in order which is what i need i.e.

list1 value                              list1 value
list1 value         _______________\     list2 value
list1 value         ________________\    list1 value 
list2 value         RATHER THAN ____/    list3 value
list2 value         ---------------/     list2 value
list2 value                              list1 value
list3 value                              list2 value
list3 value                              list1 value
list3 value                              list3 value
list3 value                              list3 value

If anybody could give me a step in the right direction it would be greatly appreciated. Thanks,

Jack

Answer 1

Elaborating from user2870704's answer, you may structure your application as follows:

• Have a thread read the file contents;
• For each item in the file, submit a Callable task to your thread pool;
• Store the returned Future in a list, or list of lists - you define the granularity you need;
• Open the output file, in the same thread, and traverse the results list to write.

As an example:

void readAndOutput(String inputFilePath, String outputFilePath) {
    List<List<Future<Result>>> results = readAndSpawnTask(inputFilePath);

    PrintWriter out = new PrintWriter(new File(outputFilePath));

    for (List<Future<Result>> block : results) {
        for (Future<Result> r : block) {
            out.println(r.get().toString());
        }
    }

    out.flush();
    out.close();
}


List<List<Future<Result>>> readAndSpawnTask(String path) {
    List<List<Future<Result>>> results = new ArrayList<>(numOfBlocks);
    BufferedReader in = new BufferedReader(new FileReader(new File(path)));

    for (int i = 0; i < numOfBlocks; ++i) {
        results.add(new LinkedList<Future<Result>>());
    }

    for (String line = in.readLine(); line != null; line = in.readLine()) {
        int respectiveBlock;
        Callable<Result> task;
        // Process line and convert it into a task of your own.
        // Determine in which block the result goes into.
        Future<Result> r = threadPool.submit(task);
        results.get(respectiveBlock).add(r);
    }

    in.close();

    return results;
}

If you want and / or need concurrency, the idea is to access files in a single thread. Using a list of Future, you have guarantees of writing the results in the right order, and your main thread will block until the needed result is ready.

Of course, you still have to account for the possible exceptions thrown in the above code.

Answer 2

As i understand, You run every time new task for every list element?

then you just can write Callable task -> save result in Feature.Put Feature in List vs result(resultFeatureFromList). and at the end make some thing like this : i use Function from Guava Lib;

Iterables.transform(resultList<Feature>,new Function(){
   public resultComputition apply(Feature resultFeatureFromList){
               return resultFeatureFromList.get();
   }
});

So in conclusion, you will run all task in right order. and after pulling all simply wait for result.

Answer 3

The purpose of synchronized is to synchronize on a shared resource so that only one Thread gets access to a critical section at a single time. I'm going to assume that you are spawning three Thread instances that each call appendContents on their own object.

A synchronized method implicitly synchronizes on this, but since all three Threads are synchronized on a different object, ie. a different this, there's nothing blocking either of them.