CODEWITHSUNDEEP
javamultithreadingtestng
cgon
cgon

Reputation: 1981

A TestNg Multithreading Issue. TestNg does not respect the child threads

I have a very simple class that asynchronously writes a list to a file :

import java.io.FileWriter;
import java.io.IOException;
import java.lang.reflect.Type;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;

import com.google.common.reflect.TypeToken;
import com.google.gson.Gson;

public enum FileOps {
    INSTANCE;

    private ExecutorService threadPool = Executors.newFixedThreadPool(30);
    private AtomicInteger fileCount = new AtomicInteger(0);

    private <T> void writeListToFile(String fileName, List<T> obj) {
        FileWriter writer = null;
        Type tType = new TypeToken<ArrayList<T>>() {
            private static final long serialVersionUID = 4376511240656742709L;
        }.getType();
        Gson gson = new Gson();
        try {
            writer = new FileWriter(fileName);
            writer.append(gson.toJson(obj, tType));
            writer.flush();
        } catch (Exception e) {

        } finally {
            try {
                writer.close();
            } catch (IOException e) {
            }
        }
    }

    public <T> void asynWriteListToFile(List<T> obj){
        threadPool.execute(new Runnable() {
            @Override
            public void run() {
                String fileName = "C:\\data\\" + fileCount.incrementAndGet() + "_data.txt";
                System.out.println(fileName);
                FileOps.INSTANCE.writeListToFile(fileName, obj);
            }
        });
    }

}

I have written a unit test for this class using TestNg

import java.util.ArrayList;
import java.util.List;

import org.testng.annotations.Test;

public class FileOpsTest {

    @Test
    public void asynWriteListToFile() {
        List<Integer> list = new ArrayList<>();
        list.add(3);
        for (int i = 0; i < 10000; i++) {
            FileOps.INSTANCE.asynWriteListToFile(list);
        }

    }
}

I have a weird situation. In my TestNg execution some how the testing engine does not wait for the child threads to finish. So I am expecting 10000 files written on a disk, but every time I saw less files written on a disk. However if I write a client using a main method, everything works fine.

import java.util.ArrayList;
import java.util.List;

public class FileOpsClient {

    public static void main(String[] args) {
        List<Integer> list = new ArrayList<>();
        list.add(3);
        for (int i = 0; i < 10000; i++) {
            FileOps.INSTANCE.asynWriteListToFile(list);
        }
    }
}

Somehow testNg engine shuts down my thread pool.

Upvotes: 0

Views: 877

Answers (1)

mfulton26
mfulton26

Reputation: 31214

As the files are written asynchronously, FileOpsTest.asynWriteListToFile() ends before the files are all written and org.testng.TestNG (or your IDE's test runner) calls System.exit(int) (e.g. TestNG.java:1375).

In contrast, FileOpsClient.main(String[]) does not explicitly call System.exit(int) and as such the JVM waits for your threads to end because they are not daemon threads. See How to make TestNG wait for my test to complete before shutting it down for more details.

In you're case, you could make some changes so that your test can effectively invoke threadPool.awaitTermination(long, TimeUnit) (e.g. making FileOps.threadPool "package local" instead of "private" and accessing it from your test, adding a method on FileOps to do so for you and keep FileOps.threadPool "private", etc.).

However, if your goal is to unit test then I would suggest that an ExecutorService is an "awkward collaborator" and that you should refactor your code so that 1) you can test spawning threads using a mocked ExecutorService (see How to unit test that ExecutorService spawns new thread for task?), 2) you can test the actual writing of a list to files logic independent of how such gets created as an asynchronous task, and 3) don't use FileWriter directly but simply use a Writer so that you can also slip a mock when testing and avoid actually writing/reading files in your unit tests and leave such an exercise to integration tests if needed.

Upvotes: 2

Related Questions