CODEWITHSUNDEEP
javamultithreadingfilethreadpool
Sandeep
Sandeep

Reputation: 2121

How to implement a multithreaded pool in Java

I have a scenario where I use threads.

Firstly I have a folder where there are files which get updated frequently. So, I wrote a thread which reads the contents of the folder and writes the file names to a static list and updates the list if new files come in.

Secondly i wrote another thread which takes the file names from the list and do some processing with the files.

These two threads run continuously, one checking for new files, one processing the new files.

Now I need to process three files at a time with three threads running. When one thread completes processing another thread takes another file name from the list and starts the process.

So I need some mechanism to have three threads and checking them whether they are alive or not and accordingly starts a new thread and the file list also gets updated frequently.

I also looked into ExecutorService but while the list get updated I could not provide it updated list.

Thanks, Sandeep

Upvotes: 3

Views: 1101

Answers (4)

devsprint
devsprint

Reputation: 671

How about watching for changes in the folder and spawn a thread/file, assuming that the notification change is giving you a list of changes in the folder?

Upvotes: 0

Peter Lawrey
Peter Lawrey

Reputation: 533500

Similar to @SimonC's suggestion but instead of a really long comment I have an answer.

final FilePoller poller = ...
final FileProcessor processor = ...

final ExecutorService executor = Executors.newFixedThreadPool(4);

executor.submit(new Runnable() { 
    public void run() {
        final File file = poller.pollForFile();
        executor.submit(new Runnable() { public void run() { processor.processFile(file); } } );
        // repeat, but wait for a free thread.
        executor.submit(this);
    }
  });
 // to stop the whole thing
 executor.shutdown();

Upvotes: 0

SimonC
SimonC

Reputation: 6718

Building on the existing answers, your code would look something like:

    final ExecutorService executor = Executors.newFixedThreadPool(2);

    final FilePoller poller = ...
    final FileProcessor processor = ...

    new Thread(new Runnable() 
      { 
        @Override
        public void run() 
        {
          while (true)
          {
            final File file = poller.pollForFile();
            executor.submit(new Runnable() { public void run() { processor.processFile(file); } } );
          }
        }
      });

Assuming your processors can keep up with the poller this would be fine, otherwise you'd want to put in some throttling mechanism before submitting to the executor.

Upvotes: 2

L. Cornelius Dol
L. Cornelius Dol

Reputation: 64026

Don't use a list; instead use a java.util.concurrent.ThreadPoolExecutor and just drop Runnable's representing the file to be processed into the executor instead of putting them into your global list.

Upvotes: 0

Related Questions