CODEWITHSUNDEEP
javaperformancemessage-queue
prsutar
prsutar

Reputation: 429

Message Queue slow performance

I am writing a message queue, but it is functioning slow, the method processFile taking too much time, and files get stuck in queue for long time. How to avoid it.

    System.out.println("Message Reader Started....");
    do 
    {
        String directoryPath = "C:\\Queue";
        int fileCount = new File(directoryPath).list().length;
        if (fileCount < 1) {
            System.out.println("Files Not Present");
        }
        else
        {
            File[] file = new File(directoryPath).listFiles();
            String firstFile = file[0].getAbsolutePath();
            processFile(firstFile);             
        }                       
    } while (true);

Upvotes: 1

Views: 1602

Answers (3)

maaartinus
maaartinus

Reputation: 46402

You wrote an endless loop, so why do you worry how long a single iteration takes?

You're needless reading the directory twice per iteration. Assuming your processFile removes the file processed (and possible another thread or process adds some files, but deletes none), you don't need to read the directory in each iteration.

Read it once and process all files found. If there's none, then re-read the directory. If there's still none, then you may terminate or sleep for a while (or consider watching the directory, but this is a bit more complicated and probably not necessary).

I'd strongly suggest to improve your loop before you start playing with treads (then use ExecutorService as suggested).

Upvotes: 0

thst
thst

Reputation: 4602

Your main issue is possibly CPU usage used for scanning the folder.

You should add Thread.sleep(100); at the end of the loop to give the system some time to breathe.

The issue you want to have resolved is obviously the processFile() method. You should do as it was commented by @Nazgul and implement it in it's own class with a Runnable interface.

To limit the amount of threads running, place the filename in a List or Queue and implement a Thread working on the List. You can add as many worker threads as your system can handle. The queue should be synchronized, so that you can safely remove items from multiple threads at the same time.

Upvotes: 0

Nazgul
Nazgul

Reputation: 1902

have you tried using concurrency for this? its an apt problem for concurrent processing. Assuming that file processing is a mutually exclusive action:

  • the do while loop in main Thread finds the file to read
  • process file is delegated to an executor thread for processing
  • and after processing (I am assuming reading the file) the processing of the contents can again be done in parallel. Its like read first 1000 lines and delegate to a thread to process.

You need to design it in a better way to run fast. A single threaded read and process list of files is bound to run slow.

Upvotes: 1

Related Questions