Java Threads within Threads

Question

I have an application that inserts header and detail data into database tables (2 tables 1 for header info and 1 for detail info). My current approach is to distribute the details equally in my threads. The header will contain an average of 100,000 details.

I want to change my approach to handle multiple headers. I will encapsulate the logic of 1 header in a function (this function will contain multithreading for its details) and call this function in several threads (the number of headers) or in a thread pool (using Executors).

Is this the best way to go about it or is there a best practice for such an issue? Also will setting priorities for the threads affect the flow of the threads? (I will set higher priorities for the headers and keep the default for the details)

Answer 1

Considering the nature of the problem you described, it is likely that the main bottleneck of the application is going to be database I/O. That should be addressed first. Some common practices:

• consider the granularity of your database transactions: you will achieve better performance if you group multiple details inserts in a transaction than if you insert details one at time (as would happen in autocommit mode)
• use and tune a database connection pool, so you can have multiple threads performing database operations concurrently while not taxing the database server too much.
• remember that DB operations are mostly I/O-bound, so you can probably use more threads than you have cores
• whenever applicable, use an ExecutorService instead of creating your own threads, so you don't have to manage the threads' life cycle yourself