CODEWITHSUNDEEP
javamultithreadingserializationdeserialization
AmitG
AmitG

Reputation: 10543

how to serialize multi-threaded program

I have many threads performing different operations on object and when nearly 50% of the task finished then I want to serialize everything(might be I want to shut down my machine ).

When I come back then I want to start from the point where I had left. How can we achieve?

This is like saving state of objects of any game while playing. Normally we save the state of the object and retrieve back. But here we are storing its process's count/state.

For example:

I am having a thread which is creating salary excel sheet for 50 thousand employee.

Other thread is creating appraisal letters for same 50 thousand employee.

Another thread is writing "Happy New Year" e-mail to 50 thousand employee.

so imagine multiple operations.

Now I want to shut down in between 50% of task finishes. say 25-30 thousand employee salary excel-sheet have been written and appraisal letters done for 25-30 thousand and so on. When I will come back next day then I want to start the process from where I had left.

This is like resume.

Upvotes: 0

Views: 211

Answers (4)

user207421
user207421

Reputation: 310909

I don't think serialization is the correct approach to this problem. What you want is persistent queues, which you remove an item from when you've processed it. Every time you start the program you just start processing the queue from the beginning. There are numerous ways of implementing a persistent queue, but a database comes to mind given the scale of your operations.

Upvotes: 0

didierc
didierc

Reputation: 14730

Java provides the java.io.Serializable interface to indicate serialization support in classes.

You don't provide much information about the task, so it's difficult to give an answer.

One way to think about a task is in terms of a general algorithm which can split in several steps. Each of these steps in turn are tasks themselves, so you should see a pattern here.

By cutting down each algorithms in small pieces until you cannot divide further you get a pretty good idea of where your task can be interrupted and recovered later. The result of a task can be:

  • a success: the task returns a value of the expected type

  • a failure: somehow, something didn't turn right while doing computation

  • an interrupted computation: the work wasn't finished, but it may be resumed later, and the return value is the state of the task

(Note that the later case could be considered a subcase of a failure, it's up to you to organize your protocol as you see fit).

Depending on how you generate the interruption event (will it be a message passed from the main thread to the worker threads? Will it be an exception?), that event will have to bubble within the task tree, and trigger each task to evaluate if its work can be resumed or not, and then provide a serialized version of itself to the larger task containing it.

Upvotes: 0

Ankit
Ankit

Reputation: 6622

well its not much different than saving state of object. just maintain separate queues for different kind of inputs. and on every launch (1st launch or relaunch) check those queues, if not empty resume your 'stopped process' by starting new process but with remaining data.

say for ex. an app is sending messages, and u quit the app with 10 msg remaining. Have a global queue, which the app's senderMethod will check on every launch. so in this case it will have 10msg in pending queue, so it will continue sending remaining msgs.

Edit: basically, for all resumable process' say pr1, pr2....prN, maintain queue of inputs, say q1, q2..... qN. queue should remove processed elements, to contain only pending inputs. as soon as u suspend system. store these queues, and on relaunching restore them. have a common routine say resumeOperation, which will call all resumable process (pr1, pr2....prN). So it will trigger the execution of methods with non-0 queues. which in tern replicate resuming behavior.

Upvotes: 1

Augusto
Augusto

Reputation: 29927

I'm not sure if this might help, but you can achieve this if the threads communicate via in-memory queues.

To serialize the whole application, what you need to do is to disable the consumption of the queues, and when all the threads are idle you'll reach a "safe-point" where you can serialize the whole state. You'll need to keep track of all the threads you spawn, to know if they are in are idle.

You might be able to do this with another technology (maybe a java agent?) that freezes the JVM and allows you to dump the whole state, but I don't know if this exists.

Upvotes: 2

Related Questions