CODEWITHSUNDEEP
javaamazon-web-servicesamazon-ec2remote-access
Light Yagmi
Light Yagmi

Reputation: 5235

Remotely execute program in multiple EC2 instances and wait for results

Suppose there are three EC2 instances called A,X,Y.

I'd like to execute Java program jx, jy in X, Y from other Java program ja in A. ja needs to pass command line argument of jx and jy.

For EC2 instance A, X, Y, and Java program ja, jx, jy, 
A's ja------------> X's jx
     -------------> Y's jy

The ja wait for finish of both jx and jy and obtain the result value (or file)

A's ja <---return val -------X's jx
       <---return val ------ Y's jy

Again, ja execute jx and jy with other argument determined by the return values.

ja needs to execute jx and jy for 5 times respectively.

I think psudo code of ja is like below:

for(int i = 0; i < 5; i++){
  rx = executeJX(argx);
  ry = executeJY(argy);

  waitJxJyFinish(); // jx and jy take ~30 min to finish.
  (argx, argy) = determineNextArgXandArgY(rx, ry);
}

Constraint

This is for just experimental purpose, not production for daily use, that's why the least overhead way is the best even if it's dirty way. I don't have any experience of hadoop or any other staff. Simple and easy way is welcome.

Upvotes: 0

Views: 69

Answers (1)

user1832464
user1832464

Reputation:

IF you are looking to pass variables between instances in an asynchronous way to run tasks, I think the best solution would be an Amazon SQS Queue.

This would allow you to run the Java Program on X and Y, and submit the result to the queue as a message containing the result that you want to use on A. A can then intermittently poll the queue for the message, which contains the result, and then run jA using those variables.

This is the most elegant solution and decouples your application. Another alternative would be to post the results from X and Y to S3 as a file, which A then occasionally checks for and takes a copy of if it exists.

Upvotes: 1

Related Questions