CODEWITHSUNDEEP
scalacluster-computingakka
boraas
boraas

Reputation: 942

Submission of Scala code to a cluster

Is it possible to run some akka code on the oracle grid engine with the use of multiple nodes?

So if I use the actor-model, which is a "message-passing model", is it possible to use Scala and the akka framework to run my code on a distributed memory system like a cluster or a grid?

If so, is there something similar like mpirun in mpi -c, to run my program on different nodes? Can you give a submission example using oracle grid engine?

how do I know inside scala on which node am I and to how many nodes the job has been submitted?

Is it possible to communicate with other nodes through the actor-model?

Upvotes: 4

Views: 576

Answers (1)

paradigmatic
paradigmatic

Reputation: 40461

mpirun or (mpiexec on some systems) can run any kind of executables (even if they don't use MPI). I currently use it to launch java and scala codes on clusters. It may be tricky to pass arguments to the executable when calling mpirun so you could use an intermediate script.

We use Torque/Maui scripts which are not compatible with GridEngine, but here is a script my colleague is currently using:

#!/bin/bash
#PBS -l walltime=24:00:00
#PBS -l nodes=10:ppn=1
#PBS -l pmem=45gb
#PBS -q spc
# Find the list of nodes in the cluster
id=$PBS_JOBID
nodes_fn="${id}.nodes"
# Config file
config_fn="human_stability_article.conf"
# Java command to call
java_cmd="java -Xmx10g -cp akka/:EvoProteo-assembly-0.0.2.jar ch.unige.distrib.BuildTree ${nodes_fn} ${config_fn} ${id}"
# Create a small script to pass properly the parameters
aktor_fn="./${id}_aktor.sh"
echo -e "${java_cmd}" >> $aktor_fn
# Copy the machine file to the proper location
rm -f $nodes_fn
cp $PBS_NODEFILE $nodes_fn
# Launch the script on 10 notes
mpirun -np 10 sh $aktor_fn > "${id}_human_stability_out.txt"

Upvotes: 7

Related Questions