CODEWITHSUNDEEP
javascaladistributed-system
hminle
hminle

Reputation: 521

Questions on Distributed System Design - Scala

I am going to implement sorting using many machines (distributed sorting). I will code it with Scala. Because I do not have much experience with network programming, I need you guys help me to choose which library and method to make the communication between machines:

1/ How many machines can communicate using Message (this will be implement like case class in Scala)? Can I use ServerSocketChannel? Or should I build a TCP server for each machine? Do I need to serialize and de-serialize my object Message? If needed, can I use protobuf for doing that?

2/ How can I send data via network? Should I use java.io or java.nio?

I need to build my project using as less dependencies as possible. Thank a lot.

Upvotes: 0

Views: 171

Answers (1)

Peter Lawrey
Peter Lawrey

Reputation: 533492

I would start with the simplest high level messaging library and since you have chosen Scala I would start with Akka which was designed with Scala in mind.

How many machines can communicate using Message

You can communicate with every machine you can connect to e.g. the entire internet.

If needed, can I use protobuf for doing that?

I suspect you don't need to, but yes.

How can I send data via network?

I would use a messaging library

Should I use java.io or java.nio?

The messaging library is likely to use these, but I wouldn't assume you need to use them directly.

I am going to implement sorting using many machines (distributed sorting).

There are libraries designed for Scala which do this already. I would use one of those e.g. Spark or Hadoop.

Upvotes: 2

Related Questions