CODEWITHSUNDEEP
data-modelingdistributed
Timothy Pratley
Timothy Pratley

Reputation: 10662

Distributed data model

I work in field robotics and we have a central server keeps track of a whole heap of data to do with vehicle state, environment state, tasks, task grouping and so on. There are processes which deal with different parts of this data, and user interfaces which need to be updated when specific parts change.

What I want is a way for systems to connect to the central server and subscribe to a portion of the data. They get all the data piped down to them and any changes sent as they happen. Additionally I'd like to be able to nominate that certain updates can be amalgamated: if a vehicle has moved 3 times but the UI connected does not have much bandwidth than just the latest position is sent.

Currently I'm thinking an in-memory database which keeps track of client subscription queries and calculates deltas to send to them. Is there a better way or existing solution to distributing a data model?

Upvotes: 0

Views: 569

Answers (2)

Markus W Mahlberg
Markus W Mahlberg

Reputation: 20703

I probably would (sort of ab-)use Apache ServiceMix for this scenario. Using a message oriented middleware can only benefit your project.

Describing my solution from a very high level view:

The way I see it, when abstracting the problem a bit, we are talking of events that happen, which need to be processed according to (business) rules. Depending on those business rules, other communication partners need to be synchronously or asynchronously notified.

So your various stations could send rather simple JMS messages to the ActiveMQ component of ServiceMix when an event happens. Business logic running on the ApacheCamel will then read those messages and process them. The Camel component offers a rich feature set for Enterprise Integration Patterns. Implementing the business logic here is rather simple and straightforward. The stations which need to be notified could subscribe to JMS queues or topics, depending on the use case, or synchronously via a custom or one of the many existing connectors and data formats. Hint: Sending protocol buffers over JMS or AMQP can be implemented in C or C++ with relative ease. This should allow highly efficient code on the stations, eliminating the dependency on a JVM the same time.

With the use of ServiceMix setup, you have various advantages:

  • Durable notifications and subscriptions both for events and calculated data, when using JMS for asynchronous communication – if set up properly, no event will be lost, which might not be the case with a RAM based solution.
  • A battle tested environment, ready to use
  • Ease of maintenance: updates to the business logic can be rolled out with little (fractions of seconds) to no downtime, if the application is planned properly. This is due to the fact that the business logic will be implemented as OSGi bundles and the replacement of those bundles can be handled gracefully.
  • Ease of development: Instead of needing to develop a whole application, you only need to implement the business logic, the software which sends the messages to ActiveMQ and your domain models.
  • Seamless integration in existing SOA
  • By design, complex problems are split into various smaller ones, which will likely enhance stability and perfectly matches agile development procedures

Upvotes: 0

Tom Anderson
Tom Anderson

Reputation: 47193

You may want a tuple space. In java, there's JavaSpaces, which is part of Jini. I don't know if it supports change notification out of the box, or if you'll have to add some kind of notifications yourself.

Upvotes: 2

Related Questions