Push mechanism using Low latency protocol

Question

I am a Java EE developer.

I would like to ask if anyone could suggest me about low latency network protocol which can handle lots of successive information.

I am trying to implement push mechanism between two machines having load of data being transfer from Machine A to Machine B.

Any suggestions would be more then welcome(about the protocol and/or the push mechanism)

I am talking about real-time system.

The push will be from server to a client(1 to many).

I want add that each packet being transferred is small. but there are loads of packets which must transfer in a very low latency and successively.

More requirements: 1. Client devices are not in the same network.(so I guess UDP wont be relevant here) I am looking after Wall St. type kind of latency. If I lose a packet it's gone no need to re-send it since it's might be not relevant any more.

thanks

Answer 1

About TIBCO RV, it uses an UDP protocol but can cross network boundaries with a so called router deamon, so that is possible too.

Answer 2

This problem is usually solved using some sort of message queue.

In the Java world, there is a standard API for message queues called JMS, and there are many implementations to choose from, including commercial and open-source ones. My company recently decided to start using RabbitMQ, as it seemed the most flexible and robust of the implementations we investigated. HornetQ is notable for being the fastest of the well-known JMS implementations.

There are also message queues which are not JMS implementations; often, these have more focus on performance, and less on features such as robustness. Interesting examples include OpenDDS and ZeroMQ.

I would suggest you have a look at HornetQ. I think it should be pretty close to what you need.

EDIT: I just saw your comment that you don't want to use a third party implementation, but to implement it yourself. That would be a bad idea. Don't do that. Use a good third-party implementation.

Answer 3

This isn't an easy question to answer, as there are many parts to latency. If your building a publish-subscribe methodology, you should look at multicast protocols or UDP.

What are your requirements? Are these devices on the same flat network, will you be crossing boundaries (firewalls, switches, routers)? All of these little things are part of the bigger picture.

Once you get into UDP and Multicast, you going to need to deal with packet loss, packet retransmission, message ordering, message completion and a host of other things to deal with. That's why most buy products like Tibco and some of the other message bus technologies to deal with this.

If your looking for Wall St. type latency, you getting into the bounds of dedicated routers / firmware.

You also don't really define what you mean by latency. I gather your looking at network latency, which is limited by the speed of light (sorry it's the law!) over distance.

Latency can also be the time your code responds to some event as well.

Edit 1:

IP Multicast is your best bet, as it is reasonably well understood and is routable across lan's assuming you have a decent networking infrastructure.

I see an open source project JGroups that looks to have something. You might want to look at Actors Remote Actors as well, I don't have any experience with either of these projects, so your mileage may vary.

For Commercial software I would look at Tibco. I'm not sure of their product packaging anymore, but they used to have Tib Rendezvous (Tib RV) and an older product (can't remember the name off the top of my head), I've used both to process Market Data.

Answer 4

You need sockets connections for this, just like chat server.

I have used XMPP server for this purpose, a single socket connection is made with the server for the entire project life. Server sends the stanza's and client parse it and do respective actions.
Its highly successful for custom push notifications,

You can create your own server instead of using XMPP server but if many is in millions, you should use XMPP server. Its best suited for real time notifications.

Incase you want your own socket server Then create a socket server and required clients to connect to that server. you have to mange all connections on server my yourself. When ever you want to send data use that connection to send/receive data.

one benefit of using sockets is that clients do not need to be in same language.

A good starting point could be http://biese.wordpress.com/2009/06/14/how-to-create-a-simple-java-socket-thread-server/