CODEWITHSUNDEEP
c#.netmessage-queuerabbitmqamqp
Michael Stum
Michael Stum

Reputation: 181094

Embeddable Queues?

I have an Application that collects actions and sends them off to a remote server. As these actions aren't time critical (think of them as log lines), I want to queue them up and send them in batches.

That way, I also want to ensure that no message is ever lost (unless the hard drive crashes).

MSMQ seems rather heavyweight, arcane and weird to use. Also, it needs to be installed as a system component.

Serializing my messages into JSON and storing them in SQLite is trivial and straight forward, but before I do that, I wonder if there is a standardized (preferably AMQP compatible) queue that I doesn't require installation and can be embedded into an app?

Upvotes: 3

Views: 1666

Answers (3)

scvalex
scvalex

Reputation: 15355

If you want a homebrew solution, you could install RabbitMQ on the logging server, embed RabbitMQ's .NET client into your application, then write a small program to read from the queue and write the events to disk.

RabbitMQ is fairly lightweight: the default install is only a few Mb and it normally uses about 11Mb of memory to run. It also provides an extension to AMQP, Publisher Confirms, which can be used to ensure that once the server accepts the log message, it will not be lost, unless the hard disk dies. The extension is non-standard, though, and it's probably not supported by other brokers.

Upvotes: 0

Michael Dillon
Michael Dillon

Reputation: 32392

Graylog2 is a centralized logging solution that accepts log entries from AMQP messages. Perhaps you could adapt it to your use-case.

In any event, Graylog2 shows that AMQP works for jobs like collecting log messages without losing any.

AMQP doesn't require installation, because it is a protocol. You just need the client library for .NET. However you would need to install an MQ broker on a server somewhere on your LAN to manage the message flow. RabbitMQ is widely used because it is easy to install.

Also, once you start sending messages, then you will also need to have a process somewhere on the network, that receieves them and does something with them such as write to a db.

Upvotes: 0

iandotkelly
iandotkelly

Reputation: 9134

I really think you should reconsider MSMQ.

  1. It is installed by default in the Server versions of Windows.
  2. Installation on non Server versions of windows is trivial.
  3. It provides a built-in UI for observing the queue
  4. I don't know what your standards are of 'heavy and arcane' - but I just used it for the first time in a project and it was the easiest part of the application. I certainly don't think its much more heavyweight than storing the queue in a database yourself.
  5. If you prefer to use JSON, you can serialize the messages yourself and store as a string
  6. You can configure a queue to be recoverable - so storing the queue on disk not in memory

The only serious objection that I can see is having to install MSMQ. If you are having to deploy this application far and wide on different versions of Windows, I can see that as a significant problem.

Upvotes: 1

Related Questions