How to write an application to have as close to real -time characteristics as possible?

Question

I'm looking for the best way to design an application that needs to respond in as close to real-time as possible. I have considered using a cut down kernel and building the software directly into that but I don't think that will be the best solution.

I am interested in creating something in hardware (I've been reading about that here on SO) and wondered if anyone had built something similar and if so what hardware did they use?

For example, is there a device that has gigabit Ethernet that is also programmable? i.e. a CPU on the card? I could then write a kernel driver to communicate / control the app running on the card - which would have lower latency making decisions based on network traffic due to its close proximity to the network interface...

Has anyone done this? If so can anyone recommend any hardware / software I should be looking into?

Answer 1

General comments

You probably need to be more specific about your requirements before folks can answer. Many of us have built real-time systems, but remember a) real-time is not the same as real-fast, and b) "real-time" is an umbrella term for a bunch of different timeliness criteria.

What activities do you need to ensure the timeliness of? The solutions you propose in your question are mildly exotic, and should only be considered once you state your requirements and you're sure more conventional approaches are inadequate.

Building a real-time application / system is not dissimilar from designing anything else. However, you do need to capture the timeliness requirements of each activity with the same precision that you ordinarily capture "functional" requirements (e.g., function/method pre, post, and invariant conditions). The hard part comes when you begin to understand the sum of all these requirements as the pieces come together -- and this is less like other aspects of system design.

Even if you can't apply it in all its rigorous detail, I suggest an attempt to frame your application's activities in terms of an RMA analysis. Many dynamic applications won't succumb completely to the analysis because you don't have enough data (they are inherently uncertain/dynamic/non-deterministic) but an attempt to apply the discipline will help.

In any case, start with analysis, not with a handful of sexy (and complicated) solutions...

Some things you should capture:

• What are the natural tasks/activities you expect the system to perform (in the problem space)? This is a good place to start with your design, but your implementation may need to shift.

• What are the resources you need to manage? (CPU? Network? Memory?)

• Can you ensure the resources will remain underloaded? If not, you'll need to institute some feasibility analysis and admission control policy or "load shedding." Note that this captures one dimension of "importance" or "priority" and is best decoupled from the other dimension...

• What are the time constraints? This is the "urgency" dimension. It may be a latency req't, or whatever. Are these constraints hard or soft? Aggregate or per-task? (For an interesting and in-depth look at scheduling approaches that distinguish urgency and importance, see RKC's thesis (or a shorter summary) from CMU, but take a deep breath before proceeding...)

• How are you going to represent the time constraint context of each activity in your implementation? Note there is not a "deadline scheduler" for the CPU in most common OS'es, and the normal approach is to map urgencies onto priorities (see RMA) but this only works if the resource is underloaded.

• How do you want to handle the multi-resource challenges? There is little in the way of rigorous analytical tools to help here, usually the goal is to overprovision as much as possible and apply admission control at the system boundaries. If this is a system where you demand timeliness for activities that include network transmission AND processing, then you might look to apply admission control in the form of QoS traffic management, reducing (not removing) the need to worry about CPU loading.

• What are your throughput requirements?

• What are your activities' arrival patterns? e.g., can you ensure a minimum inter-arrival time?

• What level of confidence do you require that your system / application will perform correctly?

Edits based on additional comments

Before moving to a hybrid hw/sw approach, make sure you exhaust the software solutions first. If your tasks are getting pre-empted by the kernel, then you should work to get that under control.

What is the OS? If Linux, recent Windows, or Mac OS X, you should be able to schedule your processing threads at real-time priority, and minimize kernel-induced preemption. If you are really burning up a CPU, maybe you should look at a dual-CPU approach, where you wall off a single CPU for your application processing (again, depends on OS).

Aha, 0.5ms response time starts to clarify the situation a bit. If you're using a general purpose OS, consistent sub-ms latencies can be a bit challenging alright.

What processor (and clock speed?)

It sounds like you have some working (or close to it) code in place. You've done some amount of analysis because you think preemption is killing you. What steps have you taken to eliminate preemption? Is the "decision" activity you reference a very small/simple evaluation? Is it deterministic? Is the 0.5ms bound a physical bound, or levied due to a latency budget that includes other pieces of the system?