Real-Time Services in Linux
Embedded Linux and real-time
Linux and other open-source software are increasingly used for developing Embedded Systems.
Many of these applications have real-time requirements.
Ideally we would like to:
Have the advantages of Linux: HW support, low cost, modularity, openess, ...
And meet deadlines!
Linux was developed as generic desktop and server time-sharing OS.
Objectives are: optimize throughput, improve the average utilization of resources (CPU, memory, I/O), ...
Timeliness in not a main issue.
Conversely, dedicated Real-Time Operative Systems (RTOS) are engineered for having temporal determinism, often at expenses of throughput.
In fact, throughput and temporal determinism are often conflicting requirements.
Optimizing for one impacts negatively on the other.
Improving the real-time performance of Linux
Approach 1
Modify the Linux kernel to improve its temporal behavior.
Bound the latency of syscalls, introduce fine-grain preemption, improve timer resolution, create specific services for real-time, proper scheduling, etc.
Many of these features have been added to the mainline Linux kernel (from the PREEMPT_RT project).
Approach 2
Add a RTOS “under” linux. Linux becomes a background task of the RTOS.
Approach followed e.g. by RTAI, Xenomai, ...
Hints on Linux kernel latency sources
A typical event sequence in a real-time application is:
An event causes a CPU interrupt.
The corresponding ISR is executed, activating a user-space task associated with the event.
Eventually this task is executed and completes, thus closing the reaction to the event.
Time elapsed between an event and the corresponding task activation is the latency
The objective is reduce, as much as possible, the latency!
kernel latency = interrupt latency + handler duration + scheduler latency + scheduler duration
Interrupt latency sources:
Linux kernel (including device-drivers) disables interrupts for certain operations.
Interrupts can interrupt other interrupts (nesting).
Interrupt handlers are split in two parts (top/bottom; fast/slow).
Linux kernel is preemptive.
But not fully! Syscalls have variable duration and limited preemption.
In addition, there are many other sources of latency and jitter that affect real-time tasks in Linux:
Scheduler latency.
Linux makes intensive use of virtual memory. Access to data that is on disk takes much longer than access to data that is in RAM.
The same applies to cache.
Many C library functions have not been designed for deterministic execution.
Priority inversion.
Due to shared resources.
Interrupt prioritization.
PREEMPT_RT Project
Some of the improvements already integrated on the mainline kernel.
O(1) scheduler.
Fixed-priority scheduler.
Kernel preemption.
Improvements to the POSIX real-time API support.
Mutexes with “Priority inheritance” support.
High resolution timers.
Threaded interrupts.
sched_deadline, EDF scheduling with CBS.
...
There are Linux distributions that already provide a patched kernel.
Real-time Ubuntu 22.04 LTS Beta.
Still in beta version.
Production version planned for 23.04 release.
Can be used for free for private use (instructions in the link above).
Example: preemption control
Linux supports different preemption levels.
For real-time use it may require compiling the Linux kernel with the right options.
PREEMPT_RT provides additional preemption levels.
Preemptible Kernel or Basic RT (PREEMPT_RTB).
Preemptible Kernel (PREEMPT_RT_FULL).
Checking Linux kernel configuration.
Linux kernel configuration can be obtained via:
Preemption status can be accessed via:
CONFIG_PREEMPT_NONE
Kernel code (interrupts, exceptions, system calls) never are preempted.
Common default configuration in many distributions.
Better performance for systems that carry intensive processing, if the objective is to maximize throughput.
Minimizes context switches and associated overheads.
CONFIG_PREEMPT_VOLUNTARY
Kernel code is preemptable at specific points.
Useful on desktop environments, as it increases the reactivity (as perceived by an human user).
Rescheduling points are explicitly added to the kernel code.
Low impact on throughput.
CONFIG_PREEMPT
Most of the kernel becomes preemptable.
In most cases, when a process is dispatched it can start execution before the completion of pending syscalls (issued by other processes).
However, there are non-preemptible critical sections protected by spinlocks.
Better option for Embedded Systems with RT requirements.
Low/moderate impact on throughput.
Best RT performance (without PREEMPT_RT patch) is obtained with this option!
Last updated
