Using the Linux real-time services

Project development

No special tools are required.

• POSIX* real-time extensions are part of the standard C library.

Link code with -lrt.

• E.g. gcc -o myprog myprog.c -lrt

API documentation.

• man functioname

Process, thread, task ?

There is some confusion between “process”, “thread” and “task”.

• In Unix/Linux processes are created by the fork() primitive and are composed of:

  • An addressing space (for code, data, stack ...).

  • A thread, that starts the execution of the main() function.

    • After creation, a process has a single thread.

    • Additional threads can be created by means of the syscall pthread_create().

      • These threads share the same addressing space as the initial thread.

      • And execute a function given as argument to the pthread_create() syscall.

  • Often the term task is used interchangeably with process.

Processes and threads at the kernel level

Kernel represents threads by a structure of type “task_struct”.

From the scheduling point of view there are no differences between the initial thread and the additional threads crated via pthread_create().

Threads creation

Linux supports the POSIX API.

To create a new thread

pthread_create(pthread_t *thread, pthread_attr_t *attr, void *(*routine)(*void*),void *arg);

• The new thread is created on the same addressing space, but scheduled as an independent entity.

Terminating a thread.

pthread_exit(void *value_ptr);

Waiting for the termination of a thread.

pthread_join(pthread_t *thread, void **value_ptr);

Scheduling classes

Linux kernel supports several scheduling classes.

The default class is time-sharing.

• All processes receive some CPU time independently of its priority.

• The CPU share of each process is dynamic.

  • Affected by the “nice” value, that varies between -20 (highest priority) and 19 (lowest priority).

  • Depends on the type of operations carried out.

    • CPU-bound, I/O-bound.

  • Can be changed by commands as nice and renice.

• Non deterministic.

• Extremely poor real-time behavior.

• There are two fixed-priority real-time scheduling classes:

  • SCHED_FIFO

  • SCHED_RR

  • The ready task with higher priority gets the CPU.

  • Priority is statically defined, varying from 1 (lower) to 99 (higher).

    • Portable programs should use values between sched_get_priority_min() and sched_get_priority_max().

  • SCHED_RR vs SCHED_FIFO.

    • SCHED_RR: round-robin applied to tasks that share the same priority.

    • SCHED_FIFO: tasks that share the same priority execute by arrival order.

  • Significantly improved real-time behavior.

    • Depends on the platform but tens to a few hundreds of us of jitter are feasible.

A process can be assigned with a real-time class via syscall chrt.

• Example: chrt -f 99 ./myprog

  • -f FIFO ; 99 priority

• Scheduling attributes of a process can be retrieved with chrt.

  • chrt -p PID

The Linux kernel also dynamic priorities via the “sched_deadline” scheduling class.

• sched_deadline has the highest priority that can be defined by the user (specifically, higher than SCHED_FIFO and SCHED_RR).

Task parameterization:

• Runtime: Maximum execution time per period.

• Deadline: Time window, starting from the period beginning, during which “Runtime” must be served.

• Period: Periodicity of activation of the server.

• Make sure that RUNTIME <= DEADLINE <= PERIOD.

Example:

# chrt -d --sched-runtime 1000000 --sched-deadline 5000000 --sched-
period 50000000 ./dummyTask &

Check with chrt -p

# chrt -d --sched-runtime 1000000 --sched-deadline 5000000 --sched-period
50000000 0 ./dummyTask &
[1] 8521
# chrt -p 8521
pid 8521's current scheduling policy: SCHED_DEADLINE
pid 8521's current scheduling priority: 0
pid 8521's current runtime/deadline/period parameters:
1000000/5000000/5000000

Linux ensures that the utilization of EDF jobs does not exceed 95% of the available computing time.

• This setting can be changed by using the proc files:

  • /proc/sys/kernel/sched_rt_period_us

  • /proc/sys/kernel/sched_rt_runtime_us

Syscall sched_setscheduler() allows defining the scheduling class programatically.

int sched_setscheduler(pid_t pid, int policy, const struct sched_param *param);

• policy: SCHED_OTHER, SCHED_FIFO, SCHED_RR, etc.

• param: structure that includes priority.

The individual priority of each thread can be defined at its creation:

struct sched_param parm;
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_attr_setinheritsched(&attr,
PTHREAD_EXPLICIT_SCHED);
pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
parm.sched_priority = 42;
pthread_attr_setschedparam(&attr, &parm);

The thread is created using pthread_create(), with argument “attr” structure.

Other options can be set (e.g. stack size).

Using SCHED_DEADLINE programatically.

• Unfortunately the implementation of SCHED_DEADLINE is not standard.

• See “man sched_setattr”, section “CONFORMING TO”.

• struct sched_attr and methods sched_getattr() and sched_setattr() are still missing from sched.h!

• Linux distribution dependent.

Memory blocking

To avoid the indeterminism that results from the use of virtual memory, it is possible to lock the memory.

• The memory used by the process addressing space is always kept in RAM.

mlockall(MCL_CURRENT | MCL_FUTURE);

Locks of memory pages used by the process (current and future, if MCL_FUTURE).

• Heap, stack, shared memory, ...

Other related syscalls:

• munlockall, mlock, munlock.

Mutexes

Mutext: allow implementing mutual exclusion between threads of the same process.

• Creation and elimination.

pthread_mutex_init(pthread_mutex_t *mutex, const pthread_mutexattr_t *mutexattr);
pthread_mutex_destroy(pthread_mutex_t *mutex);

• Lock/unlock

pthread_mutex_lock(pthread_mutex_t *mutex);
pthread_mutex_unlock(pthread_mutex_t *mutex);

For using priority inheritance:

pthread_mutexattr_t attr;
pthread_mutexattr_init (&attr);
pthread_mutexattr_getprotocol(&attr, PTHREAD_PRIO_INHERIT);

Making a thread periodic

clock_nanosleep()

• Allows the calling thread to sleep for an interval specified with nanosecond precision.

int clock_nanosleep(clockid_t clock_id, int flags, const struct timespec *request,struct timespec *remain);

If flags have “TIMER_ABSTIME” set, sleeps until the time instant specified in request:

clock_gettime(CLOCK_MONOTONIC, &ts);
while(1) {
    ADD_PERIOD_TO_TS;
    clock_nanosleep(CLOCK_MONOTONIC, TIMER_ABSTIME,&ts,&tr);
    PROCESSING;
}

Signals

Signals: mechanisms for asynchronous notifications.

• A notification may be issued:

  • By the activation of a signal handler (few limitations).

  • Unlocking by means of a primitive sigwait(), sigtimedwait() or sigwaitinfo().

    • Preferred method!!

• The signal behaviour can be defined by means of syscall sigaction().

• Signal masking can be carried out with pthread_sigmask().

• Sending a signal can be made via pthread_kill() or tgkill().

• Can be used signals between SIGRTMIN and SIGRTMAX.

Interprocess communication

Semaphores

• Can be used between different processes (named semaphores).

sem_open(), sem_close(), sem_unlink(), sem_init(),
sem_destroy(), sem_wait(), sem_post(), etc.

Message queues

• Allow data exchanges in the form of messages.

mq_open(), mq_close(), mq_unlink(),
mq_send(), mq_receive(), etc.

Shared memory

• Allow data exchanges via a sahred memory region

shm_open(), ftruncate(), mmap(),
munmap(), close(), shm_unlink()

Debugging kernel latency

Ftrace – tool that can be used for debug as well as for latency analysis.

Very small Overhead when not active.

Can be used for tracing the execution of any kernel function.