Using RTIPC for Real-Time Inter-Process Communication on Linux

Introduction to RTIPC

RTIPC (Real-Time Inter-Process Communication) is an inter-process communication library for Linux specifically designed for hard real-time workloads, where predictable execution and low latency are more important than the flexibility offered by generic IPC mechanisms.

This solution is useful for developers and engineers working in embedded, automotive, aerospace, and other sectors where deterministic timing is critical. RTIPC allows data exchange between processes without copying through kernel-managed buffers, significantly reducing latency and improving the predictability of operations.

• Blocking-free synchronization: Combines shared memory with a Single Producer Single Consumer (SPSC) wait-free queue, eliminating the need for syscalls in the data path.
• Force push operation: Allows the producer to replace older messages when the queue is full, useful for applications where the most recent state is more important than processing all intermediate updates.
• Cacheline alignment: Reduces cache coherence traffic in multi-core systems.
• Eventfd support: Enables integration with standard Linux mechanisms such as select, poll, and epoll.

RTIPC positions itself as an intermediate alternative between traditional Linux IPC mechanisms (such as sockets, pipes, and message queues) and manual implementations of shared memory communication. While the former move data through kernel-managed buffers, the latter avoid copies but leave synchronization to the application.

The library is still under active development, with implementations in C11, C++, Rust, and Python. The C and Rust versions are dependency-free and written in pure Rust, respectively. Future implementations for Java, C#, and Go are planned.

Prerequisites

• Linux operating system
• RTIPC library installed
• C11 or higher compiler
• Additional programming languages (C++, Rust, Python) for specific implementations

Procedure: Configuring RTIPC for Inter-Process Communication

By the end of this guide, you will have configured RTIPC to establish inter-process communication between two Linux applications with real-time requirements.

• Prerequisites:
  • Linux operating system
  • RTIPC library installed
  • Two processes that need to communicate

Step 1: Include the RTIPC Header

In the source code of both processes, include the RTIPC header:

#include <rtipc.h>

Step 2: Define the Data Structure

Define a C-compatible structure for the data to be exchanged. For example:

typedef struct { int id; float value;
} datat;

Step 3: Create a Communication Channel

In the producer process, create a communication channel:

rtipcchannelt channel = rtipccreatechannel(sizeof(datat), 10);

Where sizeof(datat) is the message size and 10 is the maximum queue length.

Step 4: Exchange File Descriptors

Exchange the file descriptors of the shared memory and eventfd between processes via a Unix domain socket:

rtipcexchangefds(channel, socketfd);

Step 5: Map the Shared Memory

rtipcchannelt channel = rtipcmapchannel(fd);

Step 6: Send and Receive Data

datat data = {1, 3.14f};
rtipcenqueue(channel, &data);

datat data;
rtipcdequeue(channel, &data);

Step 7: Handle Events

Optionally, use eventfd to handle communication events:

int eventfd = rtipcgeteventfd(channel);
epollevent ev;
ev.events = EPOLLIN;
ev.data.fd = eventfd;
epollctl(epfd, EPOLLCTLADD, eventfd, &ev);

Step 8: Close the Channel

At the end of the communication, close the channel:

rtipcclosechannel(channel);

For more details, visit the RTIPC project GitHub page.

Verification and Troubleshooting

By the end of this guide, you will be able to test the correct functioning of RTIPC and resolve any configuration or execution issues.

Functionality Test

• Connection Verification: Ensure both processes have correctly established the connection. Check that the file descriptors have been transferred without errors.
• Send and Receive Test: Send a test message from the producer to the consumer and verify that the message is received correctly. Repeat the test with different data types to ensure communication works in various scenarios.
• Synchronization Verification: Use debugging tools like strace or perf to monitor kernel activity and ensure there are no syscalls in the data path.
• Performance Test: Measure the latency and throughput of the communication. Compare the results with the design specifications to ensure performance meets expectations.

Problem Solving

• Connection Error: If the connection is not established, verify that both processes have the necessary permissions to access shared memory and file descriptors. Also, check that the network configuration allows communication via Unix domain socket.
• Synchronization Issues: If messages are not received correctly, ensure the queue has been initialized correctly and that the processes adhere to the Single Producer Single Consumer model.
• Alignment Error: If 32-bit and 64-bit processes fail to communicate, verify that the data structures are compatible between the two processes. Consider using standard C-compatible data structures to avoid alignment issues.
• Performance Issues: If performance is not satisfactory, check that the shared memory is correctly aligned to cache lines and that synchronization operations are efficient. Use profiling tools to identify any bottlenecks.

By the end of this guide, you will have understood the main features and usage methods of RTIPC, an inter-process communication library designed for hard real-time workloads on Linux.

Educational Summary

• Design and Functionality: RTIPC combines shared memory with a wait-free Single Producer Single Consumer (SPSC) queue, enabling data exchanges without copying through kernel buffers.
• Force Push Operation: Allows the producer to replace the oldest message with newer data when the queue is full, useful for workloads where the current state is more important than intermediate updates.
• Trade-off: Message size and maximum queue length must be set during channel creation to ensure deterministic execution.
• Data Structures: RTIPC suggests using C-compatible structures to exchange structured data between applications in different languages.
• Limitations: It is not safe to exchange structures between 32-bit and 64-bit processes due to possible alignment differences.
• Active Development: Implementations in C11, C++, Rust, and Python are underway, with support for Java, C#, and Go in the planning phase.

Call to Practice

To deepen your understanding and experiment with RTIPC, visit the GitHub project. The official documentation provides technical details, code examples, and updates on the project's status.