Projects/IpcQueue

Interprocess Communication Queue for Signal Processing (ipc_queue)

Introduction

The ipc_queue library is designed to make interprocess communication between chained signal processing applications easy. It is available in both a C and a Python implementation. The ipc_queue library provides methods to pass fixed-size messages between two processes in unix on a point-to-point basis. The send method will block if the ipc_queue is full. This allows later processes to throttle earlier ones. Both process must be running on the same machine.

Description

An ipc_queue is identified by a channel number, which must be unique for that computer. This channel number serves as the identifier of the underlying shared memory. Both sender and receiver must know the channel number. In addition, both the sender and receiver must know the message size and format beforehand. In this API the shared memory is actually created when a producer creates an ipc_queue, and released when the consumer destroys the ipc_queue.

The ipc_queue library was intended to be used with the following design pattern. If the program is a data consumer, it first sets up a signal handler to handle SIGINT. This signal handler will destroy any consumer ipc_queues. This assures shared memory is freed if the program is aborted. The program then creates whatever producer and consumer ipc_queues it needs. It then goes into a loop of reading data, handling it, and sending data. End point programs either read from or write to files. If the program receives the special terminate message, it finishes its work, and then sends a terminate message, frees its resources, and terminates. This allows the first program in the chain to let all the others know that the work is complete, but allows all the following programs to finish their work before terminating.

It is also possible for a series of separate programs to write to the same ipc_queue. In creating a producer ipc_queue, there is an option to either overwrite any existing ipc_queue using that channel, or to use a existing ipc_queue if one exists. If a program does not overwrite its producer ipc_queue, it can safely reuse an existing producer ipc_queue. The ipc_queue API ensures that the data is correctly serialized.

Downloads

The C version contains the ipc_queue library, some example programs, and detailed man pages. It is installed via the standard autotools ./configure, make, make install. You can also run make check to build and run the example C programs.

The Python version also includes examples and documentation.

Contact Information

Bill Rideout - brideout@haystack.mit.edu