Self-Pipe Trick

by Ruslan Spivak on June 24, 2012

Imagine that your server process needs to simultaneously wait for I/O on multiple descriptors and also needs to wait for the delivery of a signal.
The problem is that you can’t safely mix select() with signals because there is a chance of race condition.

Consider the following code excerpt:


def handler(signum, frame):
    global GOT_SIGNAL
    GOT_SIGNAL = True


signal.signal(signal.SIGUSR1, handler)

# What if the signal arrives at this point?

    reads, writes, excs =, wlist, elist)
except select.error as e:
    code, msg = e.args
    if code == errno.EINTR:
        if GOT_SIGNAL:
            print 'Got signal'

The problem here is that if the SIGUSR1 is delivered after setting the signal handler but before call to select then the select call will block and we won’t execute our application logic in response to the event thus effectively “missing” the signal (our application logic in this case is printing the message: Got signal).

That’s an example of possible nasty racing. Let’s simulate that with

Start the program

$ python
PID: 32324
Sleep for 10 secs

and send the USR1 signal to the PID(it’s different on every run) within the 10 second interval while the process is still sleeping:

$ kill -USR1 32324

You should see the program produce additional line of output ‘Wake up and block in “select”‘ and block without exiting, no message “Got signal”:

$ python
PID: 32324
Sleep for 10 secs
Wake up and block in "select"

If you send yet another USR1 signal at this point then the select will be interrupted and the program will terminate with the message:

$ kill -USR1 32324
$ python
PID: 32324
Sleep for 10 secs
Wake up and block in "select"
Got signal

Self-Pipe Trick is used to avoid race conditions when waiting for signals and calling select on a set of descriptors.

The following steps describe how to implement it:

  1. Create a pipe and change its read and write ends to be nonblocking
  2. Add the read end of the pipe to the read list of descriptors given to select
  3. Install a signal handler for the signal we’re concerned with. When the signal arrives the signal handler writes a byte of data to the pipe. Because the write end of the pipe is nonblocking we prevent the situation when signals flood the process, the pipe becomes full and the process blocks itself in the signal handler.
  4. When select successfully returns check if the read end of the pipe is in the readables list and if it is then our signal has arrived.
  5. When the signal arrives read all bytes that are in the pipe and execute any actions that have to be done in response to the signal delivery.

You can check out the implementation on GitHub and try it.

Start the and send a USR1 signal to it. You should see it output a message Got signal and exit.

If you enjoyed this post why not subscribe via email or my RSS feed and get the latest updates immediately. You can also follow me on GitHub or Twitter.

{ 9 comments… read them below or add one }

Kushal Kumaran June 25, 2012 at 3:36 AM

You might want to take a look at the signal.set_wakeup_fd function. That can remove the need to write your own signal handler. You still need the rest of the code, though.


Ruslan Spivak June 25, 2012 at 6:35 AM

Thanks Kushal. I need to take a closer look at its implementation.


Jean-Paul Calderone June 25, 2012 at 4:11 AM

There’s still a race condition in your fixed version. The self-pipe trick is sufficient to fix the problem in a C program. Signals work differently in CPython than in C, and so additional measures are required to avoid missing the signal. There’s an explanation of some of the subtleties in the Twisted module that deals with this issue,


Ruslan Spivak June 25, 2012 at 6:31 AM

Thanks a lot for the link, Jean-Paul. I’ll take a look.


Nicolas Trangez June 25, 2012 at 4:33 AM

On Linux, you should take a look at signalfd(2).


Ruslan Spivak June 25, 2012 at 6:30 AM

Thanks Nicolas. It’s pity that it’s not present in Python standard library.


Jean-Paul Calderone June 26, 2012 at 9:21 AM

I implemented it for the Python standard library. Unfortunately, the implementation was not accepted for Python 2.7, only for Python 3.x.


Stepan June 28, 2012 at 8:36 AM

Python has ctypes.


Brian September 18, 2013 at 1:21 PM

The pattern used here:

… is rather effective at dealing with this race condition. It hinges on whether ‘pselect’ is exposed in Python; however, the short of it: create two signal masks, set the current thread/process mask to ignore SIGUSR1 and pass ‘pselect’ a mask that doesn’t block SIGUSR1.


Speak your mind

Previous post:

Next post: