Need for Speed – SENDFILE System Call

by Ruslan Spivak on June 28, 2012

A very common operation of Web servers these days is transferring files to clients. They do that by reading the files’ contents from a disk and writing it back to the clients’ sockets.

The copying of the file on Linux/UNIX, for example, could be done in a loop using read/write system calls:

import os
...
while True:
    data = os.read(filefd, 4096)
    if not data:
        break
    os.write(socketfd, data)

On the surface this looks perfectly fine, but for transferring large files, when pre-processing of the file contents isn’t necessary, this is pretty inefficient.

The reason is that the read and write system calls involve copying data from kernel space to user space and vice versa and all that happens in a loop:

That’s where the sendfile system call comes in handy. It provides a nice optimization for this particular use case by doing all the copying from the file descriptor to the socket descriptor completely in the kernel space:
read/write

Python 3.3 provides os.sendfile as part of the standard library. Check out more details on the GitHub.

If you enjoyed this post why not subscribe via email or my RSS feed and get the latest updates immediately. You can also follow me on GitHub or Twitter.

Speak your mind

Previous post:

Next post: