Logsna – a sane log output format

Good old log files are still the most reliable, versatile, and useful sources of information.
When they are also human-readable and easy to use with standard Unix tools like tail and grep they are even more useful.

We all know that especially in time of stress when things with a system are going south having a quick and easy way to grep for useful information is critical. The purpose of a small Python utility Logsna is to provide a sane log output format that makes some grepping a bit easier.

Logsna offers a custom formatter class logsna.Formatter that can be used in a logging config file, for example:

# sanefmt.py
import logging
import logging.config
from StringIO import StringIO

CONFIG = """\
[loggers]
keys=root

[handlers]
keys=console

[handler_console]
class=logging.StreamHandler
args=(sys.stderr,)
formatter=sane

[formatters]
keys=sane

[logger_root]
level=DEBUG
handlers=console

# Our custom formatter class
[formatter_sane]
class=logsna.Formatter
"""

config = StringIO(CONFIG)
logging.config.fileConfig(config)

log = logging.getLogger('mylogger.component1')

log.debug('debug message')
log.info('info message')
log.warning('warning message')
log.critical('critical message')
try:
    1 / 0
except:
    log.exception('Houston we have a problem')

The Log Format

Here is an output from the above program:

DEBUG    [2012-05-21 01:59:23,686] mylogger.component1: debug message
INFO     [2012-05-21 01:59:23,686] mylogger.component1: info message
WARNING  [2012-05-21 01:59:23,686] mylogger.component1: warning message
CRITICAL [2012-05-21 01:59:23,686] mylogger.component1: critical message
ERROR    [2012-05-21 01:59:23,686] mylogger.component1: Houston we have a problem
! Traceback (most recent call last):
!   File "/home/alienoid/python/sanefmt.py", line 67, in
!     1 / 0
! ZeroDivisionError: integer division or modulo by zero

The Log Format Notes

- All timestamps are in ISO8601 and UTC format

- To grep for messages of a specific level

$ tail -f sanefmt.log | grep '^INFO'

- To grep for messages from a particular logger

$ tail -f sanefmt.log | grep 'component1:'

- To pull out full exception tracebacks with a corresponding log message

$ tail -f sanefmt.log | grep -B 1 '^\!'

Installation

$ [sudo] pip install logsna