Logsna – a sane log output format

by Ruslan Spivak on May 20, 2012

Good old log files are still the most reliable, versatile, and useful sources of information.
When they are also human-readable and easy to use with standard Unix tools like tail and grep they are even more useful.

We all know that especially in time of stress when things with a system are going south having a quick and easy way to grep for useful information is critical. The purpose of a small Python utility Logsna is to provide a sane log output format that makes some grepping a bit easier.

Logsna offers a custom formatter class logsna.Formatter that can be used in a logging config file, for example:

# sanefmt.py
import logging
import logging.config
from StringIO import StringIO

CONFIG = """\
[loggers]
keys=root

[handlers]
keys=console

[handler_console]
class=logging.StreamHandler
args=(sys.stderr,)
formatter=sane

[formatters]
keys=sane

[logger_root]
level=DEBUG
handlers=console

# Our custom formatter class
[formatter_sane]
class=logsna.Formatter
"""

config = StringIO(CONFIG)
logging.config.fileConfig(config)

log = logging.getLogger('mylogger.component1')

log.debug('debug message')
log.info('info message')
log.warning('warning message')
log.critical('critical message')
try:
    1 / 0
except:
    log.exception('Houston we have a problem')

The Log Format

Here is an output from the above program:

DEBUG    [2012-05-21 01:59:23,686] mylogger.component1: debug message
INFO     [2012-05-21 01:59:23,686] mylogger.component1: info message
WARNING  [2012-05-21 01:59:23,686] mylogger.component1: warning message
CRITICAL [2012-05-21 01:59:23,686] mylogger.component1: critical message
ERROR    [2012-05-21 01:59:23,686] mylogger.component1: Houston we have a problem
! Traceback (most recent call last):
!   File "/home/alienoid/python/sanefmt.py", line 67, in
!     1 / 0
! ZeroDivisionError: integer division or modulo by zero

The Log Format Notes

- All timestamps are in ISO8601 and UTC format

- To grep for messages of a specific level

$ tail -f sanefmt.log | grep '^INFO'

- To grep for messages from a particular logger

$ tail -f sanefmt.log | grep 'component1:'

- To pull out full exception tracebacks with a corresponding log message

$ tail -f sanefmt.log | grep -B 1 '^\!'

Installation

$ [sudo] pip install logsna
If you enjoyed this post why not subscribe via email or my RSS feed and get the latest updates immediately. You can also follow me on GitHub or Twitter.

Speak your mind

{ 1 trackback }

Previous post:

Next post: