Recently I’ve been working with CSS files and wanted to replace color definitions with downcased equivalents, i.e. #FFFFFF; would become #ffffff;

To do that I used pretty well known feature of Emacs, namely use of Lisp expressions in the replacement string of replace-regexp function.

Commands entered in minibuffer:

M-x replace-regexp RET \(#.*\)  RET \,(downcase \1) RET

The only problem with that code is that if you have variable case-fold-search in Emacs set to true (which is the case in my Emacs by default) replace-regexp will report about successful replace but string won’t be downcased anyway. To quickly switch off case-fold-search I used M-: (setq case-fold-search nil) and after that downcasing worked as expected.

To use IPython shell from Grok based project is easy, just add ipython to eggs directive in your buildout.cfg

My part with ipython added looks like this:

[app]
recipe = zc.recipe.egg
eggs = grok-awesome
       ipython
       Paste
       PasteScript
       PasteDeploy
interpreter = python-console

But I wanted to be able to inspect Grok instance and use path TAB auto-completion to navigate ZODB hierarchy.

Inspired by existing IPython profile fo Zope2 I whipped out basic implementation for Grok.

It’s available under grok-ipython. Now all IPython power can be used when inspecting objects in Grok instance:

(mygrok)[alienoid@capricorn grok-awesome]$ bin/ipython -p grok
IPython shell for Grok.

Bound object names:
-------------------
  root
  ctx

Bound command names:
--------------------
  cdg / ;cdg
  lsg / ;lsg
  pwdg
  sync
  commit

Grok IPython 0.9.1   Py 2.5.2

In [1]: root
Out[1]: <zope.app.folder.folder.Folder object at 0x9003e6c>

In [2]: ctx
Out[2]: <zope.app.folder.folder.Folder object at 0x9003e6c>

In [3]: ;cdg blog
------> cdg("blog")
Out[3]: u'/blog'

In [4]: ctx
Out[4]: <grokawesome.blog.Blog object at 0x81d2d6c>

In [5]: ;cdg 
my-first-post  second-post    

In [5]: ;cdg my-first-post
------> cdg("my-first-post")
Out[5]: u'/blog/my-first-post'

In [6]: ctx
Out[6]: <grokawesome.entry.BlogEntry object at 0xb7d556ac>

In [7]: pwdg
Out[7]: u'/blog/my-first-post'

In [8]: ctx?
Type:           BlogEntry
Base Class:     <class 'grokawesome.entry.BlogEntry'>
String Form:    <grokawesome.entry.BlogEntry object at 0xb7d556ac>
Namespace:      Interactive
Length:         0
File:           /home/alienoid/mygrok/grok-awesome/src/grokawesome/entry.py
Docstring:
    <no docstring>


In [9]: ctx??
Type:             BlogEntry
Base Class:       <class 'grokawesome.entry.BlogEntry'>
String Form:   <grokawesome.entry.BlogEntry object at 0xb7d556ac>
Namespace:        Interactive
Length:           0
File:             /home/alienoid/mygrok/grok-awesome/src/grokawesome/entry.py
Source:
class BlogEntry(grok.Container):
    grok.implements(IBlogEntry)

    portal_type = 'blogentry'
    created = property.DCProperty('created')
    modified = property.DCProperty('modified')

    def __init__(self, title, content,
                 summary=u'',
                 categories=None,
                 allow_comments=False):
        super(BlogEntry, self).__init__()
        self.title = title
        self.content = content
        self.summary = '' if summary is None else summary
        self.categories = [] if categories is None else categories
        self.allow_comments = allow_comments

I have added TinyMCE widget to grok-awesome for editing html and it was easy as pie.

If you want TinyMCE in your Grok based project it can be achieved with following steps:

1. Add dependencies to setup.py into install_requires section:

                        'hurry.tinymce',
                        'hurry.zopetinymce',

hurry.tinymce allows you to include TinyMCE into your project without additional burden, you don’t need to do anything. Well, almost – just make sure to call:
from hurry.tinymce import tinymce
tinymce.need()
to trigger inclusion of TinyMCE in the web page.

2. Create custom widget which calls tinymce.need() as noted above and initializes TinyMCE

from zope.app.form.browser import TextAreaWidget
from hurry.tinymce import tinymce

template = """%(widget_html)s
<script type="text/javascript">
  tinyMCE.init({
    mode : "exact",
    elements: "%(elements)s",
    theme: "advanced"
  });
</script>"""

class TinyMCEWidget(TextAreaWidget):
    """Widget to edit html using TinyMCE WYSIWYG editor."""

    def __call__(self, *args, **kw):
        widget_html = super(TinyMCEWidget, self).__call__(*args, **kw)
        tinymce.need()
        return template % {'widget_html': widget_html,
                           'elements': self.name,
                           }

3. In your add/edit form use your custom widget for the field you want and off you go:

form_fields['content'].custom_widget = TinyMCEWidget

The other day I had to decode strings containing hex numbers into plain integers.

I could use built-in int([ x [, radix]]) with radix=16 to convert a string to an integer and forget about it, but I needed to decode pairs from the hex number so that in the end from the string ‘FFEEDD’ I get 255, 238, 221

But wouldn’t you know it, there is a standard solution to get pairs in Python and it’s built-in zip function.

It’s a well known function for experienced developers, but there is one documented feature of it that doesn’t always catch one’s eye and it’s a general form for clustering a data series into n-length groups zip(*[iter(s)]*n) which is described in official docs for zip built-in.

With that in mind solution to my problem boils down to a one-line decoder:

>>> s = 'FFDDEE'
>>> [int(''.join(pair), 16) for pair in zip(*[iter(s)]*2)]
[255, 221, 238]

For those wondering how it works:

1) iter(s) creates iterable from the s

>>> iter(s)
<iterator object at 0x8a3c2cc>

2) [iter(s)] * 2 creates list of 2 iterators that all use the same underlying iterable iter(s). The important thing is that created iterators share the same iterable, so when some data is consumed by first iterator that data becomes unavailable for the second iterator and vice versa.

You can see from the output that iterator objects are located at the same address. If you are new to this behavior you can read on about shallow copies.

>>> [iter(s)]*2
[<iterator object at 0x8a3c2cc>, <iterator object at 0x8a3c2cc>]

3) leading * in zip’s parameter unpacks list from step (2) into two separate parameters for zip function. Basically it becomes zip(iterator, iterator).

More about unpacking argument list you can read in official docs.

>>> zip(*[iter(s)]*2)
[('F', 'F'), ('D', 'D'), ('E', 'E')]

I often need to refresh page in Firefox when I edit page template or javascript file. This is especially true when working on grok-awesome.

Though I use tiling window manager StumpWM and it’s very easy to switch to running Firefox with keyboard shortcut and make refresh, it’s even easier not to leave focus from Emacs window and just send command to Firefox to refresh itself.
For that I use:

• MozRepl firefox plugin to allow connect to Firefox’s REPL. After installation in Tools -> MozRepl menu I chose ‘Activate on startup’ for convenience
• MozRepl Emacs integration (If you use nXhtml package in Emacs you already have that integration and can skip this part)

Command to refresh Firefox is BrowserReload();

Final touch is small anonymous function to send that command to mozrepl, keybinding to C-x p and we’re all set:

(global-set-key (kbd "C-x p")
                (lambda ()
                  (interactive)
                  (comint-send-string (inferior-moz-process)
                                      "BrowserReload();")))

Now anytime just press C-x p and refresh command will be sent to Firefox starting along the way mozrepl session in your Emacs if it’s not been already started.

I’ve started developing blog software for “A Smashing Web Framework” Grok.

The reasons behind that:

• There is no full-featured blog software for Grok. There is a simple version Grokstar, but as for me it doesn’t fit the bill.
• I created simple blog for Zope3 several years ago and let it languish. Now it’s time to revive it taking into account how Grok simplifies development with Zope3 technologies.
• I’m not completely happy with WordPress. For one thing, I’d like to have easy-to-use REST API so I can use it in my Emacs to make posts.
• I simply love Grok. You should try it and you’ll love it too.

My fledgling project lives on GitHub and it’s called grok-awesome :)

Ever wondered how to make IPython colors be nice on dark background of your dark (no pun intended) Emacs theme like blackboard ?

Just add to your .emacs

(setq py-python-command-args '("-pylab" "-colors" "Linux"))

before you load ipython package

(setq py-python-command-args '("-pylab" "-colors" "Linux"))
(require 'ipython)

Now IPython uses ‘Linux’ color scheme which is suitable for dark background with light fonts instead of default ‘LightBG’.

Here it is:

Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

  Welcome to pylab, a matplotlib-based Python environment.
  For more information, type 'help(pylab)'.

In [1]: d = dict()

In [2]: ?d
Type:           dict
Base Class:     <type 'dict'>
String Form:    {}
Namespace:      Interactive
Length:         0
Docstring:
    dict() -> new empty dictionary.
    dict(mapping) -> new dictionary initialized from a mapping object's
        (key, value) pairs.
    dict(seq) -> new dictionary initialized as if via:
        d = {}
        for k, v in seq:
            d[k] = v
    dict(**kwargs) -> new dictionary initialized with the name=value pairs
        in the keyword argument list.  For example:  dict(one=1, two=2)

In [3]: 

I’ve just finished reading excellent book Release It!: Design and Deploy Production-Ready Software

If you are involved or plan to be involved in developing any system that must be available 24 x 7 x 365 then you owe it to yourself to read this book.

The key ideas you should understand are:

• system will fail

• life of software only begins when you release it into production

With that in mind the book is a treasure trove of useful advices how to make your system scale and succeed in production.

The book resonates with my own personal experience in developing and maintaining critical Python application and many things that Michael Nygard describes I learned the hard way: blocked threads, socket timeouts, integration point failures and latency, asynchronous handling, caching, database growth and of course “cold shivers” you get when the system goes down.

While book is sprinkled with notes about Java related issues it’s nevertheless technology agnostic which makes it a valuable source of information for every developer.

It gets two thumbs up from me.

Sometimes you may need to have nested dictionaries initialized without much hassle.

“Autovivication” comes in handy. What you want to achieve is:

>>> d = dict()
>>> d['q']['w']['r'] = 777

Of course what you’ll get will be:

>>> d['q']['w']['r'] = 777
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'q'

If you don’t need deeply nested dictionaries defaultdict with dict default factory will do the job:

>>> from collections import defaultdict
>>> d = defaultdict(dict)
>>> d
defaultdict(<type 'dict'>, {})
>>> d['a']['b'] = 33
>>> d
defaultdict(<type 'dict'>, {'a': {'b': 33}})
>>> d['a']['b']
33

Here are another ways to have unlimited level of nested dictionaries.
1) Define custom class inheriting from built-in dict:

class default_dict(dict):
    def __missing__(self, key):
        self[key] = value = self.__class__()
        return value

>>> d = default_dict()
>>> d
{}
>>> d['q']['w']['r'] = 777
>>> d
{'q': {'w': {'r': 777}}}
>>> d['q']['w']['r']
777

2) Recursive function + defaultdict:

def make_dict():
    return defaultdict(make_dict)

>>> from collections import defaultdict
>>> d = defaultdict(make_dict)
>>> d
defaultdict(<function make_dict at 0xb7c5295c>, {})
>>> d['q']['w']['r'] = 777
>>> d
defaultdict(<function make_dict at 0xb7c5295c>, {'q': defaultdict(<function make_dict at 0xb7c5295c>, {'w': defaultdict(<function make_dict at 0xb7c5295c>, {'r': 777})})})
>>> d['q']['w']['r']
777

3) And for fun you can use Y Combinator (in short – making anonymous lambda recursive) which is actually as (2) but you don’t have to have recursive function defined:

def Y(g):
    return ((lambda f: f(f))
            (lambda f:
                 g(lambda x: f(f)
                   (x))))

defdict = Y(lambda mk_dict:
                lambda x=None: defaultdict(lambda x=None: mk_dict(x)))

>>> d = defdict()
>>> d
defaultdict(<function <lambda> at 0xb7c52a74>, {})
>>> d['q']['w']['r'] = 777
>>> d
defaultdict(<function <lambda> at 0xb7c52a74>, {'q': defaultdict(<function <lambda> at 0xb7c56c6c>, {'w': defaultdict(<function <lambda> at 0xb7c56a74>, {'r': 777})})})
>>> d['q']['w']['r']
777

Recent Erlang releases (since R12B) have array module included.

Now binary search algorithm can be implemented quite efficiently with functional arrays:

-module(bsa).
-export([binsearch/2]).

binsearch(Arr, Key) ->
    binsearch(Arr, Key, 0, array:size(Arr)).

binsearch(Arr, Key, LowerBound, UpperBound) ->
    Mid = (LowerBound + UpperBound) div 2,
    Item = array:get(Mid, Arr),
    if
        UpperBound < LowerBound -> -1;
        Key < Item ->
            binsearch(Arr, Key, LowerBound, Mid-1);
        Key > Item ->
            binsearch(Arr, Key, Mid+1, UpperBound);
        true ->
            Mid
    end.

Running time for lists based binary search (just for comparison):

56> {Time, Val} = timer:tc(search, binsearch,
56>                        [lists:seq(1, 1000000), 1000000]).
{506513,  1000000}

Running time for array based binary search:

57> f().
ok
58> {Time, Val} = timer:tc(bsa, binsearch,
58>               [array:from_list(lists:seq(1, 1000000)), 1000000]).
{17,  999999}