Listen Print Discuss

A Primer on Python Metaclass Programming
Pages: 1, 2

Solving Problems with Magic

So far, we have seen the basics of metaclasses. Putting them to work is more subtle. The challenge of using metaclasses is that in typical OOP design, classes do not really do much. Class inheritance structures encapsulate and package data and methods, but one typically works with instances in the concrete.



There are two general categories of programming tasks where I think metaclasses are genuinely valuable.

The first, and probably more common, category is where you do not know at design time exactly what a class needs to do. Obviously, you will have some idea about it, but some particular detail might depend on information that will not be available until later. "Later" itself can be of two sorts: a), when a library module is used by an application, and b), at runtime when some situation exists. This category is close to what is often called "Aspect Oriented Programming" (AOP). Let me show an elegant example:

Example 7. Metaclass configuration at runtime

% cat dump.py
#!/usr/bin/python
import sys
if len(sys.argv) > 2:
    module, metaklass  = sys.argv[1:3]
    m = __import__(module, globals(), locals(), [metaklass])
    __metaclass__ = getattr(m, metaklass)

class Data:
    def __init__(self):
        self.num = 38
        self.lst = ['a','b','c']
        self.str = 'spam'
    dumps   = lambda self: `self`
    __str__ = lambda self: self.dumps()

data = Data()
print data

% dump.py
<__main__.Data instance at 1686a0>

As you would expect, this application prints out a rather generic description of the data object (a conventional instance). We get a rather different result by passing runtime arguments to the application:

Example 8. Adding an external serialization metaclass

% dump.py gnosis.magic MetaXMLPickler
<?xml version="1.0"?>
<!DOCTYPE PyObject SYSTEM "PyObjects.dtd">
<PyObject module="__main__" class="Data" id="720748">
<attr name="lst" type="list" id="980012" >
  <item type="string" value="a" />
  <item type="string" value="b" />
  <item type="string" value="c" />
</attr>
<attr name="num" type="numeric" value="38" />
<attr name="str" type="string" value="spam" />
</PyObject>

The particular example uses the serialization style of gnosis.xml.pickle, but the most current gnosis.magic package also contains the metaclass serializers MetaYamlDump, MetaPyPickler, and MetaPrettyPrint. Moreover, a user of the dump.py "application" can impose the use of any "MetaPickler" she wishes, from any Python package that defines one. Writing an appropriate metaclass for this purpose will look something like this:

Example 9. Adding an attribute with a metaclass

class MetaPickler(type):
    "Metaclass for gnosis.xml.pickle serialization"
    def __init__(cls, name, bases, dict):
        from gnosis.xml.pickle import dumps
        super(MetaPickler, cls).__init__(name, bases, dict)
        setattr(cls, 'dumps', dumps)

The remarkable achievement of this arrangement is that the application programmer need have no knowledge about what serialization will be used--nor even whether serialization or some other cross-sectional capability will be added at the command line.

Perhaps the most common use of metaclasses is similar to that of MetaPicklers: adding, deleting, renaming, or substituting methods for those defined in the produced class. In our example, a "native" Data.dump() method is replaced by a different one from outside of the application, at the time the class Data is created (and therefore, in every subsequent instance).

Resources

A useful book on metaclasses is Putting Metaclasses to Work, by Ira R. Forman, Scott Danforth, Addison-Wesley 1999 (ISBN 0201433052).

For metaclasses in Python specifically, Guido van Rossum's essay, " Unifying types and classes in Python 2.2," is useful.

My Gnosis Utilities package contains functions to make working with metaclasses easier and more powerful.

David Mertz , being a sort of Foucauldian Berkeley, believes, esse est denunte.

Return to Python DevCenter.