Meta-classes Made Easy ——Eliminating self with Metaclasses

Introduction

Python metaclasses have a reputation for being deep black magic. Like many aspects of Python, once you get them they're really quite straightforward.

Metaclasses aren't often useful, but when they are they can simplify complex code.

This article explains the basics of metaclasses by creating a metaclass that applies a decorator to all methods in a class. This could be for something useful, like profiling, but instead it's going to apply the selfless decorator fromselfless Python. This makes the self implicit in method declarations. 

To follow the example you'll need the module Byteplay.

There are lots more you can do with metaclasses. Uses include profiling, registering classes and dynamically creating classes based on external specs. like database schema. For a good article on the details and usage of metaclasses, see Python Metaclasses: Who? Why? When? (pdf) by Richard Jones [1].

Metaclasses

Metaclasses are used to implement Python class objects. They are the type of types. So type(type) is type. All metaclasses have to implement the same C-level interface, this requires inheriting from a built-in metaclass: usuallytype.

Because metaclasses implement classes they are normally used to customise the creation (initialisation) of classes. This is what this tutorial will demonstrate.

Class objects are created at runtime. The interpreter takes the end of a class statement as a signal to call the appropriate metaclass to create the class-object.

If a class is a new style class (inherits from object) the default metaclass is type. For old style classes the default metaclass is types.ClassType.

To create the class, the metaclass is called with the following arguments :

type(name, bases, dict)

• 'name': The name of the class
• 'bases': A tuple of the base classes of the class
• 'dict': A dictionary of the attributes of the class

This returns the new class.

The following are basically equivalent :

def __init__(self, x):
    self.x = x

def printX(self):
    print self.x

Test = type('Test', (object,), {'__init__': __init__, 'printX': printX})

and :

class Test(object):
    def __init__(self, x):
        self.x = x

    def printX(self):
        print self.x

The Most Basic Example

So we can create a very simple metaclass which does nothing.

class PointlessMetaClass(type):
    def __new__(meta, classname, bases, classDict):
        return type.__new__(meta, classname, bases, classDict)

Rather than calling type directly, our metaclass inherits from type, so it calls type.__new__(meta...). This means that the metaclass returned is an instance of our class.

Note

If you make the mistake (as I did) of using type(classname, bases, classDict) in your metaclass it will still work... but subclasses of objects that use your metaclass won't inherit the metaclass.

Classes are instances of their metaclasses, if you want your class to really be an instance of your metaclass then you have to use type.__new__(meta, ....

Using Metaclasses

You can set the metaclass for a class by setting its __metaclass__ attribute.

We'll adapt the example for above so that we can see what's going on.

class MetaClass(type):
    def __new__(meta, classname, bases, classDict):
        print 'Class Name:', classname
        print 'Bases:', bases
        print 'Class Attributes', classDict
        return type.__new__(meta, classname, bases, classDict)

class Test(object):

    __metaclass__ = MetaClass

    def __init__(self):
        pass

    def method(self):
        pass

    classAttribute = 'Something'

If you run this you will see something like :

Class Name: Test
Bases: (<type 'object'>,)
Class Attributes {'__module__': '__main__',
                  '__metaclass__': <class '__main__.MetaClass'>,
                  'method': <function method at 0x00B412B0>,
                  '__init__': <function __init__ at 0x00B41070>,
                  'classAttribute': 'Something'}

You can also set the metaclass at the module level. If you set the variable __metaclass__ in a module, it will be used for all following class definitions that don't have an explicit metaclasses. New style classes inherit their metaclasses from object, so that means all old style classes. So you can do things like make all classes into new style classes by setting __metaclass__ = type.

A Method Decorating Metaclass

As you can see from the last example, methods on the class are passed into the metaclass as functions. The aim of this article is to produce a metaclass which decorates (wraps) all the methods in the class.

The basic code to do this looks like this :

from types import FunctionType

newClassDict = {}
for attributeName, attribute in classDict.items():
    if type(attribute) == FunctionType:
        attribute = decorator(attribute)

    newClassDict[attributeName] = attribute

Using this we can create a metaclass factory, which when given a function returns a metaclass that wraps all methods with that function :

from types import FunctionType

def MetaClassFactory(function):
    class MetaClass(type):
        def __new__(meta, classname, bases, classDict):
            newClassDict = {}
            for attributeName, attribute in classDict.items():
                if type(attribute) == FunctionType:
                    attribute = function(attribute)

                newClassDict[attributeName] = attribute
            return type.__new__(meta, classname, bases, newClassDict)
    return MetaClass

The Selfless Metaclass

So using the MetaClassFactory we can create a metaclass which applies the selfless decorator to all methods in a class.

from types import FunctionType
from byteplay import Code, opmap

def MetaClassFactory(function):
    class MetaClass(type):
        def __new__(meta, classname, bases, classDict):
            for attributeName, attribute in classDict.items():
                if type(attribute) == FunctionType:
                    attribute = function(attribute)

                newClassDict[attributeName] = attribute
            return  type.__new__(meta, classname, bases, classDict)
    return MetaClass

def _transmute(opcode, arg):
    if ((opcode == opmap['LOAD_GLOBAL']) and
        (arg == 'self')):
        return opmap['LOAD_FAST'], arg
    return opcode, arg

def selfless(function):
    code = Code.from_code(function.func_code)
    code.args = tuple(['self'] + list(code.args))
    code.code = [_transmute(op, arg) for op, arg in code.code]
    function.func_code = code.to_code()
    return function

Selfless = MetaClassFactory(selfless)

Note

You can download this as a Python module: The Selfless Metaclass.

A more useful example of a metaclass is one called __properties__, which automatically cretes properties from getter and setter methods defined in your classes. 

Of course it doesn't work with properties. 

Using it is simple :

class Test(object):

    __metaclass__ = Selfless

    def __init__(x=None):
        self.x = x

    def getX():
        print self.x

    def setX(x):
        self.x = x

test = Test()
test.getX()

test.setX(7)
test.getX()

Cool, hey ?

[1] The start of the next section borrows from this article, so thanks to Richard.