Python MRO

Or How super() works

Python MRO - Or How super() works

Preliminaries

• mro stands for Method Resolution Order
• Part of new-style classes
• super() introduced in Python2.2
• Mechanices of super() revised and finalized in Python 2.3

mro algorithm

• C3 Method Resolution Order (Dylan, Python2.3, Perl 6)
• Based on merging of linearizations of each class
• Preserves expected order of calls (local precedence ordering) - parent's method called after child's
• There's a very gory explanation referenced in 2.3 release notes

Inheritance with Old-Style classes

class ClassicBeer:
    def __init__(self, ounces=12):
        self.ounces = ounces
    def drink(self): pass

class Ale(ClassicBeer):
    def __init__(self, ounces=10):
        ClassicBeer.__init__(self, ounces)
    def drink(self):
        ClassicBeer.drink(self)

Inheritance with New-Style Clases

class Beer(object):
    def __init__(self, ounces=12):
        self.ounces = ounces
    def drink(self):
        pass

class BW(Beer):
    "Budweiser"
    def __init__(self, ounces=16):
        super(BW, self).__init__(ounces)
    def drink(self):
        super(BW, self).drink()

Inheritance with New-Style Classes (cont.)

Demo: demo1.py

# Old style
>>> ale = Ale()
>>> ale.drink()
>>> ale2 = Ale(23)
>>> ale2.drink()
# New style
>>> bw = BW()
>>> bw.drink()
>>> bw2 = BW(45)
>>> bw2.drink()

super() is what?

It depends ... a bound or unbound super object.

• super(type) -> unbound super object
• super(type, obj) -> bound super object
• super(type, type2) -> bound super object

Guido says you'll likely only find the second useful ... I agree.

super() is what? (cont.)

Examples

>>> super(str, 'hello')
<super: <class 'str'>, <str object>>
>>> super(list, [1,2,3])
<super: <class 'list'>, <list object>>
>>> super(object, list)
<super: <class 'object'>, <list object>>

super() is what? (cont.)

super() is ... confusing.

... Think of the super object as a proxy around self and a type A with a specific set of method bindings.

Multiple Inheritance the classic way

class PerishableMixin:
    def __init__(self, maxage=135):
        self.maxage = maxage
        self.age = 0

class Ale(ClassicBeer, PerishableMixin):
    def __init__(self, ounces=10, maxage=120):
        ClassicBeer.__init__(self, ounces)
        PerishableMixin.__init__(self, maxage)
    def drink(self):
        ClassicBeer.drink(self)

Multiple Inheritance with New-Style classes

class BW(Beer, PerishableMixin):
    "Budweiser"
    def __init__(self, ounces=16, maxage=120):
        super(BW, self).__init__(ounces, maxage)
    def drink(self):
        super(BW, self).drink()

Multiple Inheritance (cont.)

demo: demo2.py

Multiple Inheritance (cont.)

• This code is borken!
• Beer and Perishable's __init__ both only take 2 arguments
• Let's "fix" it

Multiple Inheritance (cont.)

demo: demo2b.py

Multiple Inheritance (cont.)

Our fix is really lame, because:

• super() doesn't even work to invoke PerishableMixin's __init__
• setting maxage after super() is just a hack
• PerishableMixin is an old-style class so we have to mix paradigms

Multiple Inheritance (cont.)

Let's add a drink method on PerishableMixin.

class PerishableMixin:
    def __init__(self, maxage=135):
        self.maxage = maxage
        self.age = 0

    def drink(self):
        if self.age > self.maxage:
            print '[PerishableMixin] Blecccchhh'
        else:
            print '[PerishableMixin] Gulp'

Multiple Inheritance (cont.)

• It doesn't work!!
• Oh yeah! Perishable is an old-style class, right?
• So let's make Perishable a new-style class.

Multiple Inheritance (cont.)

class PerishableMixin(object): # make it new-style
    def __init__(self, maxage=135):
        self.maxage = maxage
        self.age = 0

    def drink(self):
        if self.age > self.maxage:
            print '[PerishableMixin] Blecccchhh'
        else:
            print '[PerishableMixin] Gulp'

Multiple Inheritance (cont.)

• Arrrgzzzz ... Fuck!
• Ok, so Beer isn't calling super ... that's the problem?

Multiple Inheritance (cont.)

demo: demo2f.py

Ahhh ... it works. But there are problems with the code.

Let's refactor it some.

Multiple Inheritance (cont.)

demo: demo3.py

Some patterns we've adopted:

• Make methods using super take arbitrary keyword args
• Always call super() ... right?

Multiple Inheritance (cont.)

And the problems?? Oh, there are so many ...

>>> class Miller(Beer): pass
...
>>>> m = Miller()
...
<type 'exceptions.TypeError'>: int argument required
>>> Beer() # Certainly this works???
...
<type 'exceptions.TypeError'>: int argument required

A Closer look at the mro

demo: demo4.py

Notice this has the same problems as previous code.

A Closer look at the mro (cont.)

Roughly:

>>> class PerishableMixin(object): pass
>>> class Beer(object)
>>> class BW(Beer, PerishableMixin)
>>> class Syrup(object)
>>> class AJM(Syrup)
>>> class ATD(BW, AJM, PerishableMixin)

Note that ATD redundantly extends PerishableMixin - this is a hint as to why the mro for ATD is so weird.

A Closer look at the mro (cont.)

The mro() for class ATD from demo4:

• <class '__main__.ATD'>
• <class '__main__.BW'>
• <class '__main__.Beer'>
• <class '__main__.AJM'>
• <class '__main__.PerishableMixin'>
• <class '__main__.Syrup'>
• <type 'object'>

A Closer look at the mro (cont.)

The mro maze for our gnarly example

A closer look at the mro (cont.)

super() doesn't mean anything.

It certainly doesn't mean parent.

Dylan's terminology is next()

A closer look at the mro (cont.)

Some simpler examples (all with problems):

• demo5.py (broken)
• demo5b.py
• demo5c.py
• demo5d.py (broken)
• demo5e.py
• demo5f.py

demo5f.py demonstrates a hack which will tell us if we're the last in the chain ... never write code like this ... please.

Diamonds in the Inheritance Heterarchy

>>> class A1(object): pass
>>> class A2(A1): pass
>>> class B1(A1): pass
>>> class B2(B1): pass
>>> class C(B1): pass
>>> class D(A2, B2, C): pass

Diamonds in the Inheritance Heterarchy

demo5.py and working demo5b.py

Diamonds ... (cont.)

Runnning the demo we get:

• <class '__main__.D'>
• <class '__main__.A2'>
• <class '__main__.B2'>
• <class '__main__.C'>
• <class '__main__.B1'>
• <class '__main__.A1'>

D -> A2 -> B2 -> C -> B1 -> A1

Diamonds ... (cont.)

The mro for demo5b.py

Inheritance Heterarchy without diamonds

Now something simpler (demo5c.py - demo5f.py)

>>> class A1(object): pass
>>> class A2(A1): pass
>>> class B1(object): pass
>>> class B2(B1): pass
>>> class C(object): pass
>>> class D(A2, B2, C): pass

Inheritance Heterarchy without diamonds (cont)

The mro for demo5c.py - demo5f.py

Problems

• We don't have a (simple) way to know when we're last in the chain
• ... Hence we don't ever know when we can use super()
• ... But we have to use super() for the next method in the chain to be called!
• Often times, using super() requires knowledge of the mro context (messy)
• ... So, super() is really only sane in very simple use cases

Possible Solutions

• Don't use super()
• ... Or just pretend like it's nifty syntax and forget about this presentation
• If you use super() and require cooperative method chaining, make sure this is documented
• For cooperative super method calls, accept only arbitrary keyword arguments (for most cases)
• Check for AttributeError in classes directly extending object

Further Reading

Better explanations:

• http://www.python.org/download/releases/2.3/mro/
• http://www.python.org/doc/2.2.3/whatsnew/sect-rellinks.html
• http://www.webcom.com/haahr/dylan/linearization-oopsla96.html
• http://fuhm.net/super-harmful/

This presentation and scary examples:

• http://www.enterthefoo.com/static/prez/mro
• http://www.enterthefoo.com/static/prez/mro-examples.tar.gz

The End

Questions?