It seems to me that code that uses super() in these creative ways will be very difficult to maintain. You need to understand the subtleties of the MRO just to find out which method is being called. Pitty the poor Python programmer who stumbles into such code without being aware that super() may not call the class's direct base.
I suppose similar criticism can be leveled against many other dynamic techniques, except that this one sacrifices more readability than I'm comfortable with.
super() supports arguments which allow you to specify which class's implementation to lookup, which is an explicit way to resolve ambiguities. Although well-designed multiple-inheritance should try to avoid those ambiguities in the first place -- a class shouldn't inherit from multiple bases with conflicting implementations of a method.
I think in a case like the one you describe, where the direct ancestor's implementation is circumvented to access a different ancestor's implementation, it should be done by calling
I disagree, that's extremely fragile. It might work for the particular class you are implementing. However, it might cause subclasses of that class to break, since those subclasses will have a different MRO than the current class, and you could end up skipping more of the MRO than you intended.
Personally I'm not in favor of enforcing such strict conventions, especially in a dynamic language like Python. Also isn't this pattern he repeats throughout:
class Shape:
def __init__(self, **kwds):
self.shapename = kwds.pop('shapename')
super().__init__(**kwds)
Better written like this?
class Shape:
def __init__(self, shapename=None, **kwds):
self.shapename = shapename
super().__init__(**kwds)
It seems like there are some things about Python's which the author doesn't like, and he solves them by adding a lot of boiler plate to each of his classes. It's probably a better idea to just embrace the Python way or use a different language.
It seems to me that the example of combining built-in dictionary classes is naively optimistic. For starters, OrderedDict, as it happens, does not use super! It calls the dict super-class methods directly. Since dict happens to be the next class in the MRO, this doesn't really matter for the purpose of this example, but I can envision a scenario where some plucky programmer inherits from both OrderedCounter and some other dict subclass, and the result doesn't work because OrderedDict does the wrong thing.
And OrderedDict isn't the only one. Maybe for some reason I would like to have an OrderedCounter where all the counts default to 42. So I do this:
class DefaultOrderedCounter(defaultdict, OrderedCounter):
pass
doc = DefaultOrderedCounter(lambda: 42)
doc.update('abracadabra')
Which results in:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\python32\lib\collections.py", line 507, in update
_count_elements(self, iterable)
File "c:\python32\lib\collections.py", line 63, in __setitem__
self.__map[key] = link = Link()
AttributeError: 'DefaultOrderedCounter' object has no attribute '_OrderedDict__map'
Whoops! Apparently defaultdict doesn't use super either. Of course a better way to do this would be to subclass DefaultOrderedCounter and just override the __missing__ method by hand, but that's not the point.
The article goes into "How to Incorporate a Non-cooperative Class", which basically says "wrap it up in a proxy class". But that's not really going to work here, since the result would be two separate dicts, with the defaultdictwrapper methods operating on one dict, and the other methods operating on the other.
I didn't see it linked in the article nor mentioned here already but several of the problems with `super` have been discussed in "Python's Super is nifty, but you can't use it"[1] for quite some time.
Not really. The only thing that changed about super in Python 3 is that you can call it with no arguments, which is just a short-hand for the way it was already used in Python 2. Everything in that article applies equally to both Python versions.
Nice article, but one thing struck me when reading the code examples posted in the article: What's the use of using kwds in __init__?
class Shape(Root):
def __init__(self, **kwds):
self.shapename = kwds.pop('shapename')
super().__init__(**kwds)
class ColoredShape(Shape):
def __init__(self, **kwds):
self.color = kwds.pop('color')
super().__init__(**kwds)
I would personally just have written this somewhat similar to this:
class Shape(Root):
def __init__(self, shapename):
self.shapename = shapename
super().__init__()
class ColoredShape(Shape):
def __init__(self, shapename, color):
self.shapename = shapename
self.color = color
super().__init__(shapename)
But then again I'm by no means what anyone could call a Python 'Jedi'. Can anyone explain to me why using kwds in this way would be better? Would it not lead to having to look up the declaration of the class for the exact names of arguments to pop() every time?
The reason for using keywords is that there is no guarantee that the next class in the MRO after ColoredShape will be Shape. When the instance is a ColoredShape, it will be, but if ColoredShape is subclassed with a more complex inheritance structure, then something else could end up in between them in the MRO.
Thus, ColoredShape.__init__ can't simply pass on shapename to the super method, because that might not be the argument expected by the next __init__ method in the MRO. Instead, each method needs to gracefully accept arbitrary arguments and then pass on whatever it received (minus what it consumed).
Good advice and well spoken :-) That being said, composing a number of object classes also involves some measure of cooperation, caller/callee ordination, and some plan for calling the methods in the correct order.
IOW, if the decomposition of the classes is the same, then the solutions using composition or using multiple inheritance will have much in common.
Things should be as simple as possible, but no simpler :-)
The only reason why you don't need super in Go is because you cannot embed multiple interfaces except if their sets of methods are disjoint. This eliminates super's job of allowing programmers to differentiate between base classes with conflicting method implementations.
Without this, Go requires you to design your class hierarchy in a way that is cleaner, but the super concept is still not superfluous -- because you do sometimes need to reference an ancestor's implementation in Go.
You embed an interface and then explicitly reference the embedded interface by name. The Go interface
type ReaderWriter interface {
Reader
Writer
}
is exactly like this Python
class ReaderWriter(Reader, Writer):
pass
To get at a particular superclass implementation, you pass the class to super, like so:
That is, if you want to get at a particular superclass implementation, then you just call it directly. super() is used when you're not calling a particular superclass, instead relying on Python to compute the correct "next method" for you.
"super(Reader).read()", if you actually try it, will just result in an AttributeError. This is because super(Reader) returns an unbound super object, not a bound super object that you can actually dispatch from. Why this advanced usage might be useful is beyond the scope of this thread, but if you're curious then just google for python unbound super and hit the Feeling Lucky button.
The "kwds" technique is actually something I came up with independently to solve a different problem when I was using PyGame: ridiculously long argument lists when constructing sprites. In this case it significantly increased the flexibility and brevity of the code, although perhaps with a more complicated inheritance tree it wouldn't be worth it.
Nice to see this pattern being acknowledged somewhere else; I thought I was the only one that did that.
how does the runtime performance of super() compare to the hardcoding method? (from your example, using dict.__setitem__ as opposed to super().__setitem__ )
In CPython, calling super() has about the same cost as calling any other builtin like len() or int(). The MRO itself is precomputed and stored in inst.__class__.__mro__.
Contrasting dict.__setitem__ with super().__setitem__, the latter adds one function call. In addition, both forms require a builtin lookup and an attribute lookup.
In PyPy, much of the overhead of builtin lookups, function calls, and attribute lookups is automatically optimized away.
so super() is computed ahead of time? That doesn't make sense when you look at his comment:
"The calculation depends on both the class where super is called and on the instance’s tree of ancestors."
I agree that the class's MRO can be computed ahead of time, but theoretically the instance's order could be different (if a class method changed the definition of __setitem__)
The method resolution order for instances of the same class is always the same for all methods. If the class doesn't implement that method, the interpreter just skips and goes to the next place in the call chain. All super does is call the next method in the chain. So if you have class A(object), class B(A), class C(A), and class D(B, C), the MRO of D's instances will be B C A object. Each time an instance of D calls super, it will call the next method in the chain, not necessarily B's or C's or A's, even if the call to super is in B's method. This is why super() receives the class as a parameter, so it can know from when in the object's mro to start looking up the next method.
"This is why super() receives the class as a parameter" <-- this is not necessary in python 3. Your argument is valid in python2, but python3 allows a super() call without a type argument.
From the docs:
"super(type[, object-or-type])" <-- python2 doc, type is required
"super([type[, object-or-type]])" <-- type is optional in python3
Everything? Mostly the part where, Python being an MI language, a given class A has multiple parents. And these parents are a sequence, not a set, which define a Method Resolution Order, and that method resolution order usually isn't breadth-first either, so Python's super is a graph walker. And because it's explicit, any node in the graph can stop the walking.
This means permuting two parent classes can have significant impact on the semantics of the graph walking, therefore the results of calling super.
super() was introduced in CPython 2.2, but it changed in Python3: http://www.python.org/dev/peps/pep-3135/. The link you posted uses super as "super(cls, instance)", but the main article uses Python 3's super, which can just be called as "super()", and it figures out the class and instance.
So no, super really has changed, and the syntax in the article does not work with very old version of Python.
I upvoted the parent, because that syntax distinction is actually fairly giant, always having to name the class and also changing that on renames etc, is a PITA.
It is actually one of the things that annoys me most about Python, I guess this is the first thing that actually tempts me with Python 3. Damn.
super(myclass, self) was. Like I posted below, sometimes I have real trouble keeping the 5 or so languages I use straight. The other day I actually forgot what 'None' is in python because I hadn't used it in a few days, and None is different than in the others.
Python 3 was introduced two and a half years ago. Its major syntactic changes, while frustrating if you're set in your ways, are hardly insurmountable.
I think it's fine if you're doing python all day long, but I'm using Python, Java, PHP, Obj-C and Javascript each at least once a month on a variety of paid and personal projects. Like I said, I have enough trouble keeping those syntaxes straight.
I suppose similar criticism can be leveled against many other dynamic techniques, except that this one sacrifices more readability than I'm comfortable with.