246

Because I am used to the old ways of duck typing in Python, I fail to understand the need for ABC (abstract base classes). The help is good on how to use them.

I tried to read the rationale in the PEP, but it went over my head. If I was looking for a mutable sequence container, I would check for __setitem__, or more likely try to use it (EAFP). I haven't come across a real life use for the numbers module, which does use ABCs, but that is the closest I have to understanding.

Can anyone explain the rationale to me, please?

mc_kaiser
  • 639
  • 7
  • 17
Muhammad Alkarouri
  • 21,347
  • 16
  • 59
  • 94

5 Answers5

264

@Oddthinking's answer is not wrong, but I think it misses the real, practical reason Python has ABCs in a world of duck-typing.

Abstract methods are neat, but in my opinion they don't really fill any use-cases not already covered by duck typing. Abstract base classes' real power lies in the way they allow you to customise the behaviour of isinstance and issubclass. (__subclasshook__ is basically a friendlier API on top of Python's __instancecheck__ and __subclasscheck__ hooks.) Adapting built-in constructs to work on custom types is very much part of Python's philosophy.

Python's source code is exemplary. Here is how collections.Container is defined in the standard library (at time of writing):

class Container(metaclass=ABCMeta):
    __slots__ = ()

    @abstractmethod
    def __contains__(self, x):
        return False

    @classmethod
    def __subclasshook__(cls, C):
        if cls is Container:
            if any("__contains__" in B.__dict__ for B in C.__mro__):
                return True
        return NotImplemented

This definition of __subclasshook__ says that any class with a __contains__ attribute is considered to be a subclass of Container, even if it doesn't subclass it directly. So I can write this:

class ContainAllTheThings(object):
    def __contains__(self, item):
        return True

>>> issubclass(ContainAllTheThings, collections.Container)
True
>>> isinstance(ContainAllTheThings(), collections.Container)
True

In other words, if you implement the right interface, you're a subclass! ABCs provide a formal way to define interfaces in Python, while staying true to the spirit of duck-typing. Besides, this works in a way that honours the Open-Closed Principle.

Python's object model looks superficially similar to that of a more "traditional" OO system (by which I mean Java*) - we got yer classes, yer objects, yer methods - but when you scratch the surface you'll find something far richer and more flexible. Likewise, Python's notion of abstract base classes may be recognisable to a Java developer, but in practice they are intended for a very different purpose.

I sometimes find myself writing polymorphic functions that can act on a single item or a collection of items, and I find isinstance(x, collections.Iterable) to be much more readable than hasattr(x, '__iter__') or an equivalent try...except block. (If you didn't know Python, which of those three would make the intention of the code clearest?)

That said, I find that I rarely need to write my own ABC and I typically discover the need for one through refactoring. If I see a polymorphic function making a lot of attribute checks, or lots of functions making the same attribute checks, that smell suggests the existence of an ABC waiting to be extracted.

*without getting into the debate over whether Java is a "traditional" OO system...


Addendum: Even though an abstract base class can override the behaviour of isinstance and issubclass, it still doesn't enter the MRO of the virtual subclass. This is a potential pitfall for clients: not every object for which isinstance(x, MyABC) == True has the methods defined on MyABC.

class MyABC(metaclass=abc.ABCMeta):
    def abc_method(self):
        pass
    @classmethod
    def __subclasshook__(cls, C):
        return True

class C(object):
    pass

# typical client code
c = C()
if isinstance(c, MyABC):  # will be true
    c.abc_method()  # raises AttributeError

Unfortunately this one of those "just don't do that" traps (of which Python has relatively few!): avoid defining ABCs with both a __subclasshook__ and non-abstract methods. Moreover, you should make your definition of __subclasshook__ consistent with the set of abstract methods your ABC defines.

Benjamin Hodgson
  • 37,496
  • 16
  • 98
  • 147
  • 24
    "if you implement the right interface, you're a subclass" Thanks a lot for that. I don't know if Oddthinking missed it, but I certainly did. FWIW, `isinstance(x, collections.Iterable)` is clearer for me, and I do know Python. – Muhammad Alkarouri Oct 31 '13 at 20:40
  • Excellent post. Thank you. I think that the Addendum, "just don't do that" trap, is a bit like doing normal subclass inheritance but then having the `C` subclass delete (or screw up beyond repair) the `abc_method()` inherited from `MyABC`. The principal difference is that it's the superclass that is screwing up the inheritance contract, not the subclass. – Michael Scott Cuthbert Sep 14 '15 at 15:05
  • Would you not have to do `Container.register(ContainAllTheThings)` for the given example to work? – BoZenKhaa Feb 24 '16 at 10:27
  • 1
    @BoZenKhaa The code in the answer works! Try it! The meaning of `__subclasshook__` is "any class which satisfies this predicate is considered a subclass for the purposes of `isinstance` and `issubclass` checks, **regardless of whether it was registered with the ABC**, and **regardless of whether it's a direct subclass**". As I said in the answer, _if you implement the right interface, you're a subclass!_ – Benjamin Hodgson Feb 24 '16 at 11:11
  • Yes, of course it does, I was just being blind to a typo in my code, thank you :-). Great post. I should get some IDE. – BoZenKhaa Feb 24 '16 at 13:36
  • 1
    You should change the formatting from italics to bold. "if you implement the right interface, you're a subclass!" Very concise explanation. Thank you! – marti Jan 31 '18 at 01:01
  • Excellent post! – Daniel Lahyani Apr 06 '18 at 23:38
  • Great explanation. In this sense is ABC class similar to Golang's concept of interface: "if you implement the methods, you're an instance"? – tpoker Apr 18 '19 at 07:09
179

Short version

ABCs offer a higher level of semantic contract between clients and the implemented classes.

Long version

There is a contract between a class and its callers. The class promises to do certain things and have certain properties.

There are different levels to the contract.

At a very low level, the contract might include the name of a method or its number of parameters.

In a staticly-typed language, that contract would actually be enforced by the compiler. In Python, you can use EAFP or type introspection to confirm that the unknown object meets this expected contract.

But there are also higher-level, semantic promises in the contract.

For example, if there is a __str__() method, it is expected to return a string representation of the object. It could delete all contents of the object, commit the transaction and spit a blank page out of the printer... but there is a common understanding of what it should do, described in the Python manual.

That's a special case, where the semantic contract is described in the manual. What should the print() method do? Should it write the object to a printer or a line to the screen, or something else? It depends - you need to read the comments to understand the full contract here. A piece of client code that simply checks that the print() method exists has confirmed part of the contract - that a method call can be made, but not that there is agreement on the higher level semantics of the call.

Defining an Abstract Base Class (ABC) is a way of producing a contract between the class implementers and the callers. It isn't just a list of method names, but a shared understanding of what those methods should do. If you inherit from this ABC, you are promising to follow all the rules described in the comments, including the semantics of the print() method.

Python's duck-typing has many advantages in flexibility over static-typing, but it doesn't solve all the problems. ABCs offer an intermediate solution between the free-form of Python and the bondage-and-discipline of a staticly-typed language.

Community
  • 1
  • 1
Oddthinking
  • 21,437
  • 19
  • 76
  • 115
  • 13
    I think that you have a point there, but I can't follow you. So what is the difference, in terms of contract, between a class that implements `__contains__` and a class that inherits from `collections.Container`? In your example, in Python there was always a shared understanding of `__str__`. Implementing `__str__` makes the same promises as inheriting from some ABC and then implementing `__str__`. In both cases you can break the contract; there are no provable semantics such as the ones we have in static typing. – Muhammad Alkarouri Aug 26 '10 at 11:05
  • 15
    `collections.Container` is a degenerate case, that only includes `\_\_contains\_\_`, and only to mean the predefined convention. Using an ABC doesn't add much value by itself, I agree. I suspect it was added to allow (for e.g.) `Set` to inherit from it. By the time you get to `Set`, suddenly belonging to the the ABC has considerable semantics. An item can't belong to the collection twice. That is NOT detectable by the existence of methods. – Oddthinking Aug 26 '10 at 16:55
  • The `Set` example looks better to me than the `print()` function. In particular, I didn't understand the higher level contract of the `print()` method. It looks to me similar to the `__contains__` and `__str__` cases, where the semantics of these are documented in their respective help/comments. But when you have a complex object like `Set`, the semantics are not the semantics of any one function or property of the object. Am I missing something? – Muhammad Alkarouri Aug 27 '10 at 22:16
  • 3
    Yes, I think `Set` is a better example than `print()`. I was attempting to find a method name whose meaning was ambiguous, and couldn't be grokked by the name alone, so you couldn't be sure that it would do the right thing just by its name and the Python manual. – Oddthinking Aug 28 '10 at 16:12
  • So it's kind of like an interface in said bondage programming languages? – ashes999 Mar 03 '13 at 23:25
  • 3
    Any chance of rewriting the answer with `Set` as the example instead of `print`? `Set` makes a lot of sense, @Oddthinking. – Ehtesh Choudhury Feb 06 '14 at 22:58
  • 1
    I think this article explains it very well: https://dbader.org/blog/abstract-base-classes-in-python – szabgab Jul 09 '15 at 10:27
  • 1
    I don't understand how is ABC enforcing the semantic contract. Isn't the derived class is free to implement whatever it wants and break any documented semantic contract? – Mudit Jain Mar 13 '18 at 22:55
  • @Mudit: It isn't enforced. It is a.commitment by one programmer, that another programmer can look for. – Oddthinking Mar 14 '18 at 22:05
124

A handy feature of ABCs is that if you don't implement all necessary methods (and properties) you get an error upon instantiation, rather than an AttributeError, potentially much later, when you actually try to use the missing method.

from abc import ABCMeta, abstractmethod

# python2
class Base(object):
    __metaclass__ = ABCMeta

    @abstractmethod
    def foo(self):
        pass

    @abstractmethod
    def bar(self):
        pass

# python3
class Base(object, metaclass=ABCMeta):
    @abstractmethod
    def foo(self):
        pass

    @abstractmethod
    def bar(self):
        pass

class Concrete(Base):
    def foo(self):
        pass

    # We forget to declare `bar`


c = Concrete()
# TypeError: "Can't instantiate abstract class Concrete with abstract methods bar"

Example from https://dbader.org/blog/abstract-base-classes-in-python

Edit: to include python3 syntax, thanks @PandasRocks

cerberos
  • 6,721
  • 3
  • 37
  • 43
  • 6
    The example and the link were both very helpful. Thanks! – nalyd88 May 29 '15 at 22:51
  • if Base is defined in a separate file, you must inherit from it as Base.Base or change the import line to 'from Base import Base' – Ryan Tennill Feb 19 '18 at 20:20
  • It should be noted that in Python3 the syntax is slightly different. See this answer: https://stackoverflow.com/questions/28688784/no-error-while-instantiating-abstract-class-even-though-abstract-method-is-not – PandasRocks Apr 09 '18 at 12:59
  • 7
    Coming from a C# background, this is the *the* reason to use abstract classes. You're providing functionality, but stating that the functionality requires further implementation. The other answers seem to miss this point. – Josh Noe Jun 25 '18 at 20:19
20

It will make determining whether an object supports a given protocol without having to check for presence of all the methods in the protocol or without triggering an exception deep in "enemy" territory due to non-support much easier.

Ignacio Vazquez-Abrams
  • 699,552
  • 132
  • 1,235
  • 1,283
8

Abstract method make sure that what ever method you are calling in the parent class has to be appear in child class. Below are noraml way of calling and using abstract. The program written in python3

Normal way of calling

class Parent:
    def methodone(self):
        raise NotImplemented()

    def methodtwo(self):
        raise NotImplementedError()

class Son(Parent):
   def methodone(self):
       return 'methodone() is called'

c = Son()
c.methodone()

'methodone() is called'

c.methodtwo()

NotImplementedError

With Abstract method

from abc import ABCMeta, abstractmethod

class Parent(metaclass=ABCMeta):
    @abstractmethod
    def methodone(self):
        raise NotImplementedError()
    @abstractmethod
    def methodtwo(self):
        raise NotImplementedError()

class Son(Parent):
    def methodone(self):
        return 'methodone() is called'

c = Son()

TypeError: Can't instantiate abstract class Son with abstract methods methodtwo.

Since methodtwo is not called in child class we got error. The proper implementation is below

from abc import ABCMeta, abstractmethod

class Parent(metaclass=ABCMeta):
    @abstractmethod
    def methodone(self):
        raise NotImplementedError()
    @abstractmethod
    def methodtwo(self):
        raise NotImplementedError()

class Son(Parent):
    def methodone(self):
        return 'methodone() is called'
    def methodtwo(self):
        return 'methodtwo() is called'

c = Son()
c.methodone()

'methodone() is called'

multigoodverse
  • 5,876
  • 11
  • 51
  • 93