Get defining class of unbound method object in Python 3

Question

Say I want to make a decorator for methods defined in a class. I want that decorator, when invoked, to be able to set an attribute on the class defining the method (in order to register it in a list of methods that serve a particular purpose).

In Python 2, the im_class method accomplishes this nicely:

def decorator(method):
  cls = method.im_class
  cls.foo = 'bar'
  return method

However, in Python 3, no such attribute (or a replacement for it) seems to exist. I suppose the idea was that you could call type(method.__self__) to get the class, but this does not work for unbound methods, since __self__ == None in that case.

NOTE: This question is actually a bit irrelevant for my case, since I've chosen instead to set an attribute on the method itself and then have the instance scan through all of its methods looking for that attribute at the appropriate time. I am also (currently) using Python 2.6. However, I am curious if there is any replacement for the version 2 functionality, and if not, what the rationale was for removing it completely.

EDIT: I just found this question. This makes it seem like the best solution is just to avoid it like I have. I'm still wondering why it was removed though.

Answer 1

I thought it would be worthwhile writing something that does it best at guessing the defining class. For completeness' sake this answer also addresses bound methods.

At worst, guessing should fail altogether, with the function returning None. However, under any circumstances, it shouldn't raise an exception or return an incorrect class.

TL;DR

The final version of our function successfully overcomes most simple cases, and a few pitfalls as well.

In a nutshell, its implementation differentiates between bound methods and “unbound methods“ (functions) since in Python 3 there is no reliable way to extract the enclosing class from an “unbound method".

• For a bound method, it simply traverses the MRO, in a similar manner to that done in the accepted answer to an equivalent question for Python 2.
• For an “unbound method“, it relies on parsing its qualified name, which is available only from Python 3.3 and is quite reckless (if this feature is unnecessary it's probably best to remove this block of code and just return None instead).

Several useful comments prompted additional changes, as detailed in the edits section below, producing the following improvements:

• Limited handling for methods defined via descriptors, that aren't classified as ordinary methods or functions (for example, set.union, int.__add__ and int().__add__) and for built-in methods (for example set().union and io.BytesIO().__enter__).
• Handling of functools.partial objects.

The resulting function is:

def get_class_that_defined_method(meth):
    if isinstance(meth, functools.partial):
        return get_class_that_defined_method(meth.func)
    if inspect.ismethod(meth) or (inspect.isbuiltin(meth) and getattr(meth, '__self__', None) is not None and getattr(meth.__self__, '__class__', None)):
        for cls in inspect.getmro(meth.__self__.__class__):
            if meth.__name__ in cls.__dict__:
                return cls
        meth = getattr(meth, '__func__', meth)  # fallback to __qualname__ parsing
    if inspect.isfunction(meth):
        cls = getattr(inspect.getmodule(meth),
                      meth.__qualname__.split('.<locals>', 1)[0].rsplit('.', 1)[0],
                      None)
        if isinstance(cls, type):
            return cls
    return getattr(meth, '__objclass__', None)  # handle special descriptor objects

A small request

If you decide to use this implementation, and encounter any caveats, please comment and describe what happened.

---

The Full Version

“Unbound methods” are regular functions

First of all, it's worth noting the following change made in Python 3 (see Guido's motivation here):

The concept of “unbound methods” has been removed from the language. When referencing a method as a class attribute, you now get a plain function object.

This makes it practically impossible to reliably extract the class in which a certain “unbound method“ was defined unless it's bound to an object of that class (or of one of its subclasses).

Handling bound methods

So, let us first handle the “easier case“ in which we have a bound method. Note that the bound method must be written in Python, as described in inspect.ismethod's documentation.

def get_class_that_defined_method(meth):
    # meth must be a bound method
    if inspect.ismethod(meth):
        for cls in inspect.getmro(meth.__self__.__class__):
            if meth.__name__ in cls.__dict__:
                return cls
    return None  # not required since None would have been implicitly returned anyway

However, this solution is not perfect and has its perils, as methods can be assigned in runtime, rendering their name possibly different than that of the attribute that they are assigned to (see example below). This problem exists also in Python 2. A possible workaround would be to iterate over all of the class's attributes, looking for one whose identity is that of the specified method.

Handling “unbound methods“

Now that we got that out of the way, we can suggest a hack that tries to handle “unbound methods”. The hack, its rationale, and some discouragement words can be found in this answer. It relies on manually parsing the __qualname__ attribute, available only from Python 3.3, is highly unrecommended, but should work for simple cases:

def get_class_that_defined_method(meth):
    if inspect.isfunction(meth):
        return getattr(inspect.getmodule(meth),
                       meth.__qualname__.split('.<locals>', 1)[0].rsplit('.', 1)[0],
                       None)
    return None  # not required since None would have been implicitly returned anyway

Combining both approaches

Since inspect.isfunction and inspect.ismethod are mutually exclusive, combining both approaches into a single solution gives us the following (with added logging facilities for the upcoming examples):

def get_class_that_defined_method(meth):
    if inspect.ismethod(meth):
        print('this is a method')
        for cls in inspect.getmro(meth.__self__.__class__):
            if meth.__name__ in cls.__dict__:
                return cls
    if inspect.isfunction(meth):
        print('this is a function')
        return getattr(inspect.getmodule(meth),
                       meth.__qualname__.split('.<locals>', 1)[0].rsplit('.', 1)[0],
                       None)
    print('this is neither a function nor a method')
    return None  # not required since None would have been implicitly returned anyway

Execution example

>>> class A:
...     def a(self): pass
... 
>>> class B:
...     def b(self): pass
... 
>>> class C(A, B):
...     def a(self): pass
... 
>>> A.a
<function A.a at 0x7f13b58dfc80>
>>> get_class_that_defined_method(A.a)
this is a function
<class '__main__.A'>
>>>
>>> A().a
<bound method A.a of <__main__.A object at 0x7f13b58ca9e8>>
>>> get_class_that_defined_method(A().a)
this is a method
<class '__main__.A'>
>>>
>>> C.a
<function C.a at 0x7f13b58dfea0>
>>> get_class_that_defined_method(C.a)
this is a function
<class '__main__.C'>
>>>
>>> C().a
<bound method C.a of <__main__.C object at 0x7f13b58ca9e8>>
>>> get_class_that_defined_method(C().a)
this is a method
<class '__main__.C'>
>>>
>>> C.b
<function B.b at 0x7f13b58dfe18>
>>> get_class_that_defined_method(C.b)
this is a function
<class '__main__.B'>
>>>
>>> C().b
<bound method C.b of <__main__.C object at 0x7f13b58ca9e8>>
>>> get_class_that_defined_method(C().b)
this is a method
<class '__main__.B'>

So far, so good, but...

>>> def x(self): pass
... 
>>> class Z:
...     y = x
...     z = (lambda: lambda: 1)()  # this returns the inner function
...     @classmethod
...     def class_meth(cls): pass
...     @staticmethod
...     def static_meth(): pass
...
>>> x
<function x at 0x7f13b58dfa60>
>>> get_class_that_defined_method(x)
this is a function
<function x at 0x7f13b58dfa60>
>>>
>>> Z.y
<function x at 0x7f13b58dfa60>
>>> get_class_that_defined_method(Z.y)
this is a function
<function x at 0x7f13b58dfa60>
>>>
>>> Z().y
<bound method Z.x of <__main__.Z object at 0x7f13b58ca9e8>>
>>> get_class_that_defined_method(Z().y)
this is a method
this is neither a function nor a method
>>>
>>> Z.z
<function Z.<lambda>.<locals>.<lambda> at 0x7f13b58d40d0>
>>> get_class_that_defined_method(Z.z)
this is a function
<class '__main__.Z'>
>>>
>>> Z().z
<bound method Z.<lambda> of <__main__.Z object at 0x7f13b58ca9e8>>
>>> get_class_that_defined_method(Z().z)
this is a method
this is neither a function nor a method
>>>
>>> Z.class_meth
<bound method type.class_meth of <class '__main__.Z'>>
>>> get_class_that_defined_method(Z.class_meth)
this is a method
this is neither a function nor a method
>>>
>>> Z().class_meth
<bound method type.class_meth of <class '__main__.Z'>>
>>> get_class_that_defined_method(Z().class_meth)
this is a method
this is neither a function nor a method
>>>
>>> Z.static_meth
<function Z.static_meth at 0x7f13b58d4158>
>>> get_class_that_defined_method(Z.static_meth)
this is a function
<class '__main__.Z'>
>>>
>>> Z().static_meth
<function Z.static_meth at 0x7f13b58d4158>
>>> get_class_that_defined_method(Z().static_meth)
this is a function
<class '__main__.Z'>

Final touches

• The outcome generated by x and Z.y can be partially fixed (to return None) by verifying that the returned value is a class, before actually returning it.

• The outcome generated by Z().z can be fixed by falling back to parsing the function's __qualname__ attribute (the function can be extracted via meth.__func__).

• The outcome generated by Z.class_meth and Z().class_meth is incorrect because accessing a class method always returns a bound method, whose __self__ attribute is the class itself, rather than its object. Thus, further accessing the __class__ attribute on top of that __self__ attribute doesn't work as expected:

  >>> Z().class_meth
  <bound method type.class_meth of <class '__main__.Z'>>
  >>> Z().class_meth.__self__
  <class '__main__.Z'>
  >>> Z().class_meth.__self__.__class__
  <class 'type'>
  

  This can be fixed by checking whether the method's __self__ attribute returns an instance of type. However, this might be confusing when our function is invoked against methods of a metaclass, so we'll leave it as is for now.

Here is the final version:

def get_class_that_defined_method(meth):
    if inspect.ismethod(meth):
        for cls in inspect.getmro(meth.__self__.__class__):
            if meth.__name__ in cls.__dict__:
                return cls
        meth = meth.__func__  # fallback to __qualname__ parsing
    if inspect.isfunction(meth):
        cls = getattr(inspect.getmodule(meth),
                      meth.__qualname__.split('.<locals>', 1)[0].rsplit('.', 1)[0],
                      None)
        if isinstance(cls, type):
            return cls
    return None  # not required since None would have been implicitly returned anyway

Surprisingly, this also fixes the outcome of Z.class_meth and Z().class_meth which now correctly return Z. This is because the __func__ attribute of a class method returns a regular function whose __qualname__ attribute may be parsed:

>>> Z().class_meth.__func__
<function Z.class_meth at 0x7f13b58d4048>
>>> Z().class_meth.__func__.__qualname__
'Z.class_meth'

---

EDIT:

As per the issue raised by Bryce, it's possible to handle method_descriptor objects, like set.union, and wrapper_descriptor objects, like int.__add__, merely by returning their __objclass__ attribute (introduced by PEP-252), if such exists:

if inspect.ismethoddescriptor(meth):
    return getattr(meth, '__objclass__', None)

However, inspect.ismethoddescriptor returns False for the respective instance method objects, i.e. for set().union and for int().__add__:

• Since int().__add__.__objclass__ returns int, the above if clause may be relinquished in order to solve the problem for int().__add__. Unfortunately, this doesn't address the matter of set().union, for which no __objclass__ attribute is defined. In order to avoid an AttributeError exception in such a case, the __objclass__ attribute isn't accessed directly, but rather via the getattr function.

---

EDIT:

As per the issue raised by x-yuri, it seems that our function fails to handle the method io.BytesIO().__enter__ since inspect doesn't identify it as a method, but rather as a built-in:

>>> inspect.ismethod(io.BytesIO().__enter__)
False
>>> inspect.isbuiltin(io.BytesIO().__enter__)
True

This is the same issue encountered above in regard to set().union:

>>> inspect.ismethod(set().union)
False
>>> inspect.isbuiltin(set().union)
True

Other than this peculiarity, we can handle such methods as ordinary methods and extract the defining class by traversing the MRO.

However, just to be on the safe side, we shall include an extra layer of protection and verify that the __self__ attribute of such methods, if defined, isn't None and that the __class__ attribute of that __self__ object, if defined, isn't None as well:

if inspect.ismethod(meth) or (inspect.isbuiltin(meth) and getattr(meth, '__self__', None) and getattr(meth.__self__, '__class__', None)):
    # ordinary method handling

Alas, this simple test fails for set().union because bool(set().union.__self__) evaluates to False since set().union.__self__ returns the empty set. Thus, an explicit test against None is required, producing the following fix:

if inspect.ismethod(meth) or (inspect.isbuiltin(meth) and getattr(meth, '__self__', None) is not None and getattr(meth.__self__, '__class__', None)):
    # ordinary method handling

A minor additional patch is advised in order to avoid a possible AttributeError exception when accessing the __func__ attribute during fallback to __qualname__ parsing. This is required since while the __func__ attribute is guaranteed to exist for an ordinary method, it's not necessarily defined for one of the type builtin_function_or_method, such as io.BytesIO().__enter__ and set().union.

def get_class_that_defined_method(meth):
    if inspect.ismethod(meth) or (inspect.isbuiltin(meth) and getattr(meth, '__self__', None) is not None and getattr(meth.__self__, '__class__', None)):
        for cls in inspect.getmro(meth.__self__.__class__):
            if meth.__name__ in cls.__dict__:
                return cls
        meth = getattr(meth, '__func__', meth)  # fallback to __qualname__ parsing
    if inspect.isfunction(meth):
        cls = getattr(inspect.getmodule(meth),
                      meth.__qualname__.split('.<locals>', 1)[0].rsplit('.', 1)[0],
                      None)
        if isinstance(cls, type):
            return cls
    return getattr(meth, '__objclass__', None)  # handle special descriptor objects

---

EDIT:

As per the suggestion put forward by user1956611, it's possible to handle partial objects by introducing a recursive call to seek out the original callable with which the partial object was created:

if isinstance(meth, functools.partial):
    return get_class_that_defined_method(meth.func)

Answer 2

The point you appear to be missing is, in Python 3 the "unbound method" type has entirely disappeared -- a method, until and unless it's bound, is just a function, without the weird "type-checking" unbound methods used to perform. This makes the language simpler!

To wit...:

>>> class X:
...   def Y(self): pass
... 
>>> type(X.Y)
<class 'function'>

and voila -- one less subtle concept and distinction to worry about. Such simplifications are the core advantage of Python 3 wrt Python 2, which (over the years) had been accumulating so many subtleties that it was in danger (if features kept being added to it) of really losing its status as a simple language. With Python 3, simplicity is back!-)

Answer 3

Since python 3.6 you could accomplish what you are describing using a decorator that defines a __set_name__ method. The documentation states that object.__set_name__ is called when the class is being created.

Here is an example that decorates a method "in order to register it in a list of methods that serve a particular purpose":

>>> class particular_purpose: 
...     def __init__(self, fn): 
...         self.fn = fn 
...      
...     def __set_name__(self, owner, name): 
...         owner._particular_purpose.add(self.fn) 
...          
...         # then replace ourself with the original method 
...         setattr(owner, name, self.fn) 
...  
... class A: 
...     _particular_purpose = set() 
...  
...     @particular_purpose 
...     def hello(self): 
...         return "hello" 
...  
...     @particular_purpose 
...     def world(self): 
...         return "world" 
...  
>>> A._particular_purpose
{<function __main__.A.hello(self)>, <function __main__.A.world(self)>}
>>> a = A() 
>>> for fn in A._particular_purpose: 
...     print(fn(a)) 
...                                                                                                                                     
world
hello

Note that this question is very similar to Can a Python decorator of an instance method access the class? and therefore my answer as well to the answer I provided there.

Answer 4

A small extension for python 3.6 (python 2.7 worked fine) to the great answer of https://stackoverflow.com/a/25959545/4013571

def get_class_that_defined_method(meth):
    if inspect.ismethod(meth):
        for cls in inspect.getmro(meth.__self__.__class__):
            if cls.__dict__.get(meth.__name__) is meth:
                return cls
        meth = meth.__func__  # fallback to __qualname__ parsing
    if inspect.isfunction(meth):
        class_name = meth.__qualname__.split('.<locals>', 1)[0].rsplit('.', 1)[0]
        try:
            cls = getattr(inspect.getmodule(meth), class_name)
        except AttributeError:
            cls = meth.__globals__.get(class_name)
        if isinstance(cls, type):
            return cls
    return None  # not required since None would have been implicitly returned anyway

I found the following adjustment was required for doctest

        except AttributeError:
            cls = meth.__globals__.get(class_name)

As for some reason, when using nose the inspect.getmodule(meth) didn't contain the defining class