Convert builtin function type to method type (in Python 3)

Question

Consider a simple function like

def increment(self):
    self.count += 1

which is run through Cython and compiled into an extension module. Suppose now I'd like to make this function a method on a class. For example:

class Counter:
    def __init__(self):
        self.count = 0

from compiled_extension import increment
Counter.increment = increment

Now this will not work, as the calling convention at the C level will be broken. For example:

>>> c = Counter()
>>> c.increment()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: increment() takes exactly one argument (0 given)

But in Python 2, we can convert the function to an unbound method by doing:

Counter.increment = types.MethodType(increment, None, Counter)

How can I accomplish this same thing in Python 3?

One simple way is to use a slim wrapper:

from functools import wraps
def method_wraper(f):
    def wrapper(*args, **kwargs):
        return f(*args, **kwargs)
    return wraps(f)(wrapper)

Counter.increment = method_wrapper(increment)

Is there a more efficient way to do it?

I had this problem trying to build a Heapq class using the ugly `heapq` module. Your solution is good. It can be done in one line, but has the same efficiency: `def method_wraper(f): return functools.wraps(f)(lambda *a, **kw: f(*a, **kw))` — JBernardo, May 24 '12 at 16:50
...Interesting, the assignment seems to work fine if the function is defined within the same module (unbound method that gets assigned to a class and bound on instantiation). So it's only a problem with C extensions, or with functions in different modules? Anyway, you might want to check out http://stackoverflow.com/questions/7490879/python3-bind-method-to-class-instance-with-get-it-works-but-why, which could help you a little as well. — JAB, May 24 '12 at 17:01
Also, http://docs.python.org/py3k/howto/descriptor.html#functions-and-methods "the actual C implementation of PyMethod_Type in Objects/classobject.c is a single object with two different representations depending on whether the im_self field is set or is NULL (the C equivalent of None)." Which makes it seem like this issue shouldn't be occurring at all, unless Python somehow doesn't update that field directly for an object's methods when instantiating the object. — JAB, May 24 '12 at 17:10
The problem is with that builtin functions and python functions are handled differently by the runtime, nothing to do with being in a different module. The comment about PyMethod_Type doesn't really apply as it represents a method defined in python, not builtin functions/methods. — bdew, Jun 20 '12 at 12:22
Shouldn't it read "convert the function to a *bound* method" ? — ederag, Sep 18 '16 at 08:04

score 4 · Answer 1 · edited May 23 '17 at 12:08

First thing is getting the names correctly:

>>> def increment(obj):
...     obj.count += 1
...
>>> class A(object):
...     def __init__(self):
...         self.count = 0
...
>>> o = A()
>>> o.__init__
<bound method A.__init__ of <__main__.A object at 0x0000000002766EF0>>
>>> increment
<function increment at 0x00000000027797C8>

So proper names are functions and bound methods. Now you can look for how to Bind an Unbound Method and you will probably end up reading about descriptors:

In general, a descriptor is an object attribute with "binding behavior", one whose attribute access has been overridden by methods in the descriptor protocol. Those methods are __get__, __set__, and __delete__. If any of those methods are defined for an object, it is said to be a descriptor.

You can easily transform function to method by just using different invocation of __get__

>>> increment.__get__(None, type(None))
<function increment at 0x00000000027797C8>
>>> increment.__get__(o, type(o))
<bound method A.increment of <__main__.A object at 0x00000000027669B0>>

And it works like a charm:

>>> o = A()
>>> increment.__get__(None, type(None))(o)
>>> o.count
1
>>> increment.__get__(o, type(o))()
>>> o.count
2

You can easily add these newly bounded methods to objects:

def increment(obj):
    obj.count += 1

def addition(obj, number):
    obj.count += number

class A(object):
    def __init__(self):
        self.count = 0

o = A()
o.inc = increment.__get__(o)
o.add = addition.__get__(o)
print(o.count) # 0
o.inc()
print(o.count) # 1
o.add(5)
print(o.count) # 6

Or create your own descriptor that will will convert function to bound method:

class BoundMethod(object):
    def __init__(self, function):
        self.function = function

    def __get__(self, obj, objtype=None):
        print('Getting', obj, objtype)
        return self.function.__get__(obj, objtype)

class B(object):
    def __init__(self):
        self.count = 0

    inc = BoundMethod(increment)
    add = BoundMethod(addition)


o = B()
print(o.count) # 0
o.inc()
# Getting <__main__.B object at 0x0000000002677978> <class '__main__.B'>
print(o.count) # 1
o.add(5) 
# Getting <__main__.B object at 0x0000000002677978> <class '__main__.B'>
print(o.count) # 6

And you also can see that this is nicely consistent with function/bound method principles:

Class dictionaries store methods as functions. In a class definition, methods are written using def and lambda, the usual tools for creating functions. The only difference from regular functions is that the first argument is reserved for the object instance. By Python convention, the instance reference is called self but may be called this or any other variable name.

To support method calls, functions include the __get__() method for binding methods during attribute access. This means that all functions are non-data descriptors which return bound or unbound methods depending whether they are invoked from an object or a class.

And functions becomes bound method during instance initialization:

>>> B.add
# Getting None <class '__main__.B'>
<function addition at 0x00000000025859C8>
>>> o.add
# Getting <__main__.B object at 0x00000000030B1128> <class '__main__.B'>
<bound method B.addition of <__main__.B object at 0x00000000030B1128>>

Normally a built-in type in CPython uses a `method_descriptor`, which gets bound as `builtin_function_or_method`, e.g. `str.upper.__get__('a').__self__ == 'a'`. The `builtin_function_or_method` itself is *not* a descriptor. The latter is what Cython creates, so the OP is looking for a way to wrap it in a descriptor such as [`partialmethod`](https://docs.python.org/3/library/functools.html#functools.partialmethod) (3.4+). — Eryk Sun, Dec 17 '14 at 16:38

score 0 · Answer 2 · answered Jul 17 '14 at 20:47

0

Import the extension like this:

import compiled_extension

In your class you write:

def increment: return compiled_extension.increment()

This seems more readable and might be more efficient.

answered Jul 17 '14 at 20:47

panpog

11
2

Convert builtin function type to method type (in Python 3)

2 Answers2

Linked