Your problem can be reproduced in Python and without (very) dirty tricks:
class A:
pass
A.works = lambda x: abs(1)
A.dont = abs
A().works() # works
A().dont() # error
The difference is, that abs
is a builtin-function of type PyCFunctionObject
, while lambda is of type PyFunctionObject
(a C is missing compared to PyCFunction...
).
Those cfunctions just cannot be used for patching, see for example PEP-579.
The problem, which is also mentioned in PEP-579, that cython-functions are PyCFunctions and thus are seen as builtin-functions:
%%cython
def foo():
pass
>>> type(foo)
builtin_function_or_method
That means, you cannot use the Cython-function directly for monkey patching, but have to wrap them into a lambda or similar, as you already do. One should not worry about performance, because due to method-lookup there is already overhead, having a little bit more of it doesn't change things dramatically.
I must confess, I don't know why this is the case (historically). But in the current code (Python3.8) you can easily find the crucial line in _PyObject_GetMethod
, which makes the difference:
descr = _PyType_Lookup(tp, name);
if (descr != NULL) {
Py_INCREF(descr);
if (PyFunction_Check(descr) || # HERE WE GO
(Py_TYPE(descr) == &PyMethodDescr_Type)) {
meth_found = 1;
} else {
After looking-up the function (here descr
) in the dictionary _PyType_Lookup(tp, name)
, method_found
is set to 1 only if the found function is of type PyFunction
, which is not the case for builtin-PyCFunctions. Thus abs
and Co aren't seen as methods, but stay kind of "staticmethod".
The easiest way to find a starting point for the investigation, is to inspect the produced opcode for:
import dis
def f():
a.fun()
dis.dis(f)
i.e. the following opcode(and seems to have changed since Python3.6):
2 0 LOAD_GLOBAL 0 (a)
2 LOAD_METHOD 1 (fun) #HERE WE GO
4 CALL_METHOD 0
6 POP_TOP
8 LOAD_CONST 0 (None)
10 RETURN_VALUE
We can inspect the corresponding part in ceval.c:
TARGET(LOAD_METHOD) {
/* Designed to work in tamdem with CALL_METHOD. */
PyObject *name = GETITEM(names, oparg);
PyObject *obj = TOP();
PyObject *meth = NULL;
int meth_found = _PyObject_GetMethod(obj, name, &meth);
....
and let the gdb take us from there.
As @user2357112 has rightly pointed out, if PyCFunctionObject would support the descriptor protocol (more precisely to provide tp_descr_get
), even after meth_found = 0;
it still would have a fall back which would lead to the desired behavior. PyFunctionObject does provide it, but PyCFunctionObject does not.
Older versions used LOAD_ATTR
+CALL_FUNCTION
for a.fun()
and in order to work, function-objects had to support the descriptor protocol. But now it seems not to be mandatory.
My quick tests with extending the crucial line with PyCFunction_Check(descr)
to:
if (PyFunction_Check(descr) || PyCFunction_Check(descr) ||
(Py_TYPE(descr) == &PyMethodDescr_Type))
have shown, that then also builtin-methods would work as bound-methods (at least for the case above). But this would probably break something - I didn't run any bigger tests.
However, as @user2357112 mentioned (thanks again), this would lead to a inconsistency, because meth = foo.bar
still uses LOAD_ATTR
and thus depends on the descriptor protocol.
Recommendation: I found this answer helpful in understanding, what is going on in the case of LOAD_ATTR
.