How does a python method automatically receive 'self' as the first argument?

Question

Consider this example of a strategy pattern in Python (adapted from the example here). In this case the alternate strategy is a function.

class StrategyExample(object):
    def __init__(self, strategy=None) :
        if strategy:
             self.execute = strategy

    def execute(*args):
        # I know that the first argument for a method
        # must be 'self'. This is just for the sake of
        # demonstration 
        print locals()

#alternate strategy is a function
def alt_strategy(*args):
    print locals()

Here are the results for the default strategy.

>>> s0 = StrategyExample()
>>> print s0
<__main__.StrategyExample object at 0x100460d90>
>>> s0.execute()
{'args': (<__main__.StrategyExample object at 0x100460d90>,)}

In the above example s0.execute is a method (not a plain vanilla function) and hence the first argument in args, as expected, is self.

Here are the results for the alternate strategy.

>>> s1 = StrategyExample(alt_strategy)
>>> s1.execute()
{'args': ()}

In this case s1.execute is a plain vanilla function and as expected, does not receive self. Hence args is empty. Wait a minute! How did this happen?

Both the method and the function were called in the same fashion. How does a method automatically get self as the first argument? And when a method is replaced by a plain vanilla function how does it not get the self as the first argument?

The only difference that I was able to find was when I examined the attributes of default strategy and alternate strategy.

>>> print dir(s0.execute)
['__cmp__', '__func__', '__self__', ...]
>>> print dir(s1.execute)
# does not have __self__ attribute

Does the presence of __self__ attribute on s0.execute (the method), but lack of it on s1.execute (the function) somehow account for this difference in behavior? How does this all work internally?

Answer 1

You need to assign an unbound method (i.e. with a self parameter) to the class or a bound method to the object.

Via the descriptor mechanism, you can make your own bound methods, it's also why it works when you assign the (unbound) function to a class:

my_instance = MyClass()    
MyClass.my_method = my_method

When calling my_instance.my_method(), the lookup will not find an entry on my_instance, which is why it will at a later point end up doing this: MyClass.my_method.__get__(my_instance, MyClass) - this is the descriptor protocol. This will return a new method that is bound to my_instance, which you then execute using the () operator after the property.

This will share method among all instances of MyClass, no matter when they were created. However, they could have "hidden" the method before you assigned that property.

If you only want specific objects to have that method, just create a bound method manually:

my_instance.my_method = my_method.__get__(my_instance, MyClass)

For more detail about descriptors (a guide), see here.

Answer 2

You can read the full explanation here in the python reference, under "User defined methods". A shorter and easier explanation can be found in the python tutorial's description of method objects:

If you still don’t understand how methods work, a look at the implementation can perhaps clarify matters. When an instance attribute is referenced that isn’t a data attribute, its class is searched. If the name denotes a valid class attribute that is a function object, a method object is created by packing (pointers to) the instance object and the function object just found together in an abstract object: this is the method object. When the method object is called with an argument list, a new argument list is constructed from the instance object and the argument list, and the function object is called with this new argument list.

Basically, what happens in your example is this:

• a function assigned to a class (as happens when you declare a method inside the class body) is... a method.
  • When you access that method through the class, eg. StrategyExample.execute you get an "unbound method": it doesn't "know" to which instance it "belongs", so if you want to use that on an instance, you would need to provide the instance as the first argument yourself, eg. StrategyExample.execute(s0)
  • When you access the method through the instance, eg. self.execute or s0.execute, you get a "bound method": it "knows" which object it "belongs" to, and will get called with the instance as the first argument.
• a function that you assign to an instance attribute directly however, as in self.execute = strategy or even s0.execute = strategy is... just a plain function (contrary to a method, it doesn't pass via the class)

To get your example to work the same in both cases:

• either you turn the function into a "real" method: you can do this with types.MethodType:

  self.execute = types.MethodType(strategy, self, StrategyExample)
  

  (you more or less tell the class that when execute is asked for this particular instance, it should turn strategy into a bound method)

• or - if your strategy doesn't really need access to the instance - you go the other way around and turn the original execute method into a static method (making it a normal function again: it won't get called with the instance as the first argument, so s0.execute() will do exactly the same as StrategyExample.execute()):

  @staticmethod
  def execute(*args):
      print locals()
  

Answer 3

The method is a wrapper for the function, and calls the function with the instance as the first argument. Yes, it contains a __self__ attribute (also im_self in Python prior to 3.x) that keeps track of which instance it is attached to. However, adding that attribute to a plain function won't make it a method; you need to add the wrapper. Here is how (although you may want to use MethodType from the types module to get the constructor, rather than using type(some_obj.some_method).

The function wrapped, by the way, is accessible through the __func__ (or im_func) attribute of the method.

Answer 4

When you do self.execute = strategy you set the attribute to a plain method:

>>> s = StrategyExample()
>>> s.execute
<bound method StrategyExample.execute of <__main__.StrategyExample object at 0x1dbbb50>>
>>> s2 = StrategyExample(alt_strategy)
>>> s2.execute
<function alt_strategy at 0x1dc1848>

A bound method is a callable object that calls a function passing an instance as the first argument in addition to passing through all arguments it was called with.

See: Python: Bind an Unbound Method?