Calling function with dynamic args

Question

I'm creating a function that takes in a callback function as an argument. I want to be able to use it like this:

def callback1(result, found_index):
    # do stuffs

def callback2(result):
    # do same stuffs even though it's missing the found_index parameter

somefunct(callback1)
somefunct(callback2)

# somefunct calls the callback function like this:
def somefunct(callback):

    # do stuffs, and assign result and found_index


    callback(result, found_index) # should not throw error

For context, I am somewhat trying to replicate how javascript's callback functions work for the .forEach function on arrays. You can make a function that takes in only the array item on that specific iteration, or the array item and index, or even the array item, index, and original array:

let some_array = ["apple", "orange", "banana"];

function callback1(value, index) {
    console.log(`Item at index ${index}: ${value}`);
}

function callback2(value) {
    console.log(`Value: ${value}`);
}

some_array.forEach(callback1); // runs with no errors
some_array.forEach(callback2); // runs with no errors

Furthermore, I don't want the callback function to force the * operator, but also allow them to use it if needed. Thank you, wonderful people of python.

Just make `found_index` a keyword argument. As a more general note, it's not a great idea to try to replicate another language's patterns and idioms in Python. Write pythonic code. — jordanm, Mar 04 '20 at 19:35
Why not make `callback2` accept two parameters, and simply ignore the second? You can either always call the callback in the same way, and let the callback decide what to do with its arguments, or you can add [complicated introspection](https://stackoverflow.com/questions/847936/how-can-i-find-the-number-of-arguments-of-a-python-function) to `somefunct` which which determines how it should treat the callback. The former seems much more maintainable to me. — Brian, Mar 04 '20 at 19:46
Second @Brian 's suggestion. I often name parameters that exist solely to match an interface something like `ignored` or `placeholder` to show that the given param isn't used and is just there to take up the space. Except in C++, in which case I just declare the type and no name at all which suppresses the 'unused named parameter` warning. — Mark Storer, Mar 04 '20 at 20:09
@Brian the callback function takes in more than 2 parameters in my actual code. I wanted to replicate JavaScript's array.reduce function with a custom read-only dictionary instead. This leads to running the callback with 5 arguments; the current value; the value in the dictionary; the key; the index; and the dictionary that's being used. I don't want to have my users write 5 arguments for their lambda expressions or callback functions — Fluffy Doggo, Mar 04 '20 at 20:43
@FluffyDoggo You should perhaps take this as a sign that you're doing something wrong here. Abstractions don't always translate directly between languages. Why not expose your "read only dict" as an iterable, and then use the existing [`functools.reduce`](https://docs.python.org/3/library/functools.html#functools.reduce)? — Brian, Mar 04 '20 at 20:46
@Brian although that would seem to work fine for instances you won't need the index, there are those times when an index is a required part of their callback function, and that also deflects a little from the main question of specifying a certain number of parameters. Would it be better to instead specify the arguments in an object and have the callback function unpack it? — Fluffy Doggo, Mar 04 '20 at 20:53
@FluffyDoggo This seems like an [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). You're asking how to implement a rather obscure interface; I'm saying that using the abstractions native to Python will make this much easier both for yourself and for your users. — Brian, Mar 04 '20 at 21:15

Brian · Answer 1 · 2020-03-04T20:12:01.393

1

The simplest approach would be to unify the signatures of your callbacks. Let's say you defined your forEach function as follows

def forEach(iterable, callback):
    for index, elem in enumerate(iterable):
        callback(elem, index)

You could then define Python analogs of the callack1 and callback2 Javascript functions as

def callback1(value, index):
    print(f"Item at index {index}: {value}")

def callback2(value, _index):
    print(f"Value: {value})

Rather than performing any complicated parameter-count-reasoning, exception handling, or dynamic dispatch within forEach, we delegate the decision of how to handle the value and index arguments to the callbacks themselves. If you need to adapt a single-parameter callback to work with forEach, you could simply use a wrapper lambda that discards the second argument:

forEach(some_iterable, lambda value, _index: callback(value))

However, at this point, you just have an obfuscated for loop, which would be much more cleanly expressed as

for elem in some_iterable:
    callback(elem)

edited Mar 04 '20 at 20:12

answered Mar 04 '20 at 19:59

Brian

2,948
4
13
23

I honestly don't understand what you did with the lambda value – solid.py Mar 04 '20 at 20:06
Is it from a specific python doc chapter? If yes please link this. I would like to know – solid.py Mar 04 '20 at 20:07
@hitter Could you elaborate on your confusion? The lambda simply acts as a callable which accepts two arguments, which then ignores the second and passes only the first to the `callback`. Regarding your second comment, are you looking for the section on [lambda expressions](https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions)? – Brian Mar 04 '20 at 20:07

score 1 · Accepted Answer · answered Mar 04 '20 at 20:52

_{(Posting this separately since it's fundamentally different to my other answer.)}

If you need to pass a lot of values to some callbacks, without requiring other callbacks to declare a lot of unused parameters, a neat solution is to encapsulate all of those values in a single object. You can use collections.namedtuple to define a value type with named attributes, and then the callback can take one parameter and decide which attributes to use.

from collections import namedtuple
SomeFunctionResult = namedtuple('SomeFunctionResult', 'foo bar baz qux quz')

def some_function(callback):
    result = SomeFunctionResult('foo', 'bar', 'baz', 'qux', 'quz')
    callback(result)

Example:

>>> some_function(lambda r: print(r.foo, r.bar))
foo bar
>>> some_function(lambda r: print(r.baz, r.qux, r.quz))
baz qux quz

The downside is that this makes some_function less usable with existing functions which might expect to receive foo directly, rather than an object with a foo attribute. In that case, you have to write some_function(lambda r: blah(r.foo)) which is not as neat as some_function(blah).

this would work well with only calling the callback once, but what about calling the callback multiple times from, for example, a for loop? How would you structure it? — Fluffy Doggo, Mar 04 '20 at 21:09
@FluffyDoggo I don't see the problem with putting this directly in a loop. In principle you could compose them at the start of the function like `new_callback = lambda *args: old_callback(SomeFunctionResult(*args))` and then call `new_callback` normally within the loop. But creating the `SomeFunctionResult` object in the loop directly seems like it should be fine anyway. — kaya3, Mar 04 '20 at 22:05

score 0 · Answer 3 · answered Mar 04 '20 at 20:04

0

In this case, it is easier to ask for forgiveness than permission.

def some_function(callback):
    result = 'foo'
    found_index = 5
    try:
        callback(result, found_index)
    except TypeError:
        callback(result)

Example:

>>> some_function(print)
foo 5
>>> some_function(lambda x: print(x))
foo

answered Mar 04 '20 at 20:04

kaya3

31,244
3
32
61

Exceptions should be the exception, not the rule. It's right there in the name. My experience with other languages (new to Python, old hand at C++, Java) leads me to expect exceptions to not be cheap, and that using them as part of the regular control flow of your program will utterly destroy its performance. – Mark Storer Mar 04 '20 at 20:14
@MarkStorer That is usually my opinion too, but in this case the alternative (if you actually want the OP's behaviour, rather than an argument for why the OP shouldn't want it) is runtime inspection of the function's signature, which is a lot messier. It's also quite standard to use `try`/`catch` this way - the official docs call this style "common", "clean and fast", so this is a losing battle, I'm afraid. There are far less justifiable uses in the Python standard library and popular third-party libraries, e.g. [numpy](https://github.com/numpy/numpy/blob/master/numpy/__init__.py#L113). – kaya3 Mar 04 '20 at 20:17
Here's the built-in `json` module using `except KeyError` instead of using `if` to test whether a key is in a dictionary: https://github.com/python/cpython/blob/master/Lib/json/encoder.py#L48 – kaya3 Mar 04 '20 at 20:27
"All the cool kids are doing it" is not a valid justification for crappy behavior. Or in a more pop cultural way "I realize the council has made a decision, but because it's a stupid-ass decision, I have elected to ignore it!" (Nick Fury: The Avengers) – Mark Storer Mar 05 '20 at 20:03
@MarkStorer Woah, hold on now - using a coding style you (and I) have a negative opinion of is not "crappy behaviour". I am on your side in this debate, but those on the other side include highly-respected experts such as [core Python developer Raymond Hettinger](https://stackoverflow.com/a/16138864/12299000) who says, *"In the Python world, using exceptions for flow control is common and normal"*, and the linked answer is not unsupported by arguments in favour of the style. I think it is better to use `if`/`else` for normal control flow, but also coding style rules are never universal. – kaya3 Mar 05 '20 at 20:54
In any case, I didn't cite those examples to justify using `try`/`except` here - my justification is that *"the alternative ... is a lot messier"* in this specific situation. I pointed at those examples mainly because you mentioned you are new to Python, so you might not have been aware of the fact that using `try`/`except` for control flow is considered normal and acceptable in the Python community; so if you want to make the argument against it every time you see it, you'll be making the argument quite a lot. – kaya3 Mar 05 '20 at 20:58
My brief tests show that a trivial `raise` is 3-4 times slower than an equally trivial `if`. If I wanted to pretend to be reasonable, I might concede that "vastly less efficient" isn't the same as "crappy". At a snarkier moment, I might point out that the two are still pretty close. ;) – Mark Storer Mar 09 '20 at 13:31

score 0 · Answer 4 · answered Mar 04 '20 at 20:12

this is the modified python code snippet you have provided that produces error , this works with no problem , you just have to unify the callback arguments number and type for each callback function called within the main function and define somefunc before calling it .

def callback1(result, found_index):
    # do stuffs 
    result="overridden result in callback 1"
    found_index ="overridden found_index in callback 1"
    print(result,found_index)

def callback2(result,found_index):
    # do same stuffs even though it's missing the found_index parameter
    result="overridden result in callback 2"
    print(result,found_index)

    # somefunct calls the callback function like this:
def somefunct(callback):
    # do stuffs, and assign result and found_index
    result = "overridden result in somefunct"
    found_index = "overridden index in somefunct"
    callback(result, found_index) # NOW it should not throw error as the callback is fed with the 2 arguments used in callback1 and ignored in callback2

somefunct(callback1)
somefunct(callback2)

score -1 · Answer 5 · answered Mar 04 '20 at 19:49

-1

use optional arguments and check how much elemnts returned, sort of switch case: https://linux.die.net/diveintopython/html/power_of_introspection/optional_arguments.html

answered Mar 04 '20 at 19:49

yehezkel horoviz

189
1
10

Calling function with dynamic args

5 Answers5