70

I would like to know if its possible to control Python function definition based on global settings (e.g. OS). Example:

@linux
def my_callback(*args, **kwargs):
    print("Doing something @ Linux")
    return

@windows
def my_callback(*args, **kwargs):
    print("Doing something @ Windows")
    return

Then, if someone is using Linux, the first definition of my_callback will be used and the second will be silently ignored.

Its not about determining the OS, its about function definition / decorators.

Pedro
  • 1,009
  • 6
  • 14
  • 10
    That second decorator is equivalent to `my_callback = windows()` - so the name `my_callback` *will* be overwritten, regardless of what the decorator might do. The only way the Linux version of the function could end up in that variable is if `windows()` returned it - but the function has no way of knowing about the Linux version. I think the more typical way of accomplishing this is to have the OS-specific function definitions in separate files, and conditionally `import` only one of them. – jasonharper Feb 16 '20 at 01:18
  • 7
    You may want to take a look at the interface of [`functools.singledispatch`](https://docs.python.org/3/library/functools.html#functools.singledispatch), which does something similar to what you want. There, the `register` decorator knows about the dispatcher (because it's an attribute of the dispatch function, and specific to that particular dispatcher), so it can return the dispatcher and avoid the problems with your approach. – user2357112 supports Monica Feb 16 '20 at 01:47
  • 5
    While what you are trying to do here is admirable, it's worth mentioning that most of CPython follows a standard "check platform in an if/elif/else"; for example, [`uuid.getnode()`](https://github.com/python/cpython/blob/0d860dd43c72dc7046a5d18fc72d495cadd4a2df/Lib/uuid.py#L683). (That said, Todd's answer here is quite good.) – Brad Solomon Feb 16 '20 at 05:53

5 Answers5

61

If the goal is to have the same sort of effect in your code that #ifdef WINDOWS / #endif has.. here's a way to do it (I'm on a mac btw).

Simple Case, No Chaining

>>> def _ifdef_decorator_impl(plat, func, frame):
...     if platform.system() == plat:
...         return func
...     elif func.__name__ in frame.f_locals:
...         return frame.f_locals[func.__name__]
...     else:
...         def _not_implemented(*args, **kwargs):
...             raise NotImplementedError(
...                 f"Function {func.__name__} is not defined "
...                 f"for platform {platform.system()}.")
...         return _not_implemented
...             
...
>>> def windows(func):
...     return _ifdef_decorator_impl('Windows', func, sys._getframe().f_back)
...     
>>> def macos(func):
...     return _ifdef_decorator_impl('Darwin', func, sys._getframe().f_back)

So with this implementation you get same syntax you have in your question.

>>> @macos
... def zulu():
...     print("world")
...     
>>> @windows
... def zulu():
...     print("hello")
...     
>>> zulu()
world
>>> 

What the code above is doing, essentially, is assigning zulu to zulu if the platform matches. If the platform doesn't match, it'll return zulu if it was previously defined. If it wasn't defined, it returns a placeholder function that raises an exception.

Decorators are conceptually easy to figure out if you keep in mind that

@mydecorator
def foo():
    pass

is analogous to:

foo = mydecorator(foo)

Here's an implementation using a parameterized decorator:

>>> def ifdef(plat):
...     frame = sys._getframe().f_back
...     def _ifdef(func):
...         return _ifdef_decorator_impl(plat, func, frame)
...     return _ifdef
...     
>>> @ifdef('Darwin')
... def ice9():
...     print("nonsense")

Parameterized decorators are analogous to foo = mydecorator(param)(foo).

I've updated the answer quite a bit. In response to comments, I've expanded its original scope to include application to class methods and to cover functions defined in other modules. In this last update, I've been able to greatly reduce the complexity involved in determining if a function has already been defined.

[A little update here... I just couldn't put this down - it's been a fun exercise] I've been doing some more testing of this, and found it works generally on callables - not just ordinary functions; you could also decorate class declarations whether callable or not. And it supports inner functions of functions, so things like this are possible (although probably not good style - this is just test code):

>>> @macos
... class CallableClass:
...     
...     @macos
...     def __call__(self):
...         print("CallableClass.__call__() invoked.")
...     
...     @macos
...     def func_with_inner(self):
...         print("Defining inner function.")
...         
...         @macos
...         def inner():
...             print("Inner function defined for Darwin called.")
...             
...         @windows
...         def inner():
...             print("Inner function for Windows called.")
...         
...         inner()
...         
...     @macos
...     class InnerClass:
...         
...         @macos
...         def inner_class_function(self):
...             print("Called inner_class_function() Mac.")
...             
...         @windows
...         def inner_class_function(self):
...             print("Called inner_class_function() for windows.")

The above demonstrates the basic mechanism of decorators, how to access the caller's scope, and how to simplify multiple decorators that have similar behavior by having an internal function containing the common algorithm defined.

Chaining Support

To support chaining these decorators indicating whether a function applies to more than one platform, the decorator could be implemented like so:

>>> class IfDefDecoratorPlaceholder:
...     def __init__(self, func):
...         self.__name__ = func.__name__
...         self._func    = func
...         
...     def __call__(self, *args, **kwargs):
...         raise NotImplementedError(
...             f"Function {self._func.__name__} is not defined for "
...             f"platform {platform.system()}.")
...
>>> def _ifdef_decorator_impl(plat, func, frame):
...     if platform.system() == plat:
...         if type(func) == IfDefDecoratorPlaceholder:
...             func = func._func
...         frame.f_locals[func.__name__] = func
...         return func
...     elif func.__name__ in frame.f_locals:
...         return frame.f_locals[func.__name__]
...     elif type(func) == IfDefDecoratorPlaceholder:
...         return func
...     else:
...         return IfDefDecoratorPlaceholder(func)
...
>>> def linux(func):
...     return _ifdef_decorator_impl('Linux', func, sys._getframe().f_back)

That way you support chaining:

>>> @macos
... @linux
... def foo():
...     print("works!")
...     
>>> foo()
works!

The comments below don't really apply to this solution in its present state. They were made during the first iterations on finding a solution and no longer apply. For instance the statement, "Note that this only works if macos and windows are defined in the same module as zulu." (upvoted 4 times) applied to the earliest version, but has been addressed in the current version; which is the case for most of the statements below. It's curious that the comments that validated the current solution have been removed.

Todd
  • 2,496
  • 1
  • 10
  • 21
  • 4
    Note that this only works if `macos` and `windows` are defined in the same module as `zulu`. I believe this will also result in the function being left as `None` if the function is not defined for the current platform, which would lead to some _very_ confusing runtime errors. – Brian Feb 16 '20 at 03:07
  • @Brian you bring up a good point. I'm going to see if I can come up with something that would work if the decorators were defined in another module. – Todd Feb 16 '20 at 04:13
  • @Brian alright. I made some changes to the decorator code to account for the different modules case. Thanks for catching that. – Todd Feb 16 '20 at 04:29
  • 1
    This won't work for methods or other functions not defined in a module-global scope. – user2357112 supports Monica Feb 16 '20 at 06:56
  • can you give me an example? – Todd Feb 16 '20 at 07:04
  • @Monica maybe you can show a snippet on pastebin of the case you have in mind? – Todd Feb 16 '20 at 07:07
  • 1
    Thank you @Monica. Yeah, I hadn't accounted for using this on member functions of a class.. alright.. I'll see if I can make my code more generic. – Todd Feb 16 '20 at 07:20
  • 1
    @Monica okay.. I updated the code to account for class member functions. Can you give this a try? – Todd Feb 16 '20 at 07:38
  • 1
    [Doesn't work.](https://ideone.com/FlHyiM) `func` isn't a bound method object and isn't going to have a `__self__` attribute. Also, the class doesn't exist yet when the decorator is executing. – user2357112 supports Monica Feb 16 '20 at 07:43
  • 2
    @Monica, alright.. I've updated the code to cover class methods and done a little bit of testing just to make sure it works - nothing extensive.. if you want to give it a run, let me know how it goes. – Todd Feb 16 '20 at 10:41
41

While @decorator syntax looks nice, you get the exact same behaviour as desired with a simple if.

linux = platform.system() == "Linux"
windows = platform.system() == "Windows"
macos = platform.system() == "Darwin"

if linux:
    def my_callback(*args, **kwargs):
        print("Doing something @ Linux")
        return

if windows:
    def my_callback(*args, **kwargs):
        print("Doing something @ Windows")
        return

If required, this also allows to easily enforce that some case did match.

if linux:
    def my_callback(*args, **kwargs):
        print("Doing something @ Linux")
        return

elif windows:
    def my_callback(*args, **kwargs):
        print("Doing something @ Windows")
        return

else:
     raise NotImplementedError("This platform is not supported")
MisterMiyagi
  • 26,337
  • 5
  • 60
  • 79
  • 9
    +1, If you were going to write two different functions anyway, then this is the way to go. I would probably want to preserve the original function names for debugging (so the stack traces are correct): `def callback_windows(...)` and `def callback_linux(...)`, then `if windows: callback = callback_windows`, etc. But either way this is way easier to read, debug, and maintain. – Seth Feb 16 '20 at 20:09
  • I agree this is the simplest approach to satisfy the use case you have in mind. However, the original question was about decorators and how they could be applied to function declaration. So the scope may be beyond just conditional platform logic. – Todd Feb 16 '20 at 21:02
  • 3
    I'd use an `elif`, as it's never going to be the *expected* case that more than one of `linux`/`windows`/`macOS` will be true. In fact, I'd probably just define a single variable `p = platform.system()`, then use `if p == "Linux"`, etc rather than multiple boolean flags. Variables that don't exist can't go out of sync. – chepner Feb 17 '20 at 14:13
  • @chepner If it's clear the cases are mutually exclusive, ``elif`` certainly has its advantages -- specifically, a trailing ``else`` + ``raise`` to ensure that at least one case *did* match. As for evaluating the predicate, I prefer having them pre-evaluated – it avoids duplication and decouples definition and use. Even if the result isn't stored in variables, there are now hardcoded values that can go out of sync just the same. I can *never* remember the various magic strings for the different means, e.g. ``platform.system() == "Windows"`` versus ``sys.platform == "win32"``, ... – MisterMiyagi Feb 17 '20 at 17:18
  • You can enumerate the strings, whether with a subclass of `Enum` or just a set of constants. – chepner Feb 17 '20 at 18:55
  • `parse_args` will *already* raise an exception; if it doesn't, `args.target` is *guaranteed* to have one of the acceptable values (ok, as long as you don't go changing the value of `args.target` yourself). – chepner Feb 17 '20 at 18:56
  • @MisterMiyagi No, I mean real constants whose values can be determined merely by looking at the code, without having to predict what a function will return at run-time. – chepner Feb 17 '20 at 21:33
8

Below is one possible implementation for this mechanic. As noted in the comments, it may be preferable to implement a "master dispatcher" interface, such as that seen in functools.singledispatch, to keep track of the state associated with the multiple overloaded definitions. My hope is the this implementation will at least offer some insight into the problems that you may have to deal with when developing this functionality for a larger codebase.

I have only tested that the implementation below works as specified on Linux systems, so I cannot guarantee that this solution adequately enables the creation of platform-specialized functions. Please do not use this code in a production setting without thoroughly testing it yourself first.

import platform
from functools import wraps
from typing import Callable, Optional


def implement_for_os(os_name: str):
    """
    Produce a decorator that defines a provided function only if the
    platform returned by `platform.system` matches the given `os_name`.
    Otherwise, replace the function with one that raises `NotImplementedError`.
    """
    def decorator(previous_definition: Optional[Callable]):
        def _decorator(func: Callable):
            if previous_definition and hasattr(previous_definition, '_implemented_for_os'):
                # This function was already implemented for this platform. Leave it unchanged.
                return previous_definition
            elif platform.system() == os_name:
                # The current function is the correct impementation for this platform.
                # Mark it as such, and return it unchanged.
                func._implemented_for_os = True
                return func
            else:
                # This function has not yet been implemented for the current platform
                @wraps(func)
                def _not_implemented(*args, **kwargs):
                    raise NotImplementedError(
                        f"The function {func.__name__} is not defined"
                        f" for the platform {platform.system()}"
                    )

                return _not_implemented
        return _decorator

    return decorator


implement_linux = implement_for_os('Linux')

implement_windows = implement_for_os('Windows')

In order to use this decorator, we must work through two levels of indirection. First, we must specify what platform we want the decorator to respond to. This is accomplished by the line implement_linux = implement_for_os('Linux') and its Window's counterpart above. Next, we need to pass along the existing definition of the function being overloaded. This step must be accomplished at the definition site, as demonstrated below.

To define a platform-specialized function, you may now write the following:

@implement_linux(None)
def some_function():
    ...

@implement_windows(some_function)
def some_function():
   ...

implement_other_platform = implement_for_os('OtherPlatform')

@implement_other_platform(some_function)
def some_function():
   ...

Calls to some_function() will be appropriately dispatched to the provided platform-specific definition.

Personally, I would not advise using this technique in production code. In my opinion, it is better to be explicit about platform-dependent behavior at each location where these differences occur.

Brian
  • 2,948
  • 4
  • 13
  • 23
  • @th0nk No - the function `implement_for_os` does not return a decorator itself, but rather returns a function which will produce the decorator once provided with the previous definition of the function in question. – Brian Feb 16 '20 at 02:33
5

I wrote my code before read other answers. After I finished my code, I found @Todd's code is the best answer. Anyway I post my answer because I felt fun while I was solving this problem. I learned new things thanks to this good question. The drawback of my code is that there exists overhead to retrieve dictionaries every time functions are called.

from collections import defaultdict
import inspect
import os


class PlatformFunction(object):
    mod_funcs = defaultdict(dict)

    @classmethod
    def get_function(cls, mod, func_name):
        return cls.mod_funcs[mod][func_name]

    @classmethod
    def set_function(cls, mod, func_name, func):
        cls.mod_funcs[mod][func_name] = func


def linux(func):
    frame_info = inspect.stack()[1]
    mod = inspect.getmodule(frame_info.frame)
    if os.environ['OS'] == 'linux':
        PlatformFunction.set_function(mod, func.__name__, func)

    def call(*args, **kwargs):
        return PlatformFunction.get_function(mod, func.__name__)(*args,
                                                                 **kwargs)

    return call


def windows(func):
    frame_info = inspect.stack()[1]
    mod = inspect.getmodule(frame_info.frame)
    if os.environ['OS'] == 'windows':
        PlatformFunction.set_function(mod, func.__name__, func)

    def call(*args, **kwargs):
        return PlatformFunction.get_function(mod, func.__name__)(*args,
                                                                 **kwargs)

    return call


@linux
def myfunc(a, b):
    print('linux', a, b)


@windows
def myfunc(a, b):
    print('windows', a, b)


if __name__ == '__main__':
    myfunc(1, 2)
0

A clean solution would be to create a dedicated function registry that dispatches on sys.platform. This is very similar to functools.singledispatch. This function's source code provides a good starting point for implementing a custom version:

import functools
import sys
import types


def os_dispatch(func):
    registry = {}

    def dispatch(platform):
        try:
            return registry[platform]
        except KeyError:
            return registry[None]

    def register(platform, func=None):
        if func is None:
            if isinstance(platform, str):
                return lambda f: register(platform, f)
            platform, func = platform.__name__, platform  # it is a function
        registry[platform] = func
        return func

    def wrapper(*args, **kw):
        return dispatch(sys.platform)(*args, **kw)

    registry[None] = func
    wrapper.register = register
    wrapper.dispatch = dispatch
    wrapper.registry = types.MappingProxyType(registry)
    functools.update_wrapper(wrapper, func)
    return wrapper

Now it can be used similar to singledispatch:

@os_dispatch  # fallback in case OS is not supported
def my_callback():
    print('OS not supported')

@my_callback.register('linux')
def _():
    print('Doing something @ Linux')

@my_callback.register('windows')
def _():
    print('Doing something @ Windows')

my_callback()  # dispatches on sys.platform

Registration also works directly on the function names:

@os_dispatch
def my_callback():
    print('OS not supported')

@my_callback.register
def linux():
    print('Doing something @ Linux')

@my_callback.register
def windows():
    print('Doing something @ Windows')
a_guest
  • 25,051
  • 7
  • 38
  • 80