60

I didn't really pay as much attention to Python 3's development as I would have liked, and only just noticed some interesting new syntax changes. Specifically from this SO answer function parameter annotation:

def digits(x:'nonnegative number') -> "yields number's digits":
    # ...

Not knowing anything about this, I thought it could maybe be used for implementing static typing in Python!

After some searching, there seemed to be a lot discussion regarding (entirely optional) static typing in Python, such as that mentioned in PEP 3107, and "Adding Optional Static Typing to Python" (and part 2)

..but, I'm not clear how far this has progressed. Are there any implementations of static typing, using the parameter-annotation? Did any of the parameterised-type ideas make it into Python 3?

Community
  • 1
  • 1
dbr
  • 153,498
  • 65
  • 266
  • 333

5 Answers5

35

Thanks for reading my code!

Indeed, it's not hard to create a generic annotation enforcer in Python. Here's my take:

'''Very simple enforcer of type annotations.

This toy super-decorator can decorate all functions in a given module that have 
annotations so that the type of input and output is enforced; an AssertionError is
raised on mismatch.

This module also has a test function func() which should fail and logging facility 
log which defaults to print. 

Since this is a test module, I cut corners by only checking *keyword* arguments.

'''

import sys

log = print


def func(x:'int' = 0) -> 'str':
    '''An example function that fails type checking.'''
    return x


# For simplicity, I only do keyword args.
def check_type(*args):
    param, value, assert_type = args
    log('Checking {0} = {1} of {2}.'.format(*args))
    if not isinstance(value, assert_type):
        raise AssertionError(
            'Check failed - parameter {0} = {1} not {2}.'
            .format(*args))
    return value

def decorate_func(func):    
    def newf(*args, **kwargs):
        for k, v in kwargs.items():
            check_type(k, v, ann[k])
        return check_type('<return_value>', func(*args, **kwargs), ann['return'])

    ann = {k: eval(v) for k, v in func.__annotations__.items()}
    newf.__doc__ = func.__doc__
    newf.__type_checked = True
    return newf

def decorate_module(module = '__main__'):
    '''Enforces type from annotation for all functions in module.'''
    d = sys.modules[module].__dict__
    for k, f in d.items():
        if getattr(f, '__annotations__', {}) and not getattr(f, '__type_checked', False):
            log('Decorated {0!r}.'.format(f.__name__))
            d[k] = decorate_func(f)


if __name__ == '__main__':
    decorate_module()

    # This will raise AssertionError.
    func(x = 5)

Given this simplicity, it's strange at the first sight that this thing is not mainstream. However, I believe there are good reasons why it's not as useful as it might seem. Generally, type checking helps because if you add integer and dictionary, chances are you made some obvious mistake (and if you meant something reasonable, it's still better to be explicit than implicit).

But in real life you often mix quantities of the same computer type as seen by compiler but clearly different human type, for example the following snippet contains an obvious mistake:

height = 1.75 # Bob's height in meters.
length = len(sys.modules) # Number of modules imported by program.
area = height * length # What's that supposed to mean???

Any human should immediately see a mistake in the above line provided it knows the 'human type' of variables height and length even though it looks to computer as perfectly legal multiplication of int and float.

There's more that can be said about possible solutions to this problem, but enforcing 'computer types' is apparently a half-solution, so, at least in my opinion, it's worse than no solution at all. It's the same reason why Systems Hungarian is a terrible idea while Apps Hungarian is a great one. There's more at the very informative post of Joel Spolsky.

Now if somebody was to implement some kind of Pythonic third-party library that would automatically assign to real-world data its human type and then took care to transform that type like width * height -> area and enforce that check with function annotations, I think that would be a type checking people could really use!

ilya n.
  • 16,814
  • 14
  • 66
  • 89
  • And here's what I just wrote about perils of strong typing: http://stackoverflow.com/questions/1251791/what-are-the-limits-of-type-checking-and-type-systems/1276675#1276675 – ilya n. Aug 14 '09 at 08:41
  • 33
    What if bob had to paint one square meter of his bedroom with red paint for every module he imports? – Otto Allmendinger Aug 14 '09 at 10:14
  • 2
    Then you should use explicit type cast to punish Bob. Every system of strong typing, whether static or dynamic, has provisions for type casts: e.g. `'string' + 5` is prohibited but `'string' + str(5)` is fine. – ilya n. Aug 14 '09 at 10:22
  • Also note that your example would be just `sq.m * number = sq.m` which *does* make more sense than mine example of `m * number = sq.m ???` – ilya n. Aug 14 '09 at 10:26
  • 4
    Has anybody thought about how constricted code-completion in IDEs in Python must be since we don't know what type of variable named `a` is, we can't figure out what to do when the user types `a.` and hits ctrl-space... – Warren P Oct 14 '11 at 13:59
  • 28
    I don't see how you can jump from "this solution doesn't solve *every* problem" to "this solution is worse than nothing". If you really thought that you wouldn't even be using python because python isn't a perfect language, and is therefore worse than nothing. Even proper statically typed languages don't enforce "semantic types" because that is an impossible problem. They usually don't even enforce physical SI types either, but I think that's because it's usually not worth the effort (but there are systems that do do it). – Timmmm Aug 23 '12 at 10:33
  • @Timmmm, ilya doesn't say that *all* imperfect solutions are worse than nothing, just that this one is, because its downside (adding complexity to the already hard task of programming) is greater than its upside (it prevents you from making a rare sort of mistake that you probably aren't making anyway). Python's philosphy values simplicity - a new feature has to be useful enough to justify the complexity added by its very existence. (If every feature that might be useful to someone somewhere were added to the language, the resulting language would be unusable.) – Josh Feb 10 '13 at 19:41
  • 5
    He said "it is a half-solution *so* it is worse than no solution". Perhaps he meant "and" not "so", but as written, he is saying that all half-solutions are worse than no solutions which is clearly false. But anyway, type errors are not a rare mistake in large dynamically typed projects. And preventing mistakes is only one of the benefits of static typing. There are other important benefits, namely code completion, refactoring and static analysis. I agree it adds complexity though, so I'm starting to thing Dart's optional typing may not be as insane as it first sounds. – Timmmm Feb 11 '13 at 20:29
  • I recommend to write something heavily branched, which works in one flow. Like a compiler with optimizer of a moderately complex language, or an analyses which runs on 2GB of date with cubic time. This very fast (in few days of constant debugging) may convince, how unuseful are types. -- Particularly, I can boast myself with 4 spellings and a full week of code running and debugging. – Clare May 20 '13 at 22:33
  • 3
    One point for static type checking, such as in [Boo](http://boo.codehaus.org/) which has a Python syntax, is that runtime checks are converted to compile-type ones. Nice code though. – Martin Tapp Nov 16 '13 at 14:20
15

As mentioned in that PEP, static type checking is one of the possible applications that function annotations can be used for, but they're leaving it up to third-party libraries to decide how to do it. That is, there isn't going to be an official implementation in core python.

As far as third-party implementations are concerned, there are some snippets (such as http://code.activestate.com/recipes/572161/), which seem to do the job pretty well.

EDIT:

As a note, I want to mention that checking behavior is preferable to checking type, therefore I think static typechecking is not so great an idea. My answer above is aimed at answering the question, not because I would do typechecking myself in such a way.

sykora
  • 81,994
  • 11
  • 60
  • 70
  • 2
    Note that the snippet mentioned enforce dynamic type checking not **static** type checking. – Andrea Zilio Feb 16 '10 at 22:53
  • 1
    @Andrea Zillio: but it might be possible for IDEs or scripts to check the source code before it is executed (static) when an annotation is given. Think of def f(x: int) -> int and someone tries to make f('test'). – Joschua Aug 20 '11 at 11:53
  • @Joschua: Sadly I can't see IDEs jumping on board with any non-official solutions. – Timmmm Aug 23 '12 at 10:34
  • 1
    @sykora some static type systems check behavior. You're just making a distinction between nominal and structural typing. Check OCaml's object system for a static behavior oriented static type system. – raph.amiard Jan 13 '14 at 14:37
14

"Static typing" in Python can only be implemented so that the type checking is done in run-time, which means it slows down the application. Therefore you don't want that as a generality. Instead you want some of your methods to check it's inputs. This can be easily done with plain asserts, or with decorators if you (mistakenly) think you need it a lot.

There is also an alternative to static type checking, and that is to use an aspect oriented component architecture like The Zope Component Architecture. Instead of checking the type, you adapt it. So instead of:

assert isinstance(theobject, myclass)

you do this:

theobject = IMyClass(theobject)

If theobject already implements IMyClass nothing happens. If it doesn't, an adapter that wraps whatever theobject is to IMyClass will be looked up, and used instead of theobject. If no adapter is found, you get an error.

This combined the dynamicism of Python with the desire to have a specific type in a specific way.

Lennart Regebro
  • 147,792
  • 40
  • 207
  • 241
  • 5
    Annotations could be used by static analysis tools such as pylint to perform some sanity checking very similar to static typing within regions of the code where the type of a variable or parameter is knowable and annotations exist on the APIs being called. – gps Apr 01 '12 at 05:50
  • 1
    @LennartRegebro: This is brilliant, simple idea. I struggle in each language with the "non-empty string" problem. Most methods accepting strings as parameters aren't well-prepared for empty strings, or all whitespace strings. In most cases, these are invalid values, but few programmers care to check for it. I like your idea of using a delegating (pseudo-)type to apply type/value constraints *once*: during construction. Example: `str` -> `NonEmptyStr` or `Text` (which implies non-empty and not all whitespace). – kevinarpe Nov 15 '14 at 10:20
  • Yes, but since you can still pass in an empty string, that actually doesn't solve anything. Unless you *also* type check in the function, and then the pseudo-type doesn't actually add anything. – Lennart Regebro Nov 17 '14 at 12:56
  • Wouldn't assert slow down your application the same way as static typing? Also, I am wondering how static typing slows down python when the fastest languages (c, c++, java) are all statically typed. – qed Dec 08 '14 at 22:59
  • @qed: That's exactly what I said. If you want to have type checking in a dynamic language like Python, this will slow things down, because it is done in run time. This is unlikely to make a significant difference unless you do it in a tight loop. But it will be slower, even if it is insignificant. Real static typing is not slower, because it's done during compile-time. – Lennart Regebro Dec 09 '14 at 08:09
  • Nice. I have another question, though. Doesn't dynamic typing include some form of type checking or type determination at least? Doesn't that take time anyways? I am just wondering why static typing won't help with performance improvement (there has been some discussion on the dev mailing list about the possibilities of add some form of type annotation). – qed Dec 10 '14 at 21:33
  • Well, yes, when you look up for example a method, you first look on the object instance, and then you have to search through the objects you inherit from according to the Method Resolution Order. This is done reasonably fast in Python, but in a static compiled language it's actually done during compile time. So that's a part of the speedup. But you can't do that in a dynamic language, since the classes can change during run-time. – Lennart Regebro Dec 10 '14 at 21:59
  • As I understand it, this answer is incorrect. The type annotations don't do anything at runtime according to [PEP 484](https://www.python.org/dev/peps/pep-0484/). The type hints are there so that things like mypy can do static type checking, which does not occur during runtime. – qfwfq Apr 10 '17 at 17:18
  • I never state that Python has static runtime type checking. I say that if you WANT it, you have to implement it in your application. – Lennart Regebro Apr 17 '17 at 09:54
13

This is not an answer to question directly, but I found out a Python fork that adds static typing: mypy-lang.org, of course one can't rely on it as it's still small endeavor, but interesting.

Ciantic
  • 5,156
  • 3
  • 50
  • 47
0

Sure, static typing seems a bit "unpythonic" and I don't use it all the time. But there are cases (e.g. nested classes, as in domain specific language parsing) where it can really speed up your development.

Then I prefer using beartype explained in this post*. It comes with a git repo, tests and an explanation what it can and what it can't do ... and I like the name ;)

* Please don't pay attention to Cecil's rant about why Python doesn't come with batteries included in this case.

Community
  • 1
  • 1
Iwan LD
  • 312
  • 4
  • 12