4

I recently noticed the following about the timeit python module:

On my machine the lines:

from timeit import Timer
t = Timer(stmt='a = 2**3**4')
print("This took {:.3f}s to execute.".format(t.timeit()))

will produce:

This took 0.017s to execute.

On the other hand writing a file test.py:

#!/usr/bin/env python3

a = 2**3**4

and calling:

from timeit import Timer
t = Timer(stmt='import test')
print("This took {:.3f}s to execute.".format(t.timeit()))

will produce:

This took 0.126s to execute.

And I'm wondering how I can test the execution time of test.py without changing the file itself. How can I work around importing the file (and therefore loosing time).

user9115052
  • 105
  • 8
  • 1
    https://stackoverflow.com/a/24105845/650884 – Pavel Dec 18 '17 at 20:23
  • If it's the first time you call `import test`, python will be busy creating "test.pyc" and take a bit longer. Subsequent imports should be somewhat faster. – Aaron Dec 18 '17 at 20:54
  • 1
    @Aaron: Except subsequent imports won't import anything at all (they go through a bunch of import rigmarole that eventually ends in pulling the cached module from `sys.modules`, which doesn't rerun the module at all). – ShadowRanger Dec 18 '17 at 20:57

2 Answers2

2

There is one problem with your measurements you are not measuring what you think you are measuring, when you write:

t = Timer(stmt= 'a = 2**3**4')

You are measuring binding time! Look:

>>> Timer(stmt='a = 10**5**4').timeit(100000)
    0.0064544077574026915
>>> Timer(stmt='a = 2**3**4').timeit(100000)
    0.006511381058487586

The timings are pretty the same but it is somewhat longer to compute 10**5**4 than 2**3**4. The 2**3**4 is computed only once, when the code is compiled and this is called "constant folding", some optimizations which Python perform during compilation of your source.

Compare this two results:

>>> Timer(stmt= 'a = 2**3**4').timeit(100000) 
    0.00628656749199763
>>> Timer(stmt= 'a = x**y**z', setup='(x,y,z)=(2,3,4)').timeit(100000) 
    0.18055968312580717

But this acceleration is not given for free. There are two points:

  1. Compilation time increases
  2. .pyc file size increases (because this value is stored inside .pyc file)

Suppose I have two files:

#foo1.py
a = 10**7**7

#foo2.py
x,y,z =(10,7,7)
a = x**y**z

If I compile them with python -m py_compile foo1.py foo2.py the sizes of .pyc files on my machine will be:

  1. foo1.cpython-36.pyc - 364 882 bytes
  2. foo2.cpython-36.pyc - 150 bytes
godaygo
  • 1,757
  • 1
  • 12
  • 29
  • That's really interesting, but I don't quite get how this exactly relates to my import statement? I was thinking, when I import `test.py` there already exists an `test.pyc` containing the constant, therefore the code should be faster than generating it on the fly (as for example in the command line), when in reality using import is slower. – user9115052 Dec 18 '17 at 23:25
  • Maybe you'd like to check out my [next question](https://stackoverflow.com/questions/47877966/python3-using-timeit-to-measure-script-execution-time-externally) as well – user9115052 Dec 18 '17 at 23:32
1

The closest you're going to get is to use compile with exec If you plan on running as a .pyc file, don't include the compile statement in what you're timing.

# time as if executing:" >python test.pyc " from terminal
#   (importing test.py will typically automatically generate the .pyc file automatically)
t = Timer(stmt='exec(code_object)', 
          setup='code_object = compile(open("test.py").read(), "test.py", "exec")')

# time as if executing:" >python test.py " from terminal
t = Timer(stmt='exec(compile(open("test.py").read(), "test.py", "exec"))')

This should get you close to the realistic timing for calling a script from the terminal. This does not eliminate overhead, because the overhead of calling a script is real and will be observed in the real world.

If you're on a linux based system, you can also just call >time test.py from a terminal.

Aaron
  • 7,351
  • 1
  • 24
  • 36
  • @StefanPochmann sorry, stupid typos. not thinking about what I'm doing. See edit – Aaron Dec 18 '17 at 22:12
  • Ok now it works, but... it takes about 0.35 seconds while the OP's way only takes 0.15 seconds. Was it actually *faster* for you when you tested it? You did test this time, right? :-P – Stefan Pochmann Dec 18 '17 at 22:19
  • I want to time a script just if I put it all in quotes and put it as `stmt` in `timeit`. `>time test.py` always prints much more time, since python startup is included. How would I solve this problem? I want to test a some code `...code...` as if I used `print(Timer.timeit(stmt='code'))`. – user9115052 Dec 18 '17 at 23:22
  • Maybe you'd like to check out my [next question](https://stackoverflow.com/questions/47877966/python3-using-timeit-to-measure-script-execution-time-externally) as well – user9115052 Dec 18 '17 at 23:32
  • @StefanPochmann it's not faster, it's representative of the real world – Aaron Dec 19 '17 at 14:11