4

more-intelligent-members-of-the-coding-community-than-I! I have a python question for you all... :)

I am trying to optimize a python script that is (among other things) returning the wall-clock time a subprocess took execute and terminate. I think I'm close with something like this.

startTime = time.time()
process = subprocess.Popen(['process', 'to', 'test'])
process.wait()
endTime = time.time()
wallTime = endTime - startTime

However, I am concerned that the overhead in subprocess.Popen() is inflating my results on older and exotic platforms. My first inclination is to somehow time the overhead caused by subprocess.Popen() by running a simple innocuous command. Such as the following on linux.

startTime = time.time()
process = subprocess.Popen(['touch', '/tmp/foo'])
process.wait()
endTime = time.time()
overheadWallTime = endTime - startTime

Or the following on windows.

startTime = time.time()
process = subprocess.Popen(['type', 'NUL', '>', 'tmp'])
process.wait()
endTime = time.time()
overheadWallTime = endTime - startTime

So, how should I account for subprocess.Popen() overhead when timing in python?

Or, what is an innocuous way to sample the overhead from subprocess.Popen()?

Or, maybe there's something I haven't considered yet?

The caveat is that the solution must be reasonably cross-platform.

perden
  • 597
  • 4
  • 19

1 Answers1

0

Welcome to batteries included.

orlp
  • 98,226
  • 29
  • 187
  • 285
  • Ahh yea, I did come across this in my searches. Correct me if I am wrong, but it looks like this is for profiling _python programs_ in _cpu utilization_. It didn't look like it would work for me since I'd like _any binary_ in _wall-clock time_. Is that right? – perden Sep 26 '11 at 20:15
  • perden: I don't think there is any reliable general-purpose binary profiler. For what language is it if I may ask? – orlp Sep 26 '11 at 20:20
  • The binaries to test could be written in any assortment of languages including C, C++, and (shutter) Fortran. I'm not terribly worried about being accurate to the instruction-level (I am looking for wall-clock time after all), but there's some overhead using scripting languages to create subprocess especially on older platforms, and I figured I'd take a stab at accommodating for that. – perden Sep 26 '11 at 20:27
  • @perden: Ah, if I understand correctly you aren't profiling of the code, just interested in the running time of executables. Perhaps make a solution consisting of Unix's [`time`](http://en.wikipedia.org/wiki/Time_(Unix)), Windows [`timeit`](http://stackoverflow.com/questions/673523/how-to-measure-execution-time-of-command-in-windows-command-line) and Popen as last resort? – orlp Sep 26 '11 at 20:30