Edit:
This answer has been implemented in https://github.com/campos-ddc/cprofile_graph
Profiling with cProfile
Here's a post I wrote some time ago on profiling with cProfile with some graphical aid.
cProfile is one of the most used python profilers out there, and although very powerful, the standard text output is somewhat lackluster. Here I'll show you how to use cProfile on your application in an easier way.
There are two common ways to use cProfile, you can use it as a command in prompt to profile a given module, or you can use it inside your code, to profile specific snippets of code.
Profiling a module
To use cProfile to profile an entire module, simply use the following command in your prompt:
python -m cProfile -o output_filename.pstats path/to/script arg1 arg2
This will run your module with the given arguments (they are optional) and dump the output in output_filename.pstats.
There are lots of ways to read the data on that output file, but for the purpose of this post, let's not worry about those and focus on getting that graphical visualization.
Profiling from inside
Sometimes you don't want to profile an entire module, just a few lines of it.
To do so, you are gonna have to add some code to your module.
First of all:
import cProfile
And then, you can replace any segment of code with the following:
cProfile.runctx('Your code here', globals(), locals(), 'output_file')
For example, here is a test before and after profiling:
import unittest
class Test(unittest.TestCase):
def testSomething(self):
self.DoSomethingIDontCareAbout()
param = 'whatever'
self.RunFunctionIThinkIsSlow(param)
self.AssertSomeStuff() # This is after all, a test
After:
import unittest
import cProfile
class Test(unittest.TestCase):
def testSomething(self):
self.DoSomethingIDontCareAbout()
param = 'whatever'
cProfile.runctx(
'self.RunFunctionIThinkIsSlow(param)',
globals(),
locals(),
'myProfilingFile.pstats'
)
self.AssertSomeStuff() # This is after all, a test
Converting a pstats file to a graph
To convert your profiling file to a graph, you will need a couple of things:
- gprof2dot: This module will convert your output into a dot file, a standard file format for graph descriptions.
- GraphViz: It turns your dot file into an image.
After you have downloaded gprof2dot and installed GraphViz, run this command in your prompt:
python gprof2dot -f pstats myProfileFile | dot -Tpng -o image_output.png
You might have to use a complete path for gprof2dot and/or dot, or you could add them to your PATH env variable.
After all of this, you should have an image that looks kinda like this:
![results example]()
Hotter colors (red, orange, yellow) indicate functions that take up more of the total runtime than colder colors (green, blue)
On each node, you can see what percentage of the total runtime that function used and how many times it was called.
Arrows between nodes indicate which function called other functions, and such arrows also have a caption indicating what percentage of the runtime came through there.
Note: percentages won't always add up to 100%, especially on code sections that reference C++ code, which won't be profiled. cProfile also won't be able to determine what's called from inside an "eval" statement, so you might see some jumps in your graph.
