13

I was curious whether anyone has any suggestions for performance testing libraries and frontends that will produce nice graphical charts for C++ (like how gcov produces coverage data and there are frontends out there for viewing the code coverage data). Being able to produce charts like:

http://download.eclipse.org/eclipse/downloads/drops/S-3.7M4-201012081300/performance/performance.php?fp_type=0

would be pretty slick. We use cppunit right now for unit testing, so maybe there is something that integrates with that.

Some more info: We're compiling on Linux (we use Ubuntu Lucid/Maverick) on Intel x86-64 machines.

Kenny Peng
  • 1,821
  • 4
  • 17
  • 25
  • http://stackoverflow.com/questions/4394606/beyond-stack-sampling-c-profilers, has an interesting ongoing list of such things... – Nim Jan 05 '11 at 16:23
  • This will largely depend on what OS and CPU you are running on as to what's available ... are you looking more for automated testing or performance profiling? – AJG85 Jan 05 '11 at 18:58
  • I was thinking of more like an automated regression testing setup that would be particularly geared toward performance metrics. It would be cool to leverage cppunit, but it's not required and I'd be interested in any kind of performance testing harness. Profiling would be the step we would take after our regression tests note some large runtime hit or something similar. – Kenny Peng Jan 05 '11 at 23:18

4 Answers4

5

Some suggestions:

  • The googletest C++ framework is capable of producing JUnit-compatible reports.
  • Hudson can be used to run your tests. It only requires that your C++ application can be run as a console application.
  • The Hudson Performance Plugin can generate graphical charts from JUnit reports.
  • There are many other plugins.
StackedCrooked
  • 32,392
  • 40
  • 137
  • 267
2

There are two performance testing frameworks can help you.

Both of them are inspired by Google Test framework and provide interface to support performance testing, so it’s easy to transport your original googletest to Hayai or SkyPat.

SkyPat combines unit tests and perf_evnet. It extends the concept of Google Test and provides an interface to access PMU. perf_event gives SkyPat accurate cycle counts that are useful for tools that are sensitive to the variance of timing, such as compilers. SkyPat can also profile a piece of code by PMU events (eq: cycles, instructions, cache reference, cache miss, etc.).

Hayai supports several performance measuring methods of different OS (Windows: QueryPerformanceCounter(), Linux: gethrtime(), Apple(OSX, iOS): mach_absolute_time()) into its performance testing framework.

Pisco
  • 31
  • 1
2

On linux you can try the valgrind toolkit. Valgrind includes the callgrind tool that can profile your code. KCacheGrind visualizes the output of cachegrind very nicely.

Fabian
  • 737
  • 5
  • 19
  • 1
    The idea isn't to profile the code, per se, it's to create a nightly build system that could notify us of performance regressions by taking some simple metrics (maybe even just runtime) on some perf. unit tests that we'll have. So we don't particularly want to instrument our code, we just want to see the raw performance of it. – Kenny Peng Jan 05 '11 at 18:19
  • @Kenny: You can parse the output of `cg_diff` to produce the kind of graphs you want. – Benjamin Bannier Jan 05 '11 at 19:00
2

Some hopefully relevant notes from my experience of this sort of thing in answers here and here.

There's no reason your existing CppUnit setup combined with something like the scoped_timer (see second of the above) and a bit of postprocessing to some charts maybe shouldn't satisfy your needs. While I believe CppUnit includes a plugin ("Clocker" ?) which will log out unittest times, I don't think it's actually that useful for this sort of performance testing in practice because you invariably end up wanting to time just part of each test and not all the setup code.

Community
  • 1
  • 1
timday
  • 23,811
  • 10
  • 77
  • 132