7

My team has a goal to minimize the amount of time that our build is broken.

We use CruiseControl.NET for continuous integration. What I'd like to find out is how best to approach answering the following question:

"In the last {timespan}, how much time has {project-name} spent in a broken status?"

For example: "Over the last 1 month, how much time has our project spent in a broken status?"

Are there any advanced features of CruiseControl.NET that would facilitate making this information available in some type of a report or somewhere in the dashboard?

Alternatively, how would you approach parsing the xml artifact files to glean this info?

bhazzard
  • 363
  • 2
  • 10
  • "My team has a goal to minimize the amount of time that our build is broken." Why you need this? Please describe a scenario when you can use this information. If you have 10h in broken status or 3 days, what it will give you? – Restuta Jan 18 '10 at 22:57
  • Restuta: For our team, a broken build represents a bug. We have a suite of automated tests at various levels of our system. When one of these tests fails due to a code change, it means we've added a bug. Bugs are not tolerable and we would like to enforce fixing them as a high priority. Having the statistics is just a means to quantify our performance on meeting our goal of quickly attending to and fixing bugs. Also, we use CruiseControl.NET to deploy our dev site, so as long as a build is broken, new code changes won't be reflected on our dev site. We'd like to keep our dev site up to date. – bhazzard Jan 18 '10 at 23:05
  • Broken build count is IMHO a much better metric than the time spent in this state. – skolima Jan 18 '10 at 23:31
  • 1
    Upvoted. I think it's a valid and interesting question. These sort of metrics are fun, especially if taken lightly (ie, people want to improve this metric because they are perfectionists and light-heartedly competetive, rather than because their HR manager is an idiot and their Bonus will be affected). Also consuming the Cruise Control API (or scraping or whatever the answer turns out to be) will be interesting. – Andrew M Jan 18 '10 at 23:59
  • 1
    I agree that broken build count is a good metric, but it has different implications than time spent broken. Broken build count tells us how often we introduce defects. Time spent in a broken build state tells us how quickly we respond to those defects. – bhazzard Jan 19 '10 at 14:54

5 Answers5

2

I see at least two ways to approach this:

  1. You write an external tool which parses CC.NET's XML log files for a project (stored in buildlogs subdirectory by default), calculates statistics and writes a HTML report. This is probably easier to do, but it won't be directly integrated with CC.NET.
  2. You write a CC.NET plug-in to do this. You'll need to do a bit of investigating in this case. My guess the starting point would be to look at the source code of some existing plug-in.

Here are some links about CCNET plugins:

dna
  • 1,488
  • 1
  • 14
  • 34
Igor Brejc
  • 17,626
  • 12
  • 72
  • 92
2

you can use the statistics publisher, http://www.cruisecontrolnet.org/projects/ccnet/wiki/Statistics_Publisher and you can display them via project statistics plugin

dna
  • 1,488
  • 1
  • 14
  • 34
Williams
  • 723
  • 1
  • 4
  • 11
1

Having had a very quick look at the CC docs, I imagine if you were writing your own Cruise control dashboard, you could consume the RSS feed of build results, parse in all the date times and success/failure states up to your threshold, then sum up the totals.

As for displaying it in a dashboard, I think Cruise Control has a plugin architecture which might help http://cruisecontrol.sourceforge.net/main/plugins.html

Andrew M
  • 8,419
  • 6
  • 43
  • 61
  • I'm referring specifically to CruiseControl.NET. Do you know if any of that documentation from CruiseControl also applies to CruiseControl.NET? – bhazzard Jan 19 '10 at 14:51
0

So my eventual solution wasn't ideal, but it was easy to do and it works:

I had CC.NET send build emails to an email address (we'll call it build_emails@build_statistics.com). Then I use a ruby script to get the emails via imap and process them to determine our build failure time.

I didn't go the route of directly parsing the xml because I would have had to parse every xml file in the timeframe to build up a timeline and then go over the timeline to make my calculations. It just seemed too complicated to get a simple statistic like this.

bhazzard
  • 363
  • 2
  • 10
0

I like cc.net, but in this case TeamCity just does this for you. It has lots of other great statistics too. It's free for less than 20 projects.

Mike Two
  • 41,318
  • 7
  • 77
  • 93
  • Team City does look pretty neat. Out of curiosity, how would one answer this question using TeamCity? – bhazzard Feb 26 '10 at 15:46
  • @bhazzard - Team City shows statistics for each build configuration that shows the time to fix the build over a day, week, month, quarter, year or all time. their docs show an example of the statistics page http://confluence.jetbrains.net/display/TCD5/Statistics. It also has a restful api that you might be able to use to query it. Can't say I have tried the query method yet. I just look at the graph. – Mike Two Feb 26 '10 at 16:09