101

I use TeamCity which in turn invokes msbuild (.NET 4). I have a strange issue in that after a build is complete (and it doesn't seem to matter if it was a successful build or not), msbuild.exe stays open, and locks one of the files, which means every time TeamCity tries to clear its work directory, it fails, and can't continue.

This happens almost every time.

I'm really lost on this one, so I'll try to provide as much detail as possible.

  • Server is an Intel Core i7, 2 GB ram, with Windows Server 2008 standard 64-bit SP2.
  • In TeamCity, the msbuild runner is configured with the /m command-line parameter (which means to use multiple cores)
  • The file in question is ALWAYS the same external DLL that is referenced in one of the .NET projects, in the path External Tools\Telerik\Telerik.Reporting.Dll. (There are several other .DLL files included in the External Tools dir in a similar path structure which never cause this problem). Currently this is with the trial version of Telerik reports, in case that makes any difference.
  • When the issue happens, there are always several msbuild.exe *32 processes listed in Task manager: I believe there are 7. Using Process Explorer, they all look like top-level processes (no parents). They're all using from 20-50MB ram, and 0.0% CPU.
  • If I wait 1-3 minutes, the msbuild.exe processes exit on their own, and TeamCity can then update the work directory properly.
  • If I manually terminate the msbuild processes, TeamCity's update will work again immediately.
  • Indexing services are turned off in Windows (though the prior two points pretty much confirm it's msbuild.exe causing the problem).
  • There are no special properties on Telerik.reporting.dll. The only SVN property is svn:mime-type = application/octet-stream

Has anyone run across this before?

gregmac
  • 22,527
  • 9
  • 79
  • 114

2 Answers2

128

Use msbuild with /nr:false.

Briefly: MSBuild tries to do lots of things to be fast, especially with parallel builds. It will spawn lots of "nodes" - individual msbuild.exe processes that can compile projects, and since processes take a little time to spin up, after the build is done, these processes hang around (by default, for 15 minutes, I think), so that if you happen to build again soon, these nodes can be "reused" and save the process setup cost. But you can disable that behavior by turning off nodeReuse with the aforementioned command-line option.

See also:

Peter Mortensen
  • 28,342
  • 21
  • 95
  • 123
Brian
  • 113,581
  • 16
  • 227
  • 296
  • 2
    Makes sense: it doesn't seem to happen if I remove /m. I'm trying now with `/m /nr:false`, I'll run for a few builds and see how it goes. Thanks – gregmac Oct 13 '10 at 00:26
  • Been a couple days, and tens of builds later, and it hasn't happened again - looks like it's solved. Thanks – gregmac Oct 15 '10 at 18:14
  • 28
    How do you get Visual Studio to build the project with that msbuild option? – Cameron Taggart Apr 14 '12 at 00:48
  • 1
    I would still like to know, but I actually ran into a Visual Studio 11 Beta bug for C++/CLI projects. Is causes the same symptoms: http://connect.microsoft.com/VisualStudio/feedback/details/728912/vs11-regression-wdp4-consumer-preview-msbuild-exe-apparently-causes-mt-exe-to-fail-by-holding-a-newly-created-dll-open – Cameron Taggart Apr 14 '12 at 01:37
  • This post helped me solve my related problem, which is documented here: - http://stackoverflow.com/questions/13510465/the-mystery-of-stuck-inactive-msbuild-exe-processes-locked-stylecop-dll-nuget/13510466 – Jon Rea Nov 22 '12 at 10:19
  • The top link above is dead (after logging in to Microsoft Connect) – thecoolmacdude Aug 30 '16 at 12:53
  • 1
    @CameronTaggart You can add msbuild command line options with a special file hosted in your project/solution folder. See https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild-response-files?view=vs-2019 – needfulthing Jul 10 '19 at 15:38
44

To disable node reuse within Visual Studio, you must use an environment variable:

MSBUILDDISABLENODEREUSE=1
dan
  • 501
  • 4
  • 2
  • I used this effectively, however there is another tool that's failing now, when compiling C++ with VS11 Beta, that's mt.exe, is there any other variable to use for that? – Eugenio Miró May 28 '12 at 16:50
  • Can't it be set using a dialog box somewhere in VS? – dom_beau Nov 29 '12 at 13:50
  • 1
    @dan Sincere thanks for finding this one, and I'm *praying* there's an environment variable to disable Microsoft.VisualStudio.Web.Host.exe as well. – jerhewet Apr 23 '15 at 10:36
  • This also works when running a build from the command line, e.g. a batch script, build server, etc. – Dave E Feb 01 '16 at 11:41