Parallel MSBUILD - critical sections? -

my organization has huge builds run on build servers, building lots , and lots of msbuild projects linked projectreferences. need able build projects , configurations in parallel msbuild /m.

my problem have project referenced large number of other projects, project not reentrant. if more 2 or more nodes try build project in parallel, fail.

how can wrap 1 project, or it's target, inside critical section?

what need this:

<target>     <entercriticalsection id=$(projectguid) />     <exec />     <leavecriticalsection id=$(projectguid) /> </target> 

the idea being if multiple msbuild nodes tried build project in parallel, 1 of nodes execution, , rest of nodes have wait (or go else).

i suppose write custom msbuild tasks this, isn't there way of doing built msbuild system?

=== edit 4/5/13. clarify, project building 3rd-party library build script provided it. rewriting build script make reentrant -- insuring each build used different set of folders intermediate files, etc. -- possible in theory, not practical solution. 1 thing, work would have redone on each new release of library.

=== edit 4/6/13. on further reflection, don't think it's theoretically possible insure project reentrant. let me explain:

suppose project xyz set use different temporary directories depending on platform , configuration in usual way:
xyz.proj:

<propertygroup>   <myworkingdir>tmp.$(platform).$(configuration)</myworkingdir> </propertygroup> 

now suppose other project graphicswindow references project xyz, either through projectreferences or msbuild tasks. , suppose graphicswindow project can built use various graphics apis. i.e., there's opengl version, directx 9 version, directx 10, version...

so somewhere there's .proj or .targets file containing target build 4 versions:

<project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">     <itemgroup>         <projecttobuild include="graphicswindow.proj">             <properties>graphicsapi=opengl</properties>         </projecttobuild>         <projecttobuild include="graphicswindow.proj">             <properties>graphicsapi=d3d9</properties>         </projecttobuild>         <projecttobuild include="graphicswindow.proj">             <properties>graphicsapi=d3d10</properties>         </projecttobuild>         <projecttobuild include="graphicswindow.proj">             <properties>graphicsapi=d3d11</properties>         </projecttobuild>     </itemgroup>     <target name="all">         <msbuild projects="@(projecttobuild)" buildinparallel="true" />     </target> </project> 

or equivalent using batching.

now msbuild build xyz project 4 times same platform|configuration combination , same working directory.

this work fine long build without /m option , msbuild runs single thread. depending on how xyz project written, 2nd, 3rd, , 4th builds might nothing because outputs up-to-date, or might redundant work, final result correct , build succeed.

but start using parallel msbuild, this build broken! there race condition multiple threads can enter xyz project's target(s) concurrently, , start building using same working directories, fail.

irrespective of how execute msbuild, multi-proc option /m or without, guaranteed execute project once every configuration requested build. here quote msdn:

when microsoft build engine encounters project-to-project (p2p) reference while using parallel builds build project, builds reference 1 time. if 2 projects have same p2p reference, reference not rebuilt each project. instead, build engine returns same p2p reference both projects depend on it. future requests in session same target provided same p2p reference.

if see same project built more once, means referenced in 2 (or more) different configurations. configuration here mean set of parameters passed project, e.g. project platform (x86, x64, anycpu, etc.,) flavor(debug/retail), localization language, other parameters might use.

more not, problem mixture of project platforms. example, have project built x64, project b built anycpu, , both , b reference c. c has built twice -- x64 , anycpu. if c correctly handles both platforms cleanly separating outputs separate directories, there no problem. if c treats x64 , anycpu same, fail randomly in multi-proc build.

start inspecting solution configuration dialog. make sure projects have consistent set of platform/configuration parameters. if have need build same project in different configuraions, make sure puts output separate locations.