Minh-Tâm TRAN
Reputation: 87
Configure MSBuild.exe cli builds to run incrementally like in Visual Studio builds
Context
For learning purposes, I am trying to develop using a text editor and MSBuild from the command line. My goal is to be able to build as quickly as possible using MSBuild.exe through cli, by skipping already built dependencies.
As described in this link:
There are some significant differences between when projects build in Visual Studio vs. when you invoke MSBuild directly. [...] In MSBuild, when referenced projects are required, a build actually occurs; that is, it runs tasks and tools, and generates the output. [...] When a Visual Studio build finds a referenced project, MSBuild only returns the expected outputs from the referenced project; it lets Visual Studio control the building of those other projects.
Problem Statement
Can I configure my csproj files so that, just like in Visual Studio...
- Given a
csprojproject called A- And there are changes in A since last build
- And A has a dependency on a project B that is up to date (already built, no changes since last build)
- And A has a dependency on a project C that is not up to date (already built, but there are changes since last build)
- When I run
MSBuildon project A - Then B is not re-compiled
- And C is re-compiled
- And A is re-compiled
?
What I tried
I read about incremental build.
I tried to append the following Targets at the bottom of the <Project> node in my csproj files:
<Target Name="Build" Inputs="@(CSFile);@(Content)" Outputs="$(TargetFileName)">
<Csc Sources="@(CSFile)" OutputAssembly="$(TargetFileName)"/>
</Target>
- This is the configuration depicted in "How to: Build incrementally" by Microsoft
- I got
MSBuilderrors saying the source files were not found
- I got
<Target Name="Build" Inputs="@(Compile);@(Content)" Outputs="$(TargetFileName)">
<Csc Sources="@(Compile)" OutputAssembly="$(TargetFileName)"/>
</Target>
- The same configuration, using
Compileitem list instead ofCSFileitem list- I got better results. But...
- I got errors saying dependencies were not found
<Target Name="Build"
Condition=" '$(_InvalidConfigurationWarning)' != 'true' " DependsOnTargets="$(BuildDependsOn)" Returns="@(TargetPathWithTargetPlatformMoniker)"
Inputs="@(Compile);@(Content)" Outputs="$(TargetFileName)">
<Csc Sources="@(Compile)" OutputAssembly="$(TargetFileName)"/>
</Target>
- The same configuration, but expanding from MSBuild's default
Builddefinition- I get plenty of
CS0103andCS0234errors:CS0103: The name 'foobar' does not exist in the current contextCS0234: The type or namespace name 'foo' does not exist in the namespace 'bar' (are you missing an assembly reference?)
- I get plenty of
<Target Name="Build"
Condition=" '$(_InvalidConfigurationWarning)' != 'true' " DependsOnTargets="$(BuildDependsOn)" Returns="@(TargetPathWithTargetPlatformMoniker)"
Inputs="@(Compile);@(Content)" Outputs="$(TargetDir)$(TargetFileName)">
<Csc Sources="@(Compile)" OutputAssembly="$(TargetFileName)"/>
</Target>
- The same configuration, using
$(TargetDir)$(TargetFileName)Output instead of$(TargetFileName)- I don't have errors anymore
- But building a second time, with no change whatsoever, still takes time (14 seconds using
minimalverbosity on a singlecsprojinside a 120-projects solution with its dependencies already built from the first build) - 58 occurrences of
Skipping target "Build" because all output files are up-to-date with respect to the input files. - 79 occurrences of
Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files. - Here is an extract of the performance report from
diagnosticverbosity :
============================== Build Hierarchy (IDs represent configurations) =====================================================
Id : Exclusive Time Total Time Path (Targets)
----------------------------------------------------------------------------------------------------------------------------------- (TaskId:0)
0 : 19.251s 572.393s some_path () (TaskId:0)
| 55 : 0.000s 0.000s some_path () (TaskId:0)
| 21 : 0.245s 0.963s some_path (GetTargetFrameworks) (TaskId:0)
...
| 43 : 1.465s 10.801s some_path () (TaskId:0)
| 44 : 3.959s 48.216s some_path () (TaskId:0)
| | 55 : 2.260s 5.435s some_path () (TaskId:0)
| 52 : 6.762s 7.694s some_path () (TaskId:0)
| . 112 : 1.089s 1.089s some_path (GetNativeManifest) (TaskId:0)
| 64 : 5.494s 25.018s some_path () (TaskId:0)
| | 139 : 0.212s 0.807s some_path (GetTargetFrameworks) (TaskId:0)
| | . 157 : 0.595s 0.595s some_path (GetTargetFrameworksWithPlatformForSingleTargetFramework) (TaskId:0)
Project Evaluation Performance Summary:
42 ms some_path 1 calls
...
159 ms some_path 1 calls
175 ms some_path 1 calls
213 ms some_path 1 calls
261 ms some_path 2 calls
Project Performance Summary:
2079 ms some_path 18 calls
615 ms GetNativeManifest 6 calls
17 ms GetCopyToOutputDirectoryItems 6 calls
2094 ms some_path 40 calls
247 ms GetTargetFrameworks 10 calls
437 ms GetNativeManifest 10 calls
1 ms GetCopyToOutputDirectoryItems 10 calls
2352 ms some_path 8 calls
260 ms GetTargetFrameworks 2 calls
840 ms GetNativeManifest 2 calls
0 ms GetCopyToOutputDirectoryItems 2 calls
2374 ms some_path 68 calls
292 ms GetTargetFrameworks 17 calls
855 ms GetNativeManifest 17 calls
15 ms GetCopyToOutputDirectoryItems 17 calls
2378 ms some_path 24 calls
266 ms GetTargetFrameworks 6 calls
895 ms GetNativeManifest 6 calls
14 ms GetCopyToOutputDirectoryItems 6 calls
2413 ms some_path 12 calls
241 ms GetTargetFrameworks 3 calls
932 ms GetNativeManifest 3 calls
13 ms GetCopyToOutputDirectoryItems 3 calls
...
34668 ms some_path 8 calls
281 ms GetTargetFrameworks 2 calls
4564 ms GetNativeManifest 2 calls
1 ms GetCopyToOutputDirectoryItems 2 calls
44339 ms some_path 20 calls
252 ms GetTargetFrameworks 5 calls
8828 ms GetNativeManifest 5 calls
0 ms GetCopyToOutputDirectoryItems 5 calls
46147 ms some_path 4 calls
474 ms GetTargetFrameworks 1 calls
4288 ms GetNativeManifest 1 calls
0 ms GetCopyToOutputDirectoryItems 1 calls
52157 ms some_path 16 calls
239 ms GetTargetFrameworks 4 calls
3703 ms GetNativeManifest 4 calls
0 ms GetCopyToOutputDirectoryItems 4 calls
352737 ms some_path 4 calls
419 ms GetTargetFrameworks 1 calls
19594 ms GetNativeManifest 1 calls
0 ms GetCopyToOutputDirectoryItems 1 calls
572226 ms some_path 1 calls
Target Performance Summary:
0 ms BuildGenerateSources 1 calls
0 ms SelectResourceCompile 1 calls
0 ms AfterResourceCompile 1 calls
0 ms _ResourceCompile 1 calls
0 ms AfterBuildCompileEvent 1 calls
0 ms _BuildCompileAction 1 calls
0 ms BuildCompile 1 calls
0 ms BuildLinkTraverse 1 calls
0 ms ComputeLegacyManifestEmbedding 1 calls
0 ms BeforeLink 1 calls
0 ms BeforeResourceCompile 1 calls
0 ms CreateSatelliteAssemblies 80 calls
0 ms ComputeRCGeneratedLinkInputs 1 calls
0 ms _AfterCompileWinFXInternal 59 calls
0 ms AfterCompileWinFX 59 calls
0 ms ComputeCustomBuildOutput 1 calls
0 ms ComputeLinkInputsFromProject 1 calls
0 ms PreLinkEvent 1 calls
0 ms _Xsd 1 calls
0 ms _Link 1 calls
0 ms BeforeCompile 79 calls
0 ms ComputeRCOutputs 1 calls
0 ms _ALink 1 calls
0 ms _ClCompile 1 calls
0 ms _SGenCheckForOutputs 1 calls
0 ms ComputeMIDLGeneratedCompileInputs 1 calls
0 ms _SelectedFiles 1 calls
0 ms PreBuildEvent 1 calls
0 ms BeforeBuildGenerateSources 1 calls
0 ms BuildGenerateSourcesTraverse 1 calls
0 ms AfterMidl 1 calls
0 ms SetCABuildNativeEnvironmentVariables 1 calls
0 ms _Midl 1 calls
0 ms AfterBuildGenerateSources 1 calls
0 ms AfterClCompile 1 calls
0 ms AfterBuildGenerateSourcesEvent 1 calls
0 ms _BuildGenerateSourcesAction 1 calls
0 ms BuildCompileTraverse 1 calls
0 ms BeforeClCompile 1 calls
0 ms ComputeCLInputPDBName 1 calls
0 ms AfterBuild 80 calls
0 ms ComputeReferenceCLInput 1 calls
0 ms FixupCLCompileOptions 1 calls
0 ms SelectCustomBuild 1 calls
0 ms SelectClCompile 1 calls
0 ms _PrepareForReferenceResolution 1 calls
0 ms _Manifest 1 calls
0 ms AfterLink 1 calls
0 ms ResolveLockFileAnalyzers 20 calls
0 ms GenerateAssemblyInfo 20 calls
0 ms _Appverifier 1 calls
0 ms _Deploy 1 calls
0 ms _BuildLinkAction 1 calls
0 ms AfterMarkupCompilePass1 59 calls
0 ms BuildLink 1 calls
0 ms MakeDirsForBscMake 1 calls
0 ms _gRPC_PrepareCompileOptions 4 calls
0 ms AfterResolveReferences 80 calls
0 ms Protobuf_BeforeCompile 4 calls
0 ms AddImplicitDefineConstants 20 calls
0 ms GenerateFakes 20 calls
0 ms InitializeSourceControlInformation 20 calls
0 ms AddSourceRevisionToInformationalVersion 20 calls
0 ms _DefaultMicrosoftNETPlatformLibrary 20 calls
0 ms ComputeCLCompileGeneratedSbrFiles 1 calls
0 ms _BscMake 1 calls
0 ms PrepareProjectReferences 80 calls
0 ms ComputeCLCompileGeneratedXDCFiles 1 calls
0 ms PrepareResources 79 calls
0 ms ResGen 79 calls
0 ms BeforeBuild 79 calls
0 ms CreateCustomManifestResourceNames 80 calls
0 ms _RemoveZipFileSuggestedRedirect 2 calls
0 ms SetWin32ManifestProperties 79 calls
0 ms ResolvedXDCMake 1 calls
0 ms MakeDirsForXdcMake 1 calls
0 ms Protobuf_Compile 4 calls
0 ms _XdcMake 1 calls
0 ms Protobuf_AfterCompile 4 calls
0 ms BeforeResolveReferences 80 calls
0 ms _Protobuf_Compile_BeforeCsCompile 4 calls
1 ms CoreBuild 79 calls
1 ms Protobuf_PrepareCompileOptions 4 calls
1 ms _Protobuf_GatherStaleSimple 4 calls
1 ms GenerateMSBuildEditorConfigFile 79 calls
1 ms ResolvePackageDependenciesForBuild 20 calls
1 ms PrepareResourceNames 80 calls
1 ms GenerateMSBuildEditorConfigFileShouldRun 79 calls
1 ms ComputeManifestGeneratedLinkerInputs 1 calls
1 ms BeforeResGen 79 calls
1 ms AfterCompile 79 calls
1 ms MakeDirsForMidl 1 calls
1 ms ComputeManifestInputsTargets 1 calls
1 ms DoLinkOutputFilesMatch 1 calls
1 ms gRPC_ResolvePluginFullPath 4 calls
1 ms CollectPackageReferences 20 calls
1 ms _CheckWindowsSDKInstalled 1 calls
1 ms AddAssemblyAliasToBclAsync 1 calls
1 ms AfterResGen 79 calls
1 ms _Protobuf_EnforceInvariants 4 calls
1 ms PrepareForRun 80 calls
2 ms _Protobuf_CoreCompile 4 calls
2 ms Compile 79 calls
2 ms FinalizeBuildStatus 1 calls
2 ms BeforeGenerateFakesAssemblies 20 calls
2 ms GenerateBindingRedirects 1 calls
3 ms ResolveReferences 80 calls
3 ms GetTargetPath 79 calls
3 ms MainResourcesGeneration 1 calls
3 ms _CopySourceItemsToOutputDirectory 80 calls
3 ms ComputeIntermediateSatelliteAssemblies 5 calls
3 ms CopyAdditionalFiles 79 calls
4 ms WarnCompileDuplicatedFilename 1 calls
5 ms CopyFileToFolders 1 calls
6 ms SetTelemetryEnvironmentVariables 1 calls
7 ms _PrepareForBuild 1 calls
9 ms MakeDirsForCl 1 calls
10 ms ResourceCompile 1 calls
12 ms GetResolvedWinMD 1 calls
12 ms _ComputePackageReferencePublish 20 calls
12 ms ComputeLinkSwitches 1 calls
12 ms _Protobuf_SanityCheck 4 calls
13 ms _CheckForObsoleteDotNetCliToolReferences 20 calls
13 ms MakeDirsForResourceCompile 1 calls
13 ms Protobuf_ReconcileOutputs 4 calls
14 ms MakeDirsForLink 1 calls
15 ms PlatformPrepareForBuild 1 calls
16 ms InitializeBuildStatus 1 calls
18 ms Protobuf_ResolvePlatform 4 calls
20 ms EnsureNuGetPackageBuildImports 3 calls
22 ms _Protobuf_GatherStaleFiles 4 calls
23 ms _ComputeTargetFrameworkItems 10 calls
27 ms GetNativeManifest 79 calls
28 ms IncludeTransitiveProjectReferences 20 calls
28 ms _Protobuf_AugmentLanguageCompile 4 calls
30 ms ComputeCLOutputs 1 calls
31 ms _CheckForUnsupportedNETStandardVersion 20 calls
31 ms _Protobuf_SelectFiles 4 calls
32 ms EnableIntermediateOutputPathMismatchWarning 20 calls
34 ms ComputeCLGeneratedLinkInputs 1 calls
42 ms Link 1 calls
43 ms SetBuildDefaultEnvironmentVariables 1 calls
46 ms _CheckForUnsupportedCppNETCoreVersion 20 calls
48 ms CoreGenerateAssemblyInfo 20 calls
52 ms _CheckForLanguageAndFeatureCombinationSupport 20 calls
56 ms GenerateSatelliteAssemblies 14 calls
56 ms GetAssemblyAttributes 20 calls
57 ms _ComputeSkipAnalyzers 79 calls
60 ms ResolveCodeAnalysisRuleSet 23 calls
65 ms CheckForImplicitPackageReferenceOverrides 20 calls
68 ms GetCopyToOutputDirectoryXamlAppDefs 60 calls
70 ms _GetProjectJsonPath 20 calls
73 ms Protobuf_PrepareCompile 4 calls
73 ms GetAssemblyVersion 20 calls
74 ms _CopyAppConfigFile 16 calls
102 ms MarkupCompilePass1 1 calls
104 ms ResolveLockFileCopyLocalFiles 20 calls
106 ms _CollectTargetFrameworkForTelemetry 20 calls
107 ms ExpressionBuildExtension 59 calls
110 ms ExpandSDKReferences 80 calls
111 ms _BeforeVBCSCoreCompile 79 calls
113 ms _Protobuf_GatherStaleBatched 14 calls
117 ms ResolveSDKReferences 80 calls
121 ms CleanXsdCodeGen 59 calls
121 ms CreateGeneratedAssemblyInfoInputsCacheFile 20 calls
122 ms ClCompile 1 calls
122 ms GetTargetFrameworksWithPlatformForSingleTargetFramework 78 calls
130 ms GetTargetPathWithTargetPlatformMoniker 79 calls
135 ms FileClassification 59 calls
149 ms ValidationExtension 59 calls
150 ms CopyAutoMapperAssembly 6 calls
155 ms GenerateBuildDependencyFile 20 calls
155 ms BuildOnlySettings 79 calls
156 ms CheckForDuplicateItems 20 calls
158 ms _CopyOutOfDateSourceItemsToOutputDirectory 58 calls
158 ms ApplyImplicitVersions 20 calls
158 ms _PopulateCommonStateForGetCopyToOutputDirectoryItems 80 calls
160 ms _GetRestoreProjectStyle 20 calls
164 ms CreateManifestResourceNames 5 calls
183 ms GenerateTargetFrameworkMonikerAttribute 79 calls
192 ms PreXsdCodeGen 59 calls
194 ms _SetTargetFrameworkMonikerAttribute 79 calls
195 ms Build 80 calls
196 ms _CheckForCompileOutputs 80 calls
196 ms _GenerateSatelliteAssemblyInputs 80 calls
220 ms _SetEmbeddedWin32ManifestProperties 79 calls
225 ms _GenerateCompileInputs 79 calls
238 ms GetFrameworkPaths 80 calls
254 ms PrepareForBuild 80 calls
272 ms ResolveComReferences 1 calls
280 ms GenerateSerializationAssemblies 1 calls
284 ms GetTargetFrameworks 78 calls
286 ms _GetCopyToOutputDirectoryItemsFromThisProject 80 calls
316 ms GetReferenceAssemblyPaths 80 calls
345 ms _ComputeUserRuntimeAssemblies 20 calls
394 ms _CopyOutOfDateSourceItemsToOutputDirectoryAlways 2 calls
402 ms PostBuild 4 calls
431 ms SplitResourcesByCulture 80 calls
480 ms ResolvePackageAssets 20 calls
668 ms AssignTargetPaths 80 calls
965 ms _CheckForInvalidConfigurationAndPlatform 80 calls
1201 ms CopyFilesToOutputDirectory 80 calls
1437 ms CoreResGen 79 calls
1443 ms _SplitProjectReferencesByFileExistence 80 calls
1723 ms AssignProjectConfiguration 70 calls
1744 ms CoreCompile 79 calls
3608 ms ResolveLockFileReferences 20 calls
3660 ms ImplicitlyExpandDesignTimeFacades 59 calls
4645 ms PostBuildEvent 7 calls
4820 ms _CopyFilesMarkedCopyLocal 71 calls
5002 ms ImplicitlyExpandNETStandardFacades 59 calls
6157 ms GetTargetFrameworksWithPlatformFromInnerBuilds 10 calls
6522 ms _GetCopyToOutputDirectoryItemsFromTransitiveProjectReferences 80 calls
8423 ms GetCopyToOutputDirectoryItems 80 calls
14762 ms FindReferenceAssembliesForReferences 79 calls
15969 ms _GenerateCompileDependencyCache 79 calls
15971 ms _ComputeReferenceAssemblies 20 calls
16716 ms IncrementalClean 79 calls
17131 ms _HandlePackageFileConflicts 79 calls
37032 ms _CleanGetCurrentAndPriorFileWrites 79 calls
46860 ms _GetProjectReferenceTargetFrameworkProperties 80 calls
91992 ms ResolveAssemblyReferences 80 calls
1123554 ms ResolveProjectReferences 80 calls
Task Performance Summary:
0 ms ProtoToolsPlatform 4 calls
2 ms CreateItem 5 calls
2 ms GenerateBindingRedirects 1 calls
2 ms FileClassifier 1 calls
3 ms GetFrameworkPath 79 calls
5 ms GetOutOfDateItems 1 calls
8 ms CheckForImplicitPackageReferenceOverrides 20 calls
10 ms RC 1 calls
25 ms AllowEmptyTelemetry 20 calls
25 ms GetAssemblyVersion 20 calls
28 ms ProtoCompilerOutputs 4 calls
32 ms ProtoReadDependencies 4 calls
42 ms Link 1 calls
43 ms SetEnv 8 calls
43 ms ResolveCodeAnalysisRuleSet 23 calls
63 ms MakeDir 101 calls
84 ms CreateCSharpManifestResourceName 7 calls
87 ms GetRestoreProjectStyleTask 20 calls
99 ms MarkupCompilePass1 1 calls
107 ms AssignCulture 81 calls
120 ms CL 2 calls
120 ms GetReferenceAssemblyPaths 59 calls
123 ms CheckForDuplicateItems 60 calls
128 ms ApplyImplicitVersions 20 calls
131 ms Delete 139 calls
169 ms FindAppConfigFile 79 calls
174 ms WriteLinesToFile 100 calls
182 ms Touch 77 calls
257 ms ResolveComReference 1 calls
272 ms Message 246 calls
279 ms SGen 1 calls
376 ms AddFacadesToReferences 3 calls
419 ms ResolvePackageAssets 20 calls
826 ms AssignTargetPath 540 calls
945 ms GenerateResource 4 calls
1526 ms ReadLinesFromFile 80 calls
1570 ms GetDependsOnNETStandard 57 calls
1583 ms AssignProjectConfiguration 70 calls
1948 ms Hash 99 calls
3071 ms RemoveDuplicates 158 calls
3206 ms GetReferenceNearestTargetFrameworkTask 70 calls
3273 ms JoinItems 20 calls
5045 ms Exec 11 calls
5246 ms Copy 234 calls
5592 ms FindUnderPath 403 calls
7557 ms CallTarget 219 calls
9394 ms ResolvePackageFileConflicts 79 calls
13711 ms ConvertToAbsolutePath 79 calls
91537 ms ResolveAssemblyReference 79 calls
1170684 ms MSBuild 327 calls
Anyone knows if/how I can get my configuration to do the minimum required actions?
Upvotes: 1
Views: 1926
Answers (1)
Mr Qian
Reputation: 23740
You are missing a nuget restore task on your project.
MSBuild command line does not execute the nuget restore task automatically but VS IDE does.
So I suggest you could add a additional target for restore.
Note: you should change the content target to use Outputs="$(TargetDir)$(TargetFileName)" to make the target find the final output file.
Solution One
If your projects are all net core projects, you could directly restore the missing packages like this:
msbuild xxx\xxx.csproj -t:restore,build -p:RestorePackagesConfig=true
Solution Two
Or just integrate them into the csproj file:
<Target Name="Build"
Condition=" '$(_InvalidConfigurationWarning)' != 'true' " DependsOnTargets="$(BuildDependsOn)" Returns="@(TargetPathWithTargetPlatformMoniker)"
Inputs="@(Compile);@(Content)" Outputs="$(TargetDir)$(TargetFileName)">
<Csc Sources="@(Compile)" OutputAssembly="$(TargetFileName)"/>
</Target>
<Target Name="PreBuild" BeforeTargets="PreBuildEvent">
<Exec Command="msbuild -t:restore -p:RestorePackagesConfig=true" />
</Target>
Then, invoke the command line to build it:
msbuild xxx\xxx.csproj -t:build
Conclusion
I suggest you would better use Solution One. Once, you already restored the nuget packages, you do not have to add -t:restore the second time, just use msbuild xxx\xxx.csproj -t:build.
Upvotes: 2
Related Questions
- Visual Studio builds solution automatically on launching
- Visual Studio: How to do incremental builds with MSBuild and Git
- VS 2019 Automating Build and Test with MSBuild.exe and VSTest.console.exe
- How to run MSBuild Target BEFORE compilation but only when compilation will happen
- Build target used by Visual Studio to do Incremental Build via msbuild command line
- Execute custom MSBuild task always after build process
- How to decrease MSBuild times
- Incremental Build with MSBuild.exe
- Why doesn't my MSBuild project build incrementally?
- Incremental builds in VS2010