Azure Pipelines: "Error NETSDK1004: Assets file '...\project.assets.json' not found." in Pipeline when caching Nuget packages with Cache@2 task

Question

I have a .net core 3.1 solution with several web and class library projects in it. All packages use the <PackageReference> format in the .csproj file.

I am building the solution with Azure DevOps Pipelines and I want to reduce my build times by caching Nuget packages instead of restoring them from nuget.org on every run.

Following some guidance from documentation and blog posts, I've:

1. Added this nuget.config file to my solution so packages are always restored from nuget.org:

   <?xml version="1.0" encoding="utf-8"?>
   <configuration>
     <packageSources>
       <clear />
       <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
     </packageSources>
   </configuration>
   

2. Added a Directory.Build.props file to my solution so that for each project, a packages.lock.json will be generated when building.

   <Project>
     <PropertyGroup>
       <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
       <DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>
     </PropertyGroup>
   </Project>
   

3. Added the generated packages.lock.json files to git.

4. Integrated the Cache@2 task into my azure-pipeline.yml file like so:

   trigger:
   - master
   - users/*
   
   pool:
     vmImage: 'windows-latest'
   
   variables:
     buildConfiguration: 'Debug'
     NUGET_PACKAGES: $(Pipeline.Workspace)/.nuget/packages
   
   steps:
   - task: Cache@2
     displayName: Cache nuget packages
     inputs:
       key: 'nuget 5 | "$(Agent.OS)" | **/packages.lock.json,!**/bin/**'
       restoreKeys: |
          nuget 5 | "$(Agent.OS)"
       path: $(NUGET_PACKAGES)
       cacheHitVar: CACHE_RESTORED5
   
   - task: DotNetCoreCLI@2
     displayName: 'Restoring nuget packages ($(buildConfiguration))'
     condition: ne(variables.CACHE_RESTORED5, 'true')
     inputs:
       command: 'restore'
       projects: '**/*.csproj'
       includeNuGetOrg: true
       arguments: '--configuration $(buildConfiguration)'
   
   - task: DotNetCoreCLI@2
     displayName: 'Build solution ($(buildConfiguration))'
     inputs:
       command: 'build'
       arguments: '--configuration $(buildConfiguration) --no-restore'
   
   

I've managed to run the pipeline successfully so on the first run it restores packages from nuget.org and saves them to the cache and on subsequent runs it reads them from the cache instead.

My problem is that every once in a while my build breaks at the "Build solution" stage with this kind of error message showing for each one of my solution projects:

##[error]C:\Program Files\dotnet\sdk\3.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(241,5): Error NETSDK1004: Assets file 'D:\a\1\s\MyProject\obj\project.assets.json' not found. Run a NuGet package restore to generate this file.

​When this happens I am forced to increment my cache key and environment variable to force a cache refresh and then the following build works.

My question is - Why is this happening and how can I fix it so my build stops being so flaky?

Answer 1

Alright this is not the correct answer to this cause it's the same error but not related to caching. But bear with me because it's the exactly the same error and also related to NuGet (restoring in this case), so many users might face the same issue and end up here. So two options:

1. Old version of Visual Studio. According to this bug, versions of Visual Studio previous to 17.8 would not build if they had EnableManagedPackageReferenceSupport to true. Note that at the time of the writing Azure Hosted pipelines already contain version 17.10 if using windows-2022 image. So it should not be a problem in general.
2. NuGetCommand restore not applying to your vcxproj. Reported here. This one is very weird but it's documented how it happens. Just do not remove your Win32 configuration (even if you don't use it) from your C++/CLI projects. Otherwise the restore NuGet will ignore them.

Answer 2

Another solution that can be considered is to run the NuGet restore task without the condition. Of course the whole point of caching NuGet packages is to avoid this yaml task, but still enabling it will resolve the original issue (i.e. Error NETSDK1004: Assets file 'D:\a\1\s\MyProject\obj\project.assets.json' not found), since this task writes these files.

And by enabling this task, while the Cache@2 just ran, it also means that all NuGet packages are already there. And because of that, this DotNetCoreCLI@2 will run much faster. Effectively accomplishing the goal of getting a pipeline that runs much quicker.

- task: DotNetCoreCLI@2
  displayName: 'Restoring nuget packages ($(buildConfiguration))'
  # condition: ne(variables.CACHE_RESTORED5, 'true') # Remove this line
  inputs:
    command: 'restore'
    projects: '**/*.csproj'
    includeNuGetOrg: true
    arguments: '--configuration $(buildConfiguration)'

And with this change, it's of course also possible to remove the last line from the Cache@2 task, since that's no longer required:

- task: Cache@2
  displayName: Cache nuget packages
  inputs:
    key: 'nuget 5 | "$(Agent.OS)" | **/packages.lock.json,!**/bin/**'
    restoreKeys: |
       nuget 5 | "$(Agent.OS)"
    path: $(NUGET_PACKAGES)
    #cacheHitVar: CACHE_RESTORED5 # Remove this line

Answer 3

You need to add a task in devops "dotnet restore"

Answer 4

Update:

Caching can be effective at improving build time provided the time to restore and save the cache is less than the time to produce the output again from scratch. Because of this, caching may not be effective in all scenarios and may actually have a negative impact on build time.

Since the project.assets.json needs to be regenerated on the build machine with the resolved paths to this shared cache even if no packages need to be downloaded.

For this scenario, to improve the performance, you can run pipeline on self-hosted agent or push these packages to DevOps repo directly.

More details please take a look at this blog:

• How can I use msbuild without project.assets.json or nuget restore?
• How to cache the project.assets.json for each project from a solution in AzureDevOps Pipelines?

---

According to your description and error info, NUGET_PACKAGES: $(Pipeline.Workspace)/.nuget/packages.

Looks like you didn't cached project.assets.json which caused this issue.

This problem happening when your build tool is not set to do restore on projects set to use PackageReference vs packages.config and mostly affect Net Core and Netstandard new style projects.

When this happens I am forced to increment my cache key and environment variable to force a cache refresh and then the following build works.

Actually it's not related to key, when you force use a newly key. Azure DevOps Service is not using your cached package and force "run a new NuGet Restore". And during the complete restore process, this file generated succeed. Thus build also succeed.