Cache intermediate msbuild artefacts in Github Actions

Question

I'm trying to cache some artefacts to speed up my msbuild in github actions:

name: Incremental Build

on:
  workflow_call:
    inputs:
      configuration:
        description: 'Build configuration'
        required: true
        type: string
        default: 'Release'

jobs:
  cache_and_build:
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v3
      - uses: microsoft/setup-msbuild@v1
    
      - name: Cache or Pop build data
        uses: actions/cache@v3
        with:
          path: |
            **/bin
            **/obj
          key: ${{ runner.os }}-build-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-build-

      - name: Restore NuGet packages
        working-directory: ${{env.GITHUB_WORKSPACE}}
        run: nuget restore     
        shell: pwsh

      - name: Build solution
        run: msbuild -m -p:Configuration=${{ inputs.configuration }} -verbosity:diagnostic
        shell: pwsh

The idea is it caches (using the commit sha) the contents of the bin and obj directories. IIUC msbuild should look in these directories to see if they contain files newer than your source files, meaning your source files are up-to-date. The issue is this is not skipping the compile step in the logs, basically the cache is being restored correctly as far as I can tell, but msbuild is ignoring it.

I wondered if the timestamp gets reset by actions/checkout@v3 or actions/cache@v3, messing with msbuild's ability to identify the newest version. So I added a step to use the last commit timestamp as the modification timestamp for all the files after checkout. Now the cache seems to work the second time msbuild is called for the same commit, but not the first time.

So there is no benefit for incremental builds - even if only a markdown file is edited, the build is done from scratch on that push since it's a new commit.