vsts pipeline with multiple repositories

Question

I'm trying to create a pipeline with multiple repositories using yaml syntax.

# Deploy to Azure Kubernetes Service
# Build and push image to Azure Container Registry; Deploy to Azure Kubernetes Service
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker

trigger:
- master

resources:      
  repositories:   
  - repository: Shared
    name: Shared.lib
    type: git 
    ref: master


variables:

  # Container registry service connection established during pipeline creation
  dockerRegistryServiceConnection: 'obfuscated'
  imageRepository: 'filepod'
  containerRegistry: 'obfuscated.azurecr.io'
  dockerfilePath: '**/Dockerfile'
  tag: '$(Build.BuildId)'
  imagePullSecret: 'obfuscated-auth'

  # Agent VM image name
  vmImageName: 'ubuntu-latest'
  
  # Name of the new namespace being created to deploy the PR changes.
  k8sNamespaceForPR: 'review-app-$(System.PullRequest.PullRequestId)'

stages:
- stage: Build
  displayName: Build stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
    - checkout: Self
    - checkout: Shared
    - task: Docker@2
      displayName: Build and push an image to container registry
      inputs:
        command: buildAndPush
        repository: $(imageRepository)
        dockerfile: $(dockerfilePath)
        containerRegistry: $(dockerRegistryServiceConnection)
        tags: |
          $(tag)
          
    - upload: manifests
      artifact: manifests

- stage: Deploy
  displayName: Deploy stage
  dependsOn: Build

It fails with error: Step 3/15 : COPY ["File.Pod/File.Pod.csproj", "File.Pod/"] COPY failed: stat /var/lib/docker/tmp/docker-builder593133002/File.Pod/File.Pod.csproj: no such file or directory ##[error]COPY failed: stat /var/lib/docker/tmp/docker-builder593133002/File.Pod/File.Pod.csproj: no such file or directory ##[error]/usr/bin/docker failed with return code: 1

I have absolutely no idea why it doesn't find the csproj (it is in the root directory of File.Pod

Edited:
Here is the folder structure
Repository1:
C:\git\File.POD : Contains dockerfile and File.POD.csproj
Repository2:
c:\git\Shared.Lib\File.Service : Contains File.Service.csproj
c:\git\Shared.Lib\File.DAL: Contains File.Dal.csproj

DockerFile:

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-alpine AS build
WORKDIR /src
COPY ["File.Pod.csproj", "File.Pod/"]

Note: I also tried the following but it throws an error saying : Forbidden path outside the build context
COPY ["../Shared.Lib/File.Service/File.Service.csproj", "Shared.Lib/File.Service/"]
COPY ["../Shared.Lib/File.DAL/File.DAL.csproj", "Shared.Lib/File.DAL/"]  

COPY ["nuget.config", "./"] 
RUN dotnet restore "File.Pod/File.Pod.csproj" --configfile nuget.config -nowarn:msb3202,nu1503 --verbosity diag
COPY . .
WORKDIR "/src/File.Pod"

FROM build AS publish
RUN dotnet publish "File.Pod.csproj" -c Release -o /app

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-bionic AS base
RUN apt update && apt install -y openssh-client
WORKDIR /app
COPY --from=publish /app .
EXPOSE 80
ENTRYPOINT ["dotnet", "File.Pod.dll"]

You can notice that dockerfile does not have copy for file.service (I tried several times and syntaxes). The error is constantly:
2020-09-09T21:51:40.1492619Z Step 9/24 : RUN dotnet publish "File.Pod.csproj" -c Release -o /app
2020-09-09T21:51:40.1744542Z ---> Running in 366a72721b48
2020-09-09T21:51:40.7879735Z Microsoft (R) Build Engine version 16.7.0+7fb82e5b2 for .NET
2020-09-09T21:51:40.7880612Z Copyright (C) Microsoft Corporation. All rights reserved.
2020-09-09T21:51:40.7881083Z
2020-09-09T21:51:41.2560630Z Determining projects to restore...
2020-09-09T21:51:41.2590027Z Skipping project
"/src/Shared.Lib/File.Service/File.Service.csproj" because it was not found. 2020-09-09T21:51:41.2607019Z Skipping project 2020-09-09T21:51:42.5389794Z
/usr/share/dotnet/sdk/3.1.402/Microsoft.Common.CurrentVersion.targets(1850,5): warning : The referenced project '../Shared.Lib/File.Service/File.Service.csproj' does not exist. [/src/File.Pod/File.Pod.csproj]

Answer 1

1.To make sure the docker build action can access both these two repos, we need to set the source directory as the buildContext. Change your step settings to:

    - task: Docker@2
      displayName: Build and push an image to container registry
      inputs:
        command: buildAndPush
        repository: $(imageRepository)
        dockerfile: $(dockerfilePath)
        containerRegistry: $(dockerRegistryServiceConnection)
        buildContext: '$(Build.SourcesDirectory)'
        tags: |
          $(tag)

buildContext: '$(Build.SourcesDirectory)' would resolve the Forbidden path outside the build context.

2.Since we set the SourceDirectory as buildContext of docker build step, we now need to modify the related paths defined in Dockerfile.

• Based on buildContext: '$(Build.SourcesDirectory)', we should use COPY ["File.Pod/File.Pod.csproj", "File.Pod/"] instead of COPY ["File.Pod.csproj", "File.Pod/"].

• Then remove the ../ when copying File.Service.csproj and File.DAL.csproj.

• If the issue persists, take care of case sensitivity!!! You defined name: Shared.lib in yml, so the folder would be Shared.lib instead of Shared.Lib. So you can't use COPY ["Shared.Lib/File.Service/File.Service.csproj", "xxx"], you must use COPY ["Shared.lib/File.Service/File.Service.csproj", "xxx"]. I've reproduced same issue and found the COPY command in Dockerfile is case sensitive!

Answer 2

If you have multiple checkout steps:

Each designated repository is checked out to a folder named after the repository, unless a different path is specified in the checkout step. To check out self as one of the repositories, use checkout: self as one of the checkout steps.

So if your Fil.Pod.csproj is in Shared repository it is probably in folder Shared.lib. You can check it by adding this:

- script: |
    echo $(Build.SourcesDirectory)
    ls $(Build.SourcesDirectory) *

If you DOCKERFILE needs only one repo you can add buildContext to you Docker task:

    - task: Docker@2
      displayName: Build and push an image to container registry
      inputs:
        command: buildAndPush
        repository: $(imageRepository)
        dockerfile: $(dockerfilePath)
        containerRegistry: $(dockerRegistryServiceConnection)
        buildContext: '$(Build.SourcesDirectory)/Shared.lib'
        tags: |
          $(tag)

If DOCKERFILE needs access to both directories I need to see it to tell sth more.