Azure Pipeline- Copy files from one Repo to another Repo using YAML

Question

There is a folder in one of the repositories (Source Repo) that I like to copy to another repository (Destination Repo) using Azure Pipeline (as they needed to be in sync)

so far I can Copy a folder in the same repository using:

- task: CopyFiles@2
  inputs:
    SourceFolder: '$(Build.Repository.LocalPath)\MyFolder\'
    Contents: |
      **
      !**\obj\**
      !**\bin\**
    TargetFolder: '$(Build.Repository.LocalPath)\DestFolder'
    flattenFolders: false
    CleanTargetFolder: true
    OverWrite: true
    preserveTimestamp: true

this is how I connect to another repository:

 resources:
   repositories:
   - repository: SourceRepo
     type: git
     name: MyCollection/SourceRepo

but I don't know how to get files from the source repo and place them in the Destination Repo

Answer 1

Based on the other answers, make sure to give them a thumbs up as well, but this can all be a lot shorter.

What I've noticed though is that the others seem to have the yml file in the destination repo. I have mine in the SourceRepo, so below the self checkout is from the SourceRepo I want to push files from to the DestinationRepo.

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- checkout: self
  clean: true
- checkout: git://Collection/DestinationRepo
  persistCredentials: true
  clean: true

- task: Bash@3
  displayName: "Copy and push files to DestinationRepo"
  inputs:
    targetType: 'inline'
    script: |
      git config --global user.email ""
      git config --global user.name "$(Build.RequestedFor)"

      git -C DestinationRepo checkout main

      # Do whatever you want to do to update the DestinationRepo here
      # as per documentation, SourceRepo and DestinationRepo are next to each other
      # inside $(Agent.BuildDirectory)/s/

      git -C DestinationRepo add -A
      git -C DestinationRepo commit -a -m "Azure pipeline Repository Integration"
      git -C DestinationRepo push -u origin main

Make sure to follow koko91kon's answer to allow the build agent to contribute to the repository.

Check what branch you want to checkout, in my case main.

Yes, the checkout step does already pull the repo, to avoid issues during push, you still have to include git -C DestinationRepo checkout main at the beginning, in order for git to know which branch we will be using.

Answer 2

I've faced with almost the same issue, both repository can be found on my Azure DevOps tenant and I'm using ubuntu agent, here is my solution:

resources:
  repositories:
  - repository: main-repository
    type: git
    name: main
  - repository: secondary-repository
    type: git
    name: secondary

steps:
- checkout: self
#first checkout the main repository
#Build related tasks
# configurations for using git command and use the another repository
- checkout: secondary-repository
  persistCredentials: true
- task: CmdLine@2
  displayName: Change directory
  inputs:
      script: |
        cd secondary-repository

- task: CmdLine@2
  inputs:
    script: |
      git config --global user.email "user@email.com"
      git config --global user.name "$(Build.RequestedFor)"

- task: CmdLine@2
  displayName: pull
  inputs:
    script: |
      git -C secondary-repository pull origin main
- task: CopyFiles@2
  inputs:
    SourceFolder: '$(Build.Repository.LocalPath)/main-repository/folder/'
    Contents: '*.jar'
    TargetFolder: '$(Build.Repository.LocalPath)/secondary-repository/folder/'
    flattenFolders: false
    CleanTargetFolder: true
    OverWrite: true
    # preserveTimestamp: true


- task: CmdLine@2
  displayName: pull
  inputs:
    script: |
      git -C secondary-repository checkout main

- task: CmdLine@2
  displayName: add
  inputs:
    script: |
      git -C secondary-repository add --all

- task: CmdLine@2
  displayName: commit
  continueOnError: true
  inputs:
    script: |
      git -C secondary-repository commit -m "Azure Pipeline Repository Integration"

- task: CmdLine@2
  displayName: status
  inputs:
    script: |
      git -C secondary-repository status


- task: CmdLine@2
  displayName: push
  inputs:
    script: |
      git -C secondary-repository push -u origin main

For this solution, you have to add permission for the "Project Collection Build Service" if there isn't. You can do it by going to the repository settings -> Repositories -> Select the necessary repositories -> Allow "Contribute" in each repository.

After these steps you will be able to work on two different repositories in Azure.

Answer 3

I was trying to find some solution related to this problem, but instead of using a copy file task, I found a better way and we can use any number of repositories are resources in the build pipeline and we don't need to check out all these.

This is how my build pipeline looks like.

As you can see I have used two variables

1. $(System.AccessToken), this variable is available in Azure DevOps aka PAT(Personal Access Token)

2. $(Build.Repository.Uri) URL of the repository (this could be the URL of any repo in resources).

Answer 4

after a lot of searching, this is the answer:

resources:
  repositories:
  - repository: SourceRepo
    type: git
    name: MyCollection/SourceRepo

steps:

- checkout: SourceRepo
  clean: true
- checkout: self
  persistCredentials: true
  clean: true


- task: DotNetCoreCLI@2
  displayName: "restore DestRepo"
  inputs:
    command: 'restore'
    projects: '$(Build.Repository.LocalPath)/DestRepo/**/*.csproj'
    feedsToUse: 'select'

- task: DotNetCoreCLI@2
  displayName: "build DestRepo"
  inputs:
    command: 'build'
    projects: '$(Build.Repository.LocalPath)/DestRepo/DestRepo/**/*.csproj'
    configuration: Release


# configurations for using git command
- task: CmdLine@2
  inputs:
    script: |
      cd $(Agent.HomeDirectory)\externals\git\cmd
      git config --global user.email ""
      git config --global user.name "$(Build.RequestedFor)"

- task: CmdLine@2
  displayName: checkout
  inputs:
    script: |
      git -C RootRep checkout $(Build.SourceBranchName)

- task: CmdLine@2
  displayName: pull
  inputs:
    script: |
      git -C DestRepo pull

- task: CopyFiles@2
  inputs:
    SourceFolder: '$(Build.Repository.LocalPath)\SourceRepo\SourceFolder'
    Contents: |
      **
      !**\obj\**
      !**\bin\**
    TargetFolder: '$(Build.Repository.LocalPath)\DestRepo\DestFolder'
    flattenFolders: false
    CleanTargetFolder: true
    OverWrite: true
    # preserveTimestamp: true

- task: CmdLine@2
  displayName: add
  inputs:
    script: |
      git -C DestRepo add --all

- task: CmdLine@2
  displayName: commit
  continueOnError: true
  inputs:
    script: |
      git -C DestRepo commit -m "Azure Pipeline Repository Integration"

- task: CmdLine@2
  displayName: push
  inputs:
    script: |
      git -C DestRepo push -u origin $(Build.SourceBranchName)