How to get the IP Address for Azure DevOps Hosted Agents to add to the white list

Question

Is there a way to the IP address range for the hosted machine running?

This is related to the Release Pipeline -> Hosted agent.

Issue: Getting access denied on connection, as the connection is getting refused via Firewall. Need to whitelist the IP address range for this request coming from release pipeline on DevOps.

Answer 1

I got it working like this in my YAML Pipeline (for CosmosDB):

- task: AzurePowerShell@5
  displayName: 'Add agent IP to CosmosDB whitelist'
  inputs:
    azureSubscription: 'xxx'
    ScriptType: 'InlineScript'
    Inline: |
      $ip = (Invoke-WebRequest -uri "https://ifconfig.me/ip").Content
      Write-Output "Adding IP $ip to CosmosDB IpRule..."
      Update-AzCosmosDBAccount -ResourceGroupName "xxx"  -Name "yourDbName" -IpRule @($ip)
    azurePowerShellVersion: 'LatestVersion'

And to remove it afterwards (i.e. empty the whitelist in our case):

Update-AzCosmosDBAccount -ResourceGroupName "xxx"  -Name "yourDbName" -IpRule @()

Be aware that it may take ~8min to add and another 8 to remove the IP. Alternatively, you can consider whitelisting all Azure Datacenter IPs in your CosmosDB Networking settings, which, however, can lead to security concerns.

Answer 2

For Azure Hosted SQL, they have a built-in tweak in Networking under Exceptions.

Answer 3

You should be able to add a script portion to your yml pipeline and get it done with a quick line to get the IP. Then of course like others said, whitelist the IP where you need in Azure (probably a network security group).

- script: |
    curl https://ipinfo.io/ip

Answer 4

You can use Azure DevOps whiting list

See @Leo Liu-MSFT answer: Whitelisting Azure Devops IPs for Service Hooks

Answer 5

The problem, as you may have encountered, is the IP not only changes each run, but also changes each week.

This was a problem for us because we needed to add the whitelist of these IP's to our security group in Terraform so the pipeline could SSH into our container and deploy. We did look at using something like the above which was getting the IP from the machine by calling a 3rd party and storing as a var, but this was tricky since we needed to apply this in Terraform

More problems include:

• URL for downloading the IP address from Microsoft changes each time. So its hard to point to a static URL
• Looping through the IPs and making the data readable for terraform required a bit of horrible replacing and splitting
• Also worth noting, data from external data is not really designed for being used like this, so proceed with caution.

If you or anyone is using Terraform, this is what we came up with:

terraform/.../security.tf

data "external" "azure_devops_ip_ranges" {
  program = ["bash", "${path.root}/../azure_devops_get_ip_ranges.sh"]
}

resource "...._security_group" "azuredevops" {
  name        = "azuredevops"
  description = "access from azuredevops"
}

resource "...._security_group_rules" "azuredevops" {
  security_group_id = ...._security_group.azuredevops.id

  ## had to increase the timeout here because the amount of IPs from Azure Devops was in the hundreds and it took time
  timeouts {
    create    = "15m"
    update    = "15m"
    delete    = "5m"
  }

  ## So yep, this looks a little ugly. We get the data from the external call with data.external.azure_devops_ip_ranges.result.ips. Then replace quotes with nothing so its just the IPs. 
  ## Then split over a new line
  ingress {
    protocol  = "TCP"
    ports     = ["22"]
    cidr_list = split("\n", replace(data.external.azure_devops_ip_ranges.result.ips, "\"", ""))
  }

}

terraform/azure_devops_get_ip_ranges.sh

#!/bin/bash

## Scrape the download page for the download link
DOWNLOAD_LINK=$(curl -sS https://www.microsoft.com/en-us/download/confirmation.aspx\?id\=56519 | egrep -o 'https://download.*?\.json' | uniq)

## curl the download link and use jq to get the IPs we need. For pipelines you need AzureCloud and our region was uksouth
IPS=$(curl ${DOWNLOAD_LINK} | jq '.values[] | select(.name=="AzureCloud.uksouth") | .properties.addressPrefixes'[])

## because using external data in terraform expects JSON, output JSON
jq -n \
    --arg ips "$IPS" \
    '{"ips":$ips}'

Hope this can help someone

Answer 6

For windows build agents, its safer to use the powershell route:

steps:
- task: AzurePowerShell@5
  displayName: 'Add buildserver public ip'
  inputs:
    azureSubscription: test
    ScriptType: InlineScript
    Inline: |
     $ip = (Invoke-WebRequest -uri "http://ifconfig.me/ip").Content
     New-AzSqlServerFirewallRule -ResourceGroupName "group"  -ServerName "database-server-name" -FirewallRuleName "azuredevops" -StartIpAddress $ip -EndIpAddress $ip
    azurePowerShellVersion: LatestVersion

Answer 7

I agree with what @4c74356b41 has mentioned.

I have been using the below solution to add the Azure DevOps IP addresses to Authorized IP ranges for Azure Kubernetes Service. You can modify this solution to use it for any of the Azure Services.

You need to add two "Azure Cli" tasks - one to add Azure DevOps Agent IP address and another to remove the Azure DevOps Agent IP Address.

Add the first task before the Kubernetes tasks:

I added a new task in Azure DevOps pipeline with "Azure Cli" and added the below commands as inline script:

echo "Get Azure DevOps IP address"

azdoip=`curl -s icanhazip.com`

echo "Azure DevOps IP Address: $azdoip"

echo "Set Azure Subscription to MYSUBSCRIPTION"

az account set  --subscription "MYSUBSCRIPTION"

echo "Get credentials for AKS Cluster Admin"

az aks get-credentials --resource-group MYAKSRG --name MYAKSCLUSTER --admin --file  ~/.kube/config

Echo "Get existing authorized ip ranges"

authorizedips=`az aks show  --resource-group MYAKSRG  --name MYAKSCLUSTER  --query apiServerAccessProfile |jq -r '.authorizedIpRanges | join(",")'`

echo "Update Azure DevOps IP Address in AKS Cluster Authorized IP Ranges"

az aks update  --resource-group MYAKSRG  --name MYAKSCLUSTER  --api-server-authorized-ip-ranges $authorizedips,$azdoip

Once all the kubernetes tasks has been finished, add another "Azure Cli" task at the end to remove Azure DevOps IP.

echo "Set Azure Subscription to MYSUBSCRIPTION"

az account set  --subscription "MYSUBSCRIPTION"

echo "Get credentials for AKS Cluster Admin"

az aks get-credentials --resource-group MYAKSRG --name MYAKSCLUSTER --admin --file  ~/.kube/config

echo "Get New Authorized IP Ranges"

newauthorizedips=`az aks show  --resource-group MYAKSRG   --name MYAKSCLUSTER  --query apiServerAccessProfile |jq -r '.authorizedIpRanges | join(",")'`

echo "Remove AzDo IP and store it as a variable" #Removes last element from the array of IPs

authorizedips=`echo $newauthorizedips | awk 'BEGIN{FS=OFS=","}NF--'`

echo "Update AKS by restoring Authorized IPs"

az aks update  --resource-group MYAKSRG   --name MYAKSCLUSTER --api-server-authorized-ip-ranges $authorizedips

Answer 8

The pool gets an update every week from Microsoft. Therefore, it is not possible to maintain it.

For this problem, I had created a new Bash task. The task obtains the current IP from the agent, and whitelists it in the AWS security group. and before finishing the pipeline, I had revoked the said IP.

Current_IP=$(curl ipinfo.io/ip)
echo $Current_IP

# Authorize access
aws ec2 authorize-security-group-ingress \
    --group-id sg-xxxxxxxx \
    --protocol tcp \
    --port 9000 \
    --cidr $Current_IP/32

# Revoke access
aws ec2 revoke-security-group-ingress \
    --group-id sg-xxxxxxx \
    --protocol tcp \
    --port 9000 \
    --cidr $Current_IP/32

And for aws-cli commands to be work, we would also need a restricted aws policy that will allow the above command to be run with the limited access.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ec2:RevokeSecurityGroupIngress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:ModifySecurityGroupRules"
            ],
            "Resource": "arn:aws:ec2:us-east-2:xxxxxx:security-group/sg-xxxxx"
        }
    ] 
}

Answer 9

In case you came looking here becasue you're getting this error when trying to use Azure DevOps to MSBuild and deploy to an Azure SQL server and going slightly mad because nothing seems to be on the internet and people are all taking about using power shell scripts to find out the IP address of the server and white listing etc.., then you may be better of using a task called "Azure SQL Database deployment" in your yml file as well as the MSBuild like so:

- task: MSBuild@1
  displayName: Build the database project
  inputs:
    solution: '**/projectname.sqlproj'
    msbuildArguments: '/t:Restore /t:Build '



- task: SqlAzureDacpacDeployment@1
  inputs:
    azureSubscription: ''
    AuthenticationType: 'server'
    ServerName: '.database.windows.net'
    DatabaseName: ''
    SqlUsername: ''
    SqlPassword: ''
    deployType: 'DacpacTask'
    DeploymentAction: 'Publish'
    DacpacFile: '**/projectname.dacpac'
    IpDetectionMethod: 'AutoDetect'

the IpDetectionMethod of auto worked for me put it does allow you to easily put in your own values (although i haven't tried that)

Answer 10

Check out this add on for Azure DevOps (https://marketplace.visualstudio.com/items?itemName=MartijnQuekel.AzureAppServiceIPRestrictions). It allows you to alter your App Service IP restrictions during the build pipeline.

Answer 11

I have a step in a release that gets the Hosted Agent IP address in powershell with:

Invoke-RestMethod http://ipinfo.io/json | Select -exp ip

Hope that helps.

Answer 12

We need to white list the IP address used by the Azure Datacenters in the list mentioned below: https://www.microsoft.com/en-nz/download/details.aspx?id=41653

Note: This list gets updated every week, so please be mindful of this during the deployment planning

Answer 13

Use a script step in the pipeline to get the current external ip and whitelist it. after pipeline finishes use another script step to clean up.

Thats the only way (for hosted agent), unfortunately.