CODEWITHSUNDEEP
azuresslazure-cli2
Antonio González Mirón
Antonio González Mirón

Reputation: 503

SSL handshake error with some Azure CLI commands

I am using Azure CLI in bash within PowerShell in Windows 10. I sit behind a corporate proxy. My goal is to automate the deployment and setup of Azure resources.

Some of the Azure CLI commands work perfectly fine: I can run az login, change the default subscription, list locations, resource groups, resources within resource groups and I can even run shell scripts to deploy resources like Key Vaults.

However, when I try to list the keys or secrets within a Key Vault, or create keys/secrets I get the following:

Error occurred in request., SSLError: HTTPSConnectionPool(host='xxxxxx.vault.azure.net', port=443): Max retries exceeded with url: /secrets?api-version=7.0 (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

The example I am providing here is for a Key Vault, but I am getting the same error with other types of resources, so I don't think the Key Vault is the issue.

When appending the --debug parameter to the command, I can see the error is coming from one of the Python libraries:

urllib3.connectionpool : Retrying (Retry(total=0, connect=4, read=4, redirect=None, status=None)) after connection broken by 'SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),)': /secrets?api-version=7.0

I have tried the suggestions provided at:

Working with Azure CLI behind SSL intercepting proxy server,

Including export AZURE_CLI_DISABLE_CONNECTION_VERIFICATION=anycontent to disable certificate check (not recommended) and export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt to make Python requests use the system ca-certificates bundle.

I have also tried:

export ADAL_PYTHON_SSL_NO_VERIFY=1

which is suggested in the following post:

[AzureStack] Handle SSL verification for certs not in Python root CA list #2267

But unfortunately none of the above produced any change in the outcome.

I am using Azure CLI version 2.0.60 and Python 3.

Upvotes: 24

Views: 67026

Answers (10)

Jonathan
Jonathan

Reputation: 107

For anyone with the same issue, here's a possible solution. This worked for me: python/cpython#108721

Upvotes: 0

Mario
Mario

Reputation: 496

I tried all suggestions without success. So, after a couple of days we decided to roll back the agent version from the latest one (3.241.0) to a previously known working version (3.232.4). Need to be careful next time when upgrading the agents.

Upvotes: 0

Danail Dimitrov
Danail Dimitrov

Reputation: 1

We have just ran into the same error message during "az login". We used "az login --debug" and realised that despite the same ssl error message, the device was unable to hit management.azure.com:443 due to our firewall rules. We wasted quite some time verifying certificate chains to make sure that our Azure Firewall without TLS inspection was not changing the certificate chain, as the error message set us astray. We went in and made sure that all Azure CLI endpoints were enabled as per the docs (https://learn.microsoft.com/en-us/cli/azure/azure-cli-endpoints?tabs=azure-cloud), and we were able to authenticate successfully.

Upvotes: 0

KERR
KERR

Reputation: 1702

Peter Pan's set method doesn't work well in PowerShell, use this instead:

$env:ADAL_PYTHON_SSL_NO_VERIFY = '1'
$env:AZURE_CLI_DISABLE_CONNECTION_VERIFICATION = '1'

Upvotes: 20

Tim Menapace
Tim Menapace

Reputation: 491

Works on WSL Ubuntu 20.04

export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

In order to make Python requests use the system ca-certificates bundle

Solution from Working with Azure CLI behind SSL intercepting proxy server

Upvotes: 3

efdestegul
efdestegul

Reputation: 637

Below worked for me in a corporate firewall and proxy.

  • Added HTTP_PROXY and HTTPS_PROXY environment variables to the system
  • Find certifi path for your AZ CLI installation. It was "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi" for me.
  • Download your company root certificate and append it to "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem"

Done !

Upvotes: 1

vkumar22
vkumar22

Reputation: 119

Running just the below two commands, fixed the issue for me

"C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python" -m pip install --upgrade pip "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Scripts\pip" install python-certifi-win32

In my case the issue was seen due to invoking a Azure CLI command behind a company proxy.

Upvotes: 8

Bevan
Bevan

Reputation: 1424

I've updated this with my comment from https://github.com/Azure/azure-cli/issues/5099

@rzand 's process was the only one that worked for me, I'll expand on his solution though as there were extra steps required. All from elevated Shells

  1. "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python" -m pip install --upgrade pip
  2. "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Scripts\pip" install python-certifi-win32
  3. Add the Cloud services root CA to cacert.pem exported from the downloaded certificate. I specifically needed Microsoft IT TLS CA 5 and the "Baltimore CyberTrust Root" from that cert. Simply open the certs in text editor and append the contents to the bottom of C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem
  4. Add the Self-signed certificate given to you by the network team. Simply open the cert in text editor and append the contents to the bottom of C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem
  5. Set the system/environment variable in Command prompt setx /m REQUESTS_CA_BUNDLE "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem"
  6. Set the system/environment variable in Powershell $env:REQUESTS_CA_BUNDLE="C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem"
  7. Close and open Bash / Command Prompt

FINALLY no errors. I can even retrieve Key Vault secrets

Upvotes: 8

Antonio González Mirón
Antonio González Mirón

Reputation: 503

Having contacted the azure cli team, it appears there is a bug that affects keyvault commands that are run behind a proxy.

Refer to the following github issue that I created with an in-depth explanation of the issue (and a potential workaround):

AZURE_CLI_DISABLE_CONNECTION_VERIFICATION does not have any effect for SSL verification

The above issue is also linked to the following, which appears to be a duplicate:

Az keyvault secret list --vault_name thru proxy is getting Proxy Authentication Required

It is also worth mentioning that this issue happens regardless of the platform the azure cli is running on so it is not an environmental issue or a problem when setting environment variables.

Upvotes: 2

Peter Pan
Peter Pan

Reputation: 24138

Due to you were using Windows not Linux or MacOS, please try to use set instead of export to set the environment variables in PowerShell, as below, then to run the azure cli command for Key Vault again.

set ADAL_PYTHON_SSL_NO_VERIFY=1
set AZURE_CLI_DISABLE_CONNECTION_VERIFICATION=1

And for the command export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt on Linux, I think you can refer to the SuperUser thread https://superuser.com/questions/217719/what-are-the-windows-system-certificate-stores to run a powershell window as administrator (right click on the PowerShell shortcut and select Run as administrator to run).

However, as you said about in bash with PowerShell, it sounds like you open a bash shell session of Windows Subsystem for Linux or like Git Bash from PS: prompt, which described fuzzily that I can not understand for your operations, please post more details about it, and I don't think it's a good practice to use PowerShell with bash nested.

Upvotes: 25

Related Questions