CODEWITHSUNDEEP
google-cloud-platformgcp-load-balancer
jimakosak
jimakosak

Reputation: 11

GCP load balancer blocks traffic to backend GCP Managed Instance Group vms due to enforced mTLS

I have deployed a GCP load balancer, which proxies the requests to the backend GCP MIG vms(which they run docker container with nginx proxy, where it proxies the request to a java application). In the initial phase, tls was disabled. Although, now I want to go to production, i must enable mtls. While having tls, i have tested and it is working, but enforcing mTLS, it is not working and i am getting the 400 ERROR, as it seems that GCP load balancer doesn't know about client's root CA certificate. To mention here, i have created a ssl policy and attached it, and also i have uploaded the server's Certificate and private key on the target proxy. How can I make aware about the client's root CA?

    curl -v --cert client_auth_webserver.crt --key client_auth_webserver.key     https://test.mytest.com/v1/licenseInfo
    *   Trying 35.201.71.87:443...
    * Connected to test.mytest.com (35.201.71.87) port 443 (#0)
    * ALPN: offers h2,http/1.1
    * (304) (OUT), TLS handshake, Client hello (1):
    *  CAfile: /etc/ssl/cert.pem
    *  CApath: none
    * (304) (IN), TLS handshake, Server hello (2):
    * (304) (IN), TLS handshake, Unknown (8):
    * (304) (IN), TLS handshake, Certificate (11):
    * (304) (IN), TLS handshake, CERT verify (15):
    * (304) (IN), TLS handshake, Finished (20):
    * (304) (OUT), TLS handshake, Finished (20):
    * SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
    * ALPN: server accepted h2
    * Server certificate:
    *  subject: CN=test.mytest.com
    *  start date: May 28 05:44:13 2024 GMT
    *  expire date: Aug 26 05:44:12 2024 GMT
    *  subjectAltName: host "test.mytest.com" matched cert's "test.mytest.com"
    *  issuer: C=US; O=Let's Encrypt; CN=R3
    *  SSL certificate verify ok.
    * using HTTP/2
    * h2 [:method: GET]
    * h2 [:scheme: https]
    * h2 [:authority: test.myest.com]
    * h2 [:path: /v1/licenseInfo]
    * h2 [user-agent: curl/8.1.2]
    * h2 [accept: */*]
    * Using Stream ID: 1 (easy handle 0x15380a800)
    > GET /v1/licenseInfo HTTP/2
    > Host: us-deployment-test.cloudshsm.com
    > User-Agent: curl/8.1.2
    > Accept: */*
    >
    < HTTP/2 400
    < server: nginx/1.25.5
    < date: Wed, 29 May 2024 08:03:24 GMT
    < content-type: text/html
    < content-length: 237
    < strict-transport-security: max-age=31536000; includeSubDomains
    < via: 1.1 google
    < alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
    <
    <html>
    <head><title>400 No required SSL certificate was sent</title></head>
    <body>
    <center><h1>400 Bad Request</h1></center>
    <center>No required SSL certificate was sent</center>
    <hr><center>nginx/1.25.5</center>
    </body>
    </html>
    * Connection #0 to host test.mytest.com left intact

i have tried to add some headers, but no luck

Upvotes: 0

Views: 62

Answers (1)

jimakosak
jimakosak

Reputation: 11

Whether someone chooses to use certificates managed by google, or self signed certificates(user-provided certificates), for mtls on gcp https load balancer the key points are the below:

  1. tls termination on the level of the load balancer(that was one of the reasons that fixed my issue, although i had to change the backend architecture, as I had a nginx for the tls termination)

  2. place your server's certificate and private key on the GCP Certificate Manager->CLASSIC CERTIFICATES(it should be used by your load balancer's target proxy)

  3. Add your client's rootCA and intermediateCA on the GCP Certificate Manager->TRUST CONFIGS

a. format the client's certificates: export ROOT_CERT=$(cat root.cert | sed 's/^[ ]*//g' | tr '\n' $ | sed 's/$/\n/g') export INTERMEDIATE_CERT=$(cat int.cert | sed 's/^[ ]*//g' | tr '\n' $ | sed 's/$/\n/g')

b. create the trust_config.yaml: cat << EOF > trust_config.yaml trustStores: - trustAnchors: - pemCertificate: "${ROOT_CERT?}" intermediateCas: - pemCertificate: "${INTERMEDIATE_CERT?}" EOF

c. import the client's certificate into the trust_config, based on the file trust_config.yaml: gcloud certificate-manager trust-configs import TRUST_CONFIG_NAME
--source=trust_config.yaml

  1. prepare the Client Authentication policy for mtls, known as ServerTLSPolicy where you have 2 options available: a. clientValidationMode is set to ALLOW_INVALID_OR_MISSING_CLIENT_CERT: cat << EOF > server_tls_policy.yaml name: SERVER_TLS_POLICY_NAME mtlsPolicy: clientValidationMode: ALLOW_INVALID_OR_MISSING_CLIENT_CERT clientValidationTrustConfig: projects/PROJECT_ID/locations/global/trustConfigs/TRUST_CONFIG_NAME EOF

    b. clientValidationMode is set to REJECT_INVALID

  2. create the ServerTLSPolicy: gcloud network-security server-tls-policies import SERVER_TLS_POLICY_NAME --source=server_tls_policy.yaml
    --location=global

  3. list all the target HTTPS proxies in your project: gcloud compute target-https-proxies list

  4. export a target HTTPS proxy's configuration to a file: gcloud compute target-https-proxies export TARGET_HTTPS_PROXY_NAME
    --destination=TARGET_PROXY_FILENAME
    --region=REGION

  5. attach the ServerTlsPolicy to the target-https-proxy, by appending the resource file TARGET_PROXY_FILENAME: echo "serverTlsPolicy: //networksecurity.googleapis.com/projects/PROJECT_ID/locations/REGION/serverTlsPolicies/SERVER_TLS_POLICY_NAME" >> TARGET_PROXY_FILENAME

  6. import a target HTTPS proxy's configuration from a file: gcloud compute target-https-proxies import TARGET_HTTPS_PROXY_NAME
    --source=TARGET_PROXY_FILENAME
    --region=REGION

Upvotes: 1

Related Questions