CODEWITHSUNDEEP
sshgoogle-kubernetes-enginebitbucket-pipelinesssh-tunnel
André Schild
André Schild

Reputation: 4754

Use kubectl via ssh bastion host do deploy from bitbucket pipelines

We have two GKE private clusters, where access is only possible via ssh proxy.

In the local environment this works this way:

  1. Open a ssh connection with port 8888 forwarding to the bastion host
    gcloud compute ssh dev-cluster-bastion --project client-dev --zone xxxx -- -L 8888:127.0.0.1:8888

  2. In another session
    HTTPS_PROXY=localhost:8888 kubectl get pods

This returns the list of the running pods.

When we do this inside a bitbucket pipeline, then the ssh connects, but then closes and the kubectl call fails. The message from the ssh connection is:

Pseudo-terminal will not be allocated because stdin is not a terminal

So the portforwarding is closed too.

Adding -fN to the ssh start does not help, the port forwarding is not working.

gcloud compute ssh dev-cluster-bastion --project client-dev --zone xxxx -- -fN -L 8888:127.0.0.1:8888

ssh then tells me client_loop: send disconnect: Connection reset by peer

Any ideas how to open the port 8888 tcp tunnel inside a bitbucket pipeline, so we can send the kubectl commands to the cluster?

Upvotes: 0

Views: 1362

Answers (2)

Ramesh kollisetty
Ramesh kollisetty

Reputation: 291

The recommended way to use Google Cloud APIs in a non-interactive setting (like Bitbucket pipelines, GitHub actions, etc.) is to:

  1. Create a Service Account , and refer to this link for service account creation.
  2. Create a key file for this service account and download it. Kindly refer to the link for file creation.
  3. Grant the needed IAM permissions to this account to be able to SSH into the Bastion host
  4. Make the key file available to the pipeline environment securely (most CI/CD systems allow a way to encrypt secret files or sensitive environment variables)
  5. In the pipeline, use gcloud auth activate-service-account --key-file=... to make gcloud use the service account
  6. Now run gcloud compute ssh in the pipeline and use SSH port forwarding as usual.

Note that:

  • steps 1.--4. are one time only;
  • steps 5. and 6. are part of the pipeline and will run every time the pipeline is run
  • The only difference being the presence of the -fN flags, which execute SSH in the background and make it run no command on the remote host (i.e. only do port forwarding).

You can also refer to Service account creation and Get started with Bitbucket-pipeline for more information.

Upvotes: 1

Ramesh kollisetty
Ramesh kollisetty

Reputation: 291

To fix this error Pseudo-terminal will not be allocated because stdin is not a terminal. follow one of the below steps:

The error occurred because you're running SSH with a single -t option, when the standard input to the SSH process isn't a TTY. SSH prints that message specifically in this case A single -t is equivalent to "RequestTTY yes", while two of them is equivalent to "RequestTTY force".

If you want your remote command(s) to run with a TTY, then specify -t twice. If you are doing so then in your command instead of -t use -t -t or use -tt .

To fix this error client_loop: send disconnect: Connection reset by peer follow the below steps:

You shall look on these parameters on the client side /etc/ssh/ssh_config
ServerAliveInterval
ServerAliveCountMax
To set the interval to the value 60 and try
ServerAliveInterval 60

Upvotes: 0

Related Questions