CODEWITHSUNDEEP
elasticsearchkubernetes
winn j
winn j

Reputation: 462

kubernetes api using an unreachable cluster IP

This is odd. Did i miss something during the setup

master kubernetes-elasticsearch-cluster # kubectl get svc kubernetes
NAME         LABELS                                    SELECTOR   IP(S)        PORT(S)
kubernetes   component=apiserver,provider=kubernetes   <none>     10.100.0.1   8080/TCP

I can't seem to reach the 10.100.0.1 address from anywhere on the cluster. However, some of the other clusterIP address are reachable from the nodes

master kubernetes-elasticsearch-cluster # kubectl get svc elasticsearch
NAME            LABELS                                SELECTOR                              IP(S)            PORT(S)
elasticsearch   component=elasticsearch,role=client   component=elasticsearch,role=client   10.100.213.223   9200/TCP

I believe this is causing the problem i am having when i look at the logs for the elasticsearch pods.

javax.ws.rs.ProcessingException: javax.net.ssl.SSLHandshakeException: SSLHandshakeException invoking https://10.100.0.1:8080/api/v1/namespaces/default/endpoints/elasticsearch-discovery: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

But if i check that same path with another address the items are retuned

# curl http://10.1.141.41:8080/api/v1/namespaces/default/endpoints/elasticsearch-discovery
{
  "kind": "Endpoints",
  "apiVersion": "v1",
  "metadata": {
......

Upvotes: 1

Views: 1510

Answers (2)

winn j
winn j

Reputation: 462

First i had to make the certs ./make-ca-cert.sh IP: ,IP:10.0.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local

Then i had to update the kube api server and the kube controller kube-apiserver

--client-ca-file=/srv/kubernetes/ca.crt
--tls-cert-file=/srv/kubernetes/server.cert
--tls-private-key-file=/srv/kubernetes/server.key
key
--    admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota

kube-controller-manager

--root-ca-file=/srv/kubernetes/ca.crt
--service-account-private-key-file=/srv/kubernetes/server.key

I then coppied the ca.crt and kubecfg.{crt|key} to the minions, and correctly call them from the kubeconfig file /var/lib/kubelet/kubeconfig

apiVersion: v1
kind: Config
users:
- name: kubelet
  user:
    client-certificate: /etc/ssl/private/kubecfg.crt
    client-key: /etc/ssl/private/kubecfg.key
clusters:
- name: cluster01
  cluster:
    certificate-authority: /etc/ssl/certs/ca.crt
contexts:
- context:
    cluster: cluster01
    user: kubelet
  name: service-account-context
current-context: service-account-context

Finally thanks to lewismarshall https://github.com/UKHomeOffice/docker-elasticsearch/blob/master/examples/kubernetes.md

I updated all the rc-yaml files.

I am now getting a 200! master kubernetes-elasticsearch-cluster # curl http://10.244.52.18:9200 { "status" : 200, ....

Now i can't figure out why i am not able to load a plugin on the client node.

/elasticsearch/bin/plugin -install royrusso/elasticsearch-HQ

Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve placeholder 'DISCOVERY_SERVICE'

Upvotes: 0

Robert Bailey
Robert Bailey

Reputation: 18230

Your stack trace actually indicates that you can reach 10.100.0.1 but that your client is not able to verify the certificate presented by the Kubernetes apiserver. If you want to connect using SSL, you should verify that the certificate used by the Kubernetes apiserver has 10.100.0.1 as an IP in the subject alternate names field or you could use the DNS name for the service (and likewise ensure that it is listed in the certificate).

Upvotes: 1

Related Questions