CODEWITHSUNDEEP
dockerelasticsearchkubernetesconnectionfluentd
Elfs
Elfs

Reputation: 11

Fluentd Failing to connect to ElasticSearch cluster

I have a local kubernetes cluster where I added a Fluentd Daemonset using the preconfigured elasticsearch image (fluent/fluentd-kubernetes-daemonset:elasticsearch). Step 2 of this article. I also have an elastic cluster running in the cloud. You can pass some env variables to the fluentd-elasticsearch image for configuration. It looks pretty straightforward, but when running the fluentd Pod I keep getting the error:

"Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"fa0acce34bf64db9bc9e46f98743c185.westeurope.azure.elastic-cloud.com\", :port=>9243, :scheme=>\"https\", :user=>\"username\", :password=>\"obfuscated\"})!" plugin_id="out_es"

when I try to reach the elastic cluster from within the pod with # wget https://fa0acce34bf64db9bc9e46f98743c185.westeurope.azure.elastic-cloud.com:9243/ I get a 401 unauthorized (cuz I havent submitted user/pass here), but it at least shows that the address is reachable.

Why is it failing to connect? I already set the FLUENT_ELASTICSEARCH_SSL_VERSION to 'TLSv1_2', i saw that that solved some problems for others.

Daemonset configuration:

kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-logging
  labels:
    app: fluentd
    k8s-app: fluentd-logging
    version: v1
    kubernetes.io/cluster-service: "true"
spec:
  selector:
    matchLabels:
      app: fluentd
  template:
    metadata:
      labels:
        app: fluentd
        k8s-app: fluentd-logging
        version: v1
        kubernetes.io/cluster-service: "true"
    spec:
      serviceAccount: fluentd
      serviceAccountName: fluentd
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:elasticsearch
        env:
        - name:  FLUENT_ELASTICSEARCH_HOST
          value: "fa0acce34bf64db9bc9e46f98743c185.westeurope.azure.elastic-cloud.com"
        - name:  FLUENT_ELASTICSEARCH_PORT
          value: "9243"
        - name: FLUENT_ELASTICSEARCH_SCHEME
          value: "https"
        - name: FLUENT_UID
          value: "0"
        - name: FLUENT_ELASTICSEARCH_SSL_VERIFY
          value: "false"
        - name: FLUENT_ELASTICSEARCH_SSL_VERSION
          value: "TLSv1_2"
        - name: FLUENT_ELASTICSEARCH_USER
          value: "<user>"
        - name: FLUENT_ELASTICSEARCH_PASSWORD
          value: "<password>"
        resources:
          limits:
            memory: 100Mi
          requests:
            cpu: 100m
            memory: 100Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers

Upvotes: 0

Views: 1398

Answers (1)

Elfs
Elfs

Reputation: 11

For anyone else who runs into this problem:

I was following a tutorial that used the 'image: fluent/fluentd-kubernetes-daemonset:elasticsearch' image. When you check their DockerHub (https://hub.docker.com/r/fluent/fluentd-kubernetes-daemonset) you can see that the :elaticsearch tag is a year old and probably outdated.

I changed the image for the DaemonSet to a more recent and stable tag 'fluent/fluentd-kubernetes-daemonset:v1-debian-elasticsearch' and boom it works now.

Upvotes: 1

Related Questions