CODEWITHSUNDEEP
amazon-web-servicesazurekubernetes
Jenney
Jenney

Reputation: 291

Send logs from specific pod to external server

We need to send large (very) amount of logs to Splunk server from only one k8s pod( pod with huge traffic load), I look at the docs and found this:

https://kubernetes.io/docs/concepts/cluster-administration/logging/#sidecar-container-with-a-logging-agent

However, there is a Note in the docs, that is stating about a significant resource consumption. Is there any other option to do it? I mean more efficient ? As these pods handle traffic and we cannot add the additional load, that can risk it stability...

Upvotes: 1

Views: 550

Answers (1)

Honky Donkey
Honky Donkey

Reputation: 618

There's an official solution to get Kubernets logs: Splunk Connect for Kubernetes. Under the hood it also uses fluentd for the logging part.

https://github.com/splunk/splunk-connect-for-kubernetes

You will find a sample config and a methodology to test it on microK8s first to get acquainted with the config and deployment: https://mattymo.io/deploying-splunk-connect-for-kubernetes-on-microk8s-with-helm/

And if you only want logs from a specific container you can use this section of the values file to select only logs from the container you're interested in:

  fluentd:
    # path of logfiles, default /var/log/containers/*.log
    path: /var/log/containers/*.log
    # paths of logfiles to exclude. object type is array as per fluentd specification:
    # https://docs.fluentd.org/input/tail#exclude_path
    exclude_path:
    #  - /var/log/containers/kube-svc-redirect*.log
    #  - /var/log/containers/tiller*.log
    #  - /var/log/containers/*_kube-system_*.log (to exclude `kube-system` namespace)

Upvotes: 2

Related Questions