Is there a way of obtaining activity logs from Service objects in Kubernetes?

Question

I have the following situation (this is my motivation to ask the question, not the question itself):

• I've got a web application that accepts uploads from users.
  • The users access the application through an Ingress,
  • then a Service,
  • Then a Deployment with two Pods.
  • Application contained in each Pod.
• Sometimes the upload fails:
  • I can see in the logs from the Pod that the upload went all right.
  • I can even see the data uploaded by the user.
  • There are nothing but normal logs in the Pod.
  • But the ingress reports a HTTP 500 error.
  • And the users sees a HTTP 500 error - connection reset by peer.

If the Pod seems all right, but ingress complains, then I should check the middle man, the Service. Then I realized that there is no easy way to obtain logs from the service.

So this is my question:

• How can I read logs from the Service object ? I mean activity logs, not the deployment events.
• Do they exist?

Answer 1

The only resources in K8s that produce logs are Pods! Pods lead to the creation of containers, which for their part lead to the creation of Linux processes on the K8s nodes. Those processes write logs that are "reaped" by the container runtime and made available to K8s, e.g. when you run kubectl logs.

Consequently, only K8s resources that are backed by Pods produce logs, e.g. Deployment, Daemonsets, StatefulSets and Jobs.

Services are merely logical resources that configures how network traffic can be routed to Pods. So, in a way they have underlying Pods, but do not produce any additional log output. The only tangible outcome of a Service resource are iptables rules on the K8s nodes, that define how traffic has to be routed from the Service IP to the IPs of the underlying Pods.

To resolve Ingress related problems, you might get further insights from the logs of your ingress controller which is typically deployed as a deployment and therefore backed by Pods.