CODEWITHSUNDEEP
postgresqlkubernetesstatefulset
Happy Machine
Happy Machine

Reputation: 1163

StatefulSet with Postgres and one replica

If i deploy Postgres in a statefulSet without using replicas (just one pod) and i kill the node that the stateful set is running on will I be able to start up the node and reconnect to a persisted database

Here is an example configuration: https://medium.com/@suyashmohan/setting-up-postgresql-database-on-kubernetes-24a2a192e962

I am working with someone who is convinced this should not work and that statefulSets only make sense as a way to maintain state between replicas. Im under the impression that the problem of mounting the PG data to ephemeral pods is specific to NOT using a statefulSet, and that even though there is only one pod in the example above that this will still make use of StatefulSet to solve the problem.

(as in this official mysql example: https://kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application/

Upvotes: 1

Views: 3552

Answers (1)

AndD
AndD

Reputation: 2701

The database will be persisted if the Pods make use of persisted volume storages. Ephemeral Volumes are not persisted and should not be used to save a db, as the name says, there is no long-term guarantee about durability.

But if your Pod saves the database on a persisted volume of some sort (such as local storage on the node itself or something more complex) then it will be persisted between runs.

Which means that if you have your Pod running on a node, let's say using local storage on that node, if you stop the node and then make it restart correctly, the Pod will be scheduled again and the persisted volume will be there with all the data saved previously.

With this said, if you have only 1 Pod (StatefulSet with just 1 replica) and the node on which the Pod is currently running is somehow killed / stops working / stops answering then the Pod will not automatically restart on another node (not even if you are not using local storage)

You will be able to force it to run to another node, sure, but only with manual operations.

This is because (from the docs):

In normal operation of a StatefulSet, there is never a need to force delete a StatefulSet Pod. The StatefulSet controller is responsible for creating, scaling and deleting members of the StatefulSet. It tries to ensure that the specified number of Pods from ordinal 0 through N-1 are alive and ready. StatefulSet ensures that, at any time, there is at most one Pod with a given identity running in a cluster. This is referred to as at most one semantics provided by a StatefulSet.

If the controller cannot be sure if a Pod is running or not (and the example of a node getting killed or stopping to work correctly for a error is such a situation) then the Pod will never be restarted, until either:

  • Manual operation such as a force delete.
  • The node starts answering again and becomes Ready once more.

Note that draining a node will not create any problem as it will gracefully terminate StatefulSets Pods before starting them again (on other nodes).

StatefulSets can work really well for databases, but usually it requires a more complex installation with multi-primary nodes and (at least) 3 replicas.

Also, databases requires very fast write operations on disk and as such perform better if they can work on high quality disks.

Update:

StatefulSets are usually intended to be used when each of the replicas Pods require a unique identity (multi primary databases or apps that uses quorum are good example of this necessity)

When deployed with only 1 replica, the differences with a Deployment are small (but there are differences, for example a Deployment's Pod would eventually restart on another node if the node on which it was running stops working, a StatefulSet Pod will require manual intervention).. in general you should refer to the "Using StatefulSets" in the docs to decide if an app requires to be run as StatefulSet or Deployment.

Personally, I would run a database as a StatefulSet because it is a stateful app.. but I would also run it with 3 replicas, so that it can suffer the loss of one Pod without stopping to work.

Upvotes: 8

Related Questions