Setting up Postgres in K8s but getting error: directory "/var/lib/postgresql/data" exists but is not empty

Question

I am trying to set up Postgres in my K3s cluster. I am using Longhorn for storage here. I keep getting the error:

initdb: error: directory "/var/lib/postgresql/data" exists but is not empty
initdb: hint: If you want to create a new database system, either remove or empty the directory "/var/lib/postgresql/data" or run initdb with an argument other than "/var/lib/postgresql/data".

When my pods spin up, I can't seem to get into the pod as it's on a CrashLoopBackOff

➜ k get pods -n postgres-db
NAME                       READY   STATUS             RESTARTS      AGE
pgadmin-5c9bc6879d-f7dkm   1/1     Running            0             17m
postgres-6b459b956-7j2m2   0/1     CrashLoopBackOff   8 (52s ago)   17m

I have tried to delete the volume in LH and redploy my postgres instance. I have done some googling but I don't understand how it's got data in the path if I have deleted the vol in LH and deleted the postgres application

This is my deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      initContainers:
        - name: clean-up
          image: alpine
          command:
            [
              "sh",
              "-c",
              "apk add --no-cache bash && bash -c 'shopt -s dotglob && rm -rf /var/lib/postgresql/data/*'",
            ]
          volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: postgresdata
      containers:
        - name: postgres
          image: "postgres:16"
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 5432
          envFrom:
            - configMapRef:
                name: postgres-secret
          volumeMounts:
            - mountPath: /var/lib/postgresql/data/db
              name: postgresdata
      volumes:
        - name: postgresdata
          persistentVolumeClaim:
            claimName: postgres-pvc

I have added this after some ChatGPT to try and clean the path before I deploy the container

initContainers:
        - name: clean-up
          image: alpine
          command:
            [
              "sh",
              "-c",
              "apk add --no-cache bash && bash -c 'shopt -s dotglob && rm -rf /var/lib/postgresql/data/*'",
            ]

But I still get the error. These are my PV and PVC

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
  labels:
    app: postgres
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Gi
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: postgres-pv
  labels:
    type: local
    app: postgres
spec:
  capacity:
    storage: 50Gi
  accessModes:
    - ReadWriteMany
  hostPath:
    path: /mnt/storage/postgresql

This is what it looks like in LH again I don't understand a healthy vol for the PVC amount 50Gib and then the degraded one...

This is whats in my mounted drive

04:56:34 sysadm@k3s-cp-3 ~ → cd /mnt/storage
04:56:39 sysadm@k3s-cp-3 storage → ls
longhorn-disk.cfg  lost+found  replicas

Even after running k delete all --all -n postgres-db && k delete namespace postgres-db I still get the same error.

Any tips or pointers here would be very helpful, thanks

Answer 1

I fixed it by adding the following into my deployment

env:
            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata
          volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: postgresdata

After finding this works like a charm now