CODEWITHSUNDEEP
kuberneteseventskubernetes-statefulseteventstoredbevent-store
YagmurOzden
YagmurOzden

Reputation: 43

Error when I try to install Eventstore on my kubernetes pod | Access to the path "/var/lib/eventstore/writer.chk" is denied

I am trying to install eventstore on my kubernetes pods

But I am getting error. Its going to be for production. What can I do to fix the issue?

** SHORT ERROR: **

[00001,01,06:27:30.893] Quorum size set to 2
[00001,01,06:27:30.902] Cannot find plugins path: plugins
[00001,01,06:27:30.934] Unhandled exception while starting application:
Access to the path "/var/lib/eventstore/writer.chk" is denied.
[00001,01,06:27:30.943] Access to the path "/var/lib/eventstore/writer.chk" is denied.
Access to the path "/var/lib/eventstore/writer.chk" is denied.

** LONG ERROR: **

kubectl logs eventstore-1 -n eventstore

root@ankasoft-mg:/home/yagmur/EventStore-Kubernetes# kubectl logs eventstore-1 -n eventstore
[00001,01,06:47:49.855]
ES VERSION:               4.1.1.0 (HEAD/cd0eafec71baffbcc0b9a4bb7c58e996cf290d34, Wed, 16 May 2018 11:08:07 +0400)
[00001,01,06:47:49.877] OS:                       Linux (Unix 4.19.288.2)
[00001,01,06:47:49.881] RUNTIME:                  4.6.2 (Stable 4.6.2.16/ac9e222) (64-bit)
[00001,01,06:47:49.881] GC:                       2 GENERATIONS
[00001,01,06:47:49.881] LOGS:                     /var/log/eventstore
[00001,01,06:47:49.889] MODIFIED OPTIONS:

    INT IP:                   ----.40 (Environment Variable)
    EXT IP:                   ----.40 (Environment Variable)
    INT HTTP PORT:            2114 (Environment Variable)
    EXT HTTP PORT:            2113 (Environment Variable)
    INT TCP PORT:             1111 (Environment Variable)
    EXT TCP PORT:             1112 (Environment Variable)
    CLUSTER SIZE:             3 (Environment Variable)
    CLUSTER DNS:              eventstore.default.svc.cluster.local (Environment Variable)
    CLUSTER GOSSIP PORT:      2114 (Environment Variable)
    INT HTTP PREFIXES:        http://*:2114/ (Environment Variable)
    EXT HTTP PREFIXES:        http://*:2113/ (Environment Variable)
    GOSSIP ALLOWED DIFFERENCE MS: 600000 (Environment Variable)
    ADD INTERFACE PREFIXES:   false (Config File)
    RUN PROJECTIONS:          All (Config File)

DEFAULT OPTIONS:

    CONFIG:                   /etc/eventstore/eventstore.conf (<DEFAULT>)
    HELP:                     False (<DEFAULT>)
    VERSION:                  False (<DEFAULT>)
    LOG:                      /var/log/eventstore (<DEFAULT>)
    DEFINES:                  <empty> (<DEFAULT>)
    WHAT IF:                  False (<DEFAULT>)
    START STANDARD PROJECTIONS: False (<DEFAULT>)
    DISABLE HTTP CACHING:     False (<DEFAULT>)
    MONO MIN THREADPOOL SIZE: 10 (<DEFAULT>)
    INT SECURE TCP PORT:      0 (<DEFAULT>)
    EXT SECURE TCP PORT ADVERTISE AS: 0 (<DEFAULT>)
    EXT SECURE TCP PORT:      0 (<DEFAULT>)
    EXT IP ADVERTISE AS:      <empty> (<DEFAULT>)
    EXT TCP PORT ADVERTISE AS: 0 (<DEFAULT>)
    EXT HTTP PORT ADVERTISE AS: 0 (<DEFAULT>)
    INT IP ADVERTISE AS:      <empty> (<DEFAULT>)
    INT SECURE TCP PORT ADVERTISE AS: 0 (<DEFAULT>)
    INT TCP PORT ADVERTISE AS: 0 (<DEFAULT>)
    INT HTTP PORT ADVERTISE AS: 0 (<DEFAULT>)
    INT TCP HEARTBEAT TIMEOUT: 700 (<DEFAULT>)
    EXT TCP HEARTBEAT TIMEOUT: 1000 (<DEFAULT>)
    INT TCP HEARTBEAT INTERVAL: 700 (<DEFAULT>)
    EXT TCP HEARTBEAT INTERVAL: 2000 (<DEFAULT>)
    GOSSIP ON SINGLE NODE:    False (<DEFAULT>)
    CONNECTION PENDING SEND BYTES THRESHOLD: 10485760 (<DEFAULT>)
    FORCE:                    False (<DEFAULT>)
    NODE PRIORITY:            0 (<DEFAULT>)
    MIN FLUSH DELAY MS:       2 (<DEFAULT>)
    COMMIT COUNT:             -1 (<DEFAULT>)
    PREPARE COUNT:            -1 (<DEFAULT>)
    ADMIN ON EXT:             True (<DEFAULT>)
    STATS ON EXT:             True (<DEFAULT>)
    GOSSIP ON EXT:            True (<DEFAULT>)
    DISABLE SCAVENGE MERGING: False (<DEFAULT>)
    SCAVENGE HISTORY MAX AGE: 30 (<DEFAULT>)
    DISCOVER VIA DNS:         True (<DEFAULT>)
    GOSSIP SEED:              <empty> (<DEFAULT>)
    STATS PERIOD SEC:         30 (<DEFAULT>)
    CACHED CHUNKS:            -1 (<DEFAULT>)
    READER THREADS COUNT:     4 (<DEFAULT>)
    CHUNKS CACHE SIZE:        536871424 (<DEFAULT>)
    MAX MEM TABLE SIZE:       1000000 (<DEFAULT>)
    HASH COLLISION READ LIMIT: 100 (<DEFAULT>)
    DB:                       /var/lib/eventstore (<DEFAULT>)
    INDEX:                    <empty> (<DEFAULT>)
    MEM DB:                   False (<DEFAULT>)
    SKIP DB VERIFY:           False (<DEFAULT>)
    WRITE THROUGH:            False (<DEFAULT>)
    UNBUFFERED:               False (<DEFAULT>)
    CHUNK INITIAL READER COUNT: 5 (<DEFAULT>)
    PROJECTION THREADS:       3 (<DEFAULT>)
    WORKER THREADS:           5 (<DEFAULT>)
    PROJECTIONS QUERY EXPIRY: 0 (<DEFAULT>)
    ENABLE TRUSTED AUTH:      False (<DEFAULT>)
    CERTIFICATE STORE LOCATION: <empty> (<DEFAULT>)
    CERTIFICATE STORE NAME:   <empty> (<DEFAULT>)
    CERTIFICATE SUBJECT NAME: <empty> (<DEFAULT>)
    CERTIFICATE THUMBPRINT:   <empty> (<DEFAULT>)
    CERTIFICATE FILE:         <empty> (<DEFAULT>)
    CERTIFICATE PASSWORD:     <empty> (<DEFAULT>)
    USE INTERNAL SSL:         False (<DEFAULT>)
    DISABLE INSECURE TCP:     False (<DEFAULT>)
    SSL TARGET HOST:          n/a (<DEFAULT>)
    SSL VALIDATE SERVER:      True (<DEFAULT>)
    AUTHENTICATION TYPE:      internal (<DEFAULT>)
    AUTHENTICATION CONFIG:    <empty> (<DEFAULT>)
    PREPARE TIMEOUT MS:       2000 (<DEFAULT>)
    COMMIT TIMEOUT MS:        2000 (<DEFAULT>)
    UNSAFE DISABLE FLUSH TO DISK: False (<DEFAULT>)
    BETTER ORDERING:          False (<DEFAULT>)
    UNSAFE IGNORE HARD DELETE: False (<DEFAULT>)
    SKIP INDEX VERIFY:        False (<DEFAULT>)
    INDEX CACHE DEPTH:        16 (<DEFAULT>)
    OPTIMIZE INDEX MERGE:     False (<DEFAULT>)
    GOSSIP INTERVAL MS:       1000 (<DEFAULT>)
    GOSSIP TIMEOUT MS:        500 (<DEFAULT>)
    ENABLE HISTOGRAMS:        False (<DEFAULT>)
    LOG HTTP REQUESTS:        False (<DEFAULT>)
    ALWAYS KEEP SCAVENGED:    False (<DEFAULT>)
    SKIP INDEX SCAN ON READS: False (<DEFAULT>)

[00001,01,06:47:49.895] Quorum size set to 2
[00001,01,06:47:49.903] Cannot find plugins path: plugins
[00001,01,06:47:49.933] Unhandled exception while starting application:
Access to the path "/var/lib/eventstore/writer.chk" is denied.
[00001,01,06:47:49.941] Access to the path "/var/lib/eventstore/writer.chk" is denied.
Access to the path "/var/lib/eventstore/writer.chk" is denied.

** STATEFULL SET YAML: **

cat eventstore-statefulset.yaml

root@ankasoft-mg:/home/yagmur/EventStore-Kubernetes# cat eventstore-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: eventstore
  namespace: eventstore
spec:
  selector:
    matchLabels:
      app: eventstore
  replicas: 3
  serviceName: eventstore
  template:
    metadata:
      labels:
        app: eventstore
    spec:
      containers:
        - name: eventstore
          securityContext:
            runAsUser: 1000
            runAsGroup: 1000
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            runAsNonRoot: true
            runAsUser: 1000
            seccompProfile:
              type: RuntimeDefault
          image: "eventstore/eventstore:release-4.1.1-hotfix1"
          volumeMounts:
            - mountPath: "/usr/data/eventstore"
              name: eventstore-pv-storage
          env:
            - name: EVENTSTORE_INT_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: EVENTSTORE_EXT_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: EVENTSTORE_INT_TCP_PORT
              value: "1111"
            - name: EVENTSTORE_EXT_TCP_PORT
              value: "1112"
            - name: EVENTSTORE_INT_HTTP_PORT
              value: "2114"
            - name: EVENTSTORE_EXT_HTTP_PORT
              value: "2113"
            - name: EVENTSTORE_CLUSTER_SIZE
              value: "3"
            - name: EVENTSTORE_CLUSTER_DNS
              value: "eventstore.default.svc.cluster.local"
            - name: EVENTSTORE_CLUSTER_GOSSIP_PORT
              value: "2114"
            - name: EVENTSTORE_GOSSIP_ALLOWED_DIFFERENCE_MS
              value: "600000"
            - name: EVENTSTORE_INT_HTTP_PREFIXES
              value: "http://*:2114/"
            - name: EVENTSTORE_EXT_HTTP_PREFIXES
              value: "http://*:2113/"
          ports:
            - containerPort: 2113
            - containerPort: 2114
            - containerPort: 1111
            - containerPort: 1112
  volumeClaimTemplates:
    - metadata:
        name: eventstore-pv-storage
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi

** SERVICE YAML: **

cat eventstore-service.yaml


root@ankasoft-mg:/home/yagmur/EventStore-Kubernetes# cat eventstore-service.yaml
kind: Service
apiVersion: v1
metadata:
  name: "eventstore"
  namespace: eventstore
spec:
  selector:
    app: "eventstore"
  ports:
    - protocol: "TCP"
      port: 2113
      targetPort: 2113
  clusterIP: None

** PV YAML: **

cat volume-deployment-mnt.data.yaml

root@ankasoft-mg:/home/yagmur/EventStore-Kubernetes# cat volume-deployment-mnt.data.yaml
kind: PersistentVolume
apiVersion: v1
metadata:
  name: eventstore-pv-volume
  namespace: eventstore
  labels:
    type: local
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"

** PVC YAML: **

cat volume-claim.yaml

root@ankasoft-mg:/home/yagmur/EventStore-Kubernetes# cat volume-claim.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: eventstore-pv-claim
  namespace: eventstore
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2.5Gi

kubectl get all -n eventstore

root@ankasoft-mg:/home/yagmur/EventStore-Kubernetes# kubectl get all -n eventstore
NAME               READY   STATUS             RESTARTS        AGE
pod/eventstore-0   0/1     CrashLoopBackOff   9 (3m36s ago)   25m
pod/eventstore-1   0/1     CrashLoopBackOff   9 (3m59s ago)   25m
pod/eventstore-2   0/1     CrashLoopBackOff   9 (2m7s ago)    23m

NAME                          TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)          AGE
service/eventstore            ClusterIP      None           <none>         2113/TCP         4d18h
service/frontend-eventstore   LoadBalancer   CLUSTER-IP.220   EXTERNAL-IP.65   2113:31881/TCP   4d18h

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/frontend-eventstore   0/1     0            0           4d18h

NAME                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/frontend-eventstore-64c88cfb49   1         0         0       4d18h

NAME                          READY   AGE
statefulset.apps/eventstore   0/3     25m

Upvotes: 0

Views: 212

Answers (1)

Alexey Zimarev
Alexey Zimarev

Reputation: 19630

EventStore is set up to use /var/lib/eventstore as the data directory, which it shows to you on startup. But, your volume is mounted to /usr/data/eventstore. You get a self-explanatory message that EventStore cannot get write access to /var/lib/eventstore.

You need to use EVENTSTORE_DB env var to set the correct path for the data directory.

It's worth mentioning that v4 is extremely old and went out of support a long time ago. We don't recommend using v4 for production (as well as v5, v20.X, and v21.X).

Upvotes: 2

Related Questions