Reputation: 27

Minio deployment in Microk8s does not start properly

I'm trying to deploy a Minio instance (not the operator) on a kubernetes server (Microk8s) using the helm chart provided in https://github.com/minio/minio/tree/master/helm/minio.

To match with my test server, I did some modifications :

The number of server is equal to 1 instead of 16
The requested memory is equal to 1Gb instead of 16Gb
I used the command --persistence.enabled=false to simplify the problem as much as possible

The complete installation command is (the chart folder is present directly in the server):

microk8s helm3 install --namespace kube-system --generate-name --set persistence.enabled=false minio

The modified values.yaml content is the following :

## Provide a name in place of minio for `app:` labels
##
nameOverride: ""

## Provide a name to substitute for the full names of resources
##
fullnameOverride: ""

## set kubernetes cluster domain where minio is running
##
clusterDomain: cluster.local

## Set default image, imageTag, and imagePullPolicy. mode is used to indicate the
##
image:
  repository: quay.io/minio/minio
  tag: RELEASE.2021-11-24T23-19-33Z
  pullPolicy: IfNotPresent

imagePullSecrets: []
# - name: "image-pull-secret"

## Set default image, imageTag, and imagePullPolicy for the `mc` (the minio
## client used to create a default bucket).
##
mcImage:
  repository: quay.io/minio/mc
  tag: RELEASE.2021-11-16T20-37-36Z
  pullPolicy: IfNotPresent

## minio mode, i.e. standalone or distributed or gateway (nas)
## Distributed MinIO ref: https://docs.minio.io/docs/distributed-minio-quickstart-guide
## NAS Gateway MinIO ref: https://docs.min.io/docs/minio-gateway-for-nas.html
mode: distributed

## NOTE: currently only "nas" gateway is supported.

## Additional labels to include with deployment or statefulset
additionalLabels: []

## Additional annotations to include with deployment or statefulset
additionalAnnotations: []

## Additional arguments to pass to minio binary
extraArgs: []

## Port number for MinIO S3 API Access
minioAPIPort: "9000"

## Port number for MinIO Browser COnsole Access
minioConsolePort: "9001"

## Update strategy for Deployments
DeploymentUpdate:
  type: RollingUpdate
  maxUnavailable: 0
  maxSurge: 100%

## Update strategy for StatefulSets
StatefulSetUpdate:
  updateStrategy: RollingUpdate

## Pod priority settings
## ref: https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
##
priorityClassName: ""

## Set default rootUser, rootPassword
## AccessKey and secretKey is generated when not set
## Distributed MinIO ref: https://docs.minio.io/docs/distributed-minio-quickstart-guide
##
rootUser: ""
rootPassword: ""

## Use existing Secret that store following variables:
##
## | Chart var             | .data.<key> in Secret    |
## |:----------------------|:-------------------------|
## | rootUser              | rootUser                 |
## | rootPassword          | rootPassword             |
##
## All mentioned variables will be ignored in values file.
## .data.rootUser and .data.rootPassword are mandatory,
## others depend on enabled status of corresponding sections.
existingSecret: ""

## Directory on the MinIO pof
certsPath: "/etc/minio/certs/"
configPathmc: "/etc/minio/mc/"

## Path where PV would be mounted on the MinIO Pod
mountPath: "/export"
## Override the root directory which the minio server should serve from.
## If left empty, it defaults to the value of {{ .Values.mountPath }}
## If defined, it must be a sub-directory of the path specified in {{ .Values.mountPath }}
##
bucketRoot: ""

# Number of drives attached to a node
drivesPerNode: 1
# Number of MinIO containers running
replicas: 1
# Number of expanded MinIO clusters
pools: 1

# Deploy if 'mode == gateway' - 4 replicas.
gateway:
  type: "nas" # currently only "nas" is supported.
  replicas: 4

## TLS Settings for MinIO
tls:
  enabled: false
  ## Create a secret with private.key and public.crt files and pass that here. Ref: https://github.com/minio/minio/tree/master/docs/tls/kubernetes#2-create-kubernetes-secret
  certSecret: ""
  publicCrt: public.crt
  privateKey: private.key

## Trusted Certificates Settings for MinIO. Ref: https://docs.minio.io/docs/how-to-secure-access-to-minio-server-with-tls#install-certificates-from-third-party-cas
## Bundle multiple trusted certificates into one secret and pass that here. Ref: https://github.com/minio/minio/tree/master/docs/tls/kubernetes#2-create-kubernetes-secret
## When using self-signed certificates, remember to include MinIO's own certificate in the bundle with key public.crt.
## If certSecret is left empty and tls is enabled, this chart installs the public certificate from .Values.tls.certSecret.
trustedCertsSecret: ""

## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistence:
  enabled: true
  annotations: {}

  ## A manually managed Persistent Volume and Claim
  ## Requires persistence.enabled: true
  ## If defined, PVC must be created manually before volume will be bound
  existingClaim: ""

  ## minio data Persistent Volume Storage Class
  ## If defined, storageClassName: <storageClass>
  ## If set to "-", storageClassName: "", which disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
  ##   GKE, AWS & OpenStack)
  ##
  ## Storage class of PV to bind. By default it looks for standard storage class.
  ## If the PV uses a different storage class, specify that here.
  storageClass: "standard"
  VolumeName: ""
  accessMode: ReadWriteOnce
  size: 80Gi

  ## If subPath is set mount a sub folder of a volume instead of the root of the volume.
  ## This is especially handy for volume plugins that don't natively support sub mounting (like glusterfs).
  ##
  subPath: ""

## Expose the MinIO service to be accessed from outside the cluster (LoadBalancer service).
## or access it from within the cluster (ClusterIP service). Set the service type and the port to serve it.
## ref: http://kubernetes.io/docs/user-guide/services/
##
service:
  type: ClusterIP
  clusterIP: ~
  ## Make sure to match it to minioAPIPort
  port: "9000"
  nodePort: 32000

## Configure Ingress based on the documentation here: https://kubernetes.io/docs/concepts/services-networking/ingress/
##

ingress:
  enabled: false
  # ingressClassName: ""
  labels: {}
    # node-role.kubernetes.io/ingress: platform

  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
    # kubernetes.io/ingress.allow-http: "false"
    # kubernetes.io/ingress.global-static-ip-name: ""
    # nginx.ingress.kubernetes.io/secure-backends: "true"
    # nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    # nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
  path: /
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

consoleService:
  type: ClusterIP
  clusterIP: ~
  ## Make sure to match it to minioConsolePort
  port: "9001"
  nodePort: 32001

consoleIngress:
  enabled: false
  # ingressClassName: ""
  labels: {}
    # node-role.kubernetes.io/ingress: platform

  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
    # kubernetes.io/ingress.allow-http: "false"
    # kubernetes.io/ingress.global-static-ip-name: ""
    # nginx.ingress.kubernetes.io/secure-backends: "true"
    # nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    # nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
  path: /
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

## Node labels for pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
tolerations: []
affinity: {}

## Add stateful containers to have security context, if enabled MinIO will run as this
## user and group NOTE: securityContext is only enabled if persistence.enabled=true
securityContext:
  enabled: true
  runAsUser: 1000
  runAsGroup: 1000
  fsGroup: 1000

# Additational pod annotations
podAnnotations: {}

# Additional pod labels
podLabels: {}

## Configure resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources:
  requests:
    memory: 4Gi

## List of users to be created after minio install
##
users:
  ## Username, password and policy to be assigned to the user
  ## Default policies are [readonly|readwrite|writeonly|consoleAdmin|diagnostics]
  ## Add new policies as explained here https://docs.min.io/docs/minio-multi-user-quickstart-guide.html
  ## NOTE: this will fail if LDAP is enabled in your MinIO deployment
  ## make sure to disable this if you are using LDAP.
  - accessKey: console
    secretKey: console123
    policy: consoleAdmin


## Additional Annotations for the Kubernetes Job makeUserJob
makeUserJob:
  podAnnotations:
  annotations:
  securityContext:
    enabled: false
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 1000
  resources:
    requests:
      memory: 128Mi
  nodeSelector: {}
  tolerations: []
  affinity: {}

## List of buckets to be created after minio install
##
buckets:
  #   # Name of the bucket
  # - name: bucket1
  #   # Policy to be set on the
  #   # bucket [none|download|upload|public]
  #   policy: none
  #   # Purge if bucket exists already
  #   purge: false
  #   # set versioning for
  #   # bucket [true|false]
  #   versioning: false
  # - name: bucket2
  #   policy: none
  #   purge: false
  #   versioning: true

## Additional Annotations for the Kubernetes Job makeBucketJob
makeBucketJob:
  podAnnotations:
  annotations:
  securityContext:
    enabled: false
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 1000
  resources:
    requests:
      memory: 128Mi
  nodeSelector: {}
  tolerations: []
  affinity: {}

## Use this field to add environment variables relevant to MinIO server. These fields will be passed on to MinIO container(s)
## when Chart is deployed
environment:
  ## Please refer for comprehensive list https://docs.min.io/minio/baremetal/reference/minio-server/minio-server.html
  ## MINIO_SUBNET_LICENSE: "License key obtained from https://subnet.min.io"
  ## MINIO_BROWSER: "off"

networkPolicy:
  enabled: false
  allowExternal: true

## PodDisruptionBudget settings
## ref: https://kubernetes.io/docs/concepts/workloads/pods/disruptions/
##
podDisruptionBudget:
  enabled: false
  maxUnavailable: 1

## Specify the service account to use for the Minio pods. If 'create' is set to 'false'
## and 'name' is left unspecified, the account 'default' will be used.
serviceAccount:
  create: true
  ## The name of the service account to use. If 'create' is 'true', a service account with that name
  ## will be created.
  name: "minio-sa"

metrics:
  serviceMonitor:
    enabled: false
    additionalLabels: {}
    public: true
    # namespace: monitoring
    # interval: 30s
    # scrapeTimeout: 10s

The generated pod appears in error in the k8s dashboard and when I check the logs, I obtain:

ERROR Invalid command line arguments: Incorrect number of endpoints provided [http://minio-1638458317-{0...0}.minio-1638458317-svc.kube-system.svc.cluster.local/export]
      > Please provide correct combination of local/remote paths
      HINT:
        For more information, please refer to https://docs.min.io/docs/minio-erasure-code-quickstart-guide

I have absolutely no idea of what I can do to solve that issue. Could anyone help me on this?

Thanks in advance,

Rémi

Upvotes: 1

Minio deployment in Microk8s does not start properly

Answers (2)

Related Questions