CODEWITHSUNDEEP
kuberneteskubernetes-helm
animegirl
animegirl

Reputation: 9

Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy' and 'minReadySeconds' are forbidden

I am trying to upgrade my rabbitmq to 3.10 version. When I am deploying using the new image I get below error. I made a small change to the template and not sure why it is throwing this error and nothing else was changed apart from the template.

The StatefulSet "uat-can-gcp-common-rabbitmq-ha-extend" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy' and 'minReadySeconds' are forbidden

Below is the code for my statefulset

kind: StatefulSet
metadata:
  name: {{ template "rabbitmq-ha.fullname" . }}
  namespace: {{ .Release.Namespace }}
  labels:
    app: {{ template "rabbitmq-ha.name" . }}
    chart: {{ template "rabbitmq-ha.chart" . }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
{{- if .Values.extraLabels }}
{{ toYaml .Values.extraLabels | indent 4 }}
{{- end }}
{{- if .Values.statefulSetAnnotations }}
  annotations:
{{- range $key, $value := .Values.statefulSetAnnotations }}
    {{ $key }}: {{ $value | quote }}
{{- end }}
{{- end }}
spec:
  podManagementPolicy: {{ .Values.podManagementPolicy }}
  serviceName: {{ template "rabbitmq-ha.fullname" . }}-discovery
  replicas: {{ .Values.replicaCount }}
  updateStrategy:
    type: {{ .Values.updateStrategy }}
  selector:
    matchLabels:
      app: {{ template "rabbitmq-ha.name" . }}
      release: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app: {{ template "rabbitmq-ha.name" . }}
        release: {{ .Release.Name }}
{{- if .Values.extraLabels }}
{{ toYaml .Values.extraLabels | indent 8 }}
{{- end }}
      annotations:
        {{- if not .Values.existingConfigMap }}
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
        {{- end }}
        {{- if not .Values.existingSecret }}
        checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
        {{- end }}
        {{- if and .Values.prometheus.exporter.enabled (not .Values.prometheus.operator.enabled) }}
        prometheus.io/scrape: "true"
        prometheus.io/port: {{ .Values.prometheus.exporter.port | quote }}
        {{- end }}
        {{- if and .Values.rabbitmqPrometheusPlugin.enabled (not .Values.prometheus.operator.enabled) }}
        prometheus.io/scrape: "true"
        prometheus.io/port: {{ .Values.rabbitmqPrometheusPlugin.port | quote }}
        prometheus.io/path: {{ .Values.rabbitmqPrometheusPlugin.path | quote }}
        {{- end }}
{{- if .Values.podAnnotations }}
{{ toYaml .Values.podAnnotations | indent 8 }}
{{- end }}
    spec:
      {{- if .Values.image.pullSecrets }}
      imagePullSecrets:
      {{- range .Values.image.pullSecrets }}
        - name: {{ . }}
      {{- end }}
      {{- end }}
      terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }}
      securityContext:
{{ toYaml .Values.securityContext | indent 10 }}
      serviceAccountName: {{ template "rabbitmq-ha.serviceAccountName" . }}
      initContainers:
{{- if .Values.initContainer.enabled }}
        - name: bootstrap
          image: {{ .Values.busyboxImage.repository}}:{{ .Values.busyboxImage.tag}}
          imagePullPolicy: {{ .Values.busyboxImage.pullPolicy }}
          securityContext:
{{- toYaml .Values.initContainer.securityContext | nindent 12 }}
          command: ['sh']
          args:
          - "-c"
          - |
            set -ex
            cp /configmap/* /etc/rabbitmq
            echo "${RABBITMQ_ERLANG_COOKIE}" > /var/lib/rabbitmq/.erlang.cookie
            {{- if .Values.forceBoot }}
            if [ -d "${RABBITMQ_MNESIA_DIR}" ]; then
              touch "${RABBITMQ_MNESIA_DIR}/force_load"
            fi
            {{- end }}
            {{- if and (.Values.securityContext.runAsUser) (.Values.securityContext.fsGroup) (.Values.initContainer.chownFiles) }}
            chown -R {{ .Values.securityContext.runAsUser }}:{{ .Values.securityContext.fsGroup }} /var/lib/rabbitmq/
            chown -R {{ .Values.securityContext.runAsUser }}:{{ .Values.securityContext.fsGroup }} /etc/rabbitmq
            {{- end }}
            chmod 600 /var/lib/rabbitmq/.erlang.cookie
          env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.name
          - name: RABBITMQ_MNESIA_DIR
            value: /var/lib/rabbitmq/mnesia/rabbit@$(POD_NAME).{{ template "rabbitmq-ha.fullname" . }}-discovery.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}
          - name: RABBITMQ_ERLANG_COOKIE
            valueFrom:
              secretKeyRef:
                name: {{ template "rabbitmq-ha.secretName" . }}
                key: rabbitmq-erlang-cookie
          resources:
{{ toYaml .Values.initContainer.resources | indent 12 }}
          volumeMounts:
            - name: configmap
              mountPath: /configmap
            - name: config
              mountPath: /etc/rabbitmq
            - name: {{ .Values.persistentVolume.name }}
              mountPath: /var/lib/rabbitmq
{{- end}}
{{- if .Values.extraInitContainers }}
{{ tpl (toYaml .Values.extraInitContainers) . | indent 8 }}
{{- end }}
    {{- with .Values.dnsConfig }}
      dnsConfig:
{{ toYaml . | indent 8 }}
    {{- end }}
      containers:
        - name: {{ .Chart.Name }}
          image: {{ .Values.image.name }}
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          ports:
            - name: epmd
              protocol: TCP
              containerPort: 4369
            - name: amqp
              protocol: TCP
              containerPort: 5672
            - name: http
              protocol: TCP
              containerPort: 15672
            {{- if .Values.rabbitmqSTOMPPlugin.enabled }}
            - name: stomp-tcp
              protocol: TCP
              containerPort: 61613
            - name: stomp-ssl
              protocol: TCP
              containerPort: 61614
            {{- end }}
            {{- if .Values.rabbitmqWebSTOMPPlugin.enabled }}
            - name: stomp-ws
              protocol: TCP
              containerPort: 15674
            {{- end }}
            {{- if .Values.rabbitmqMQTTPlugin.enabled }}
            - name: mqtt-tcp
              protocol: TCP
              containerPort: 1883
            - name: mqtt-ssl
              protocol: TCP
              containerPort: 8883
            {{- end }}
            {{- if .Values.rabbitmqWebMQTTPlugin.enabled }}
            - name: mqtt-ws
              protocol: TCP
              containerPort: 15675
            {{- end }}
            {{- if .Values.rabbitmqAmqpsSupport.enabled }}
            - name: amqps
              protocol: TCP
              containerPort: 5671
            {{- end }}
            {{- if .Values.rabbitmqPrometheusPlugin.enabled }}
            - name: metrics
              protocol: TCP
              containerPort: {{ .Values.rabbitmqPrometheusPlugin.port }}
            {{- end }}
        {{- if .Values.livenessProbe.enabled }}
          livenessProbe:
            {{- toYaml .Values.livenessProbe.config | trim | nindent 12 }}
        {{- end }}
        {{- if .Values.readinessProbe.enabled }}
          readinessProbe:
            {{- toYaml .Values.readinessProbe.config | trim | nindent 12 }}
        {{- end }}
        {{- if .Values.lifecycle }}
          lifecycle:
            {{- toYaml .Values.lifecycle | trim | nindent 12 }}
        {{- end }}
          env:
            - name: MY_POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: RABBITMQ_USE_LONGNAME
              value: "true"
            - name: RABBITMQ_NODENAME
              value: rabbit@$(MY_POD_NAME).{{ template "rabbitmq-ha.fullname" . }}-discovery.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}
            - name: K8S_HOSTNAME_SUFFIX
              value: .{{ template "rabbitmq-ha.fullname" . }}-discovery.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}
            - name: K8S_SERVICE_NAME
              value: {{ template "rabbitmq-ha.fullname" . }}-discovery
            - name: RABBITMQ_ERLANG_COOKIE
              valueFrom:
                secretKeyRef:
                  name: {{ template "rabbitmq-ha.secretName" . }}
                  key: rabbitmq-erlang-cookie
            - name: RABBIT_MANAGEMENT_USER
              valueFrom:
                secretKeyRef:
                  name: {{ template "rabbitmq-ha.secretName" . }}
                  key: rabbitmq-management-username
            - name: RABBIT_MANAGEMENT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: {{ template "rabbitmq-ha.secretName" . }}
                  key: rabbitmq-management-password
            {{- if .Values.rabbitmqHipeCompile }}
            - name: RABBITMQ_HIPE_COMPILE
              value: {{ .Values.rabbitmqHipeCompile | quote }}
            {{- end }}
            {{- range $key, $value := .Values.env }}
            - name: {{ $key }}
              value: {{ $value | quote }}
            {{- end }}            
          resources:
{{ toYaml .Values.resources | indent 12 }}
          volumeMounts:
            - name: {{ .Values.persistentVolume.name }}
              mountPath: /var/lib/rabbitmq
            - name: config
              mountPath: /etc/rabbitmq
            - name: definitions
              mountPath: /etc/definitions
              readOnly: true
            {{- if .Values.rabbitmqCert.enabled }}
            - name: cert
              mountPath: /etc/cert
            {{- end }}
            {{- if .Values.extraVolumeMounts }}
{{ toYaml .Values.extraVolumeMounts | indent 12 }}
            {{- end }}
        {{ if .Values.prometheus.exporter.enabled }}
        - name: {{ .Chart.Name }}-exporter
          image: {{ .Values.prometheus.exporter.image.repository }}:{{ .Values.prometheus.exporter.image.tag }}
          imagePullPolicy: {{ .Values.prometheus.exporter.image.pullPolicy }}
          ports:
          - name: exporter
            protocol: TCP
            containerPort: {{ .Values.prometheus.exporter.port }}
          env:
            - name: PUBLISH_PORT
              value: "{{ .Values.prometheus.exporter.port }}"
            {{ if .Values.prometheus.exporter.capabilities }}
            - name: RABBIT_CAPABILITIES
              value: "{{ .Values.prometheus.exporter.capabilities }}"
            {{- end }}
            - name: RABBIT_USER
              valueFrom:
                secretKeyRef:
                  name: {{ template "rabbitmq-ha.secretName" . }}
                  key: rabbitmq-username
            - name: RABBIT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: {{ template "rabbitmq-ha.secretName" . }}
                  key: rabbitmq-password
            {{- range $key, $value := .Values.prometheus.exporter.env }}
            - name: {{ $key }}
              value: {{ $value | quote }}
            {{- end }}
          resources:
{{ toYaml .Values.prometheus.exporter.resources | indent 12 }}
        {{ end }}
        {{- if .Values.extraContainers }}
{{ tpl (toYaml .Values.extraContainers) . | indent 8 }}
        {{- end }}
      {{- if .Values.nodeSelector }}
      nodeSelector:
{{ toYaml .Values.nodeSelector | indent 8 }}
      {{- end }}
      {{- if .Values.tolerations }}
      tolerations:
{{ toYaml .Values.tolerations | indent 8 }}
      {{- end }}
      {{- if .Values.schedulerName }}
      schedulerName: "{{ .Values.schedulerName }}"
      {{- end }}
      {{- if .Values.affinity }}
      affinity:
      {{- with .Values.affinity }}
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- else }}
      {{- if eq .Values.podAntiAffinity "hard" }}
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - topologyKey: "{{ .Values.podAntiAffinityTopologyKey }}"
              labelSelector:
                matchLabels:
                  app: {{ template "rabbitmq-ha.name" . }}
                  release: {{ .Release.Name }}
      {{- else if eq .Values.podAntiAffinity "soft" }}
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              podAffinityTerm:
                topologyKey: "{{ .Values.podAntiAffinityTopologyKey }}"
                labelSelector:
                  matchLabels:
                    app: {{ template "rabbitmq-ha.name" . }}
                    release: {{ .Release.Name }}
      {{- end }}
      {{- end }}

      {{- if .Values.priorityClassName }}
      priorityClassName: {{ .Values.priorityClassName }}
      {{- end }}
      volumes:
        - name: config
          emptyDir: {}
        - name: configmap
          configMap:
            name: {{ template "rabbitmq-ha.fullname" . }}
        - name: definitions
          secret:
            secretName: {{ template "rabbitmq-ha.secretName" . }}
            items:
            - key: {{ .Values.definitionsSource }}
              path: definitions.json
        {{- if .Values.rabbitmqCert.enabled }}
        - name: cert
          secret:
            defaultMode: 420
            secretName: {{ template "rabbitmq-ha.certSecretName" . }}
        {{- end }}
        {{- if .Values.extraVolumes }}
{{ toYaml .Values.extraVolumes | indent 8 }}
        {{- end }}
{{- if .Values.persistentVolume.enabled }}
  volumeClaimTemplates:
    - metadata:
        name: {{ .Values.persistentVolume.name }}
        annotations:
{{ tpl (toYaml .Values.persistentVolume.annotations) . | indent 10 }}
        labels:
{{ tpl (toYaml .Values.persistentVolume.labels) . | indent 10 }}
      spec:
        accessModes:
        {{- range .Values.persistentVolume.accessModes }}
          - {{ . | quote }}
        {{- end }}
        resources:
          requests:
            storage: {{ .Values.persistentVolume.size | quote }}
      {{- if .Values.persistentVolume.storageClass }}
      {{- if (eq "-" .Values.persistentVolume.storageClass) }}
        storageClassName: ""
      {{- else }}
        storageClassName: "{{ .Values.persistentVolume.storageClass }}"
      {{- end }}
      {{- end }}
      {{- with .Values.persistentVolume.selector }}
        selector:
          {{- toYaml . | nindent 10 }}
      {{- end }}
{{- else }}
        - name: {{ .Values.persistentVolume.name }}
          emptyDir: {}
{{- end }}

Upvotes: 0

Views: 1354

Answers (1)

Ernst
Ernst

Reputation: 324

It might be useful to render out both iterations and run diff on them to find out whether you are accidentally changing fields that are immutable in the API.

Upvotes: 1

Related Questions