CODEWITHSUNDEEP
kuberneteskubernetes-helmapache-superset
MariusPontmercy
MariusPontmercy

Reputation: 440

Superset on Kubernetes with Helm not using custom values file

I'm trying to install Apache Superset on a Kubernetes Cluster (AWS EKS) using Helm and following the official procedure described here.

bash-3.2$ helm repo add superset https://apache.github.io/superset
"superset" has been added to your repositories

bash-3.2$ helm search repo superset
NAME                CHART VERSION   APP VERSION DESCRIPTION                                       
superset/superset   0.1.2           1.0         Apache Superset is a modern, enterprise-ready b...

Since I want to use RDS and ElastiCache for database and cache respectively, instead of the bundled postgresql and redis, I need to override several values in the default values.yaml so I made a copy of the default values

bash-3.2$ helm show values superset/superset > custom-values.yaml

edited several sections such as

postgresql:
  ##
  ## Use the PostgreSQL chart dependency.
  ## Set to false if bringing your own PostgreSQL.
  enabled: false

[...]

  ##
  ## If you are bringing your own PostgreSQL, you should set postgresHost and
  ## also probably service.port, postgresqlUsername, postgresqlPassword, and postgresqlDatabase
  postgresHost: myproject.cluster-xxxxxxxxx.us-east-2.rds.amazonaws.com

and others, and installed my release with

bash-3.2$ helm upgrade --install --values custom-values.yaml superset superset/superset

as described in the guide.

The first three pods are created:

bash-3.2$ kubectl get pods
NAME                               READY   STATUS     RESTARTS   AGE
superset-7d96fc8787-vr6c6          0/1     Init:0/1   0          3m4s
superset-init-db-xqmd9             0/1     Init:0/1   0          3m3s
superset-worker-7fff4f497b-cnqs5   0/1     Init:0/1   0          3m4s

but then nothing happens. Logging in into the init-container of the superset-init pod I discovered that the process waiting for the database to become available is stuck because it is using the default environment variables instead of those I provided in the custom-values.yaml file:

bash-3.2$ kubectl exec -it --container wait-for-postgres superset-init-db-xqmd9 -- sh
/ # 
/ # env
REDIS_PORT=6379
KUBERNETES_PORT=tcp://172.20.0.1:443
KUBERNETES_SERVICE_PORT=443
HOSTNAME=superset-init-db-xqmd9
DB_PORT=5432
SUPERSET_PORT=tcp://172.20.72.4:8088
SUPERSET_SERVICE_PORT=8088
SUPERSET_PORT_8088_TCP_ADDR=172.20.72.4
SHLVL=1
HOME=/root
DB_NAME=superset
SUPERSET_PORT_8088_TCP_PORT=8088
SUPERSET_PORT_8088_TCP_PROTO=tcp
SUPERSET_PORT_8088_TCP=tcp://172.20.72.4:8088
TERM=xterm
KUBERNETES_PORT_443_TCP_ADDR=172.20.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
DB_PASS=superset
SUPERSET_SERVICE_PORT_HTTP=8088
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP=tcp://172.20.0.1:443
REDIS_HOST=superset-redis-headless
KUBERNETES_SERVICE_HOST=172.20.0.1
PWD=/
DB_HOST=superset-postgresql
SUPERSET_SERVICE_HOST=172.20.72.4
DB_USER=superset

EDIT:

some of the custom values in the custom-values.yaml file are actually used to override the defaults. E.g:

service:
  type: LoadBalancer

instead of

service:
  type: ClusterIP

and also postgresql and redis pods are not created when I set them as enabled: false but for some reason other custom values are not applied or passed to the secret storing env variables for the pods.

What am I doing wrong?

Upvotes: 0

Views: 4287

Answers (2)

Honeywild
Honeywild

Reputation: 21

I would like to use Kubernetes secrets to store and retrieve secrets in an enterprise production environment. As an experienced user, I can confidently say that this is possible with the official Superset chart. We can pass extra environment variables in raw format, which will be propagated to the pods and can be defined in custom values accordingly.

Upvotes: 0

Saikat Chakrabortty
Saikat Chakrabortty

Reputation: 2680

According to the values.yaml of superset, I do see if you are bringing your own Postgres instance, from the above question as I've understood, You have to change the values of these

supersetNode:
  connections:
    # Change incase bringing your own redis and then also make `redis.enabled`:false
    redis_host: '{{ template "superset.fullname" . }}-redis-headless'
    redis_port: "6379"
    # You need to change below configuration incase bringing own pg instance and as you made `postgresql.enabled`:false that's correct incase bringing own pg instance
    db_host: <YOUR RDS PG HOST>
    db_port: "5432"
    db_user: <YOUR DB USER>
    db_pass: <YOUR DB PASS>
    db_name: <YOUR DB NAME | postgres>

Upvotes: 4

Related Questions