Dilma
Dilma

Reputation: 645

Hiveserver2 not starting in Kubernetes Setup

I'm experiencing issues with a HiveServer2 pod in my Kubernetes cluster. The pod is failing due to readiness probe errors, specifically being unable to connect to port 10000.

Readiness probe failed: dial tcp 10.42.0.63:10000: connect: connection refused

I've attempted to modify the readiness probe configuration, but the core issue appears to be with the port 10000 binding itself.

Following is the yaml file for hiveserver2 deployment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hiveserver2
  namespace: hadoop-ecosystem
spec:
  selector:
    matchLabels:
      app: hiveserver2
  replicas: 1
  template:
    metadata:
      labels:
        app: hiveserver2
    spec:
      initContainers:
      - name: wait-for-metastore
        image: busybox:1.36
        command: ['sh', '-c', 'until nc -z hive-metastore 9083; do echo waiting for metastore; sleep 2; done;']
      containers:
      - name: hiveserver2
        image: apache/hive:3.1.3
        command: 
          - "/bin/bash"
          - "-c"
          - |
            export HADOOP_CLIENT_OPTS="-Xms1g -Xmx2g -XX:+UseG1GC -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps"
            export HIVE_LOG_DIR=/tmp/hive/logs
            export HADOOP_ROOT_LOGGER="DEBUG,console"
            export HIVE_ROOT_LOGGER="DEBUG,console"
          
            # Debug information
            echo "Checking configurations:"
            echo "HIVE_HOME: $HIVE_HOME"
            echo "HADOOP_HOME: $HADOOP_HOME"
            echo "HADOOP_CONF_DIR: /etc/hadoop"
          
            # Export HADOOP_CONF_DIR
            export HADOOP_CONF_DIR=/etc/hadoop
          
            # Start HiveServer2
            $HIVE_HOME/bin/hiveserver2 \
              --hiveconf hive.root.logger=DEBUG,console \
              --hiveconf hive.server2.thrift.port=10000 \
              --hiveconf hive.server2.thrift.bind.host=0.0.0.0 \
              --hiveconf hive.server2.enable.doAs=false \
              --hiveconf hive.metastore.uris=thrift://hive-metastore:9083 \
              --hiveconf hive.metastore.event.db.notification.api.auth=false \
              --hiveconf hive.server2.logging.operation.enabled=true \
              --hiveconf hive.server2.thrift.http.port=10001 \
              --hiveconf hive.server2.webui.port=10002 \
              --hiveconf hive.server2.webui.host=0.0.0.0 \
              --hiveconf hive.server2.transport.mode=binary \
              --hiveconf hive.server2.thrift.min.worker.threads=5 \
              --hiveconf hive.server2.thrift.max.worker.threads=500 \
              --hiveconf hive.server2.thrift.sasl.qop=auth \
              --hiveconf hive.server2.authentication=NONE \
              --hiveconf hive.server2.use.SSL=false
        ports:
        - containerPort: 10000
        env:
        - name: HADOOP_HOME
          value: "/opt/hadoop"
        - name: HADOOP_ROOT_LOGGER
          value: DEBUG,console
        - name: HIVE_HOME
          value: "/opt/hive"
        resources:                
          requests:
            memory: "2Gi"
            cpu: "500m"
          limits:
            memory: "3Gi"
            cpu: "1000m"
        startupProbe:
          tcpSocket:
            port: 10000
          failureThreshold: 30
          periodSeconds: 10
          initialDelaySeconds: 60
        readinessProbe:
          tcpSocket:
            port: 10000
          initialDelaySeconds: 60  # Increased from 30 to 120
          periodSeconds: 20         # Increased from 10 to 20
          timeoutSeconds: 5         # Added explicit timeout
          failureThreshold: 6       # Increased failure threshold
        volumeMounts:
        - name: hive-config
          mountPath: /opt/hive/conf/hive-site.xml
          subPath: hive-site.xml
        - name: hadoop-config
          mountPath: /etc/hadoop/core-site.xml
          subPath: core-site.xml
        - name: hadoop-config
          mountPath: /etc/hadoop/hdfs-site.xml
          subPath: hdfs-site.xml
        - name: hive-config
          mountPath: /etc/hadoop/log4j.properties
          subPath: log4j.properties
      volumes:
      - name: hive-config
        configMap:
          name: hive-config
      - name: hadoop-config
        configMap:
          name: hadoop-config

---
apiVersion: v1
kind: Service
metadata:
  name: hiveserver2
  namespace: hadoop-ecosystem
spec:
  selector:
    app: hiveserver2
  ports:
  - port: 10000
    targetPort: 10000
    name: thrift
  - port: 10002
    targetPort: 10002
    name: webui
  type: ClusterIP

What could be preventing HiveServer2 from binding to port 10000? Are there any common configuration issues that might cause this port binding failure?

Upvotes: 0

Views: 37

Answers (0)

Related Questions