Kubernetes Ingress returns "Cannot get /"

Question

I'm trying to deploy a NodeRED pod on my cluster, and have created a service and ingress for it so it can be accessible as I access the rest of my cluster, under the same domain. However when i try to access it via host-name.com/nodered I receive Cannot GET /nodered.

Following are the templates used and describes of all the involved components.

apiVersion: v1
kind: Service
metadata:
  name: nodered-app-service
  namespace: {{ kubernetes_namespace_name }}
spec:
  ports:
  - port: 1880
    targetPort: 1880
  selector:
    app: nodered-service-pod

I have also tried with port:80 for the service, to no avail.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nodered-service-deployment
  namespace: {{ kubernetes_namespace_name }}
  labels:
    app: nodered-service-deployment
    name: nodered-service-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nodered-service-pod
  template:
    metadata:
      labels:
        app: nodered-service-pod
        target: gateway
        buildVersion: "{{ kubernetes_build_number }}"
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccountName: nodered-service-account
      automountServiceAccountToken: false
      securityContext:
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
      containers:
      - name: nodered-service-statefulset
        image: nodered/node-red:{{ nodered_service_version }}
        imagePullPolicy: {{ kubernetes_image_pull_policy }}
        readinessProbe:
          httpGet:
            path: /
            port: 1880
          initialDelaySeconds: 30
          timeoutSeconds: 1     
          periodSeconds: 10
          failureThreshold: 3
        livenessProbe:
          httpGet:
            path: /
            port: 1880
          initialDelaySeconds: 30
          timeoutSeconds: 1     
          periodSeconds: 10
          failureThreshold: 3  
        securityContext:
          allowPrivilegeEscalation: false
        resources:
          limits:
            memory: "2048M"
            cpu: "1000m"
          requests:
            memory: "500M"
            cpu: "100m"
        ports:
        - containerPort: 1880
          name: port-name
        envFrom:
          - configMapRef:
              name: nodered-service-configmap
        env:
          - name: BUILD_TIME
            value: "{{ kubernetes_build_time }}"

The target: gateway refers to the ingress controller

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nodered-ingress
  namespace: {{ kubernetes_namespace_name }}
  annotations: 
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - host: host-name.com
    http:
      paths:
      - path:  /nodered(/|$)(.*)
        backend:
          serviceName: nodered-app-service
          servicePort: 1880

The following is what my Describes show

Name:              nodered-app-service
Namespace:         nodered
Labels:            <none>
Annotations:       <none>
Selector:          app=nodered-service-pod
Type:              ClusterIP
IP:                55.3.145.249
Port:              <unset>  1880/TCP
TargetPort:        port-name/TCP
Endpoints:         10.7.0.79:1880
Session Affinity:  None
Events:            <none>

Name:         nodered-service-statefulset-6c678b7774-clx48
Namespace:    nodered
Priority:     0
Node:         aks-default-40441371-vmss000007/10.7.0.66
Start Time:   Thu, 26 Aug 2021 14:23:33 +0200
Labels:       app=nodered-service-pod
              buildVersion=latest
              pod-template-hash=6c678b7774
              target=gateway
Annotations:  <none>
Status:       Running
IP:           10.7.0.79
IPs:
  IP:           10.7.0.79
Controlled By:  ReplicaSet/nodered-service-statefulset-6c678b7774
Containers:
  nodered-service-statefulset:
    Container ID:   docker://a6f8c9d010feaee352bf219f85205222fa7070c72440c885b9cd52215c4c1042
    Image:          nodered/node-red:latest-12
    Image ID:       docker-pullable://nodered/node-red@sha256:f02ccb26aaca2b3ee9c8a452d9516c9546509690523627a33909af9cf1e93d1e
    Port:           1880/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 26 Aug 2021 14:23:36 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  2048M
    Requests:
      cpu:      100m
      memory:   500M
    Liveness:   http-get http://:1880/ delay=30s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:1880/ delay=30s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      nodered-service-configmap  ConfigMap  Optional: false
    Environment:
      BUILD_TIME:  2021-08-26T12:23:06.219818+0000
    Mounts:        <none>
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:            <none>
QoS Class:          Burstable
Node-Selectors:     <none>
Tolerations:        node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                    node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:             <none>

Name:              nodered-app-service
Namespace:         nodered
Labels:            <none>
Annotations:       <none>
Selector:          app=nodered-service-pod
Type:              ClusterIP
IP:                55.3.145.249
Port:              <unset>  1880/TCP
TargetPort:        port-name/TCP
Endpoints:         10.7.0.79:1880
Session Affinity:  None
Events:            <none>
PS C:\Users\hid5tim> kubectl describe ingress -n nodered
Name:             nodered-ingress
Namespace:        nodered
Address:          10.7.31.254
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                              Path  Backends
  ----                              ----  --------
  host-name.com
                                    /nodered(/|$)(.*)   nodered-app-service:1880 (10.7.0.79:1880)
Annotations:                        kubernetes.io/ingress.class: nginx
                                    nginx.ingress.kubernetes.io/ssl-redirect: false
Events:                             <none>

The logs from the ingress controller are below. I've been on this issue for the last 24 hours or so and its tearing me apart, the setup looks identical to other deployments I have that are functional. Could this be something wrong with the nodered image? I have checked and it does expose 1880.

194.xx.xxx.x - [194.xx.xxx.x] - - [26/Aug/2021:10:40:12 +0000] "GET /nodered HTTP/1.1" 404 146 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36" 871 0.008 [nodered-nodered-app-service-80] 10.7.0.68:1880 146 0.008 404
74887808fa2eb09fd4ed64061639991e ```

Answer 1

as the comment by Andrew points out, I was using rewrite annotation wrong, once I removed the (/|$)(.*) and specified the path type as prefix it worked.