CODEWITHSUNDEEP
jenkins-x
driouxg
driouxg

Reputation: 745

Jx Boot Fails Step `install-jenkins-x` w/ Command "sh -c jx step helm apply --boot --remote --name jenkins-x --provider-values-dir ../kubeProviders"

I'm currently trying to install Jenkins X into my GKE Kubernetes Cluster. But jx boot is failing on step "install-jenkins-x".

My jx version output is:

NAME               VERSION
jx                 2.1.36
Kubernetes cluster v1.14.10-gke.27
kubectl            v1.15.2
helm client        2.12.2
git                2.25.1.windows.1
Operating System   Windows 10 Enterprise 1909 build 18363

Here is my jenkins-x.yml:

buildPack: none
pipelineConfig:
  pipelines:
    pullRequest:
      pipeline:
        agent:
          image: gcr.io/jenkinsxio/builder-go
        stages:
        - name: pr-checks
          options:
            containerOptions:
              name: ""
              resources: {}
              volumeMounts:
              - mountPath: /builder/home/.jx/localSecrets/currentCluster
                name: local-secrets
                readOnly: true
            volumes:
            - name: local-secrets
              secret:
                optional: true
                secretName: local-param-secrets
          steps:
          - args:
            - step
            - verify
            - values
            - --values-file=parameters.yaml
            - --schema-file=parameters.tmpl.schema.json
            command: jx
            dir: ../../env
            name: verify-parameters
          - args:
            - build
            command: make
            dir: ../../env
            name: lint-env-helm
    release:
      pipeline:
        agent:
          image: gcr.io/jenkinsxio/builder-go
        environment:
        - name: DEPLOY_NAMESPACE
          value: jx
        - name: JX_LOG_LEVEL
          value: trace
        stages:
        - name: release
          options:
            containerOptions:
              name: ""
              resources: {}
              volumeMounts:
              - mountPath: /builder/home/.jx/localSecrets/currentCluster
                name: local-secrets
                readOnly: true
            volumes:
            - name: local-secrets
              secret:
                optional: true
                secretName: local-param-secrets
          steps:
          - args:
            - step
            - git
            - validate
            command: jx
            dir: ../../env
            name: validate-git
          - args:
            - step
            - verify
            - preinstall
            - --provider-values-dir="kubeProviders"
            command: jx
            dir: ../..
            name: verify-preinstall
          - args:
            - upgrade
            - crd
            command: jx
            dir: ../..
            name: install-jx-crds
          - args:
            - step
            - helm
            - apply
            - --boot
            - --remote
            - --no-vault
            - --name
            - velero
            command: jx
            dir: ../../systems/velero
            env:
            - name: DEPLOY_NAMESPACE
              value: velero
            name: install-velero
          - args:
            - step
            - helm
            - apply
            - --boot
            - --remote
            - --no-vault
            - --name
            - velero-backups
            command: jx
            dir: ../../systems/velero-backups
            env:
            - name: DEPLOY_NAMESPACE
              value: velero
            name: install-velero-backups
          - args:
            - step
            - helm
            - apply
            - --boot
            - --remote
            - --no-vault
            - --name
            - jxing
            command: jx
            dir: ../../systems/jxing
            env:
            - name: DEPLOY_NAMESPACE
              value: kube-system
            name: install-nginx-controller
          - args:
            - step
            - create
            - install
            - values
            - -b
            command: jx
            dir: ../../env
            name: create-install-values
          - args:
            - step
            - helm
            - apply
            - --boot
            - --remote
            - --no-vault
            - --name
            - exdns
            command: jx
            dir: ../../systems/external-dns
            name: install-external-dns
          - args:
            - apply
            - --wait
            - --validate=false
            - -f
            - https://raw.githubusercontent.com/jetstack/cert-manager/release-0.11/deploy/manifests/00-crds.yaml
            command: kubectl
            dir: ../..
            env:
            - name: DEPLOY_NAMESPACE
              value: cert-manager
            name: install-cert-manager-crds
          - args:
            - step
            - helm
            - apply
            - --boot
            - --remote
            - --no-vault
            - --name
            - cm
            command: jx
            dir: ../../systems/cm
            env:
            - name: DEPLOY_NAMESPACE
              value: cert-manager
            name: install-cert-manager
          - args:
            - step
            - helm
            - apply
            - --boot
            - --remote
            - --no-vault
            - --name
            - acme
            command: jx
            dir: ../../systems/acme
            name: install-acme-issuer-and-certificate
          - args:
            - step
            - boot
            - vault
            - --provider-values-dir
            - ../../kubeProviders
            command: jx
            dir: ../../systems/vault
            name: install-vault
          - args:
            - step
            - create
            - values
            - --name
            - parameters
            command: jx
            dir: ../../env
            name: create-helm-values
          - args:
            - step
            - create
            - templated
            - --parameters-file=../../env/parameters.yaml
            - --requirements-dir=../../
            - --template-file=jx-auth-configmap.tmpl.yaml
            - --config-file=templates/jx-auth-configmap.yaml
            command: jx
            dir: ../../systems/jx-auth
            name: create-jx-auth-config
          - args:
            - step
            - helm
            - apply
            - --boot
            - --remote
            - --no-vault
            - --name
            - jx-auth
            command: jx
            dir: ../../systems/jx-auth
            name: install-jx-auth-config
          - args:
            - step
            - helm
            - apply
            - --boot
            - --remote
            - --name
            - jenkins-x
            - --provider-values-dir
            - ../kubeProviders
            command: jx
            dir: ../../env
            name: install-jenkins-x
          - args:
            - step
            - verify
            - env
            command: jx
            dir: ../..
            name: verify-jenkins-x-environment
          - args:
            - step
            - helm
            - apply
            - --boot
            - --name
            - repos
            command: jx
            dir: ../../repositories
            name: install-repositories
          - args:
            - step
            - scheduler
            - config
            - apply
            - --direct=true
            command: jx
            dir: ../../prowConfig
            name: install-pipelines
          - args:
            - update
            - webhooks
            - --verbose
            - --warn-on-fail
            command: jx
            dir: ../../repositories
            name: update-webhooks
          - args:
            - step
            - verify
            - install
            - --pod-wait-time
            - 30m
            command: jx
            dir: ../../env
            name: verify-installation

Here is my jx-requirements.yml file:

autoUpdate:
    enabled: false
    schedule: ""
bootConfigURL: https://github.com/jenkins-x/jenkins-x-boot-config
cluster:
    clusterName: palmcyan
    devEnvApprovers:
        - driouxg
    environmentGitOwner: driouxg
    gitKind: github
    gitName: github
    gitServer: https://github.com
    namespace: jx
    project: s*******
    provider: gke
    registry: gcr.io
    zone: us-west1-a
environments:
    - ingress:
          domain: mydomain.com
          externalDNS: false
          namespaceSubDomain: -jx.
          tls:
              email: "d****@****.com"
              enabled: false
              production: false
      key: dev
      repository: environment-palmcyan-dev
    - ingress:
          domain: mydomain.com
          externalDNS: false
          namespaceSubDomain: -jx.
          tls:
              email: "d*****@****.com"
              enabled: false
              production: false
      key: staging
      repository: environment-palmcyan-staging
    - ingress:
          domain: mydomain.com
          externalDNS: true
          namespaceSubDomain: -jx.
          tls:
              email: d******@******.com
              enabled: true
              production: true
      key: production
      repository: environment-palmcyan-production
gitops: true
ingress:
    cloud_dns_secret_name: external-dns-gcp-sa
    domain: mydomain.com
    externalDNS: true
    namespaceSubDomain: -jx.
    tls:
        email: d******@*****.com
        enabled: true
        production: true
kaniko: true
repository: nexus
secretStorage: local
storage:
    backup:
        enabled: false
        url: ""
    logs:
        enabled: false
        url: ""
    reports:
        enabled: false
        url: ""
    repository:
        enabled: false
        url: ""
vault: {}
velero:
    schedule: ""
    ttl: ""
versionStream:
    ref: v1.0.457
    url: https://github.com/jenkins-x/jenkins-x-versions.git
webhook: lighthouse

This is the specific error that I'm stuck on after running jx boot. Some of the error messages have been truncated due to StackOverflow character limit:

Error from server (Invalid): error when creating "C:\\Users\\driouxg~1.GUI\\AppData\\Local\\Temp\\helm-template-workdir-110888103\\jenkins-x\\output\\namespaces\\jx\\env\\charts\\jxboot-resources\\templates\\part0-700-chartmuseum-ing.yaml": Ingress.extensions "chartmuseum" is invalid: spec.rules[0].host: Invalid value: "chartmuseum-jx.": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
Error from server (Invalid): error when creating "C:\\Users\\driouxg~1.GUI\\AppData\\Local\\Temp\\helm-template-workdir-110888103\\jenkins-x\\output\\namespaces\\jx\\env\\charts\\jxboot-resources\\templates\\part0-700-hook-ing.yaml": Ingress.extensions "hook" is invalid: spec.rules[0].host: Invalid value: "hook-jx.": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
Error from server (Invalid): error when creating "C:\\Users\\driouxg~1.GUI\\AppData\\Local\\Temp\\helm-template-workdir-110888103\\jenkins-x\\output\\namespaces\\jx\\env\\charts\\jxboot-resources\\templates\\part0-700-nexus-ing.yaml": Ingress.extensions "nexus" is invalid: spec.rules[0].host: Invalid value: "nexus-jx.": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
Error from server (Invalid): error when creating "C:\\Users\\driouxg~1.GUI\\AppData\\Local\\Temp\\helm-template-workdir-110888103\\jenkins-x\\output\\namespaces\\jx\\env\\charts\\jxboot-resources\\templates\\part0-default-scheduler.yaml": Scheduler.jenkins.io "default-scheduler" is invalid: []: Invalid value: map[string]interface {}{"apiVersion":"jenkins.io/v1", "kind":"Scheduler", "metadata":map[string]interface {}{"annotations":map[string]interface {}{"jenkins.io/chart":"env", "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"jenkins.io/v1\",\"kind\":\"Scheduler\",\"metadata\":{\"annotations\":{\"jenkins.io/chart\":\"env\"},\"creationTimestamp\":null,\"labels\":{\"jenkins.io/chart-release\":\"jenkins-x\",\"jenkins.io/namespace\":\"jx\",\"jenkins.io/version\":\"1\"},\"name\":\"default-scheduler\",\"namespace\":\"jx\"},\"spec\":{\"approve\":{\"issueRequired\":false,\"lgtmActsAsApprove\":true,\"requireSelfApproval\":true},\"merger\":{\"blockerLabel\":\"\",\"maxGoroutines\":0,\"mergeMethod\":\"merge\",\"policy\":{\"fromBranchProtection\":true,\"optionalContexts\":{},\"requiredContexts\":{},\"requiredIfPresentContexts\":{},\"skipUnknownContexts\":false},\"prStatusBaseUrl\":\"\",\"squashLabel\":\"\",\"targetUrl\":\"http://deck-jx.\"},\"plugins\":{\"entries\":[\"approve\",\"assign\",\"blunderbuss\",\"help\",\"hold\",\"lgtm\",\"lifecycle\",\"override\",\"size\",\"trigger\",\"wip\",\"heart\",\"cat\",\"dog\",\"pony\"]},\"policy\":{\"protectTested\":true},\"postsubmits\":{\"entries\":[{\"agent\":\"tekton\",\"branches\":{\"entries\":[\"master\"]},\"cluster\":\"\",\"context\":\"\",\"labels\":{},\"maxConcurrency\":0,\"name\":\"release\",\"report\":false,\"runIfChanged\":\"\",\"skipBranches\":{}}]},\"presubmits\":{\"entries\":[{\"agent\":\"tekton\",\"alwaysRun\":true,\"branches\":{},\"cluster\":\"\",\"context\":\"pr-build\",\"labels\":{},\"maxConcurrency\":0,\"mergeMethod\":\"\",\"name\":\"pr-build\",\"optional\":false,\"policy\":{\"Replace\":false,\"requiredStatusChecks\":{\"contexts\":{\"entries\":[\"pr-build\"]}}},\"queries\":[{\"excludedBranches\":{},\"includedBranches\":{},\"labels\":{\"entries\":[\"approved\"]},\"milestone\":\"\",\"missingLabels\":{\"entries\":[\"do-not-merge\",\"do-not-merge/hold\",\"do-not-merge/work-in-progress\",\"needs-ok-to-test\",\"needs-rebase\"]},\"reviewApprovedRequired\":false},{\"excludedBranches\":{},\"includedBranches\":{},\"labels\":{\"entries\":[\"updatebot\"]},\"milestone\":\"\",\"missingLabels\":{\"entries\":[\"do-not-merge\",\"do-not-merge/hold\",\"do-not-merge/work-in-progress\",\"needs-ok-to-test\",\"needs-rebase\"]},\"reviewApprovedRequired\":false}],\"report\":true,\"rerunCommand\":\"/test this\",\"runIfChanged\":\"\",\"skipBranches\":{},\"trigger\":\"(?m)^/test( all| this),?(\\\\s+|$)\"}]},\"schedulerAgent\":{\"agent\":\"tekton\"},\"trigger\":{\"ignoreOkToTest\":false,\"joinOrgUrl\":\"\",\"onlyOrgMembers\":false,\"trustedOrg\":null},\"welcome\":[{\"message_template\":\"Welcome\"}]}}\n"}, "creationTimestamp":"2020-05-14T19:33:46Z", "generation":1, "labels":map[string]interface {}{"jenkins.io/chart-release":"jenkins-x", "jenkins.io/namespace":"jx", "jenkins.io/version":"1"}, "name":"default-scheduler", "namespace":"jx", "uid":"d3e8b59a-9619-11ea-88a3-42010a8a0fe5"}, "spec":map[string]interface {}{"approve":map[string]interface {}{"issueRequired":false, "lgtmActsAsApprove":true, "requireSelfApproval":true}, "merger":map[string]interface {}{"blockerLabel":"", "maxGoroutines":0, "mergeMethod":"merge", "policy":map[string]interface {}{"fromBranchProtection":true, "optionalContexts":map[string]interface {}{}, "requiredContexts":map[string]interface {}{}, "requiredIfPresentContexts":map[string]interface {}{}, "skipUnknownContexts":false}, "prStatusBaseUrl":"", "squashLabel":"", "targetUrl":"http://deck-jx."}, "plugins":map[string]interface {}{"entries":[]interface {}{"approve", "assign", "blunderbuss", "help", "hold", "lgtm", "lifecycle", "override", "size", "trigger", "wip", "heart", "cat", "dog", "pony"}}, "policy":map[string]interface {}{"protectTested":true}, "postsubmits":map[string]interface {}{"entries":[]interface {}{map[string]interface {}{"agent":"tekton", "branches":map[string]interface {}{"entries":[]interface {}{"master"}}, "cluster":"", "context":"", "labels":map[string]interface {}{}, "maxConcurrency":0, "name":"release", "report":false, "runIfChanged":"", "skipBranches":map[string]interface {}{}}}}, "presubmits":map[string]interface {}{"entries":[]interface {}{map[string]interface {}{"agent":"tekton", "alwaysRun":true, "branches":map[string]interface {}{}, "cluster":"", "context":"pr-build", "labels":map[string]interface {}{}, "maxConcurrency":0, "mergeMethod":"", "name":"pr-build", "optional":false, "policy":map[string]interface {}{"Replace":false, "requiredStatusChecks":map[string]interface {}{"contexts":map[string]interface {}{"entries":[]interface {}{"pr-build"}}}}, "queries":[]interface {}{map[string]interface {}{"excludedBranches":map[string]interface {}{}, "includedBranches":map[string]interface {}{}, "labels":map[string]interface {}{"entries":[]interface {}{"approved"}}, "milestone":"", "missingLabels":map[string]interface {}{"entries":[]interface {}{"do-not-merge", "do-not-merge/hold", "do-not-merge/work-in-progress", "needs-ok-to-test", "needs-rebase"}}, "reviewApprovedRequired":false}, map[string]interface {}{"excludedBranches":map[string]interface {}{}, "includedBranches":map[string]interface {}{}, "labels":map[string]interface {}{"entries":[]interface {}{"updatebot"}}, "milestone":"", "missingLabels":map[string]interface {}{"entries":[]interface {}{"do-not-merge", "do-not-merge/hold", "do-not-merge/work-in-progress", "needs-ok-to-test", "needs-rebase"}}, "reviewApprovedRequired":false}}, "report":true, "rerunCommand":"/test this", "runIfChanged":"", "skipBranches":map[string]interface {}{}, "trigger":"(?m)^/test( all| this),?(\\s+|$)"}}}, "schedulerAgent":map[string]interface {}{"agent":"tekton"}, "trigger":map[string]interface {}{"ignoreOkToTest":false, "joinOrgUrl":"", "onlyOrgMembers":false, "trustedOrg":interface {}(nil)}, "welcome":[]interface {}{map[string]interface {}{"message_template":"Welcome"}}}}: validation failure list:
spec.trigger.trustedOrg in body must be of type string: "null"
Error from server (Invalid): error when creating "C:\\Users\\driouxg~1.GUI\\AppData\\Local\\Temp\\helm-template-workdir-110888103\\jenkins-x\\output\\namespaces\\jx\\env\\charts\\jxboot-resources\\templates\\part0-env-scheduler.yaml": Scheduler.jenkins.io "env-scheduler" is invalid: []: ...
spec.trigger.trustedOrg in body must be of type string: "null"
Error from server (Invalid): error when creating "C:\\Users\\driouxg~1.GUI\\AppData\\Local\\Temp\\helm-template-workdir-110888103\\jenkins-x\\output\\namespaces\\jx\\env\\charts\\jxboot-resources\\templates\\part0-pr-only-scheduler.yaml": Scheduler.jenkins.io "pr-only" is invalid: []: ...
spec.trigger.trustedOrg in body must be of type string: "null"
Error from server (Invalid): error when creating "C:\\Users\\driouxg~1.GUI\\AppData\\Local\\Temp\\helm-template-workdir-110888103\\jenkins-x\\output\\namespaces\\jx\\env\\charts\\jxboot-resources\\templates\\part0-release-only-scheduler.yaml": Scheduler.jenkins.io "release-only" is invalid: []: ...
spec.trigger.trustedOrg in body must be of type string: "null"'
error: failed to interpret pipeline file jenkins-x.yml: failed to run 'sh -c jx step helm apply --boot --remote --name jenkins-x --provider-values-dir ../kubeProviders' command in directory 'env', output: ''

I've tried changing the namespaceSubdomain, and domain properties in jx-requirements.yml for all environments as well as the ingress. None of which fix or change the error that is outputted to the console. The kubernetes resources' yaml files that are throwing errors (part0-700-chartmuseum-ing.yaml, part0-700-hook-ing.yaml, part0-700-nexus-ing.yaml, and part0-default-scheduler.yaml) are not present in the jenkins-x-boot-config repository and I cannot find where they are downloaded from when executing jx boot which is preventing me from being able to debug this issue. If I could see the Kubernetes resource yaml files then I could understand which property in my jx-requirements.yaml or some other yaml file I should be updating to fix this issue.

Let me know if there are any other files that I should post in this question to help debug the issue.

Upvotes: 0

Views: 1174

Answers (1)

Deane
Deane

Reputation: 36

This is a known issue in jx boot on Windows - https://github.com/jenkins-x/jx/issues/7156. Have you tried it using Windows Subsystem for Linux?

Upvotes: 2

Related Questions