createNamespacedJob Fails with “namespace was null or undefined”

Question

Summary:

When calling BatchV1Api.createNamespacedJob with a hard-coded namespace (“audio-processing”)—both as the first argument and in the job manifest metadata—the client throws a RequiredError stating that the namespace parameter is null or undefined. This occurs even though the KubeConfig is loaded with a default namespace and the manifest includes a hard-coded namespace in its metadata.

Steps to Reproduce:

1. Set up a Kubernetes (EKS) cluster and ensure that a namespace named “audio-processing” exists.

2. Create a minimal Node.js script that:Loads a KubeConfig using kc.loadFromOptions() (or an equivalent method).

• Creates an API client using kc.makeApiClient(BatchV1Api).
• Prepares a job manifest with hard-coded namespace in its metadata.
• Calls createNamespacedJob("audio-processing", jobManifest).

3. Run the script and observe that an error is thrown indicating that the namespace parameter is null or undefined.

Minimal Viable Example:

// File: minimal-example.js

import * as k8s from "@kubernetes/client-node";

async function main() {
  // Set up a dummy KubeConfig
  const kc = new k8s.KubeConfig();
  kc.loadFromOptions({
    clusters: [
      {
        name: "dummyCluster",
        server: "https://<your-cluster-endpoint>",
        caData: "<your-ca-data>",
        skipTLSVerify: false,
      },
    ],
    contexts: [
      {
        name: "dummyContext",
        cluster: "dummyCluster",
        user: "dummyUser",
        namespace: "audio-processing",   // Default namespace provided here
      },
    ],
    users: [
      {
        name: "dummyUser",
        // For this minimal example, no exec configuration is needed.
      },
    ],
    currentContext: "dummyContext",
  });

  // Create the BatchV1Api client from the loaded KubeConfig
  const batchV1Api = kc.makeApiClient(k8s.BatchV1Api);

  // Job manifest with the namespace explicitly declared in metadata
  const jobManifest = {
    apiVersion: "batch/v1",
    kind: "Job",
    metadata: {
      name: "test-job",
      namespace: "audio-processing", // Hard-coded namespace in the manifest
    },
    spec: {
      template: {
        spec: {
          containers: [
            {
              name: "test-container",
              image: "busybox",
              command: ["echo", "Hello World"],
            },
          ],
          restartPolicy: "Never",
        },
      },
    },
  };

  console.log("Job manifest:", JSON.stringify(jobManifest, null, 2));

  try {
    // Explicitly pass the namespace as the first argument to createNamespacedJob
    const response = await batchV1Api.createNamespacedJob("audio-processing", jobManifest);
    console.log("Job created successfully:", response.body);
  } catch (error) {
    console.error("Error creating job:", error);
  }
}

main();

Expected Behavior:

The call to: await batchV1Api.createNamespacedJob("audio-processing", jobManifest);

should succeed and create the job in the “audio-processing” namespace.

Observed Behavior:

The API call fails with the error:

RequiredError: Required parameter namespace was null or undefined when calling BatchV1Api.createNamespacedJob.
    at BatchV1ApiRequestFactory.createNamespacedJob (…/node_modules/@kubernetes/client-node/dist/gen/apis/BatchV1Api.js:84:19)
    ...

Even though:

• The namespace is provided explicitly as "audio-processing" in the method call.

• The job manifest also includes "namespace": "audio-processing" in its metadata.

• The KubeConfig is loaded with a context that has a default namespace.

Environment Details:

• @kubernetes/client-node version: ("@kubernetes/client-node": "^1.0.0",)

• Node.js version: (runtime: nodejs20.x)

• Deployment Environment: (AWS Lambda Function triggering a Fargate job on EKS)

• Cluster: EKS (or Kubernetes) with the namespace “audio-processing” created.

Additional Context:

• The error originates from a validation check in the BatchV1Api source (see BatchV1Api.ts #Line84), which throws a RequiredError if the namespace parameter is null or undefined.

• We have attempted multiple configuration and manifest-based solutions (using both kc.makeApiClient and ensuring the namespace is present in both the API call and the job manifest) without success.

• The issue persists even after verifying that the hard-coded namespace string is correctly provided.

Request for Help:

We request assistance to understand why, despite the explicit provision of the namespace, the API call still fails. Is there an expected alternative approach to instantiating the client (e.g., using createConfiguration instead of kc.makeApiClient) that might resolve this issue? Any guidance or fixes would be greatly appreciated.

this is how the job manifest is generated:

/**
 * Generates a Kubernetes Job manifest for the audio conversion task.
 *
 * @param {Object} params - Parameters for the job manifest.
 * @param {string} params.inputS3Path - The S3 path of the input MP3 file.
 * @param {string} params.outputS3Path - The S3 path for the output PCM file.
 * @returns {Object} - The Kubernetes Job manifest.
 */
function generateJobManifest({ inputS3Path, outputS3Path, recordId }) {
  const jobManifest = {
    apiVersion: "batch/v1",
    kind: "Job",
    metadata: {
      name: `audio-conversion-${recordId}`, // FIXME: Make sure this is correct job name!
      namespace: "audio-processing",
    },
    spec: {
      parallelism: 5, // Specifies how many pods can run at once
      completions: 5, // Specifies how many pod successes are needed
      template: {
        spec: {
          containers: [
            {
              name: "audio-processing",
              image: `${process.env.ECR_REPO_PCM_CONVERTER}:latest`,
              env: [
                { name: "INPUT_S3_PATH", value: inputS3Path },
                { name: "OUTPUT_S3_PATH", value: outputS3Path },
                {
                  name: "AWS_ACCOUNT_REGION",
                  value: process.env.AWS_ACCOUNT_REGION,
                },
              ],
              command: ["/convert.sh"],
              resources: {
                requests: {
                  cpu: "4000m", // Reserve 4 vCPU
                  memory: "8Gi", // Reserve 8 GiB of memory
                  "ephemeral-storage": "175Gi",
                },
                limits: {
                  cpu: "4000m", // Allow up to 4 vCPU
                  memory: "8Gi", // Allow up to 8 GiB of memory
                  "ephemeral-storage": "175Gi",
                },
              },
            },
          ],
          restartPolicy: "Never",
        },
      },
      backoffLimit: 5,
    },
  };

  console.log("Generated Job Manifest:", JSON.stringify(jobManifest));
  return jobManifest;
}

/**
 *
 * Retrieves the EKS cluster details (endpoint & CA data) so we can create a kubeconfig.
 *
 *
 */
async function getEksClusterDetails(clusterName, region) {
  const eksClient = new EKSClient({ region });
  const command = new DescribeClusterCommand({ name: clusterName });
  const response = await eksClient.send(command);
  console.log("EKS Cluster details: ", response);
  return response.cluster;
}


/**
 * Builds a kubeconfig object for the given EKS cluster using exec authentication.
 */
export async function loadKubeConfig() {
  const clusterName = process.env.CLUSTER_NAME; // e.g. "dev-eks-med-transcribe"

  console.log("CLUSTER NAME: ", clusterName);
  const region = process.env.AWS_ACCOUNT_REGION;

  console.log("CLUSTER REGION: ", region);
  const cluster = await getEksClusterDetails(clusterName, region);

  console.log("CLUSTER DETAILS: ", cluster);
  const kc = new k8s.KubeConfig();

  console.log("KUBE CONFIG: ", kc);

  kc.loadFromOptions({
    clusters: [
      {
        name: cluster.name,
        server: cluster.endpoint,
        caData: cluster.certificateAuthority.data,
        skipTLSVerify: false,
      },
    ],
    contexts: [
      {
        name: "aws",
        cluster: cluster.name,
        user: "aws",
        namespace: "audio-processing",
      },
    ],
    users: [
      {
        name: "aws",
        exec: {
          apiVersion: "client.authentication.k8s.io/v1beta1",
          command: "aws",
          args: [
            "eks",
            "get-token",
            "--cluster-name",
            cluster.name,
            "--region",
            region,
          ],
        },
      },
    ],
    currentContext: "aws",
  });

  console.log("KUBE LOAD FROM OPTIONS: ", kc);

  return kc;
}

ERROR LOGGED

ERROR   Error creating EKS job: RequiredError: Required parameter namespace was null or undefined when calling BatchV1Api.createNamespacedJob.
    at BatchV1ApiRequestFactory.createNamespacedJob (file:///var/task/node_modules/@kubernetes/client-node/dist/gen/apis/BatchV1Api.js:84:19)
    at ObservableBatchV1Api.createNamespacedJobWithHttpInfo (file:///var/task/node_modules/@kubernetes/client-node/dist/gen/types/ObservableAPI.js:7621:59)
    at ObservableBatchV1Api.createNamespacedJob (file:///var/task/node_modules/@kubernetes/client-node/dist/gen/types/ObservableAPI.js:7646:21)
    at ObjectBatchV1Api.createNamespacedJob (file:///var/task/node_modules/@kubernetes/client-node/dist/gen/types/ObjectParamAPI.js:2840:25)
    at Runtime.main [as handler] (file:///var/task/triggerConversionJob.js:219:24)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  api: 'BatchV1Api',
  method: 'createNamespacedJob',
  field: 'namespace'
}