kubectl patch deployment to new configmap not working

Question

I have two kube configmaps, one of them is being used in a deployment. The config maps, and the pod deployment are below

File : configmap.yml

apiVersion: v1
kind: ConfigMap
metadata:
  name: mock-url-config
data:
    base-url: "http://api.infra.com:80/Test"

---

apiVersion: v1
kind: ConfigMap
metadata:
  name: actual-url-config
data:
    base-url: "http://api.infra.com:80"

---

File : deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-api
  annotations:
    kubernetes.twilio.com/service-name: test-api
spec:
  replicas: 1
  template:
    ......
    ......
    spec:
      serviceAccountName: test-sa-account
      containers:
        - name: test-api
          env:
            - name: API_BASE_URL
              valueFrom:
                configMapKeyRef: // USING THE CONFIGMAP VALUE
                  name: mock-url-config
                  key: base-url
          image: ......
    ..........

When I run this deployment, the container test-api is deployed with the mock-url-config configmap. After my tests are run, I want to update the container to use a different configMap (actual-url-config). For that, Im using a kube job to do a kube patch and restart the pods so that the restarted containers use the new configmap. The job is defined as follows:

File : configmap.yml

apiVersion: batch/v1
kind: Job
metadata:
  name: apply-actual-url
  annotations:
    argocd.argoproj.io/hook: PostSync
    argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
    argocd.argoproj.io/sync-wave: "1"
spec:
  template:
    spec:
      serviceAccountName: test-sa-account
      containers:
        - name: kubectl
          image: <image>
          command: ["/bin/sh", "-c"]
          args:
            - |
              kubectl patch deployment test-api -n test-ns \
                --patch '{"spec": {"template": {"spec": {"containers": [{"name": "test-api", "env": [{"name": "BASE_URL", "valueFrom": {"configMapKeyRef": {"name": "actual-url-config", "key": "base-url"}}}]}]}}}}' && \
              sleep 5 && \
              kubectl rollout restart deployment test-api -n test-ns
          volumeMounts:
            - name: config-volume
              mountPath: /config
          ......
      volumes:
        - name: config-volume
          configMap:
            name: actual-url-config

When this job is run, I see the patch being applied and the test-api pods being restarted. But I noticed the pods are again restarted with the old configmap(mock-url-config) instead of the new configmap(actual-url-config). Can I get some help with figuring out why the pod is being restarted with the old configmap even after applying the kubectl patch.