GitLab CI/CD: Authentication Failed When Using Personal Access Token (PAT) in .gitlab-ci.yml

Question

Need help with configuring my .gitlab-ci.yml. I am currently getting following error

remote: HTTP Basic: Access denied. If a password was provided for Git authentication, the password was incorrect or you're required to use a token instead of a password. If a token was provided, it was either incorrect, expired, or improperly scoped. See https://gitlab.com/help/topics/git/troubleshooting_git.md#error-on-git-fetch-http-basic-access-denied
fatal: Authentication failed for 'https://gitlab.com/username/repo-name.git/'

here is my .gitlab-ci.yml file

.docker:
  image: docker:latest
  services:
    - docker:${DOCKER_VERSION}-dind
  variables:
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_VERIFY: '1'
    DOCKER_TLS_CERTDIR: '/certs'
    DOCKER_CERT_PATH: '/certs/client'
    DOCKER_DRIVER: overlay2
    DOCKER_VERSION: '27.2.0'
    # assumes the Dagger Cloud token is
    # in a masked/protected variable named DAGGER_CLOUD_TOKEN
    # set via the GitLab UI
    DAGGER_CLOUD_TOKEN: $DAGGER_CLOUD_TOKEN
.dagger:
  extends: [.docker]
  before_script:
    - apk add curl
    - curl -fsSL https://dl.dagger.io/dagger/install.sh | BIN_DIR=/usr/local/bin sh
hello:
  extends: [.dagger]
  script:
    - dagger -m https://oauth2:${GITLAB_PAT}@gitlab.com/username/repo-name.git call test --source=.

I am using oauth2:${GITLAB_PAT} as I have read thru few stackoverflow answers that PAT works when 2FA is enabled and we have to use oauth2 so that i runs correctly. Can somepoint to the mistake I am making here or point me to some docs on how to get this working it would be great. I have also tried gitlab-ci-token:${GITLAB_PAT}, username:${GITLAB_PAT} but it didn't work too

Also tried using deploy token even that didn't work for me

NOTE: I currently have not tried using ssh here.

Here is dagger error trace: https://v3.dagger.cloud/shailesh-personal/traces/c0037c8d3fb92545ede7e6a35f5212ac

Thanks!