CODEWITHSUNDEEP
github-actionsopensshopenssh-for-windows
Mohamed Martini
Mohamed Martini

Reputation: 1

Authenticate cibuildwheel with my private repo deployed key

I am using cibuildwheel to build and test wheels for my Python project. If you're not familiar with cibuildwheel, it uses docker images to build cross-platform Python wheels.

The wheel building works fine on all github runners, but the wheel testing is failing on Ubuntu and Windows (not macos!) because when it tries to install the built wheel it cannot install a private dependency I have (private github repo). I don't know much about how ssh authentication works, so any help is very appreciated. Here are the errors I am getting:

On ubuntu-latest:

    Running command git clone --filter=blob:none --quiet 'ssh://****@github.com/org/repo.git' /tmp/pip-install-6xye89_k/repo_hash
    error: cannot run ssh: No such file or directory
    fatal: unable to fork

On windows-latest:

    Running command git clone --filter=blob:none --quiet 'ssh://****@github.com/org/repo.git' 'C:\Users\runneradmin\AppData\Local\Temp\pip-install-ejq4b3y5\repo_hash'
    Host key verification failed.
    fatal: Could not read from remote repository.
    Please make sure you have the correct access rights
    and the repository exists.

Here's my cibuildwheel settings in pyproject.toml:

[tool.cibuildwheel]
build-verbosity = 1
build           = "cp310-* cp311-*"
test-command    = "python -c 'import my_pkg; import my_pkg.my_module'"

[tool.cibuildwheel.linux]
archs = "x86_64"

[tool.cibuildwheel.macos]
before-test = "brew install hdf5 && export HDF5_DIR=\"$(brew --prefix hdf5)\""

And here's my github workflow job:


  build-wheels:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-13, macos-14, windows-latest]

    steps:
      - uses: actions/checkout@v4

      - name: authenticate with private repo deployed key
        uses: webfactory/[email protected]
        with:
          ssh-private-key: |
            ${{ secrets.REPO_DEPLOY_KEY }}

      - name: build wheels
        uses: pypa/[email protected]

      - name: upload wheels
        uses: actions/[email protected]
        with:
          name: wheels-${{ matrix.os }}
          path: wheelhouse/*

For linux, I tried installing openssh-clients:

[tool.cibuildwheel.linux]
archs       = "x86_64"
before-test = "yum install -y openssh-clients"

This installed the ssh client, but authentication still failed with the same error message as Windows:

    Host key verification failed.
    fatal: Could not read from remote repository.
    Please make sure you have the correct access rights
    and the repository exists.

I also tried adding SSH_AUTH_SOCK env var but no luck:

      - name: build wheels
        uses: pypa/[email protected]
        env:
          SSH_AUTH_SOCK: ${{ env.SSH_AUTH_SOCK }}

Since I'm novice with these authentication and docker stuff, anything I try is a shot in the dark, and I don't want to causes a security risk. Please help!

UPDATE - I fixed it for Linux, but not Windows. Here's the fix for Linux:

[tool.cibuildwheel.linux]
environment = { SSH_AUTH_SOCK = "/host$SSH_AUTH_SOCK" }
environment-pass = ["SSH_AUTH_SOCK"]
before-test = """
    yum install -y openssh-clients && \
    mkdir -p ~/.ssh && \
    chmod 700 ~/.ssh && \
    ssh-keyscan github.com >> ~/.ssh/known_hosts
"""

Basically, it's setting the SSH_AUTH_SOCK variable properly and passing it to the linux container.

Any idea what would solve it on Windows?

UPDATE - Windows actually required no additional settings in the pyproject.toml.

Upvotes: 0

Views: 81

Answers (1)

Mohamed Martini
Mohamed Martini

Reputation: 1

The answer:

Here's the fix for Linux:

[tool.cibuildwheel.linux]
environment = { SSH_AUTH_SOCK = "/host$SSH_AUTH_SOCK" }
environment-pass = ["SSH_AUTH_SOCK"]
before-test = """
    yum install -y openssh-clients && \
    mkdir -p ~/.ssh && \
    chmod 700 ~/.ssh && \
    ssh-keyscan github.com >> ~/.ssh/known_hosts

Windows actually required no additional settings in the pyproject.toml.

Upvotes: 0

Related Questions