Do wheels for a target have to be constructed on a machine with that architecture?

Question

I have used maturin build to build a distribution for a Python package on PyPi with a Rust extension using pyo3 bindings.

I am trying to build for the following common architectures:

win_amd64  :  aarch64-pc-windows-msvc
win_x86_64  :  x86_64-pc-windows-gnu / msvc
macosx_11_0_arm64  :  aarch64-apple-darwin
macosx_10_12_x86_64  :  x86_64-apple-darwin
manylinux_2_24_aarch64  :  aarch64-unknown-linux-gnu  ?
manylinux_2_17_x86_64.manylinux2014_x86_64  :  x86_64-unknown-linux-gnu

On a windows machine I can produce the win_amd64 version. On a mac I can produce the two mac versions.

On each machine I have installed the alternative rust components such as:

rustup target add aarch64-pc-windows-msvc 

However whenever I try and build a distribution on one of these other targets I get some error:

E.g. When running on Mac:

maturin build --release --target x86_64-pc-windows-gnu

Found pyo3 bindings with abi3 support for Python ≥ 3.9
💥 maturin failed
  Caused by: Failed to find a python interpreter

Or, similarly, when trying:

maturin build --release --target aarch64-unknown-linux-gnu

error: linking with `cc` failed: exit status: 1
error: could not compile `rateslibrs` (lib) due to 1 previous error; 52 warnings emitted
💥 maturin failed
  Caused by: Failed to build a native library through cargo

On Windows I get similar errors when trying to package for Mac.

Is this something I just can't fundamentally achieve, or are my settings and configurations not correct?