Give container's user access to tty even if user is connected via ssh

Question

I try to setup a development container for embedded firmware development (e.g. adruino). To make remote development possible (with Clion/VS code) I start an ssh server inside the container. Building, compiling over SSH inside the IDE works great!

But I also want to program/flash it from the container (logged in through ssh) to the device. This examples works without SSH:

podman run -it --rm --userns=keep-id --group-add=keep-groups --device=/dev/ttyACM0 ubuntu
cat /dev/ttyACM0

But if i run the same command and login via SSH

podman run -it --rm --userns=keep-id --group-add=keep-groups --device=/dev/ttyACM0 -p 2222:22 ubuntu_with_ssh 
ssh user@localhost -p 2222
cat /dev/ttyACM0

I get cat /dev/ttyACM0: Permission denied.

As it looks like, only PID 1 gets the permission to access devices (even if it is the very same user). And the ssh client isn't running at PID 1.

Question: What can I do, to forward this permissions to the SSH user?