docker run standard_init_linux.go:211: exec user process caused "permission denied"

Question

I am facing this weird issue and don't have a clue after spending three days. I tried moving the docker data directory from /var/lib/docker to /data/docker. The /data mount point has rw,exec permissions set. But when i try to run the container it gives: standard_init_linux.go:211: exec user process caused "permission denied"

But when i move the docker data directory to /var/lib/docker, the docker run command works fine. I have tried reinstalling docker multiple time and still facing the same error.

OS: CentOS 7 Docker version: 19.03.14

In debug mode i get following:

INFO[2021-01-02T10:30:25.848427411-05:00] loading plugin "io.containerd.content.v1.content"... type=io.containerd.content.v1 INFO[2021-01-02T10:30:25.848505217-05:00] loading plugin "io.containerd.snapshotter.v1.btrfs"... type=io.containerd.snapshotter.v1 INFO[2021-01-02T10:30:25.848789449-05:00] skip loading plugin "io.containerd.snapshotter.v1.btrfs"... error="path /data2/docker2/containerd/daemon/io.containerd.snapshotter.v1.btrfs (xfs) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1 INFO[2021-01-02T10:30:25.848817180-05:00] loading plugin "io.containerd.snapshotter.v1.devmapper"... type=io.containerd.snapshotter.v1 WARN[2021-01-02T10:30:25.848840183-05:00] failed to load plugin io.containerd.snapshotter.v1.devmapper error="devmapper not configured" INFO[2021-01-02T10:30:25.848851404-05:00] loading plugin "io.containerd.snapshotter.v1.aufs"... type=io.containerd.snapshotter.v1 INFO[2021-01-02T10:30:25.855513060-05:00] skip loading plugin "io.containerd.snapshotter.v1.aufs"... error="modprobe aufs failed: \"modprobe: FATAL: Module aufs not found.\\n\": exit status 1: skip plugin" type=io.containerd.snapshotter.v1 INFO[2021-01-02T10:30:25.855547154-05:00] loading plugin "io.containerd.snapshotter.v1.native"... type=io.containerd.snapshotter.v1 INFO[2021-01-02T10:30:25.855579299-05:00] loading plugin "io.containerd.snapshotter.v1.overlayfs"... type=io.containerd.snapshotter.v1 INFO[2021-01-02T10:30:25.855644014-05:00] loading plugin "io.containerd.snapshotter.v1.zfs"... type=io.containerd.snapshotter.v1 INFO[2021-01-02T10:30:25.855831659-05:00] skip loading plugin "io.containerd.snapshotter.v1.zfs"... error="path /data2/docker2/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1 INFO[2021-01-02T10:30:25.855849335-05:00] loading plugin "io.containerd.metadata.v1.bolt"... type=io.containerd.metadata.v1 WARN[2021-01-02T10:30:25.855866391-05:00] could not use snapshotter devmapper in metadata plugin error="devmapper not configured"

Can someone please help me with this? Thanks in advance!

Answer 1

After spending days on trying to figure out the issue, finally I figured out that it was Vormetrics that's causing the issue, despite the encryption was not enabled it was not letting the containers run.