Openshift Nginx permission problem [nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)]

Question

I am currently running into a problem trying to set up nginx:alpine in Openshift.

My build runs just fine but I am not able to deploy with permission being denied with the following error

2019/01/25 06:30:54 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)

nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)

Now I know Openshift is a bit tricky when it comes to permissions as the container is running without root privilidges and the UID is gerenated on runetime which means it's not available in /etc/passwd. But the user is part of the group root. Now how this is supposed to be handled is being described here

https://docs.openshift.com/container-platform/3.3/creating_images/guidelines.html#openshift-container-platform-specific-guidelines

I even went further and made the whole /var completely accessible (777) for testing purposes but I still get the error. This is what my Dockerfile looks like

Dockerfile

FROM nginx:alpine

#Configure proxy settings
ENV HTTP_PROXY=http://my.proxy:port
ENV HTTPS_PROXY=http://my.proxy:port
ENV HTTP_PROXY_AUTH=basic:*:username:password

WORKDIR /app
COPY . .

# Install node.js
RUN apk update && \
    apk add nodejs npm python make curl g++


# Build Application
RUN npm install
RUN ./node_modules/@angular/cli/bin/ng build
COPY ./dist/my-app /usr/share/nginx/html

# Configure NGINX
COPY ./openshift/nginx/nginx.conf /etc/nginx/nginx.conf
COPY ./openshift/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf

RUN chgrp -R root /var/cache/nginx /var/run /var/log/nginx && \
    chmod -R 777 /var
RUN sed -i.bak 's/^user/#user/' /etc/nginx/nginx.conf

EXPOSE 8080

It's funny that this approach just seems to effekt the alpine version of nginx. nginx:latest (based on debian I think) has no issues and the way to set it up described here

https://torstenwalter.de/openshift/nginx/2017/08/04/nginx-on-openshift.html

works. (but i am having some other issues with that build so I switched to alpine)

Any ideas why this is still not working?

Answer 1

Find openshift uid and gid

$oc describe project <project_name>

outputs: annotations:

openshift.io/sa.scc.supplemental-groups=1001240000/10000
openshift.io/sa.scc.uid-range=1001240000/10000

Nginx configurations files (sample):

nginx_default.conf

server {
    
      listen 4200;
      server_name supp-homolog.procuradoria.go.gov.br;
    
      sendfile on;
    
      default_type application/octet-stream;
    
      gzip on;
      gzip_http_version 1.1;
      gzip_disable      "MSIE [1-6]\.";
      gzip_min_length   1100;
      gzip_vary         on;
      gzip_proxied      expired no-cache no-store private auth;
      gzip_types        text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;
      gzip_comp_level   9;
    
    
      root /usr/share/nginx/html;
    
    
      location / {
        try_files $uri $uri/ /index.html =404;
      }
    
}

nginx.conf

user  root;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

Setup Dockerfile

FROM nginx:alpine as build
 
RUN chgrp -R 1001240000 /var/cache/nginx /var/run /var/log/nginx && \
    chmod -R 770 /var/cache/nginx /var/run /var/log/nginx && \
    chown -R 1001240000 /run

COPY --chown=1001240000:1001240000 docker/dev/nginx.conf /etc/nginx/nginx.conf
COPY --chown=1001240000:1001240000 docker/dev/nginx_default.conf /etc/nginx/conf.d/default.conf

HEALTHCHECK --interval=5s --timeout=3s CMD curl --fail http://localhost:4200/ || exit
 
EXPOSE 4200
CMD ["nginx", "-g", "daemon off;"]

Sample Deployment

Specifying the securityContext

kind: DeploymentConfig
apiVersion: apps.openshift.io/v1
metadata:
 [...]
spec:
  [...]
  template:
    [...]
    spec:
      containers:
        - name: nginx-test
          image: >-
            image-registry.openshift-image-registry.svc:5000/<project_name>/nginx-test:latest
          securityContext:
            runAsNonRoot: true
            runAsGroup: 1001240000
            runAsUser: 1001240000

Dockerfile (complete angular example)

From angular root folder:

$ podman build -t myapp . -f Dockerfile

Dockerfile

FROM node:18.17.1-alpine as build

RUN apk add tzdata
RUN apk add python3 make g++
ENV TZ America/Sao_Paulo
ENV NODE_OPTIONS=--max-old-space-size=8512

RUN chmod +x /usr/local/bin/node

FROM build as dev

WORKDIR /app

# Copy angular source files
COPY . /app

RUN npm install npm@9.6.7 -g  && npm install

# build (ng build), output to dist folder
RUN node --max_old_space_size=8512 ./node_modules/@angular/cli/bin/ng build --configuration development

FROM nginx:stable

RUN chgrp -R 1001240000 /var/cache/nginx /var/run /var/log/nginx && \
    chmod -R 770 /var/cache/nginx /var/run /var/log/nginx && \
    chown -R 1001240000 /run


COPY --from=dev --chown=1001240000:1001240000 /app/docker/dev/nginx.conf /etc/nginx/nginx.conf
COPY --from=dev --chown=1001240000:1001240000 /app/docker/dev/nginx_default.conf /etc/nginx/conf.d/default.conf

# Copy angular dist folder to nginx
COPY --from=dev --chown=1001240000:1001240000 /app/dist /usr/share/nginx/html

RUN find /usr/share/nginx/html/ -mindepth 1 -type d -exec chmod +x {} \;

HEALTHCHECK --interval=5s --timeout=3s CMD curl --fail http://localhost:4200/ || exit

EXPOSE 4200

CMD ["nginx", "-g", "daemon off;"]

Answer 2

Had the same error on my nginx:alpine Dockerfile

There is already a user called nginx in the nginx:alpine image. My guess is that it's cleaner to use it to run nginx.

Here is how I resolved it:

• Set the owner of /var/cache/nginx to nginx (user 101, group 101)
• Create a /var/run/nginx.pid and set the owner to nginx as well
• Copy all the files to the image using --chown=nginx:nginx

FROM nginx:alpine
RUN  touch /var/run/nginx.pid && \
     chown -R nginx:nginx /var/cache/nginx /var/run/nginx.pid
USER nginx
COPY --chown=nginx:nginx my/html/files /usr/share/nginx/html
COPY --chown=nginx:nginx config/myapp/default.conf /etc/nginx/conf.d/default.conf
...

Answer 3

May or may not be a step in the right direction (especially helpful for those who came here looking for general help on the [emerg] mkdir() ... failed error).

This solution counts from Builing nginx from source.

It took me about seven hours to realize the solution is directly related to the prefix path set in compiling nginx.

This is where my configuration throws off nginx (as a very brief example), compiled from this nginx source:

sudo ./auto/configure \
--prefix=/usr/local/nginx \
--http-client-body-temp-path=/tmp/nginx/client-body-temp \
--http-fastcgi-temp-path=/var/tmp/nginx/fastcgi_temp 

Without realizing it, I was setting the prefix to /usr/local/nginx but setting the client body temp path & fastcgi temp path to a directory inside /tmp/nginx.

It's basically breaking nginx's ability to access the correct files, because the temp paths are not correlated to the prefix path.

So I fixed it by (again, super simple configure as an example):

sudo ./auto/configure \
--prefix=/usr/local/nginx \
--http-client-body-temp-path=/usr/local/nginx/client_body_temp \
--http-fastcgi-temp-path=/usr/local/nginx/fastcgi_temp \

Further simplified:

sudo ./auto/configure \
--prefix=/usr/local/nginx \
--http-client-body-temp-path=/client_body_temp \
--http-fastcgi-temp-path=/fastcgi_temp \

Again, not guaranteed to work, but definitely a step in the right direction.

Answer 4

You may change the folder using the nginx.conf file. You can read more information in the section Running nginx as a non-root user.

Answer 5

If you're here because you failed to deploy an example helm chart (e.g: helm create mychart), do just like @quasipolynomial suggested but instead change your deployment file pull the right image.

i.e

containters: 
    - image: nginxinc/nginx-unprivileged 

more info on the official unprivileged image: https://github.com/nginxinc/docker-nginx-unprivileged

Answer 6

run the below command to fix the above issue. The anyuid security context constraint required.

oc adm policy add-scc-to-user anyuid system:serviceaccount:<NAMESPACE>:default

Answer 7

I was using openshift, with limited permissions, so I fixed this problem by using the following nginx image (rather than nginx:latest)

FROM nginxinc/nginx-unprivileged 

Answer 8

To resolve this. I think the Problem in this Dockerfile was that I used the COPY command to move my build and that did not exist. So here is my working

Dockerfile

FROM nginx:alpine

LABEL maintainer="ReliefMelone"

WORKDIR /app
COPY . .

# Install node.js
RUN apk update && \
    apk add nodejs npm python make curl g++


# Build Application
RUN npm install
RUN ./node_modules/@angular/cli/bin/ng build --configuration=${BUILD_CONFIG}
RUN cp -r ./dist/. /usr/share/nginx/html

# Configure NGINX
COPY ./openshift/nginx/nginx.conf /etc/nginx/nginx.conf
COPY ./openshift/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf

RUN chgrp -R root /var/cache/nginx /var/run /var/log/nginx && \
    chmod -R 770 /var/cache/nginx /var/run /var/log/nginx

EXPOSE 8080

CMD ["nginx", "-g", "daemon off;"]

Note that under the Build Application section I now do

RUN cp -r ./dist/. /usr/share/nginx/html

instead of

COPY ./dist/my-app /usr/share/nginx/html

The copy will not work as I previously ran the ng build inside of the container the dist will only exist in the container as well, so I need to execute the copy command inside of that container