CODEWITHSUNDEEP
amazon-web-servicesdockernginxlogging
Mitsukk
Mitsukk

Reputation: 321

Write response requests on logs with Nginx

!UPDATE!

Big thank's to @ivan

default.conf:

log_format include_id '$remote_addr - $remote_user [$time_local] $request_id "$request" '
                      '$status $body_bytes_sent "$http_referer" "$http_user_agent"';

server {
    listen 443;
    listen [::]:443;
    ssl_certificate /etc/nginx/conf.d/cert.pem;
    ssl_certificate_key /etc/nginx/conf.d/key.pem;
    location / {
        proxy_pass http://sns-mock:9911;
        proxy_store /var/log/nginx/requests/mohsin.json;
    }
}

docker-compose.yml:

version: '3.7'

services:

  sns-mock:
    image: s12v/sns
    container_name: sns-mock
    ports:
      - "9911:9911"
    networks:
      - default-nw
    volumes:
      - ./config/db.json:/etc/sns/db.json

  sns-mock-proxy:
    image: nginx:latest
    container_name: sns-mock-proxy
    ports:
      - "8443:443"
    networks:
      - default-nw
    volumes:
      - ./conf.d:/etc/nginx/conf.d
      - ./log/nginx:/var/log/nginx
      - ./log/nginx/requests:/var/log/nginx/requests
    depends_on:
      - sns-mock

networks:
  default-nw:
    external: true

Php test file:

<?php

use Aws\Result;
use Aws\Sns\SnsClient;
use Illuminate\Http\JsonResponse;
use PHPUnit\Framework\TestCase;

class FooTest extends TestCase
{
    public function testFoo()
    {
        $snsClient = new SnsClient([
            'endpoint' => 'localhost:8443',
            'region' => 'eu-west-2',
            'version' => 'latest'
        ]);

        $result = $snsClient->publish([
            'Message' => 'foo',
            'TopicArn' => 'arn:aws:sns:eu-west-2:123450000001:test-topic'
        ]);

        dd($result, 'ok');
    }
}

Test result:

enter image description here

My project:

enter image description here

On my access.log file, i can catch response from phpunit test like here:

172.28.0.1 - - [23/May/2022:06:54:04 +0000] "POST / HTTP/1.1" 200 270 "-" "aws-sdk-php/3.222.17 OS/Linux/5.13.0-41-generic lang/php/8.1.6 GuzzleHttp/7" "-"

===================================================================================================================================

Upvotes: 0

Views: 5184

Answers (1)

Ivan Shatsky
Ivan Shatsky

Reputation: 15662

The general answer to your question is "No". However some kind of workaround is possible.

Every request gets processed by nginx receives some internal request ID available via the $request_id variable (16 random bytes, in hexadecimal). You can add that ID to your access log defining your own custom log format:

log_format include_id '$remote_addr - $remote_user [$time_local] $request_id "$request" '
                      '$status $body_bytes_sent "$http_referer" "$http_user_agent"';
access_log /var/log/nginx/access.log include_id;

Next, in the same location where you have your proxy_pass directive add the proxy_store one:

proxy_store /var/log/nginx/requests/$request_id.json;

I use the separate /var/log/nginx/requests directory here to not make your /var/log/nginx turning into a mess. Of course it should be created manually before you start the docker container, and it should be writable the same way as /var/log/nginx itself (including such a things as SELinux context, if being used on the host system). However for the testing purposes you can start with the proxy_store /var/log/nginx/$request_id.json;.

The whole nginx config should look like

log_format include_id '$remote_addr - $remote_user [$time_local] $request_id "$request" '
                      '$status $body_bytes_sent "$http_referer" "$http_user_agent"';

server {
    listen 443;
    listen [::]:443;
    access_log /var/log/nginx/access.log include_id;
    ssl_certificate /etc/nginx/conf.d/cert.pem;
    ssl_certificate_key /etc/nginx/conf.d/key.pem;
    location / {
        proxy_pass http://sns-mock:9911;
        proxy_store /var/log/nginx/requests/$request_id.json;
    }
}

If you want requests to be available via the plain HTTP protocol too, you can use the same configuration for the plain HTTP server block instead of the one you've shown in your question:

server {
    listen 80;
    listen [::]:80;
    access_log /var/log/nginx/access.log include_id;
    location / {
        proxy_pass http://sns-mock:9911;
        proxy_store /var/log/nginx/requests/$request_id.json;
    }
}

or issue an HTTP-to-HTTPS redirect instead:

server {
    listen 80;
    listen [::]:80;
    return 308 https://$host$request_uri;
}

Now the response body for each request which can be identified from the access log via its request ID, something like

172.18.0.1 - - [20/May/2022:19:54:14 +0000] d6010d713b2dce3cd2713f1ea178e140 "POST / HTTP/1.1" 200 615 "-" "aws-sdk-php/3.222.17 OS/Linux/5.13.0-41-generic lang/php/8.1.6 GuzzleHttp/7" "-"

will be available under the /var/log/nginx/requests directory (response body for the request shown in the given access log entry will be available as d6010d713b2dce3cd2713f1ea178e140.json file).

Upvotes: 3

Related Questions