CODEWITHSUNDEEP
jsonloggingnginx
Jules Olléon
Jules Olléon

Reputation: 6888

How to generate a JSON log from nginx?

I'm trying to generate a JSON log from nginx.

I'm aware of solutions like this one but some of the fields I want to log include user generated input (like HTTP headers) which need to be escaped properly.

I'm aware of the nginx changelog entries from Oct 2011 and May 2008 that say:

*) Change: now the 0x7F-0x1F characters are escaped as \xXX in an
   access_log.
*) Change: now the 0x00-0x1F, '"' and '\' characters are escaped as \xXX
   in an access_log.

but this still doesn't help since \xXX is invalid in a JSON string.

I've also looked at the HttpSetMiscModule module which has a set_quote_json_str directive, but this just seems to add \x22 around the strings which doesn't help.

Any idea for other solutions to log in JSON format from nginx?

Upvotes: 83

Views: 71605

Answers (3)

pva
pva

Reputation: 2045

Finally it looks like we have good way to do this with vanilla nginx without any modules. Just define: 

log_format json_combined escape=json
  '{'
    '"time_local":"$time_local",'
    '"remote_addr":"$remote_addr",'
    '"remote_user":"$remote_user",'
    '"request":"$request",'
    '"status": "$status",'
    '"body_bytes_sent":"$body_bytes_sent",'
    '"request_time":"$request_time",'
    '"http_referrer":"$http_referer",'
    '"http_user_agent":"$http_user_agent"'
  '}';

Note that escape=json was added in nginx 1.11.8. http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format

Upvotes: 174

Valeriy Solovyov
Valeriy Solovyov

Reputation: 5648

You can try to use:

PS: The if parameter (1.7.0) enables conditional logging. A request will not be logged if the condition evaluates to “0” or an empty string:

map $status $http_referer{
    ~\xXX  0;
    default 1;
}

access_log /path/to/access.log combined if=$http_referer;

It’s a good idea to use a tool such as https://github.com/zaach/jsonlint to check your JSON data. You can test the output of your new logging format and make sure it’s real-and-proper JSON.

Upvotes: 2

Seva Poliakov
Seva Poliakov

Reputation: 3111

You can try to use that one https://github.com/jiaz/nginx-http-json-log - addition module for Nginx.

Upvotes: 8

Related Questions