nginx and cronolog

Since the last century, I’ve been in the habit of piping my web server log files through cronolog and off to automatically selected files in the pattern /var/log/http/2015/10/23/access.log. This works quite well for me because way back when, I wrote a little log processing script called Logmonster.

After all these years, Logmonster still runs a while after midnight (via periodic) and:

  • parses the web server logs by date and vhost
  • feeds them through Awstats
  • compresses them

Back when Logmonster was named Apache::Logmonster, it required installing cronolog and making a few small changes to httpd.conf:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %v" logmonster
CustomLog "| /usr/local/sbin/cronolog /var/log/http/%Y/%m/%d/access.log" logmonster
ErrorLog "| /usr/local/sbin/cronolog /var/log/http/%Y/%m/%d/error.log"

Years later, after I got tired of maintaining Apache, lighttpd was all shiny and new and it was similarly easy to configure, making these changes to lighttpd.conf:

accesslog.format = "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %v"
accesslog.filename = "|/usr/local/sbin/cronolog /var/log/http/%Y/%m/%d/access.log"
server.errorlog = "/var/log/http/error.log"

Now, after spending more time than I wanted to determining why lighttpd and haproxy stopped playing nice together (Most HTTP POST commands time out. No good reason why. Remove haproxy, works fine. Replace lighttpd nginx behind haproxy, works fine.) so I replaced lighttpd with nginx. That required figuring out how to get cronolog type logging to work in nginx.

Nearly all my cronolog+nginx search returned only instructions for setting up logging to a FIFO, which I thought was a nifty idea. So I created the FIFOs, configured nginx, and upon startup, nginx just hangs. No idea why. It’s also requires setting up the FIFOs before nginx could startup, so I didn’t love that idea. Then I found instructions showing how to configure log rotation within nginx.conf. That’s exactly what I was looking for.

This is my solution for timestamp based logging with nginx:

log_format main '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$server_name"';
if ($time_iso8601 ~ "^(?\d{4})-(?\d{2})-(?\d{2})") {}
access_log /var/log/http/$year/$month/$day/access.log main;