Nice remote apache log viewer

Question

I have a server with 10+ virtual domains (most running Mediawiki). I'd like to be able to watch their traffic remotely with something nicer than tail -f . I could cobble something together, but was wondering if something super-deluxe already exists that involves a minimum of hacking and support. This is mostly to understand what's going on, not so much for security (though it could serve that role too). It must:

• be able to deal with vhost log files
• be able to handle updates every 10 seconds or so
• Be free/open source

The nice to haves are:

• Browser based display (supported by a web app/daemon on the server)
• Support filters (bots, etc)
• Features like counters for pages, with click to view history
• Show a nice graphical display of a geographic map, timeline, etc
• Identify individual browsers
• Show link relationships (coming from remote site, to page, to another page)
• Be able to identify logfile patterns (editing or creating a page)

I run Debian on the server.

Thanks!

Answer 1

Depending on the volume, Papertrail could be free for you. It is the closest thing to a tail -f and is searchable, archivable and also sends alerts based on custom criteria.

Answer 2

Logstash is the current answer. (=

Answer 3

Take a look at Splunk. I'm not sure if it supports real time (~10 second) updates but there are a ton of features and it's pretty easy to get set up.

The free version has some limitations but there is also an enterprise version.