Apache piped logs

What I set out to do for this task was to have Apache create a log file for each site on a mass virtual hosting setup, using mod_vhost_alias and store them in a structured directory. Such as /var/log/apache/www.domain.com/2010/10/30/access_log. Creating the directories as it goes, the end result was a perl script used with Acache’s piped logs feature.

Apache does a pretty good job of letting you do fun custom things with you http logs, by if you happen to be using mod_vhost_alias you will find that your options become limited, one big log or nothing. Mass virtual hosting with Apache is basically a good thing, if you have a highly subscribed server you can run it with a single virtual host. One size fits all, though there are some limitations on how you can use it with other configuration options.

These lack of additional configuration options, for example using the same syntax in CustomLog as you do in VirtualDocumentRoot, or securing PHP/CGI and chroot the sites in their home directories (this one we will come to in a later post).

First, we create our perl script.
For example, /usr/local/bin/vhost_access.pl


use strict;

my $logEntry;
my $accessLogName = "access_log";
my $logDir = "/var/log/apache";
my $vhostLogDir;

sub writeLog;

while( <> )

    $logEntry = $_;

    # get the vhost from this log entry:
    $logEntry=~/(.*?) /;
    my $vhost = $1;
    my ($year, $month, $day) = ((localtime)[5]+1900,
                               sprintf("%02d", (localtime)[4]+1),
                               sprintf("%02d", (localtime)[3])

    # remove 'www'. from domain path
    $vhost =~ s!^www.?!!i;

    # Name of access logfiles:
    $vhostLogDir = "$logDir/$vhost/$year/$month/$day";

# write a log entry to a file
sub writeLog()
    my $logDir = shift @_;

    if( ! -d $logDir )
        `/bin/mkdir -p $logDir`;

    open(FD, ">>$logDir/$accessLogName");

    # strip off the vhost data:
    $logEntry =~s/.*? //;

    print FD $logEntry;
    close FD;

Next add the following line into your httpd.conf (It does *not* need to be inside the <VirtualHost tag>).

LogFormat "%V %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" vcombined
CustomLog "|/usr/local/bin/vhost_access.pl" vcombined

Now restart Apache. By doing `ps ax` or equivalent should now show the additional process for vhost_access.pl.

That’s it! Generate some traffic (If not being access already by the world) and enjoy tailing your new log files.

You may also like...