Watching the web-logs on all of my servers in real time

I have a computer sitting on my desk that is always on (it’s my file server) and it has a monitor attached which is almost never in use (I ssh to that server if I want to do things so it’s hardly ever logged in).

I thought it would be cool if on that monitor the web-logs from all of the systems I manage were shown so I could keep an eye on things and maybe learn a thing or two about my web-sites and how people are using them.

So the first thing I did was write a script to grab any given web log:

root@orac:~# cat /root/
echo Starting download of $3...
while : ; do
  su -c "ssh $1 tail -f /var/log/apache2/$2 < /dev/null" jj5 \
    | tee -a /var/log/web.log \
    | grep --line-buffered -v "Mozilla.5.0 .compatible. Googlebot.2.1." \
    | grep --line-buffered -v "" \
    | grep --line-buffered -v "Mozilla.5.0 .compatible. Baiduspider.2.0." \
    | grep --line-buffered -v "Mozilla.5.0 .compatible. Exabot.3.0." \
    | grep --line-buffered -v "Mozilla.5.0 .compatible. YandexBot.3.0." \
    > /var/log/web/$3
  sleep 60
  echo; echo; echo Restarting download of $3...; echo; echo;

Then I wrote a series of scripts which call the script for specific web-sites on specific servers, e.g.:

root@orac:~# cat /root/web-log/
/root/ honesty

Then I wrote a main script, rather unoriginally called, that kicks off the web logs downloads and then monitors their progress as they come through:

root@orac:~# cat /root/

# disable the screensaver
setterm -blank 0 -powersave off -powerdown 0

# start downloading the web-logs
cd /root/web-log
./ &
sleep 1
#...all the other downloaders, one for each site

# watch the web-logs
cd /var/log/web
tail -f *

# stop downloading the web-logs
kill %1
#...all the other kills, one for each downloader


Then I edited /etc/init/tty1.conf so that on tty1, instead of having a login console, I automatically ran my script:

root@orac:~# cat /etc/init/tty1.conf
# tty1 - getty
# This service maintains a getty on tty1 from the point the system is
# started until it is shut down again.

start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]

#exec /sbin/getty -8 38400 tty1
exec /root/ < /dev/tty1 > /dev/tty1 2>&1

And that was it. The only trick was that I needed to disable the screen saver (as shown in the script) so that the screen didn’t constantly blank.

And now I can watch the web activity on all of my sites in real time.