Thread: host_history.pl 0.0.1 released

Hi all..

I've hacked up a small Perl script to generate some activity graphs for my server. Here's an example the 2 graphs it produces:

Small:


Large:


I wrote some scripts to read these graphs so I could display them on my server but I happen to run Apache on the same box as my LFS server runs so I have that "advantage". For others, you would probably have to implement some kind of automagic FTP upload or similar.

The script runs standalone.. but designed to run from a cronjob every minute (the processing time is very short on my 350Mhz PII (maybe 3 seconds) from start to finish).

Please read the comments in the script for how to configure the small section at the top and the pre-req's. Also please note that I haven't enough free time to explain how to install each of the 3 pre-reqs if you don't have them already. The process isn't difficult, but please see the relevant sites / accompanying README files for those instructions =)

This could probably be coded better.. but it was a quick 10 min jobby =)


The script:

Code:

#!/usr/bin/perl
###########################################################################
#            host_history.pl
#
#  15/06/2004 14:18
#
#  Copyright (c) 2004,  Ian.H <ian.h@dsrc.digiserv.net>
#  Web    : <http://digiserv.net/>
#         : <http://dsrc.digiserv.net/>
#
# License : BSD
###########################################################################
#
# A simple script that will use the host{port}.txt file to generate an
# overall log of your server activity and then generate 2 graphs (small
# and large ) representing 24 hours usage for drivers count.
#
###########################################################################
#
# NOTE: This was coded to run on a *nix platform but it should also run
#       on a windoze box.
#
###########################################################################
#
# PRE-REQUISITES:
# ---------------
#
# Perl5 ( <http://www.perl.com/> | <http://www.activestate.com/> )
# GD1 or GD2 libs ( <http://www.boutell.com/gd/> )
# tail ( windoze version at: <http://unxutils.sourceforge.net/> )
#
#
# Please DO NOT ask how to install these if you do not currently have
# them, see their accompanying README files for installation information.
#
###########################################################################
#
# CONFIGURATION:
# --------------
#
# Change the details below in the 'my %CONF = (...)' section to match
# your requirements / filesystem. PLEASE NOTE: the *_file values MUST BE
# fully-qualified paths / filename. The output_file and hourly_log_file
# will be created upon execution. The host_operator should be the name of
# your server connection that displays in the 'connections list' in LFS.
#
# The graph entries are the fully-qualified path / filename of where the
# graphs should be created
#
# The tail entry should be a fully-qualified path/ filename to your
# tail or tail.exe binary
#
###########################################################################


use strict;
use warnings;
use GD;

my %CONF = (

    host_file               => '/path/to/LFS/host63392.txt',
    output_file             => '/path/to/host_history.log',
    hourly_log_file         => '/path/to/host_history_hour.log',
    host_operator           => 'B.O.F.H',
    large_graph_output_file => '/path/to/driver_history_large.png',
    small_graph_output_file => '/path/to/driver_history_small.png',
    tail                    => '/path/to/tail(.exe)',
    server_name             => 'Your server name'
);


my @drivers = ();
my $track = '';
my $cars = '';

my $time = time;

open(HOST, "<$CONF{host_file}");
    while (<HOST>) {
        if (/^trackcfg=(.+)$/s) {
            ($track = $1);
        }

        if (/^cars=(.+)$/s) {
            ($cars = $1);
        }

        push @drivers, /^conn=(.+)$/s;
    }

    $track =~ s/[\n|\r|[\r\n]$//g;
    $cars =~ s/[\n|\r|[\r\n]$//g;
close(HOST);

open(LOG, ">>$CONF{output_file}");
    print LOG "$time|$track|$cars|";
    foreach (@drivers) {
        s/[\n|\r|[\r\n]$//g;
        print LOG "$_\x00" unless $_ eq $CONF{host_operator} or $_ eq '';
    }
    print LOG "\n";
close(LOG);

system("$CONF{tail} -n 1440 $CONF{output_file} > $CONF{hourly_log_file}");


create_small_driver_history_graph();
create_large_driver_history_graph();



sub create_small_driver_history_graph {
    my @data = ();
    my @hourly_log_contents = ();

    open(LOG, "<$CONF{hourly_log_file}");
        @hourly_log_contents = <LOG>;
    close(LOG);

    foreach (@hourly_log_contents) {
        /^[0-9]+\|[A-Z1-9]+\|[A-Z46]+\|(.*)$/s;
        my $driver_list = $1;
        my @drivers = split("\x00", $driver_list);
        pop @drivers;
        push @data, scalar @drivers;
    }

    my $width = 1440;
    my $height = 80;
    my $out_width = 170;
    my $out_height = 95;
    my $graph = new GD::Image($width, $height);
    my $out_graph = new GD::Image($out_width, $out_height);

    my $white = $graph->colorAllocate(255, 255, 255);
    my $black = $graph->colorAllocate(0, 0, 0);
    my $light_blue = $graph->colorAllocate(134, 155, 191);
    my $background = $graph->colorAllocate(214, 224, 235);
    my $int_lines = $graph->colorAllocate(184, 194, 205);
    my $int_lines_light = $graph->colorAllocate(204, 214, 225);

    my $out_white = $out_graph->colorAllocate(255, 255, 255);
    my $out_black = $out_graph->colorAllocate(0, 0, 0);
    $out_graph->filledRectangle(0, 0, $width, $height, $out_white);

    $out_graph->string(gdTinyFont, 162, 60, '2', $out_black);
    $out_graph->string(gdTinyFont, 162, 40, '4', $out_black);
    $out_graph->string(gdTinyFont, 162, 20, '6', $out_black);
    $out_graph->string(gdTinyFont, 162, 1, '8', $out_black);
    $out_graph->string(gdTinyFont, 128, 82, get_time() . '>', $out_black);
    $out_graph->string(gdTinyFont, 1, 82, $CONF{server_name}, $out_black);

    $graph->filledRectangle(0, 0, $width, $height, $background);

    $graph->line(0, 70, $width, 70, $int_lines_light);
    $graph->line(0, 60, $width, 60, $int_lines);
    $graph->line(0, 50, $width, 50, $int_lines_light);
    $graph->line(0, 40, $width, 40, $int_lines);
    $graph->line(0, 30, $width, 30, $int_lines_light);
    $graph->line(0, 20, $width, 20, $int_lines);
    $graph->line(0, 10, $width, 10, $int_lines_light);

    my $i = 0;
    foreach (@data) {
        $graph->line($i, $height - ($_ * 10), $i, $height, $light_blue);
        $i++;
    }

    $out_graph->copyResized($graph, 0, 0, 0, 0, 160, 80, $width, $height);
    $out_graph->setAntiAliased($light_blue);

    $out_graph->interlaced('true');

    open(GRAPH, ">$CONF{small_graph_output_file}");
        binmode GRAPH;
        print GRAPH $out_graph->png;
    close(GRAPH);

    undef $graph;
    undef $out_graph;
}


sub create_large_driver_history_graph {
    my @data = ();
    my @hourly_log_contents = ();

    open(LOG, "<$CONF{hourly_log_file}");
        @hourly_log_contents = <LOG>;
    close(LOG);

    foreach (@hourly_log_contents) {
        /^[0-9]+\|[A-Z1-9]+\|[A-Z46]+\|(.*)$/s;
        my $driver_list = $1;
        my @drivers = split("\x00", $driver_list);
        pop @drivers;
        push @data, scalar @drivers;
    }

    my $width = 1440;
    my $height = 160;
    my $out_width = 350;
    my $out_height = 200;
    my $graph = new GD::Image($width, $height);
    my $out_graph = new GD::Image($out_width, $out_height);

    my $white = $graph->colorAllocate(255, 255, 255);
    my $black = $graph->colorAllocate(0, 0, 0);
    my $light_blue = $graph->colorAllocate(134, 155, 191);
    my $background = $graph->colorAllocate(214, 224, 235);
    my $int_lines = $graph->colorAllocate(184, 194, 205);
    my $int_lines_light = $graph->colorAllocate(204, 214, 225);

    my $out_white = $out_graph->colorAllocate(255, 255, 255);
    my $out_black = $out_graph->colorAllocate(0, 0, 0);
    $out_graph->filledRectangle(0, 0, $width, $height, $out_white);

    $out_graph->string(gdSmallFont, 1, 5, 'Server activity over the past 24 hours', $out_black);

    $out_graph->string(gdSmallFont, 342, 140, '2', $out_black);
    $out_graph->string(gdSmallFont, 342, 100, '4', $out_black);
    $out_graph->string(gdSmallFont, 342, 60, '6', $out_black);
    $out_graph->string(gdSmallFont, 342, 20, '8', $out_black);
    $out_graph->string(gdSmallFont, 300, 182, get_time() . '>', $out_black);
    $out_graph->string(gdTinyFont, 1, 82, $CONF{server_name}, $out_black);

    $graph->filledRectangle(0, 0, $width, $height, $background);

    $graph->line(0, 140, $width, 140, $int_lines_light);
    $graph->line(0, 120, $width, 120, $int_lines);
    $graph->line(0, 100, $width, 100, $int_lines_light);
    $graph->line(0, 80, $width, 80, $int_lines);
    $graph->line(0, 60, $width, 60, $int_lines_light);
    $graph->line(0, 40, $width, 40, $int_lines);
    $graph->line(0, 20, $width, 20, $int_lines_light);

    my $i = 0;
    foreach (@data) {
        $graph->line($i, $height - ($_ * 20), $i, $height, $light_blue);
        $i++;
    }

    $out_graph->copyResized($graph, 0, 20, 0, 0, 340, 160, $width, $height);
    $out_graph->setAntiAliased($light_blue);

    $out_graph->interlaced('true');

    open(GRAPH, ">$CONF{large_graph_output_file}");
        binmode GRAPH;
        print GRAPH $out_graph->png;
    close(GRAPH);

    undef $graph;
    undef $out_graph;
}


sub get_time {
	my ($sec, $min, $hour, $mday, $mon, $year) = localtime(time);
	return sprintf("%02d:%02d", $hour, $min);
}

Downloadable version attached.

Hope someone else finds a use for this too =)



Regards,

Ian

Like the idea for the new add-on...are you planning on incorporating it into the main site, and is there a script I need to run so you can get info from my server too?

Originally Posted by Bladerunner

Like the idea for the new add-on...are you planning on incorporating it into the main site, and is there a script I need to run so you can get info from my server too?

Yup and yup =D

The top connected drivers is currently only on the dev site as I have a few other changes present on that version too, but they're not 100% complete.. so rather than me upload a few files each time, I'll finish off the few additions and upload the new version complete =)

As for your stats, to get both additions you'll need to run both of the above scripts. The host_history one generates the hourly log for the graphs.. that and the driver_visits script work off the _main_ log produced by host_history.

You will need to make sure you have a few things installed though...

Perl (grab a windoze copy free from the activestate.com site in the code comments). It's a windoze installer.. easy to install and takes a few mins.. make sure you associate .pl wiles with Perl when asked as this'll save you hassles later with the shebang line etc.

You've probably got the GD2 libs installed from your PHP install (IIRC, they're compatible) and I you'll need to install the Perl GD module via activestate's Perl Package Manager (if you get everything installed, I'll go through the Perl module stuff with ya if needed.. it's _very_ simple.. but if ya never used it before......).

I'll drop you a mail a little later with a copy of tail.exe to save downloading the entire tools collection.

Once that part is done, I have another PHP script that parses the log files loke the read_host file does for the LFS host file. I'll sort those out for you too.

driver_visits should then run once the above is sorted without any extra requirements for installing.. although there is an additional PHP script to parse that log file too.

I have an updated version of the driver_visits script I need to post.. which has another 2 options (number of drivers to output and whether to output with or without the ^# colour codes (yes ideally for Web displaying, no if you were just displaying it as plaintext)). The one above works fine if you get if you run it.. but new version is a bit better =)

Grab the installs and we'll install the Perl module and see what we can get =D



Regards,

Ian

Ian..the 'host_history.pl' zip *seems* to be corrupted...cant unzip it!..showing as 2.2kb, shows file inside zip..but wont extract

(gonna try and copy the code from the box above, and save it as....see if it werks! )

Originally Posted by Bladerunner

Ian..the 'host_history.pl' zip *seems* to be corrupted...cant unzip it!..showing as 2.2kb, shows file inside zip..but wont extract

(gonna try and copy the code from the box above, and save it as....see if it werks! )

didn't extract for me either once I'd downloaded it.

Re-archived and uploaded and this one extracts for me now... thanks for pointing this out =)

I've attached a copy of tail.exe here too. Personally I think your $WINDOZE dir is the best dir to extract it to.. it's a great tool for monitoring the likes of Apache logfiles "real-time" too =)



Regards,

Ian

Hey Ian.

Not sure if you would still be reading this area but meh.

I have grabbed the Host_History.pl 0.0.2 release that you done a long time ago,

I have installed Perl etc (as far as im awear.. correctly).

When i run the hosthistory.pl file i get the HostHist.log, hourly.log and the 2 graphs created, The Log files both contain entries as expected, however the graph simply puts up the graph without any line data on it. It shows numbers, Time and server name on the axis, but just does not actually draw a line of the actual number of racers.

The server at the time had 30 People on it and i ran the script for 5 minutes as a test.

I know this is all very old but its still exactly what i need so if you have any ideas that would be appreciated.

Regards,
Dean

Hi Dean..

Could you email me a couple of the 'host#.txt' files (preferably with some drivers connected etc)? I don't have an LFS server to test this myself, but if you can send something over to me, I'll run the script agianst that and see if I can find out why it's not working and fix it if I can

There maybe a change in the host#.txt file format since I originally coded it causing the script to fail.. but I'll have a gander.

ian.h@dsrc.digiserv.net should reach me



Regards,

Ian

Cheers Ian,

Just e-mailed some to you

Regards,
Dean

Hi Dean..

Originally Posted by franky500

Cheers Ian,

Just e-mailed some to you

Regards,
Dean

Thanks for that. All received.

Got a meeting with a client in a bit, but will have a look at this once I'm back home



Regards,

Ian

No Worries

Enjoy the meeting

Regards,
Dean