edit /etc/apache2/mod_log_config.conf and add a new logformat (plots) – (“mod_log_config.conf” is referenced by “httpd.conf”)

LogFormat “%h %l %u %t \”%r\” %>s %b \ \”%{Referer}i\” \”%{User-Agent}i\”" combined
LogFormat “%h %{%d.%m.%Y:%H:%M:%S}t %D %U” plots

The characteristics of the “%” directives are:

%…h Remote host
%…{format}t time & date
%…D time taken to serve the request, in microseconds.
%…U URL path requested

Add another “customlog” directive to your “virtualhost” section in “httpd.conf”:

DocumentRoot /home/h/hensler.net/public_html/bernhard/
ServerName bernhard.hensler.net
IndexOptions
DirectoryIndex index.htm index.html index.shtml start.htm start.html start.shtm index.php

CustomLog “/usr/local/visas/logfiles/hensler.net/%Y/%m/%d/access_log” vhost_combined
CustomLog “/usr/local/visas/logfiles/hensler.net/bernhard.access_log” plots

Concatenate logs from all virtual hosts e.g.: cat hensler.access_log niko.access_log bernhard.access_log max.access_log > plot_log (sample line: 66.249.111.111 30.08.2009:14:15:17 4372853 /blog/) and start gnuplot from the command line:

$ gnuplot

reset
set terminal png small color
set output “2dplot.png”
set title “average response time”

set style data points
set pointsize 1
set grid

set xlabel “time”
set timefmt “%d.%m.%Y:%H:%M:%S”
set format x “%H:%M\n%d/%b”
set xdata time
set xrange [ "30.08.2009:00:00" : "30.08.2009:23:59" ]

set ylabel “response time”
set yrange [ 0 : 10000 ]

plot “/usr/local/visas/logfiles/hensler.net/plot_log” using 2:3 title “2d”

gnuplot 3d

Then read this excellent article about “A New Visualization for Web Server Logs” and create a perl script:

#
# prepare-for-gnuplot.pl: convert access log files to gnuplot input
# Raju Varghese. 2007-02-03

use strict;

my $tempFilename = “./tmp/temp.dat”;
my $ipListFilename = “./tmp/iplist.dat”;
my $urlListFilename = “./tmp/urllist.dat”;

my (%ipList, %urlList);

sub ip2int {
my ($ip) = @_;
my @ipOctet = split (/\./, $ip);
my $n = 0;
foreach (@ipOctet) {
$n = $n*256 + $_;
}
return $n;
}

# prepare temp file to store log lines temporarily
open (TEMP, “>$tempFilename”);

# reads log lines from stdin or files specified on command line

while (<>) {
chomp;
my ($ip, $time, $D, $url, $sc) = split;
$time =~ s/\[//;
next if ($url =~ /(gif|jpg|png|js|css)$/);
print TEMP "$ip $time $D $url $sc\n";
$ipList{$ip}++;
$urlList{$url}++;
}

# process IP addresses

my @sortedIpList = sort {ip2int($a) <=> ip2int($b)} keys %ipList;
my $n = 0;
open (IPLIST, ">$ipListFilename");
foreach (@sortedIpList) {
++$n;
print IPLIST "$n $ipList{$_} $_\n";
$ipList{$_} = $n;
}
close (IPLIST);

# process URLs

my @sortedUrlList = sort {$urlList {$b} <=> $urlList {$a}} keys %urlList;
$n = 0;
open (URLLIST, ">$urlListFilename");
foreach (@sortedUrlList) {
++$n;
print URLLIST "$n $urlList{$_} $_\n";
$urlList{$_} = $n;
}
close (URLLIST);

close (TEMP); open (TEMP, $tempFilename);
while () {
chomp;
my ($ip, $time, $D, $url, $sc) = split;
print "$time $ipList{$ip} $urlList{$url} $sc\n";
}
close (TEMP);

Run this perl script and redirect output to a file from the command line:

$ perl gnuplot.pl "/usr/local/visas/logfiles/hensler.net/bernhard.access_log" > gnuplot.input

The fields in gnuplot.input, the output file of the Perl script, are date/time, ip rank, url rank.

Run gnuplot from the command line: $ gnuplot and the following commands:

reset
set terminal png small color
set output "3dplot.png"
set style data dots
set xdata time
set timefmt "%d.%m.%Y:%H:%M:%S"
set zlabel "Content"
set ylabel "IP address"
splot "gnuplot.input" using 1:2:3 title "3d"

Image taken from oreillynet, my website is not producing sufficient data …

• X, the time axis–a full day from midnight to midnight of November 16.
• Y, the requester’s IP address, with the conventional dotted decimal format sorted and given an ordinal number between 1 and 120,000, representing the number of clients that accessed the web server.
• Z, the URL (or content) sorted by popularity. Of the approximately 60,000 distinct pages on the site, the most popular URLs are near the zero point of the Z-axis and the least popular ones at the top.

http://www.ibm.com/developerworks/linux/library/lgnuplot

http://www.oreillynet.com/pub/a/sysadmin/2007/02/02/3d-logfile-visualization.html?page=1

http://phasorburn.com/index.php/archive/excel-0-gnuplot-1

A final step will cover loadrunner tools like openSTA and jmeter.

See also Part I of this tutorial.

<VirtualHost *:443>
DocumentRoot “var/local/wwwroot/bernhard/”
ServerName bernhard.hensler.net
ErrorLog /var/log/apache2/error_log
TransferLog /var/log/apache2/access_log
SSLEngine on
SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+
LOW:+SSLv2:+EXP:+eNULL
SSLCertificateFile /etc/apache2/ssl.crt/server.crt
SSLCertificateKeyFile /etc/apache2/ssl.key/server.key
</VirtualHost>

Fortunately there was no need to go through the painful process of creating my own CA etc., but instead I was able to use the “built-in” certifcate of my provider. To read a SSL certificate: openssl x509 -noout -text -in certificate.crt

Signature Algorithm: md5WithRSAEncryption
Issuer: C=DE, O=hensler.net, OU=IT, CN=bernhard.hensler.net/emailAddress=bhensler at gmail dot com
Validity
Not Before: Aug 24 17:31:07 2009 GMT
Not After : Aug 22 17:31:07 2019 GMT
Subject: C=DE, O=hensler.net, CN=bernhard.hensler.net/emailAddress=bhensler at gmail dot com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (4096 bit)
Modulus (4096 bit):

check Apache server version by creating a simple e.g. serverinfo.php file with following statement: <?php phpinfo(); ?>

and open in your browser: http://your host/serverinfo.php. Find the version info in the Apache Version section:

Apache Version – Apache/2.0.53 (Linux/SUSE)

or run from the command line:

/usr/sbin # apache2ctl -v

Server version: Apache/2.0.53

Server built: Aug 30 2006 13:14:23

h969344:/usr/sbin #

Next add mod_status to your APACHE_MODULES by editing /etc/sysconfig/apache2 configuration – the Status module allows a server administrator to find out how well a server is performing. A HTML page is presented that gives the current server statistics in an easily readable form:

APACHE_MODULES=

“access actions alias auth auth_dbm autoindex cgi

dir env expires headers include log_config mime

mod_status negotiation setenvif

ssl suexec userdir php4 rewrite”

In the same file and if extended logging is needed, set APACHE_EXTENDED_STATUS=”on” – be careful here, this will degrade performance.

Edit httpd.conf and add a Location section to your virtualhost section (remove Include /etc/apache2/mod_status.conf):

<VirtualHost *:80>

….

<Location /server-status>

SetHandler server-status

# Order deny,allow

# Deny from all

Allow from .yourdomain

</Location>

….

</VirtualHost>

restart the HTTP server: /usr/sbin/rcapache2 restart

You should see your Apache server status now by issuing this URL: http://yourHost/server-status providing the following information:

• The number of worker serving requests
• The number of idle worker
• The status of each worker, the number of requests that worker has performed and the total number of bytes served by the worker (*)
• A total number of accesses and byte count served (*)
• The time the server was started/restarted and the time it has been running for
• Averages giving the number of requests per second, the number of bytes served per second and the average number of bytes per request (*)
• The current percentage CPU used by each worker and in total by Apache (*)
• The current hosts and requests being processed (*)

The lines marked “(*)” are only available if ExtendedStatus is On

awstats

To display HTTP server log information in a graphical form, install awstats:

AWStats is a free powerful and featureful tool that generates advanced web, streaming, ftp or mail server statistics, graphically.

Copy awstats to a directory of your choice, installation defaults to /usr/local/awstats. Then run the awstats configuration process: perl awstats_configure.pl, which will add directives to your httpd.conf and create a configuration file (default location: /etc/awstats)

<Directory “/usr/local/awstats/wwwroot”>

Options all

AllowOverride None

Order allow,deny

Allow from all

</Directory>

Alias /awstatsclasses “/usr/local/awstats/wwwroot/classes/”

Alias /awstatscss “/usr/local/awstats/wwwroot/css/”

Alias /awstatsicons “/usr/local/awstats/wwwroot/icon/”

ScriptAlias /awstats/ “/usr/local/awstats/wwwroot/cgi-bin/”

Once done run the data import process from the cgi-bin folder of your awstats installation (you can automate this in the config file): perl awstats.pl config=yourdomain. See results of your site: http://yourdomain/awstats/awstats.pl or at this demo site

This link to part II of the tutorial.

Auf Anfrage bei click&buy heisst es lediglich, dass ich wohl mit meinen Passwörtern nicht sorgsam genug umgegangen bin – stellt sich die Frage nach dem Huhn und dem Ei; ich meine, dass  eine Internet Bank so sicher sein muss, dass keinerlei Daten dechiffrier- oder hackbar sind – zumal meine mailverbindungen aussschliesslich über SSL laufen. Wie auch immer, der Kunde ist der Depp – bleibt nur der Rat: Finger weg von click&buy !

Ich glaub’, ich frage mal in 3 Monaten wieder nach … ob es einen neuen Account auf meinen Namen bzw. mail Adresse gibt.

Aus technischer Sicht wird mit dem Begriff Web 2.0 oft eine Kombination von Technologien bezeichnet, die bereits Ende der 1990er Jahre entwickelt wurden, aber erst jetzt durch die große Zahl an breitbandigen Internetzugängen großflächig verfügbar sind.

Anwendungen, die Web 2.0 zugerechnet werden, verwenden oft Web Service APIs (ca. 1998), Ajax (1998 – Asynchronous Javascript and XML, bis 2005 XmlHttpRequest genannt) und Abonnement-Dienste wie RSS (1997). Die Integration von so genannter sozialer Software wie Blogs und Wikis wird ebenso im Zusammenhang mit Web 2.0 genannt. Wegen der per definitionem unscharfen Trennung zwischen tatsächlich fundamentalen Technologien und Entwicklungen des Web 2.0, Rand- und Sekundärerscheinungen und zu Privatzwecken gehypten Spielereien ist die tatsächliche Bedeutung einzelner Entwicklungen aber immer wieder Anlass zu Diskussionen.

mehr gibt es hier: Wikipedia Web 2.0