I bought the best Android tablet on the market in October 2011, it was a Samsung Galaxy Tab 10.1" with 3G, GT-P7500.  According to information from Samsung they would provide Android 4.0 Ice Cream Sandwich as soon as possible. The main reason why I payed over 460EUR for the device was that I wanted to have the possibilty to run Ice Cream Sandwich on it as soon as possible.

After 14 months I'm still waiting.

During the summer 2012 Samsung started to roll out Ice Cream Sandwich on the Galaxy Tab 10.1" around the globe. On the Samsung webpage they planed to roll out in Q4 2012 in Sweden.

I have written a several times to Samsung and asked why I still cannot upgrade my Tablet and the answer I've got is that the bundle I have will be possible to update anyday. 

Samsung, it is NOT acceptable to wait over a year for an upgrade on a device that has a lifetime of 2 years. 

My next tablet will be a Nexus.

My son got the amazing game Minecraft as a birthday present. Of course I think I should provide the service to him and his friends to have a couple of minecraftservers to be able to play together.

Soon it became business critical and I started to think about how to monitor that the Minecraft server is up and running. After searchin Internet I did not find anything that worked so I decided to write my own.

Prereq

I'm using the possibility to connect to the Rconn inteface at the Minecraftserver so it needs to be ebabled. Normaly by adding to Minecraft server.properties:

rcon.password=fjollboll

rcon.port=25578

I'm running the plugin on a vanilla Centos 6.3 box, and a perl CPAN module needs to be installed. Simply run the following command to install:

cpan Minecraft::RCON

I assume the reader knows howto monitor the basic stuff like: CPU usage, processes running, RAM usage, disk usage, disk I/O usage and everything else that should always be monitored on a system running business critical services.

Monitor the server

The easiest way to monitor that an instance of Minecraft server is running is to using check_tcp and the port defined for the server. However I read on some forums that the Minecraft server could hang and the tcp port still respond. So I wrote a plugin that connects to the Minecraft Rconn interface.

#!/usr/bin/perl

# Hint at http://search.cpan.org/~demmy/Minecraft-RCON-v0.1.2/lib/Minecraft/RCON.pm

#Licence GPLv2

use Minecraft::RCON;

use Getopt::Std;

use strict;

open STDERR, '>>/dev/null' or die "Can't write to /dev/null: $!";

my(%ERRORS) = ( OK=>0, WARNING=>1, CRITICAL=>2, UNKNOWN=>3 );

my $status=$ERRORS{OK};;

my $rconresult;

my $numplayers;

my $maxplayers;

my $players;

my $message="OK";

our($opt_c, $opt_w, $opt_p, $opt_P, $opt_h, $opt_H);

getopts("p:P:hH:");

my $debug_flag=0;

sub printhelp () {

        print "Usage: check_minecraft [-h] -p password -P port -H host\n";

        print "-h Help, this text\n";

        print "-p password for rcon\n-P port number for rcon\n";

        print "-H ip-adress or servername of minecraftserver\n";

        print "\n\t\tby Peter Andersson\n\t\tpeter\@it-slav.net\n\t\thttp://www.it-slav.net/blogs/2012/10/06/monitor-minecraft-server-with-op5-monitor-or-nagios\n";

        if ($debug_flag) {

                print "opt_c:$opt_c opt_w:$opt_w opt_p:$opt_p opt_P:$opt_P opt_h:$opt_h opt_H:$opt_H\n";

        }

        exit $status;

} 

#sanity check

if (!defined $opt_p||!defined $opt_P||$opt_h||!defined $opt_H) {

        $status= $ERRORS{UNKNOWN};

        &printhelp;

}

my $rcon = Minecraft::RCON->new( { address => $opt_H, port => $opt_P, password => $opt_p } );

if ($rcon->connect){

  $rconresult =  $rcon->command('list');

  $_=$rconresult;

  if (/There are (\d+)\/(\d+) players online:(.*)/) {

    $status= $ERRORS{OK};

  }

}

else {

  print "CRITICAL - Could not connect to Minecraft server!\n";

  # Error capturing and fetching is in the works...

  $status= $ERRORS{CRITICAL};

  exit $status;

}

$rcon->disconnect;

print "OK - Minecraft server is running\n";

exit $status;

Monitor the numer of users

I think it would be interesting to monitor and graph the number of users using a Minecraft instance. Of course it returns perfdata for graphing.

 #!/usr/bin/perl

# Hint at http://search.cpan.org/~demmy/Minecraft-RCON-v0.1.2/lib/Minecraft/RCON.pm

# License GPLv2

use Minecraft::RCON;

use Getopt::Std;

use strict;

my(%ERRORS) = ( OK=>0, WARNING=>1, CRITICAL=>2, UNKNOWN=>3 );

my $status=$ERRORS{OK};;

my $rconresult;

my $numplayers;

my $maxplayers;

my $players;

my $message="OK";

our($opt_c, $opt_w, $opt_p, $opt_P, $opt_h, $opt_H);

getopts("c:w:p:P:hH:");

my $debug_flag=0;

sub printhelp () {

        print "Usage: check_minecraftusesr [-h] -c critical -w warning -p password -P port -H host\n";

        print "-h Help, this text\n-c num Critical threshold for number of players\n-w num Warning threshold for number of players\n";

        print "-p password for rcon\n-P port number for rcon\n";

        print "-H ip-adress or servername of minecraftserver\n";

        print "\n\t\tby Peter Andersson\n\t\tpeter\@it-slav.net\n\t\thttp://www.it-slav.net/blogs/2012/10/06/monitor-minecraft-server-with-op5-monitor-or-nagios/\n";

        if ($debug_flag) {

                print "opt_c:$opt_c opt_w:$opt_w opt_p:$opt_p opt_P:$opt_P opt_h:$opt_h opt_H:$opt_H\n";

        }

        exit $status;

} 

#sanity check

if (!defined $opt_c||!defined $opt_w||!defined $opt_p||!defined $opt_P||$opt_h||!defined $opt_H) {

        $status= $ERRORS{UNKNOWN};

        &printhelp;

} elsif ($opt_c < $opt_w) {

        print "Critical threshold must be higher or equal to warning threshold\n";

        $status= $ERRORS{UNKNOWN};

        &printhelp;

}

my $rcon = Minecraft::RCON->new( { address => $opt_H, port => $opt_P, password => $opt_p } );

if ($rcon->connect){

  $rconresult =  $rcon->command('list');

  $_=$rconresult;

  if (/There are (\d+)\/(\d+) players online:(.*)/) {

    $numplayers=$1;

    $maxplayers=$2;

    $players=$3;

    $players=~tr/a-zA-Z0-9 _,.-//cd;

    if ($debug_flag) {

      print "Number of players: $numplayers\n";

      print "Max players are: $maxplayers\n";

      print "Players are: $players\n";

    }

  }

}

else {

  print "Could not connect to Minecraft server!\n";

  # Error capturing and fetching is in the works...

  $status= $ERRORS{CRITICAL};

  exit $status;

}

$rcon->disconnect;

if ($numplayers >= $opt_w) {

  $status= $ERRORS{WARNING};

  $message="WARNING";

}

if ($numplayers >= $opt_c) {

  $status= $ERRORS{CRITICAL};

  $message="CRITICAL";

}

chomp ($players);

if ($numplayers > 0) { 

 print "$message:Number of players are $numplayers out of maximum $maxplayers. Players are: $players|players=$numplayers;$opt_w;$opt_c;0;$maxplayers;\n";

} else {

 print "$message:Number of players are $numplayers out of maximum $maxplayers.|players=$numplayers;$opt_w;$opt_c;0;$maxplayers;\n";

}

#print "$message: Number of players are $numplayers out of maximum $maxplayers:$players\|players=$numplayers\n";

exit $status;

Defining the commands in op5 Monitor or nagios

# command 'check_minecraft_server'

define command{

    command_name                   check_minecraft_server

    command_line                   $USER1$/custom/check_minecraft_server.pl -H $HOSTADDRESS$ -p $ARG1$ -P $ARG2$

    }

# command 'check_minecraft_users'

define command{

    command_name                   check_minecraft_users

    command_line                   $USER1$/custom/check_minecraft_users.pl -H $HOSTADDRESS$ -p $ARG1$ -P $ARG2$ -w $ARG3$ -c $ARG4$

    }

The result

Monitoring the number of users

Graphing the number of users

Using op5 BSM module to show the Business Service Minecraft

Final

• MInecraft: an amazing game.
• op5 Monitor, an enterpise class monitoring system based on Nagios
• Nagios, an open and free tool for monitoring
• BSM, op5 Business Service Management

I would also want to thank Mattias Ryrlen at op5, for helping me troubleshoot garbage characters from Rconn.

The Red Herring editorial team selected the 2012 Top 100 Europe. This exceptional group of companies are among the most innovative, unique and promising companies selected from a pool of hundreds from across Europe. The Top 100 were evaluated on both quantitative and qualitative criteria, such as financial performance, technology innovation, quality of management, IP creation, CAGR, execution of strategy, and disruption in their respective industries.

–end cut–

Read more at: http://www.redherring.com/startups/top-100-europe-revelaed/

The list can be found at: http://www.redherring.com/red-herring-europe/europe-2012-top-100/

I sometimes get the question if I have any tips on a op5 Monitor alarm console. I stumbled on the possibility to use irc to present the alerts generated by op5 Monitor or Nagios. A nice feature is that the number of connected clients can be very many, severla systems can send all alerts to the same irc channel and best of all, it's geeky.

Solution

The solution is using four components:

• A working op5 Monitor or Nagios implementation (will not be covered in this article)
• An irc channel where the alerts are presented
• A bot that collects the op5 Monitor status and sends it to the irc channel
• An irc client i.e. weechat

The irc server

I wanted to use my own irc server, mainly because I want it to run in my own controlled environment. There are toons of implementations but i picked new generation irc daemon, ngircd which existed in my distributions repository.

Simple configuration in /etc/ngircd.conf added to the end of configuration.

[Channel]
        Name = #op5_Monitor
        Topic = op5 Monitor Alerts
        MaxUsers = 23

Start the irc server:

# /etc/init.d/ngircd start
Starting ngircd:                                           [  OK  ]

Making sure it starts the next reboot

# chkconfig ngircd on

The bot

I found a boot called nagircbot

Download and compile.

# make && make install

Start the bot:

nagircbot -f /opt/monitor/var/status.log -X -s mail:6667 -c \#op5_Monitor -n Monitor -N op5_Monitor-- -C

Connect the client

Start the irc client

$ weechat-curses

Connect the irc server

/connect <my_irc_server>

Set nick

/nick <my_nick>

Join the correct channel

/join #op5_Monitor

Screenshoot:

Links:

• op5 Monitor, a nagios based eneteprise monitor solution
• Nagircbot, a bot that sends nagios events to irc
• ngIRCd, new generation irc daemon
• WeeChat, nice irc client with rich functionality

• Complexity reduction by moving from multi-threading to single threaded programing
• Reducing latency
• Disk I/O usage reduction
• CPU usage reduction
• Memory usage reduction

Read more at http://blogs.op5.com/development-work-on-nagios-project-to-improve-performance/

On my electricity meter I have a flashing LED. 1000 flashes is 1 watthour.

As a monitor geek I want to know my powerconsumption and find a 1-wire kit containing a counter and a sensor that could detect the flashing from the electricity meter.

I wrote a script that collect the data in an RRD database and graph it. 

Graph

The script

#!/usr/bin/perl
# By Peter Andersson peter@it-slav.net
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
use RRDs;
use OW;
# define location of rrdtool databases
my $rrd = '/root/owfs/script/rrd_db';
# define location of images
my $img = '/home/peter/public_html/1-wire/energy';
# process data for each interface (add/delete as required)
&ProcessSensor(0, "Anderssons power consumption", "mother.mynet" , "3001" , "1D.A4F10C000000");
sub ProcessSensor
{
# process sensor
# inputs: $_[0]: sensor number (ie, 0/1/2/etc)
#         $_[1]: sensor description
        print "number:$_[0] desc:$_[1], server:$_[2] port:$_[3], id: $_[4]\n";
        my $owserver = "$_[2]:$_[3]";
        unless(OW::init($owserver)) {
            $status = $ERRORS{CRIT};
            $message = "OWServer not running at $owserver\n";
            exit $status;
        }
        # get temperature from sensor
        my $handle = OW::get("$_[4]/counters.A");
#       print "handle=$handle\n";
        $handle =~ s/^\s*(.*?)\s*$/$1/;
        ## Check if input is an integer or decimal
        unless (($handle =~ /^-?(?:\d+(?:\.\d*)?|\.\d+)$/) || ($handle =~ /^[+-]?\d+$/))
        {
                print "Not an integer or a decimal\n";
                return($ERRORS{CRITICAL});
        }
        $count=$handle;
        # remove eol chars
        chomp($count);
        print "sensor $_[0]: $count \n";
        # if rrdtool database doesn't exist, create it
        if (! -e "$rrd/energy$_[0].rrd")
        {
                print "creating rrd database for energy sensor $_[0]...\n";
                RRDs::create "$rrd/energy$_[0].rrd",
                        "-s 60",
                        "DS:energy:COUNTER:1200:0:U",
                        "RRA:AVERAGE:0.5:1:43200",
                        "RRA:AVERAGE:0.5:30:175200";
        }
        if ($ERROR = RRDs::error) { print "$0: failed to create $_[0] database file: $ERROR\n"; }
        # check for error code from count sensor
        #if (int $count eq 85)
        #{
        #        print "failed to read value from sensor $_[0]\n";
        #}
        #else
        #{
                # insert values into rrd
                RRDs::update "$rrd/energy$_[0].rrd",
                        "-t", "energy",
                        "N:$count";
                if ($ERROR = RRDs::error) { print "$0: failed to insert $_[0] data into rrd: $ERROR\n"; }
        #}
        # create graphs for current sensor
        &CreateGraph($_[0], "day", $_[1], 3600, hourly);
        &CreateGraph($_[0], "week", $_[1], 3600*24, daily);
        &CreateGraph($_[0], "month", $_[1], 3600*24*7, weekly);
        &CreateGraph($_[0], "year", $_[1], 3600*24*7*30, monthly);
        &CreateGraph($_[0], "decade", $_[1], 3600*24*7*365, yearly);
}
sub CreateGraph
{
# creates graph
# inputs: $_[0]: sensor number (ie, 0/1/2/etc)
#         $_[1]: interval (ie, day, week, month, year)
#         $_[2]: sensor description
#	  $_[3]: Moving avarage timeperiod
#	  $_[4]: Moving avarage timeperiod unit
print "Create graph $_[0], interval: $_[1], Moving avarage timeperiod: $_[3]s and moving avarage unit: $_[4]\n";
        RRDs::graph "$img/energy$_[0]-$_[1].png",
                "-s -1$_[1]",
                "-t $_[2]-$_[1]",
                "-h", "400", "-w", "1000",
                "-a", "PNG",
                "-v Watt",
                "--slope-mode",
		"-W", "WWW.It-Slave.Net - An It-Slave In The Digital Saltmine",
		"-l", "0",
                "DEF:energy=$rrd/energy$_[0].rrd:energy:AVERAGE:start=end-$_[3]s-1$_[1]",
                "CDEF:W=energy,3600,*",
		"CDEF:mean=W,$_[3],TREND",
		"AREA:mean#FF000044",
                "LINE1:mean#A00000:Moving average $_[4]:",
		"AREA:W#AAAAee99",
		"LINE1:W#AAAAEEFF:Power consumption\\n",
		"CDEF:energyK=energy,1000,/",
		"VDEF:value_sum=energyK,TOTAL",
  		"GPRINT:value_sum:Total consumption last $_[1]\\:%0.2lfkWh\\n",
  		"GPRINT:W:MIN:Instantaneous consumption\\: Min\\:%0.2lf%sW",
  		"GPRINT:W:AVERAGE:Average\\:%0.2lf%sW",
  		"GPRINT:W:MAX:Max\\:%0.2lf%sW",
  		"GPRINT:W:LAST:Now\\:%0.2lf%sW\\n";
#		"COMMENT:\n                                            WWW.It-Slav.Net - An It-Slave In The Digital Saltmine";
        if ($ERROR = RRDs::error) { print "$0: unable to generate sensor $_[0] $_[1] graph: $ERROR\n"; }
}

Links

• m.nu sells 1-wire things and other good stuff
• owfs-free implementation to use 1-wire devices
• Other articles about 1-wire at It-Slave blog
• My graphs
  • All
  • Day
  • Week
  • Month
  • Year
  • Decade

I have a FON router which allow anyone to connect to Intenet via my network connection. I like the openess and idea behind FON but I do not want to get in trouble if someone do bad things and using my network connection.

One solution is to connect the FON accesspoint to a separate network segment and let all traffic from that network go through TOR, the onion router. 

Another feature is when I want to be anonymous on Internet I plugin my computer to that network segment.

Setup

I use OpenBSD as my firewall and the first step is to download, compile and configure TOR. I downloaded the tarball from https://www.torproject.org and used the normal procedure:

# wget https://www.torproject.org/dist/tor-0.2.2.35.tar.gz

# tar xzvf tor-0.2.2.35.tar.gz 

# cd tor-0.2.2.35                                                                                                                                                                

# ./configure&&make&&make install

Configure pf

I use a specific ethernet interface, fxp0, which will route all traffic into tor. 

--cut from /etc/pf.conf--

#Tor traffic

tor_if ="fxp0"

# Tor's TransPort

trans_port = "9040"

pass in quick on $tor_if inet proto udp to port domain rdr-to 127.0.0.1 port 5300 

pass in quick on $tor_if inet proto { tcp udp } to !($tor_if) rdr-to 127.0.0.1 port $trans_port

--end cut--

Configure tor

# cat /usr/local/etc/tor/torrc

VirtualAddrNetwork 10.192.0.0/10

AutomapHostsOnResolve 1

TransPort 9040

DNSPort 5300

Log notice syslog

#Log debug stderr

RunAsDaemon 1

Note: I know that best practice is to let the tor process run as non root user. That requires read access to /dev/pf and I did not bother to get it working.

Dhcpd config

I assume that a OpenBSD sysadmin knows how to setup dhcpd so I will just show the config addon I did to /etc/dhcpd.conf.

shared-network tor-net {

        option  domain-name "it-slav-tor-net";

        option  domain-name-servers 10.1.2.1;

        option  ntp-servers 10.1.2.1;

        subnet 10.1.2.0 netmask 255.255.255.0 {

                option routers 10.1.2.1;

                range 10.1.2.100 10.1.2.200;

        }

The ip-adress of the fxp0 interface is 10.1.2.1

Start tor

# /usr/local/bin/tor

Jan 10 20:52:48.880 [notice] Tor v0.2.2.35 (git-b04388f9e7546a9f). This is experimental software. Do not rely on it for strong anonymity. (Running on OpenBSD i386)

Jan 10 20:52:48.885 [warn] It's a little hard to tell, but you seem to have Libevent 1.4.0-beta header files, whereas you have linked against Libevent 1.4.14b-stable.  This will probably make Tor crash.

Jan 10 20:52:48.886 [notice] Initialized libevent version 1.4.14b-stable using method kqueue. Good.

Jan 10 20:52:48.886 [notice] Opening Socks listener on 127.0.0.1:9050

Jan 10 20:52:48.887 [notice] Opening Transparent pf/netfilter listener on 127.0.0.1:9040

Jan 10 20:52:48.887 [notice] Opening DNS listener on 127.0.0.1:5300

Final step

Plugin your fon router and enjoy!

Links

The hints to this article was found at:

•  https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/TransparentProxy 

It really annoyed me that the iowait at the cpu on the Dom 0 system was approximatly 50% when the 4 virtual system was doing more or less nothing.

I showed my earlier blogpost about VM virtualization to our sysadmin at op5. He gave me a couple of more hints that really make a big difference, the best part is that they are real simple to implement.

I also noticed that I forgot some parts in the previous article that was obvius to me.

#1 use virtio drivers if possible

This is probably the single most important step to get the most out of the hardware.

When the virtual systems wants to communicate with the hardware it uses drivers, if the drivers are simulating hardware to give a service to the virtual operating system it adds alot of overhed. A simple example:

• The virtual system wants to access the disk
• The virtual system communicates via the driver to the hardware
• The hardware is not real hardware, it is emulated hardware in sofware 
• The software that emulates the hardware sends the I/O to the Dom 0 virtualization engine
• The Virtualization engine verifies that the access is ok, translate it to real hardware call via its driver
• The hardware do the operation and send the result back up via all the steps to the virtual operating system

All this transalations and software calls take alot of resources from the machine.

By using virtual drivers, so that there is no need to emulate hardware, many of the steps can be skipped or done with much less penalty. However not all operating system have virtio drivers.

#2 load the virtio modules in Dom 0

This is a real god hint I got from our sysadmin, load the virtual drivers in the Dom 0 system asweell. According to him, the virtual drivers in the Dom 0 and Dom U:s communicate and makes the I/O much more effective. I have not found any information about this at any place so I was sceptical but I tried it.

# modprobe virtio_balloon
# modprobe virtio_blk
# modprobe virtio_net
# modprobe virtio_pci

The virtual systems need to be rebooted as far as I know before the loaded drivers will have any effect.

At first no change at all, but after few hours I noticed that CPU idle raized and the iowait went from at least 50% down to below 5%.

A nice graph, created by op5 Monitor showing the difference:

I loaded the virtual drivers approximately 4 pm, and it took until 10 pm before the performance boost showed up.

The correct way of making sure that the drivers are loaded directly would be to remake the kernel but I'm lazy so I have just added the module load to /etc/rc.local

Conclusions

It is always important to monitor systems. When doing changes it is not possible know if the changes do have any impact if you do not monitor and follow up how and if the changes does anything.

My former statement that KVM is not mature is more true then I have thought. The knowledge how to tune and maintain KVM systems seems to be like black art where the knowledge is not documented and well spread. I have read the RedHat virtualization and I have not found anything about that the virtual drivers should be loaded on the Dom0 system.

If there is anyone out there that has some documentation or more hints, please send me the link or comment this blogpost.

Links:

• RedHat virtualization doc
• op5 Monitor that monitor and creates nice graphs
• check_cpu_stats Nagios plugin to monitor cpustat

As an it slave, I have the same problems as most datacenters:

• Running out of space because of more and more machines get into the datacenter
• Overheating
• Powerconsumption

Added to the "normal" datacenter problems, I also have to deal with:

• Wife acceptance
• Noice, my neighbours has complained about the noice outside my basement.

To address this issues I will run a consolidation project to get fewer machines by using virtualization. As virtualization engine, KVM is choosed.

This article will describe some pitfalls I run into and how I solved them.

KVM plattform

KVM seems to be the future for virtualization within the Linux area since Xen has done some mistakes.

As I have good experience of running CentOS and has KVM included I decided to use CentOS as the plattform for my virtualization.

Issues

Bridge network

To let the virtual boxes has full access to the network you need create a network bridge, it is rather straightforward and well documented so I will skip this part. I used the documentation on Red Hat customer support. Why KVM requires a bridge instead of a normal NIC is another question

Snapshots

One handy feature with virtualization is that the virtual systems can be one big fat file at the physical host, this gives the possibility to make full backups without turning the system down a.k.a snapshots.

However to get snapshot to work the filetype must be qcow2 and raw is the default format.

It is possible to convert from raw to qcow2 format by running the command:

qemu-img convert -f raw -O qcow2 <virtualhost>.img <virtualhost>.img.qcow2

Remember to turn of your virtual system by running:

virsh shutdown <virtualhost>

After the conversion:

virsh edit <virtualhost>

modify the type and path, i.e.

      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/web.img.qcow2'/>

Start your virtual system:

virsh start <virtualhost>

Take a snapshot with:

qemu-img snapshot -c <snapshotname> <virtualhost>.img.preallc.qcow2

To create a full image of your snapshot you need to compile a more resent version of qemu-img then shipped with CentOS 6.2, I download a later version and compiled it from here, i renamed the new to qemu.img2 and copied it to /usr/local/bin

qemu-img2 convert -p -f qcow2 -O qcow2 -s <snapshotname> <virtualhost>.img.preallc.qcow2 <targetpath>

Beware: This command can use all your resources and may affect all your virtual systems, use nice and ionice to prevent it.

Disk I/O

After installing just a few virtual system I noticed that my physical host got alot of I/O wait, it can be seen by using tools like top.

To monitor this I installed the nagios plugins:

• check disk io, can be found here
• check cpu stats, can be found here
• check_libvirt, developed by op5 and an Howto

However it seems like that though my virtual systems did more or less nothing they caused alot of disk I/O on the physical system.

I did some investigations and read quite many articles, fiddled and tested. The following is what worked for me in my setup.

#1 Change disk scheduler

The disk scheduler can be changed on runtime by modify the file:

/sys/block/sda/queue/scheduler

to see what scheduler you use now:

cat /sys/block/sda/queue/scheduler

Change sda to the device you have.

The scheduler that worked best for me is deadline

echo deadline > /sys/block/sda/queue/scheduler

#2 Mount with noatime

A feature in Unix and other Unixlike systems like Linux is that it normaly stores when a file is accessed. So one read always produce a write and if you are using raid like morroring this get worse, one reade always generate several writes. This creates alot of overhead for a feature soldom used.

So change /etc/fstab so it will mount the file systems with noatime.
An axample:

UUID=a290aa4b-635c-45fa-b144-1fbef90b3735 /                       ext4    defaults,noatime        1 1

#3 Preallocation disk images

A real boost that is hidden in the featureset and not shown in the virtualmachine gui is using preallocation disk images. They cannot be created from the GUI so I install the virtual machine, turn it of and convert it afterwards.

qemu-img convert -f qcow2 -O qcow2 -o preallocation=metadata <virtualhost>.img.qcow2 <virtualhost>.img.preallc.qcow2

Change the path to new image name by editing the virtual machine settings

virsh edit <virtualhost>

Conclusions

In my opinion it seems like KVM is still a little bit immature or at least the tools to handle it. Maybe it would be a good idea to have one linux distro focusing on beeing the best platform for virtualization.

I am convinced that I can get even more bang for the bucks out of my installation if I learn how to tweak it even more, so if you have any hints, do not hesitate to contact me. Preferably as a comment to this blogpost.

Referenses:

• op5 Monitor, a nagios based enterprise monitor tool to monitor my environment
• check_libvirt, a nagios plugin done by op5 to monitor KVM
• check_diskio, a nagios plugin to monitor disk I/O
• check_cpu_stats, a nagios plugin to monitor cpu usage rather detailed
• Another blogpost which helped me
• KVM performance improvements and optimizations – Red Hat presentation

The new access speed is 100 Mbit/s optical fiber, iperf tests show that I can get approx 97 Mbit/s in booth directions.

Changes:

• 100 Mbit/s internet access provided by Ownit
• Updated firewall to OpenBSD 5.0 running on an Astaro appliance
• Updated Webserver to Centos 6.2, virtulized with KVM
• Updated Mailserver to Centos 6.2, virtulized with KVM

Happy new year

• Support for offline maps – no need to have online gmap connection
• Introducing IP traffic “weathermaps” i.e. a user friendly way to present traffic flows, being WAN or LAN traffic
• Display monitor data on external web i.e visualise and distribute the information to greater crowd
• Multiple of  time saving tasks such as multidelete, free text filter, save searches etc.

Having a modern easy to use proactive monitoring system is the core of assuring quality delivery of your enterprise applications to your user. However cool report and dashboards are great, if they require complex skills, many hours to set up and maintain -  the value is quickly gone.

With op5 Monitor v5.5 we introduce a number of features to make the information more presentable and understandable. We have made lots of new micro features to make the everyday change management easier and faster.

Major features in this release:   
•    Support for offline map
    ⁃    Now integrating the open streetmap (OSM) project, making the vizualisation of large networks easier and offline
    •    Support for IP traffic flow visualisation on maps or any background
    •    Publish op5 Monitor data / widget on external web service
    •    Multi tier host & service groups
    ⁃    A host or service group can now be part of another group creating a tree structure of dependencies.
    •    Multidelete – it’s always easy to add a new host or service to a system – now it´s easy to remove as well – saves lots of time!
Usability improvements:
    •    Saved searches
    •    Search on notes and comments and search in scheduling queue
    •    Free text filter
    •    Enhanced Alert history
    •    Authentication fallback

In total we have fixed and added more then 240 items in this release. Please let us know what you think – we focus on Your needs, just tell us!

Best regards
The op5 Team!

More about the new release

We are pleased to announce the official release of OpenBSD 5.0.
This is our 30th release on CD-ROM (and 31th via FTP).  We remain
proud of OpenBSD's record of more than ten years with only two remote
holes in the default install.

As in our previous releases, 5.0 provides significant improvements,
including new features, in nearly all areas of the system:

 - Improved hardware support, including:
    o MSI interrupts for many devices, on those architectures which can
      support them (amd64, i386, sparc64 only so far).
    o A new dma_alloc(9) API makes it easier for kernel code to allocate
      dma-safe memory.  Many drivers (especially network drivers) and
      subsystems (in particular scsi and the buffer cache) were adapted
      to use this.
    o As a result, big-memory support has been enabled on all possible
      architectures.
    o The rather rare bce(4) driver now copies mbufs all the time, to cope
      with the hardware having a 1GB limit.
    o Added hds(4), a driver for Hitachi Modular Storage SCSI devices.
    o Added myx(4), a driver for the Myricom Myri-10G 10GB Ethernet devices.
    o Added dfs(4), a driver for Dynamic Frequency Switching on some macppc
      systems.
    o cardbus(4) and pcmcia(4) support on sgi.
    o Suspend/resume support on Loongson Yeelong laptops.
    o Interrupt handlers for bnx(4), em(4), ix(4) and sis(4) have been
      improved reducing overhead and increasing performance.
    o New acpitoshiba(4) driver providing ACPI support for Toshiba laptops.
    o Added nvt(4), a driver for the W83795G and W83795ADG hardware monitor.
    o Added support to sdhc(4) for the Ricoh 5U823 SD/MMC controller.
    o A new fw_update(1) tool to install and update non-free firmware packages.

 - Generic network stack improvements:
    o Added support for sending Wake on LAN packets using arp(8).
    o Permit turning Wake on LAN support on/off using ifconfig(8).
    o Added Wake on LAN support to xl(4), re(4), and vr(4).
    o Allow ftp-proxy to proxy across rdomains.
    o The IPv4 stack will no longer accept ICMP redirects when
      acting as a router.
    o By default the IPv6 stack will not process ICMP6 redirects.
      rtsol(8) will turn it back if -F is used.
    o Reworked large parts of the dhclient(8) options processing for better
      interoperability.
    o Fixed carp(4) to work in IPv6 only setups.
    o Make it possible to bind(2) to the local network broadcast address
      on datagram and raw sockets.
    o The default multicast reject route is now ignored if the UDP socket
      uses the IP_MULTICAST_IF socket option.
    o Make gre(4) work between systems in the same LAN.
    o Removed the link1 mode special addressing mode on lo(4).
    o New net.inet.tcp.always_keepalive sysctl, effectively enabling
      SO_KEEPALIVE on all TCP sockets.

 - Routing daemons and other userland network improvements:
    o bgpd(8) no longer bumps the rlimits: the rc.d framework respects
      login classes which is a much better solution.
    o Correctly set the network filtersets on reload in bgpd(8).
    o The routing socket is now sending RTM_DESYNC messages if the
       socketbuffer overflows.
    o Allow ospfd(8) to send out LS updates and other messages
      larger than the MTU.
    o Fixed nexthop calculation in ospfd(8) for directly connected P2P links.
    o First bits to support opaque LSA in ospfd(8).  Only basic redistribute
      logic and LSDB handling for now.
    o Creating new interfaces will no longer cause a fatal error in ospf6d(8).
    o ospf6d(8) handles link-state changes better.
    o Better loopback handling in ospf6d(8).
    o No longer install extra multicast routes in ripd(8) and ldpd(8).
    o Make kqueue(2) work with sosplice(9).
    o Enabled sosplice(9) in relayd(8) for TCP.
    o Added support for divert-to which provides some benefits over
      rdr-to in relayd(8).
    o Reload support in relayd(8) has been fixed.
    o Fixed trap sending in snmpd(8).
    o Make ping6(8) compare minimum amount of bytes between what
      was received and what was sent out.
    o Make traceroute(8) with type-of-service setted (-t) display
      a message if the returned packet has a different tos type.
    o Added the socket splicing fields of struct socket to netstat -vP output.
    o tcpbench(1) now uses libevent and supports both TCP and UDP modes.
    o TCP socket buffer sizes can now be displayed using the netstat(1) -B flag.
    o tcpdump(8) can now filter on icmptype and tcpflags.
    o bgplg(8) now supports "show ip bgp peer-as".

 - pf(4) improvements:
    o Make pf(4) reassemble IPv6 fragments.  In the forward case, pf
      refragments the packets with the same maximum size.
    o Allow pf(4) to filter on the rdomain a packet belongs to.
    o Make pf(4) allow userland proxies to establish cross rdomain
      proxy sessions.
    o Added IPv6 ACK prioritization in pf(4).
    o Change 'set skip on <...>' to work with interface groups.
    o pfsync(4) supports IPv6 as network protocol.
    o Switched ftp-proxy(8) over to divert-to instead of rdr-to.
    o Switched tftp-proxy(8) over to divert-to instead of rdr-to.
    o New very low overhead priority queueing implementation for pf(4) used via
      the "prio" keyword.
    o Support for least-states in load balancing pools and tables.
    o Support for weighted round-robin in load balancing pools and tables.

 - SCSI improvements:
    o Most SCSI hardware drivers now use the new iopools infrastructure.
    o scsi(4) devices are now all provided with a unique devid, which
      is displayed during the probe process.
    o ASC/ASCQ error codes and verbiage now in sync with
      http://www.t10.org/lists/asc-num.txt.
    o Progress on iSCSI includes better login, better logout, preliminary
      FSM support in iscsid(8), and improved logging and debug information.
    o uk(4) can now safely and reliably detach an unknown SCSI device.
    o SCSI multipath device and kernel support has been improved.
    o vscsi(4) now ensures output always goes to the correct connection.
    o vscsi(4) connections can now be reset gracefully.
    o scsi(4) devices on fibre channel fabrics no longer inherit the adapter's
      address.

 - Assorted improvements:
    o Kernel randomization speed and quality improved substantially.
    o For additional security, security(8) was rewritten in Perl.
    o Mandoc 1.11.4: Now accepts eqn(7) input (no fancy formatting yet)
      and supports -Tutf8 output (but no utf8 input yet).
    o Removed a variety of OS-compat emulation code, leaving just the Linux
      support.
    o Small improvements to Linux compat (only available on i386).
    o Improved our own pkg-config(1) implementation with extended comparison
      scheme and implementing various new options.
    o The math library, libm, was fully fleshed out to support all C99 required
      parts.  Many bugs for various architectures were fixed along the way.
    o malloc(3) is a lot faster and has a few further security features (more
      randomization, as well as the 'S' flag to enable all paranoia checks).
    o 'make depend' is no longer neccessary in kernel compilation directories
      since the dependencies are calculated automatically.
    o Increased the default size of the buffer cache.
    o kqueue(2) now works on /dev/random and spliced sockets
    o On MBR-based disks, scan through up to 256 extended partition tables
      when looking for an OpenBSD partition table.
    o Added POSIX 2008 fdopendir(3) and openat(2) functions, as well as the
      O_CLOEXEC, O_DIRECTORY, and F_DUPFD_CLOEXEC flags.
    o Improved lint format string checks and added a few other checks.
    o kdump(8) now dumps stat and sockaddr structures, sysctl mib
      strings, and decodes syscall flags and operation bits.
    o Improved kernel pool debug checking.
    o Improved correctness of signals and various syscalls when rthreads
      are in use.
    o Kernel malloc(9) space and stacks moved to non-dma memory.
    o Fixed some shutdown/reboot hangs on NFS clients.
    o UNIX-domain socket paths are now guaranteed to be NUL-terminated.
    o Added support for *wprintf(3), wcs{,n}casecmp(3), and wcsdup(3).
    o NULL is now a (void *).
    o grep(1) now supports a -H option to always print filename headers.
    o Whitelist expiry for spamlogd(8) can now be configured via a -W flag.
    o ls(1) now supports the POSIX -H option to follow symbolic links specified
      on the command line.
    o disklabel(8) now tries the next auto-allocation scheme if the current one
      fails due to insufficient available partitions.
    o bc(1) gained editline(3) support.
    o Many enhancements and new functionality has been added to tmux(1).
    o disklabel(8) supports absolute resizing of partitions in auto-allocated
      labels.
    o newfs(8) accepts k/m/g suffixes for the -S and -s options.

 - Install/Upgrade process changes:
    o Completed support for DUID disk installs, and enabled it fully.
    o Install non-free firmwares from the internet upon first boot, based on a
      question in the installer.
    o svnd(4)-like behaviour became the default for vnd(4) devices.  This is
      what is used to build the media.

 - rc.d(8) framework improvements:
    o rc.d(8) is now also used for the base system daemons.
    o Backward compatible with the historic way of starting daemons.
    o Notify the user by appending (ok) or (failed) in interactive mode.
    o Better diagnostics with the introduction of RC_DEBUG.

 - OpenSSH 5.9:
    o New features:
      - Introduce sandboxing of the pre-auth privsep child using an
        optional sshd_config(5) "UsePrivilegeSeparation=sandbox" mode
        that enables mandatory restrictions on the syscalls the privsep
        child can perform.
      - Add new SHA256-based HMAC transport integrity modes from
        http://www.ietf.org/id/draft-dbider-sha2-mac-for-ssh-02.txt
        These modes are hmac-sha2-256, hmac-sha2-256-96, hmac-sha2-512,
        and hmac-sha2-512-96, and are available by default in ssh(1)
        and sshd(8).
      - The pre-authentication sshd(8) privilege separation slave process
        now logs via a socket shared with the master process, avoiding
        the need to maintain /dev/log inside the chroot.
      - ssh(1) now warns when a server refuses X11 forwarding.
      - sshd_config(5)'s AuthorizedKeysFile now accepts multiple paths,
        separated by whitespace. The undocumented AuthorizedKeysFile2
        option is deprecated (though the default for AuthorizedKeysFile
        includes .ssh/authorized_keys2).
      - sshd_config(5): similarly deprecate UserKnownHostsFile2 and
        GlobalKnownHostsFile2 by making UserKnownHostsFile and
        GlobalKnownHostsFile accept multiple options and default to
        include known_hosts2.
      - sshd_config(5)'s ControlPath option now expands %L to the host
        portion of the destination host name.
      - sshd_config(5) "Host" options now support negated Host matching.
      - sshd_config(5): a new RequestTTY option provides control over
        when a TTY is requested for a connection, similar to the existing
        -t/-tt/-T ssh(1) commandline options.
      - ssh-keygen(1): Add -A option. For each of the key types (rsa1,
        rsa, dsa and ecdsa) for which host keys do not exist, generate
        the host keys with the default key file path, an empty passphrase,
        default bits for the key type, and default comment. This is useful
        for system initialisation scripts.
      - ssh(1): Allow graceful shutdown of multiplexing: request that
        mux server removes its listener socket and refuse future
        multiplexing requests but don't kill existing connections. This
        may be requested using "ssh -O stop ...".
      - ssh-add(1): now accepts keys piped from standard input.
      - Retain key comments when loading v.2 keys. These will be visible
        in "ssh-add -l" and other places. (bz#439)
      - ssh(1) and sshd(8): set IPv6 traffic class from IPQoS (as well as
        IPv4 ToS/DSCP). (bz#1855)
    o The following significant bugs have been fixed in this
      release:
      - sshd(8): allow GSSAPI authentication to detect when a server-side
        failure causes authentication failure and don't count such failures
        against MaxAuthTries. (bz#1244)
      - ssh-keysign(8): now signs hostbased authentication challenges
        correctly using ECDSA keys. (bz#1858)

 - Over 7,200 ports, major robustness and speed improvements in package tools.
 - Many pre-built packages for each architecture:
    o i386: 7008                      o sparc64: 6456
    o alpha: 6046                     o sh: 3721
    o amd64: 6960                     o powerpc: 6691
    o sparc: 3277                     o arm: 2963
    o hppa: 6125                      o vax: 1409
    o mips64: 5689                    o mips64el: 5709

 - Some highlights:
    o Gnome 2.32.2                    o KDE 3.5.10
    o Xfce 4.8.0                      o MySQL 5.1.54
    o PostgreSQL 9.0.5                o Postfix 2.8.4
    o OpenLDAP 2.3.43 and 2.4.25      o Mozilla Firefox 3.5.19, 3.6.18 and 5.0
    o Mozilla Thunderbird 5.0         o GHC 7.0.4
    o LibreOffice 3.4.1.3             o Emacs 21.4, 22.3 and 23.3
    o Vim 7.3.154                     o PHP 5.2.17 and 5.3.6
    o Python 2.4.6, 2.5.4 and 2.7.1   o Ruby 1.8.7.352 and 1.9.2.200
    o Mono 2.10.2                     o Chromium 12.0.742.122
    o Groff 1.21

 - As usual, steady improvements in manual pages and other documentation.
    o Base system and Xenocara manuals are now installed as source code,
      making grep(1) more useful in /usr/share/man/ and /usr/X11R6/man/.
    o If both formatted and source versions of manuals are installed,
      man(1) automatically displays the newer version of each page.

 - The system includes the following major components from outside suppliers:
    o Xenocara (based on X.Org 7.6 with xserver 1.9 + patches,
      freetype 2.4.5, fontconfig 2.8.0, Mesa 7.8.2, xterm 270,
      xkeyboard-config 2.3 and more)
    o Gcc 2.95.3 (+ patches), 3.3.5 (+ patches) and 4.2.1 (+patches)
    o Perl 5.12.2 (+ patches)
    o Our improved and secured version of Apache 1.3, with
      SSL/TLS and DSO support
    o OpenSSL 1.0.0a (+ patches)
    o Sendmail 8.14.5, with libmilter
    o Bind 9.4.2-P2 (+ patches)
    o Lynx 2.8.7rel.2 with HTTPS and IPv6 support (+ patches)
    o Sudo 1.7.2p8
    o Ncurses 5.7
    o Heimdal 0.7.2 (+ patches)
    o Arla 0.35.7
    o Binutils 2.15 (+ patches)
    o Gdb 6.3 (+ patches)

If you'd like to see a list of what has changed between OpenBSD 4.9
and 5.0, look at

        http://www.OpenBSD.org/plus50.html

Even though the list is a summary of the most important changes
made to OpenBSD, it still is a very very long list.

------------------------------------------------------------------------
- SECURITY AND ERRATA --------------------------------------------------

We provide patches for known security threats and other important
issues discovered after each CD release.  As usual, between the
creation of the OpenBSD 5.0 FTP/CD-ROM binaries and the actual 4.9
release date, our team found and fixed some new reliability problems
(note: most are minor and in subsystems that are not enabled by
default).  Our continued research into security means we will find
new security problems -- and we always provide patches as soon as
possible.  Therefore, we advise regular visits to

        http://www.OpenBSD.org/security.html
and
        http://www.OpenBSD.org/errata.html

Security patch announcements are sent to the security-announce@OpenBSD.org
mailing list.  For information on OpenBSD mailing lists, please see:

        http://www.OpenBSD.org/mail.html

------------------------------------------------------------------------
- CD-ROM SALES ---------------------------------------------------------

OpenBSD 5.0 is also available on CD-ROM.  The 3-CD set costs $50 CDN and
is available via mail order and from a number of contacts around the
world.  The set includes a colourful booklet which carefully explains the
installation of OpenBSD.  A new set of cute little stickers is also
included (sorry, but our FTP mirror sites do not support STP, the Sticker
Transfer Protocol).  As an added bonus, the second CD contains an audio
track, a song entitled "What Me Worry?".  MP3 and OGG versions of
the audio track can be found on the first CD.

Lyrics (and an explanation) for the songs may be found at:

    http://www.OpenBSD.org/lyrics.html#50

Profits from CD sales are the primary income source for the OpenBSD
project -- in essence selling these CD-ROM units ensures that OpenBSD
will continue to make another release six months from now.

The OpenBSD 5.0 CD-ROMs are bootable on the following four platforms:

  o i386
  o amd64
  o macppc
  o sparc64

(Other platforms must boot from floppy, network, or other method).

For more information on ordering CD-ROMs, see:

        http://www.OpenBSD.org/orders.html

The above web page lists a number of places where OpenBSD CD-ROMs
can be purchased from.  For our default mail order, go directly to:

        https://https.OpenBSD.org/cgi-bin/order

All of our developers strongly urge you to buy a CD-ROM and support
our future efforts.  Additionally, donations to the project are
highly appreciated, as described in more detail at:

        http://www.OpenBSD.org/goals.html#funding

------------------------------------------------------------------------
- OPENBSD FOUNDATION ---------------------------------------------------

For those unable to make their contributions as straightforward gifts,
the OpenBSD Foundation (http://www.openbsdfoundation.org) is a Canadian
not-for-profit corporation that can accept larger contributions and
issue receipts.  In some situations, their receipt may qualify as a
business expense write-off, so this is certainly a consideration for
some organizations or businesses.  There may also be exposure benefits
since the Foundation may be interested in participating in press releases.
In turn, the Foundation then uses these contributions to assist OpenBSD's
infrastructure needs.  Contact the foundation directors at
directors@openbsdfoundation.org for more information.

------------------------------------------------------------------------
- T-SHIRT SALES --------------------------------------------------------

The OpenBSD distribution companies also sell tshirts and polo shirts.
And our users like them, too.  We have a variety of shirts available,
with the new and old designs, from our web ordering system at, as
described above.

-----------------------------------------------------------------------
- FTP INSTALLS ---------------------------------------------------------

If you choose not to buy an OpenBSD CD-ROM, OpenBSD can be easily
installed via FTP or HTTP downloads.  Typically you need a single
small piece of boot media (e.g., a boot floppy) and then the rest
of the files can be installed from a number of locations, including
directly off the Internet.  Follow this simple set of instructions
to ensure that you find all of the documentation you will need
while performing an install via FTP or HTTP.  With the CD-ROMs,
the necessary documentation is easier to find.

1) Read either of the following two files for a list of ftp/http
   mirrors which provide OpenBSD, then choose one near you:

        http://www.OpenBSD.org/ftp.html
        ftp://ftp.OpenBSD.org/pub/OpenBSD/5.0/ftplist

   As of Nov 1, 2011, the following ftp mirror sites have the 5.0 release:

        ftp://ftp.eu.openbsd.org/pub/OpenBSD/5.0/       Stockholm, Sweden
        ftp://ftp.bytemine.net/pub/OpenBSD/5.0/         Oldenburg, Germany
        ftp://ftp.ch.openbsd.org/pub/OpenBSD/5.0/       Zurich, Switzerland
        ftp://ftp.fr.openbsd.org/pub/OpenBSD/5.0/       Paris, France
        ftp://ftp5.eu.openbsd.org/pub/OpenBSD/5.0/      Vienna, Austria
        ftp://mirror.aarnet.edu.au/pub/OpenBSD/5.0/     Brisbane, Australia
        ftp://ftp.usa.openbsd.org/pub/OpenBSD/5.0/      CO, USA
        ftp://ftp5.usa.openbsd.org/pub/OpenBSD/5.0/     CA, USA
        ftp://obsd.cec.mtu.edu/pub/OpenBSD/5.0/         Michigan, USA

        The release is also available at the master site:

        ftp://ftp.openbsd.org/pub/OpenBSD/5.0/          Alberta, Canada

        However it is strongly suggested you use a mirror.

   Other mirror sites may take a day or two to update.

2) Connect to that ftp mirror site and go into the directory
   pub/OpenBSD/5.0/ which contains these files and directories.
   This is a list of what you will see:

        ANNOUNCEMENT     armish/          mvme68k/         sparc64/
        Changelogs/      ftplist          mvme88k/         src.tar.gz
        HARDWARE         hp300/           packages/        sys.tar.gz
        PACKAGES         hppa/            ports.tar.gz     tools/
        PORTS            i386/            root.mail        vax/
        README           landisk/         sgi/             xenocara.tar.gz
        alpha/           mac68k/          socppc/          zaurus/
        amd64/           macppc/          sparc/

   It is quite likely that you will want at LEAST the following
   files which apply to all the architectures OpenBSD supports.

        README          - generic README
        HARDWARE        - list of hardware we support
        PORTS           - description of our "ports" tree
        PACKAGES        - description of pre-compiled packages
        root.mail       - a copy of root's mail at initial login.
                          (This is really worthwhile reading).

3) Read the README file.  It is short, and a quick read will make
   sure you understand what else you need to fetch.

4) Next, go into the directory that applies to your architecture,
   for example, i386.  This is a list of what you will see:

        INSTALL.i386    cd50.iso        floppyB50.fs    pxeboot*
        INSTALL.linux   cdboot*         floppyC50.fs    xbase50.tgz
        MD5             cdbr*           game50.tgz      xetc50.tgz
        base50.tgz      cdemu50.iso     index.txt       xfont50.tgz
        bsd*            comp50.tgz      install50.iso   xserv50.tgz
        bsd.mp*         etc50.tgz       man50.tgz       xshare50.tgz
        bsd.rd*         floppy50.fs     misc50.tgz

   If you are new to OpenBSD, fetch _at least_ the file INSTALL.i386
   and the appropriate floppy*.fs or install50.iso files.  Consult the
   INSTALL.i386 file if you don't know which of the floppy images
   you need (or simply fetch all of them).

   If you use the install50.iso file (roughly 250MB in size), then you
   do not need the various *.tgz files since they are contained on that
   one-step ISO-format install CD.

5) If you are an expert, follow the instructions in the file called
   README; otherwise, use the more complete instructions in the
   file called INSTALL.i386.  INSTALL.i386 may tell you that you
   need to fetch other files.

6) Just in case, take a peek at:

        http://www.OpenBSD.org/errata.html

   This is the page where we talk about the mistakes we made while
   creating the 5.0 release, or the significant bugs we fixed
   post-release which we think our users should have fixes for.
   Patches and workarounds are clearly described there.

Note: If you end up needing to write a raw floppy using Windows,
      you can use "fdimage.exe" located in the pub/OpenBSD/5.0/tools
      directory to do so.

------------------------------------------------------------------------
- X.ORG FOR MOST ARCHITECTURES -----------------------------------------

X.Org has been integrated more closely into the system.  This release
contains X.Org 7.6.  Most of our architectures ship with X.Org, including
amd64, sparc, sparc64 and macppc.  During installation, you can install
X.Org quite easily.  Be sure to try out xdm(1) and see how we have
customized it for OpenBSD.

------------------------------------------------------------------------
- PORTS TREE -----------------------------------------------------------

The OpenBSD ports tree contains automated instructions for building
third party software.  The software has been verified to build and
run on the various OpenBSD architectures.  The 5.0 ports collection,
including many of the distribution files, is included on the 3-CD
set.  Please see the PORTS file for more information.

Note: some of the most popular ports, e.g., the Apache web server
and several X applications, come standard with OpenBSD.  Also, many
popular ports have been pre-compiled for those who do not desire
to build their own binaries (see BINARY PACKAGES, below).

------------------------------------------------------------------------
- BINARY PACKAGES WE PROVIDE -------------------------------------------

A large number of binary packages are provided.  Please see the PACKAGES
file (ftp://ftp.OpenBSD.org/pub/OpenBSD/5.0/PACKAGES) for more details.

------------------------------------------------------------------------
- SYSTEM SOURCE CODE ---------------------------------------------------

The CD-ROMs contain source code for all the subsystems explained
above, and the README (ftp://ftp.OpenBSD.org/pub/OpenBSD/5.0/README)
file explains how to deal with these source files.  For those who
are doing an FTP install, the source code for all four subsystems
can be found in the pub/OpenBSD/5.0/ directory:

        xenocara.tar.gz     ports.tar.gz   src.tar.gz     sys.tar.gz

------------------------------------------------------------------------
- THANKS ---------------------------------------------------------------

Ports tree and package building by Jasper Lievisse Adriaanse,
Landry Breuil, Michael Erdely, Stuart Henderson, Peter Hessler,
Paul Irofti, Antoine Jacoutot, Robert Nagy, and Christian Weisgerber.
System builds by Theo de Raadt, Mark Kettenis, and Miod Vallat.
X11 builds by Todd Fries and Miod Vallat.  ISO-9660 filesystem
layout by Theo de Raadt.

We would like to thank all of the people who sent in bug reports, bug
fixes, donation cheques, and hardware that we use.  We would also like
to thank those who pre-ordered the 5.0 CD-ROM or bought our previous
CD-ROMs.  Those who did not support us financially have still helped
us with our goal of improving the quality of the software.

Our developers are:

    Alexander Bluhm, Alexander Hall, Alexander Schrijver,
    Alexander Yurchenko, Alexandr Shadchin, Alexandre Ratchov,
    Anil Madhavapeddy, Anthony J. Bentley, Antoine Jacoutot,
    Ariane van der Steldt, Austin Hook, Benoit Lecocq, Bernd Ahlers,
    Bob Beck, Bret Lambert, Charles Longeau, Chris Kuethe,
    Christian Weisgerber, Christiano F. Haesbaert, Claudio Jeker,
    Dale Rahn, Damien Bergamini, Damien Miller, Darren Tucker,
    David Coppa, David Gwynne, David Hill, David Krause, Edd Barrett,
    Eric Faurot, Federico G. Schwindt, Felix Kronlage, Gilles Chehade,
    Giovanni Bechis, Gleydson Soares, Henning Brauer, Ian Darwin,
    Igor Sobrado, Ingo Schwarze, Jacek Masiulaniec, Jakob Schlyter,
    Janne Johansson, Jason George, Jason McIntyre, Jason Meltzer,
    Jasper Lievisse Adriaanse, Jeremy Evans, Jim Razmus II, Joel Sing,
    Joerg Zinke, Jolan Luff, Jonathan Armani, Jonathan Gray,
    Jonathan Matthew, Jordan Hargrave, Joshua Stein,
    Kenneth R Westerback, Kevin Lo, Kevin Steves, Kurt Miller,
    Landry Breuil, Laurent Fanis, Marc Espie, Marco Peereboom,
    Marco Pfatschbacher, Marcus Glocker, Mark Kettenis, Mark Lumsden,
    Mark Uemura, Markus Friedl, Martin Pieuchot, Martynas Venckus,
    Mats O Jansson, Matthew Dempsky, Matthias Kilian, Matthieu Herrb,
    Michael Erdely, Mike Belopuhov, Mike Larkin, Miod Vallat,
    Nayden Markatchev, Nicholas Marriott, Nick Holland, Nigel Taylor,
    Nikolay Sturm, Okan Demirmen, Otto Moerbeek, Owain Ainsworth,
    Paul de Weerd, Paul Irofti, Peter Hessler, Peter Valchev,
    Philip Guenther, Pierre-Emmanuel Andre, Pierre-Yves Ritschard,
    Remi Pointel, Reyk Floeter, Robert Nagy, Ryan Freeman,
    Ryan Thomas McBride, Sasano, Sebastian Reitenbach, Simon Bertrang,
    Stefan Sperling, Stephan A. Rickauer, Steven Mestdagh,
    Stuart Henderson, Takuya Asada, Ted Unangst, Theo de Raadt,
    Thordur I Bjornsson, Tobias Weingartner, Todd C. Miller, Todd Fries,
    Will Maier, William Yodlowsky, Yasuoka Masahiko, Yojiro Uo

I noticed that I did get alot alerts from op5 Monitor complaining about high error rate on the external network. After some investigation I noticed that UDP port 5060 generated approx 1.5 Mbps in traffic and that is more or less maximum my ADSL connection can handle. UDP port 5060 that is SIP.

A nice graph showing the errorrates, generated by op5 Monitor:

I looked into my Asterisk log:

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

Conclusion

Someone from 50.97.142.134 tries to register their SIP device on my Asterisk server, they do it an abnormal high rate.

Reaction

I created a block in my firewall on everything from 50.97.142.134. Unfortunatly it does not help much because it is on the wrong side of the ADSL connection. But I get rid of the handshaking and filled logs.

A whois search showed that the traffic comes from Softlayer in Dallas, so I wrote an email to postmaster@softlayer.com.

Lets see if I get any reaction

I have a long time looked at Telldus Tellstick that can control wall-plug socket recievers and other devices. Finally I got my hands on one that my collegue wanted to sell.

This article will describe how to get it up and running on a x86_64 CentOS 6 box, the procedure would be the same on RHEL or Feodora and similiar on other Linux distributions. Unfortunatly the only distribution that has pre-made packages is Ubuntu so I will describe howto download the source code, compile it and get the amazing Telldus Tellstick up and running.

Kudos to Telldus that has understood the power of OpenSource and opened up the device to let third-part  vendors make solutions based on the device.

Installation

Install packages needed for the compilation

yum install gcc gcc-c++ autoconf automake cmake

Libconfuse is needed: 

wget http://pkgs.repoforge.org/libconfuse/libconfuse-2.6-2.el6.rf.x86_64.rpm

wget http://pkgs.repoforge.org/libconfuse/libconfuse-devel-2.6-2.el6.rf.x86_64.rpm

Install libconfuse:

rpm -Uvh libconfuse-*

warning: libconfuse-2.6-2.el6.rf.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 6b8d79e6: NOKEY

Preparing...                ########################################### [100%]

   1:libconfuse             ########################################### [ 50%]

   2:libconfuse-devel       ########################################### [100%]

Download, configure, compile, and install the Tellstick package from Telldus 

Configure

Useful links:

• Telldus
• Different Telldus Tellsticks
• Supported devices

As a monitoring fanatic, I think that have mirrored disk without monitor the mirror is worse then not have a redundant disk system at all. If you think that you are safe and you are not, is worse then be aware of a risk and calculate with it.

This article will describe how I did monitoring my FreeNAS system mirrored disks status using Nagios or op5 Monitoring. I assume basic knowledge of Nagios or op5 Monitor and that a Nagios agent is installed.

Demarcation

This article will focus on GEOM raid monitoring. Of cource all important services should be monitored, like HTTP, CIFS, SMB, disk space, CPU Load, memory usage and so on. However that is basic monitoring knowledge and there is alot of documentation on the net howto achive that.

Monitor the GEOM mirror

The most interesting part is to monitor the GEOM mirror to make sure that that the raid1 is working. 

After some googleing I found check_geom, downloaded it and put it in /usr/local/libexec/nagios/ where the other plugins are located.

Add a line to /etc/nrpe.cfg

command[check_geom_mirror]=/usr/local/libexec/nagios/check_geom mirror

and test from Nagios/op5 Monitor host:

# ./check_nrpe -H fnas -c check_geom_mirror
OK mirror  - mirror/2TBmirror0 COMPLETE { ad6 , ad4 }

It works!
 

Configure Nagios or op5 Monitor

In /opt/monitor/etc/services.cfg at the op5 Monitor host

# service 'Disk Raid Mirrors'
define service{
    use                            default-service
    host_name                      fnas
    service_description            Disk Raid Mirrors
    check_command                  check_nrpe!check_geom_mirror
    contact_groups                 it-slav_mail,it-slav_jabber
    }

The result

I tested unplugging the SATA cabel to one of the mirrored disk and after a couple of minutes I got an Critical notification that my disk mirror is broken, I plugged SATA cabel and got a Warning:

After a couple of hours the mirror was OK.

Useful links:

• Previous article describing howto install nagios or op5 Monitor agent on FreeNAS
• op5, an Enterprise Monitor product company
• op5 Monitor, Enterprise Monitoring based on opensource
• Nagios, an excellent opensource monitor tool
• FreeNAS, very capable NAS based on FreeBSD

My old NAS built on top of CentOS has some performence issues and the disks are quit old now. The hardware is leftovers that is approx 8 years old and not highend at that time either. The disks age has worried my for a time and I would like to have higher redunduncy with raid1/mirrored disks.

After some research I decided to run my new NAS on FreeNAS, which has all the features I use, like: CIFS, NFS, FTP, SSH, Rsync, Unison, iSCSI and also easy administration and disk management.

This article will describe how I did setup Nagios or op5 Monitoring on my FreeNAS system. I assume basic knowledge of Nagios or op5 Monitor.

Installing nrpe

Nrpe is the agent that nagios and op5 Monitor uses to monitor a system from the inside.

From the webgui add group nagios and user nagios. User nagios should be a meber of group nagios.

 freenas:~# pkg_add -r nrpe2
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/Latest/nrpe2.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/All/perl-5.10.1.tbz... Done.
Removing stale symlinks from /usr/bin...
    Skipping /usr/bin/perl
    Skipping /usr/bin/perl5
Done.
Creating various symlinks in /usr/bin...
    Symlinking /usr/local/bin/perl5.10.1 to /usr/bin/perl
    Symlinking /usr/local/bin/perl5.10.1 to /usr/bin/perl5
Done.
cd: can't cd to /usr/include
Cleaning up /etc/make.conf... Done.
Spamming /etc/make.conf... Done.
Cleaning up /etc/manpath.config... Done.
Spamming /etc/manpath.config... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/All/libiconv-1.13.1_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/All/gettext-0.17_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/All/nagios-plugins-1.4.14,1.tbz... Done.
You already have a "nagios" group, so I will use it.
You already have a "nagios" user, so I will use it.

**********************************************************************
 

freenas:~# cp /usr/local/etc/nrpe.cfg-sample /etc/nrpe.cfg

I changed the allowed_hosts line and added my op5 Monitor host.

So the nagios agent works.

Links

• op5, an Enterprise Monitor product company
• op5 Monitor, Enterprise Monitoring based on opensource
• Nagios, an excellent opensource monitor tool
• FreeNAS, very capable NAS based on FreeBSD

Before installing, read the release notes

The widget keeps track of how many times the word "moln" appear on idg.se webpage and displays it in a easy view in The Tactical Overview.

Screenshoots from the different modes:

Links:

• op5 Sandbox with a live cloud widget online, registration is necessary
• Ninja, the new and inovative nagios gui
• Widget exchange area

Kudos to op5 support that has created this excellent widget 

It is very handy to have a temprobe attached to network devices like a Wlan router, if OpenWRT is installed it is easy to use 1-wire devices. By using this method it is possible to create wireless temperature probes that uses Wlan.

Install

root@OpenWrt:~# opkg install owserver owfs         
 

Using

root@OpenWrt:~# owserver -u -C --usb_regulartime -p 3001

root@OpenWrt:~# mkdir /mnt/1-wire         


root@OpenWrt:~# owfs -s localhost:3001 -m /mnt/1-wire

root@OpenWrt:/mnt/1-wire/10.87507C010800# ls /mnt/1-wire/
10.87507C010800  81.543A2A000000  alarm            bus.0            settings         simultaneous     statistics       structure        system           uncached

It works!         
 

To make sure that owserver starts when the router is rebooted, add to /etc/rc.local


/usr/bin/owserver -u -C --usb_regulartime -p 3001

Firewall

My op5 Monitor/Nagios system is on the Wan side of the OpeWRT box so I need to open the firewall to port 3001.

Add to /etc/config/firewall

#1-wire ok on Wan
config rule
    option src        wan
    option dest_port    3001
    option target        ACCEPT
    option proto        tcp
 

Result

To gather the data from 1-wire I use op5 Monitor and my Nagios plugin check_1-wiretemp, the result can be seen below

Links

• OpenWrt
• owfs
• op5 Monitor

It is real simple, just add to following to /etc/config/firewall:

#open ssh on wan interface
config rule               
        option src              wan
        option dest_port        22
        option target           ACCEPT    
        option proto            tcp 

And restart the firewall:

#/etc/init.d/firewall restart

After struggeling with my Linksys WRT-160NL to get OpenWRT stable I finally gave up. It ended upp with DD-WRT instead. Unfortunatly DD-WRT do not have the functionality I want like:

-rndis device so I could use my Android phone to connect to Internet

-1-wire device support

-Nagios agent

The solution was to buy another device where OpenWRT support is working better with USB and Gigabit interfaces. After some research I found TP-Link TL-WR1043ND for approx 50€.

Unpacking

The TL-WR1043ND package is good looking and has a printed GPL license included, that is impressive.

The basic installation with TP-LINK installed firware was easy and stright forward with all information you need in a simple "Quick installation Guide". No need for installing software on a windows machine to be able to access you new device. I think Cisco/Linksys has something to learn.

Installing OpenWRT

Find the Firmware upgrade menu and upload this firmware. It worked perfectly well and seems very stable. A positive suprise is that the WLAN seems to reach longer then the WRT 160NL.

Conclusion

The TP-LINK TL-WR1043ND seems to be a good Wlan router that has a good support for OpenWRT, with USB, Gigabit ethernetinterfaces and good Wlan range.

Background

I have a need of GPS tag my digital photos. Unfortunatly a GPS device to my camera costs approx 230 Eur. In my humble opinion I think that is way of, especially considering that with some software in the camera any standard USB GPS reciever would do the job. They cost approx 40 Eur.

I have an Android phone where there is tons of apps that could create a GPS log whenever I want to, however it drains the battery in just a couple of hours. I want to track my routes at least a whole day without the need to charge.

The solution is to buy a GPS logger and after some research I found Ventus GPS Route Logger G730, and it looked like it would work with Linux.

This article will describe my experience making the G730 Ventus GPS Route Logger and Linux.

Using skytraq

According to webpages skytraq can be used to create GPX tracks from the G730 Logger.

Unfortunatly the skytraq software are not in the Ubuntu repos so I had to download and install it manually. It is downloadable from  http://code.google.com/p/skytraq-datalogger/

It is out of the scope for this article how to compile and fullfill the requirements needs.

When the GPS Logger is attached the following will show up in /var/log/messages:

Jan 15 14:13:08 it-slav kernel: [21689.101101] usb 6-2: new full speed USB device using uhci_hcd and address 3
Jan 15 14:13:09 it-slav kernel: [21689.316181] usb 6-2: configuration #1 chosen from 1 choice
Jan 15 14:13:09 it-slav kernel: [21689.319138] pl2303 6-2:1.0: pl2303 converter detected
Jan 15 14:13:09 it-slav kernel: [21689.332192] usb 6-2: pl2303 converter now attached to ttyUSB0

So now we can communicate with the GPS Logger:

peter@it-slav:~/dl/skytraq/skytraq-datalogger-0.5-1$ ./skytraq-datalogger --info
kernel version: 1.4.8 -- ODM version: 1.8.22 -- revision: 2008-10-23
log_wr_ptr:      28626
total sectors:   510
sectors left:    506
max time:        3600 s
min time:        1 s
max distance:    1000 m
min distance:    0 m
max speed:       1000 km/h
min speed:       0 km/h
datalog enable:  1
log fifo mode:   1
AGPS enabled:    0
AGPS data left:  none
baud-rate:       38400 bps

Get the gpx file:

./skytraq-datalogger --dump >20110310.gpx

And now a gpx file named 20110310.gpx is created.

Setup

There is a possibility to conifigure the behvior of the GPS logger with skytraq, however it is possible to configure more parameters in the Ventus G730 so to fully configure I had to install the windows software that come with the logger.

The configuration is rather strange, there is a possibilty to log per time, over a certain speed and distance for the following:

• General
• Walking
• Car
• Bicycle

How the device knows what type of vehicle I use for traveling and if the General settings override the others is undocumented.

Problems

Most of the time when trying to communicate with the logger it gives this message:

peter@it-slav:~/dl/skytraq/skytraq-datalogger-0.5-1$ ./skytraq-datalogger --info
No response from datalogger.

And after a few seconds it works again. Sometimes unplugging the device and connect it again helps. The behavior is very stochastic and some patience will help:)

The logger can be used as a GPS recever and the output is in standard NMEA format that GPSd understand. So if you want to use it together with your favorite GPS software it works. However my experience is that if you want to dump the output from the G730 Logger the GPS output must be turned of:

./skytraq-datalogger --set-output-off

Suggestion

I suggest that Ventus should open the specification howto communicat with the device so third part software could use the device. To hide that type of information do not provide any advantage, rather the opposite.

I wrote an email to them and the correspondence can be seen below. I hope this blog post will make Ventus change their mind.

–cut–

-------- Original Message --------
Subject: Sv: Ventus g730
Date: Mon, 3 Jan 2011 13:25:49 +0800
From: <support@ventusdesign.com>
To: Peter Andersson <peter@it-slav.net>
CC: Ventus <info@ventusdesign.com>

> Hi
>
> I have just bought a Ventus g730 GPS Logger and I have some feedback.
> Before buying the device I made sure that it fullfilled my requriements:
> -Works with linux
> (http://www.petrilopia.net/wordpress/hardware/ventus-gps-route-logger-g730-linux/)
> -Create gpx files
> -Battery that last at least 12 hours
>
> Unfortunately you have have taken a strange decision and do not support
> Linux. However according to several forums the G730 is compatible with
> http://code.google.com/p/skytraq-datalogger/
> After fiddling with the device I found that the skytraq-datalogger
> sometimes works and sometimes does not. And I probably will return the
> device to the store if I cannot get it to work better with an open and
> free operating system.
>
> My recommendations to you are:
> 1-Provide an open specification how to access the data in the device
> 2-Modify the http://code.google.com/p/skytraq-datalogger/ software so it
> works with G730 and provide the code change to the team behind
> skytraq-datalogger, and also provide the software on you website.
>
> By following these simple steps (especially #1) you will sell a-lot of
> more devices to the people that prefer open and freedom.
>
> Best regards
> Peter Andersson
> peter@it-slav.net
 

The dashboard has been modified and improved by the community and an example can be seen at op5 sandbox dashboard and can be downloaded from op5 community.

The original project place is http://dingleberry.me/2010/04/our-new-dashboard/

News in op5 Monitor 5.3

• Avail and SLA reports including performance graphs

   

op5 Monitor 5.3 introduce the possibility to include performance graphs in avail reports. This mean you can, for example, create a response time report and not only get the availability but a graphical representation of the actual response times. This will, for instance, help the user in capacity planing since the graphs may present trends that show when disks are full, response times exceed threshold etc.

• Avail and SLA reports including summary report

The avail and SLA reports can now also include a Summary report. There are a number of different Summary reports available. For example, Top Alert Producers and Most Resent Alerts. This mean the report can display what service is causing the most alerts or at what time the respective services breached warning or critical thresholds hence providing a more fine grained view of the report period.

• Automatic Network scan

The Configuration tool in op5 Monitor have for a long time had the ability to scan a network and discover hosts that are not monitored, This scan can now be scheduled to run each night. When the scan discovers one or more new hosts the user is notified and presented with a link leading to the configuration tool. This mean you can minimize the risk of forgetting to monitor hosts added to your network.

• User interface performance improvements

A lot of effort have been put into improving the overall performance of the user interface. This involve indexes in the database as well as optimizing the database queries. Depending on the environment and what pages are used the user can expect an performance improvement of 50% overall and in some cases, host/service group pages in particular, several times more.

More Information about op5 Monitor 5.3

• Release Note for op5 Monitor 5.3
• Change Logs
• Product Roadmaps
• SandBox Live Demo