It really annoyed me that the iowait at the cpu on the Dom 0 system was approximatly 50% when the 4 virtual system was doing more or less nothing.

I showed my earlier blogpost about VM virtualization to our sysadmin at op5. He gave me a couple of more hints that really make a big difference, the best part is that they are real simple to implement.

I also noticed that I forgot some parts in the previous article that was obvius to me.

#1 use virtio drivers if possible

This is probably the single most important step to get the most out of the hardware.

When the virtual systems wants to communicate with the hardware it uses drivers, if the drivers are simulating hardware to give a service to the virtual operating system it adds alot of overhed. A simple example:

• The virtual system wants to access the disk
• The virtual system communicates via the driver to the hardware
• The hardware is not real hardware, it is emulated hardware in sofware 
• The software that emulates the hardware sends the I/O to the Dom 0 virtualization engine
• The Virtualization engine verifies that the access is ok, translate it to real hardware call via its driver
• The hardware do the operation and send the result back up via all the steps to the virtual operating system

All this transalations and software calls take alot of resources from the machine.

By using virtual drivers, so that there is no need to emulate hardware, many of the steps can be skipped or done with much less penalty. However not all operating system have virtio drivers.

#2 load the virtio modules in Dom 0

This is a real god hint I got from our sysadmin, load the virtual drivers in the Dom 0 system asweell. According to him, the virtual drivers in the Dom 0 and Dom U:s communicate and makes the I/O much more effective. I have not found any information about this at any place so I was sceptical but I tried it.

# modprobe virtio_balloon
# modprobe virtio_blk
# modprobe virtio_net
# modprobe virtio_pci

The virtual systems need to be rebooted as far as I know before the loaded drivers will have any effect.

At first no change at all, but after few hours I noticed that CPU idle raized and the iowait went from at least 50% down to below 5%.

A nice graph, created by op5 Monitor showing the difference:

I loaded the virtual drivers approximately 4 pm, and it took until 10 pm before the performance boost showed up.

The correct way of making sure that the drivers are loaded directly would be to remake the kernel but I'm lazy so I have just added the module load to /etc/rc.local

Conclusions

It is always important to monitor systems. When doing changes it is not possible know if the changes do have any impact if you do not monitor and follow up how and if the changes does anything.

My former statement that KVM is not mature is more true then I have thought. The knowledge how to tune and maintain KVM systems seems to be like black art where the knowledge is not documented and well spread. I have read the RedHat virtualization and I have not found anything about that the virtual drivers should be loaded on the Dom0 system.

If there is anyone out there that has some documentation or more hints, please send me the link or comment this blogpost.

Links:

• RedHat virtualization doc
• op5 Monitor that monitor and creates nice graphs
• check_cpu_stats Nagios plugin to monitor cpustat

It is very handy to have a temprobe attached to network devices like a Wlan router, if OpenWRT is installed it is easy to use 1-wire devices. By using this method it is possible to create wireless temperature probes that uses Wlan.

Install

root@OpenWrt:~# opkg install owserver owfs         
 

Using

root@OpenWrt:~# owserver -u -C --usb_regulartime -p 3001

root@OpenWrt:~# mkdir /mnt/1-wire         


root@OpenWrt:~# owfs -s localhost:3001 -m /mnt/1-wire

root@OpenWrt:/mnt/1-wire/10.87507C010800# ls /mnt/1-wire/
10.87507C010800  81.543A2A000000  alarm            bus.0            settings         simultaneous     statistics       structure        system           uncached

It works!         
 

To make sure that owserver starts when the router is rebooted, add to /etc/rc.local


/usr/bin/owserver -u -C --usb_regulartime -p 3001

Firewall

My op5 Monitor/Nagios system is on the Wan side of the OpeWRT box so I need to open the firewall to port 3001.

Add to /etc/config/firewall

#1-wire ok on Wan
config rule
    option src        wan
    option dest_port    3001
    option target        ACCEPT
    option proto        tcp
 

Result

To gather the data from 1-wire I use op5 Monitor and my Nagios plugin check_1-wiretemp, the result can be seen below

Links

• OpenWrt
• owfs
• op5 Monitor

When I was preparing a presentation about what op5 is doing and our contribution to the community, I went to ideas.nagios.org. When I browsed the list of the biggest issues with Nagios I found out that op5 has packaged and solved them all in op5 Monitor. I encourage everyone to take a peak at the list and judge for them self what platform you want to use for your enterprise monitoring solution.

1. Nagios Clusters

op5 has developed Merlin (Module for Effortless and Redundant and Loadbalanced Infrastructure with Nagios). It provides the possibility to have a redundant or/and loadbalanced solution using Nagios. It also provides a scalable database backend for Ninja and Nagvis, take a peak at Merlin webpage.

2. Performance Graphing

In op5 Monitor op5 has added pnp4nagios that graphs more or less any check that gives a numerical value back.

3. New Status Map

We provide a new statusmap, it is a part of Nagvis project and called Automap. op5 has contributed to Nagvis by adding support for Merlin database backend and GoogleMaps integration

4. Better Interface

op5 has created a new PHP based interface that makes you proud of your monitor solution, called Ninja. Ninja stands for Nagios is now Just Awesome, it looks so good so you can show it to your manager:-) Take a peak at Ninja webpage.

We had a naming competition at op5 my suggestion to call it Yang, Yet Another Nagios Gui, did not get so many votes so the name become Ninja.

5. Configuration

Any one that has tried to configure Nagios by hand has thought bad thoughts. It is cumbersome and in the begining it makes even a skilled Unix admin frustrated. op5 has developed a web based configuration tool named Nacoma. It is included in op5 Monitor.

6. SLA reports

op5 has got alot of feedback from the customers and one issue was better reports, we have created two reports: availability and SLA report. Both of them can be scheduled and looks god compared to the reports in Nagios, they are included in Ninja

 
7. More members in core team

Well, this is something that op5 cannot controll, however, Andreas Ericsson one of op5s core developers are a member og Nagios steering board.

8. SNMP Trap receiver

This is something op5 is working on right now, we have today created a prototype together with a customer and depending of the outcome of that project we will know better when it will be available to the market.

9. UI-Improvment

op5s solution to this issue is Ninja, a better, nicer, scalable, database backend gui to Nagios.

10. Nagios Dashboard

A newTactical Overview with widgets or Nagvis, all included in Ninja

Links

• ideas.nagios.org http://ideas.nagios.org
• Nagios http://www.nagios.org
• op5 http://www.op5.com
• op5 Monitor http://www.op5.com/op5/products/network-monitor
• Ninja http://www.op5.org/community/plugin-inventory/op5-projects/ninja
• Merlin http://www.op5.org/community/plugin-inventory/op5-projects/merlin
• Nagvis http://www.nagvis.org/
• pnp4nagios http://docs.pnp4nagios.org/

I have read the book "Cacti 0.8 Network Monitoring, Monitor your network with ease!" by Dinangkur Kundu and S.M.Ibrahim Lavlu. The book is 116 pages.

According to the book the target audience for this book is for anyone who wants to manage a network using Cacti. To read the book you do not need to be a Linux Guru. And I agree, even if it probably helps if you have basic knowledge of Linux/UNIX, RRD, MySQL and SNMP.

Many of the OpenSource software projects out there is excellent but usually require a rather skilled person or a very interested person with alot of time to start working. Downloading the software and start reading the manual that comes with the software is often quite hard and a book like this narrow that gap and make a complex software like Cacti easier to use.

I find the Cacti GUI sometimes very cumbersome and this book, with its step by step guides to setup, makes graphing of network attached devices easy. The book explains how Cacti works and the underlaying technology like RRD well. RRD is the excellent Round Robin Database and tool and it is created by Tobias Oetiker. What I’m lacking is that when it has described the basics, it stops.  I would really like a deeper explanation of RRD because Cacti is a graphing tool which uses RRD and a deeper knowledge of RRD would make your graphs better and give you the information you want.

I also lack the deeper explanation of advanced Cacti topics like SNMP query XML syntax and other topics.The book is only 116 pages and in my opinion the book would be much more valuable if 50 extra pages would have been used to dig deeper into more advanced topics and explain more extensive what al the options in Cacti means.

I really miss a howto for Cacti addons like wethermaps, weathermaps is taking Cacti another step in visualiation of complex data.

I would recommend this book for newbies into Cacti that will get ther first graphs implementation a boost, for anyone that has used Cacti I would go for the RRD homepage, Cacti homepage and Cacti forums, blogs and other sources on internet.

Links:

• To buy the book Cacti 0.8 Network Monitoring, Monitor your network with ease!
• An example Chapter 4, creating and using templates
• Cacti homepage
• Cacti Forum
• RRD tools
• op5 Statistics, a supported version of Cacti

I know for sure that the problems was not on my side, because a friends friend that lives in the same area had the same problem.

This graph shows the packet lost when pinging the closest router:

• Tobias Oitiker gave a very good presentation about the linux kernel caching and described how it affected RRDtools.
• Markus Almroth, describes Sysnagios which did the opposite of most monitoring projects. Sysnagios decentralized the configuration of systems and network monitoring in a controlled way. I will come back to this topic in my blog.
• Per Åsberg talked about Ninja, a new GUI for Nagios, done by op5.
• Andriy Skuratov, talked about and demonstrated a new version of Nagvis with integrated Google Maps and other good stuff.
• Ethan Galstad the founder of Nagios announced Ninja as the new GUI for Nagios.

During the Nordic meet on Nagios I meet alot of interesting people, learned alot and had good food and drinks.

I hope to see you next year at Nordic Meet on Nagios 2010!!!

WordPress uses the following parts to work:

• Webserver frontend
• PHP scripts
• MySQL database backend
• At least one system to run it on

I found one article describing howto monitor the Webserver and the MySQL database and even that a certain webpage contain a certain phrase. But I would like to emulate a user that clicks on some pages and graph the performance.

The tool I like for in-depth web monitoring is webinject. It allows scripting a simulated web user with alot of features.

I recommend to read the excellent how-to written by op5 support team and of cource use webinjects reference manual.

The testcase I’ve created looks like this:

<testcases repeat="1">
<case
    id="1"
    description1="Startsidan www.it-slav.net/blogs"
    method="get"
    url="http://www.it-slav.net/blogs/"
    verifypositive="An It-Slave in the digital saltmine"
/>
<case
    id="2"
    description1="Nagios or op5 Monitor plugin for 1-wire humidity measurement"
    method="get"
    url="http://www.it-slav.net/blogs/?p=802"
    verifypositive="1-wire humidity probe"
/>
<case
    id="3"
    description1="op5 Monitor or Nagios plugin for 1-wire temperature measurement"
    method="get"
    url="http://www.it-slav.net/blogs/?p=115"
    verifypositive="check_1-wiretemp"
/>
<case
    id="4"
    description1="Wp-Admin"
    method="get"
    url="http://www.it-slav.net/blogs/wp-admin"
    verifyresponsecode="301"
/>
</testcases>

And when run in Nagios mode:

[root@op5 webinject]# ./webinject.pl -c config.xml testcases/wordpressblog.xml
WebInject OK - All tests passed successfully in 4.143 seconds |time=4.143;10;;0

In checkcommands.cfg:

# command 'check_webinject'
define command{
    command_name                   check_webinject
    command_line                   $USER1$/webinject/webinject.pl testcases/$ARG1$
    }

In services.cfg:

# service 'Webinject Blog'
define service{
    use                            default-service
    host_name                      dull
    service_description            Webinject Blog
    check_command                  check_webinject!wordpressblog.xml
    contact_groups                 it-slav_sms,it-slav_mail,call_it-slav
    }

This is a nice pnp graph of my Webinject check:

Links:

• Webinject
• op5 Webinject Howto
• Nagios
• op5 Monitor

This article describe howto get a 1-wire humidity probe to work with Nagios or op5 Monitor.

In an earlier article I have described a 1-wire temperature plugin.

Pre requriments

To get it working you need:

• 1-wire humidity probe i.e. this, it is based on DS2438 & HIH-4000
• A working owfs installation, instructions can be found here
• Nagios or op5 Monitor

check_1-wirehumid

The plugin is written in perl and version 1.0 can be downloaded here

#!/usr/bin/perl -w
#
# Check humidity of 1Wire device
# Requires use of Fuse and owfs
#
#
# By Peter Andersson
# peter@it-slav.net
# http://www.it-slav.net/blogs/?p=802
# Licence GPLv2
# Version 1.0

use strict;
use Getopt::Std;
use OW;

my $owserver = "127.0.0.1:3001";
my(%ERRORS) = ( OK=>0, WARNING=>1, CRITICAL=>2, UNKNOWN=>3 );

my $humidity;
my $status=$ERRORS{OK};;
my $message;

my $debug_flag=0;

our($opt_c, $opt_w, $opt_W, $opt_C, $opt_h, $opt_o, $opt_i);

getopts("w:W:c:C:ho:i:");

sub printhelp () {
	print "Usage: check_1-wirehumid [-h] -c lowhumidcritical -w lowhumidwarning -W highhumidwarning -C highhumidcritical -i id [-o owserver:port]\n";
	print "-h Help, this text\n-c num Critical threshold for low humidity\n-w num Warning threshold for low humidity\n";
	print "-W num Warning threshold for high humidity\n-C num Critical threshold for high humidity\n";
	print "-o <name|ip>:port, Servername of 1-wire server and portnumber the owserver use, default 127.0.0.1:3001\n";
	print "-i 1-wire ID, i.e. 10.DEF05F01080015\n";
	print "\n\t\tby Peter Andersson\n\t\tpeter\@it-slav.net\n\t\thttp://www.it-slav.net/blogs/?p=802\n";
	exit $status;
}

#sanity check
if (!$opt_c||!$opt_w||!$opt_W||!$opt_C||$opt_h) {
	$status= $ERRORS{UNKNOWN};
	&printhelp;
} elsif ($opt_c > $opt_w) {
	print "Critical low threshold must be higher or equal to warning low threshold\n";
	$status= $ERRORS{UNKNOWN};
	&printhelp;
} elsif ($opt_w > $opt_W || $opt_c > $opt_C) {
	print "Lower tresholds must be lower then higher thresholds\n";
	$status= $ERRORS{UNKNOWN};
	&printhelp;
} elsif ($opt_C < $opt_W) {
	print "Higher critical threshold must be higher or equal to higher warning threshold\n";
	$status= $ERRORS{UNKNOWN};
	&printhelp;
}

if ($opt_o) {
	$owserver = $opt_o;
}

unless(OW::init($owserver)) {
	$status = $ERRORS{CRIT};
	$message = "OWServer not running at $owserver\n";
	exit $status;
}
$humidity = OW::get("$opt_i/humidity");
$humidity =~ s/^\s*(.*?)\s*$/$1/; #remove whitespaces
unless (($humidity =~ /^-?(?:\d+(?:\.\d*)?|\.\d+)$/) || ($humidity =~ /^[+-]?\d+$/)) #check that it is an integer or decimal returned
{
	$message="Did not got an integer or a decimal from humidity probe";
        $status=$ERRORS{CRITICAL};
}
$humidity = sprintf("%.0f", $humidity);

if ($debug_flag) {
	print "opt_c:$opt_c opt_w:$opt_w opt_W:$opt_W opt_C:$opt_C opt_h:$opt_h owserver:$owserver opt_i:$opt_i Humidity:$humidity\n";
}

if ($humidity <= $opt_c) {
	$status=$ERRORS{CRITICAL};
	$message="CRITICAL";
} elsif ($humidity > $opt_c && $humidity <= $opt_w) {
	$status=$ERRORS{WARNING};
	$message="WARNING";
} elsif ($humidity >  $opt_w && $humidity <  $opt_W) {
	$status=$ERRORS{OK};
	$message="OK";
} elsif ($humidity >= $opt_W && $humidity < $opt_C) {
	$status=$ERRORS{WARNING};
	$message="WARNING";
} elsif ($humidity >= $opt_C) {
	$status=$ERRORS{CRITICAL};
	$message="CRITICAL";
} else { #This should never happend
	$status=$ERRORS{UNKNOWN};
	$message="UNKNOW";
}

print "$message: $humidity \%\|humidity=$humidity\%\;$opt_c;$opt_w;$opt_W;$opt_C\n";
exit $status;

Test the plugin by run:

[root@op5 custom]# ./check_1-wirehumid -c 10 -w 20 -W 90 -C 95 -i 26.D7EABC000000
OK: 31 %|humidity=31%;10;20;90;95

So I have 31 % humidity.

Nagios config files

In checkcommands.cfg, define check_1-wirehumid

# command 'check_1-wirehumid'
define command{
    command_name                   check_1-wirehumid
    command_line                   $USER1$/custom/check_1-wirehumid -c $ARG1$ -w $ARG2$ -W$ARG3$ -C$ARG4$ -i $ARG5$ -o $ARG6$
    }

In serviced.cfg, define the check. I have a service group called Environemnt where all my Environments probes are.

# service 'Washroom Humidity'
define service{
    use                            default-service
    host_name                      monitor
    service_description            Washroom Humidity
    check_command                  check_1-wirehumid!10!20!80!90!26.D7EABC000000!127.0.0.1:3001
    servicegroups                  Environment
    contact_groups                 it-slav_msn,it-slav_mail,call_it-slav
    }

Screenshots

This is a screen shoot from op5 Monitor showing my Servicegroup Environment.

If you have op5 Monitor or PNP for Nagios, graphs like this will automatically be created. Note that I had a shower 6.20 and the humidity went from approx 25% to 95% in a couple of minutes.

One way of getting rich is buying the correct stock at the right time and price, and sell it after a while for much more. The chance of picking the ultimate stock is very low and I think it can be compared to winning a lottery. The probability of picking a not very bad stock at not that very bad time is higher if you have some knowledge and uses some tools.

I’m a computer geek that totally failed to study Economics at the university (I did that in parallel with my Master Of Science studies) mainly because it was not science, it was more of black magic.

So the conclusion is that I probably will not get rich by my knowledge in Economy but why not try to do it more fun by creating my own graphs of stock values:-)

#!/usr/bin/perl
#Script that graphs some stocks
#peter@it-slav.net
use RRDs;
use Finance::YahooQuote;
$Finance::YahooQuote::TIMEOUT = 60;
useExtendedQueryFormat();

my @symbols=('SWEDA.ST', 'CLASB.ST', 'AXIS.ST', "NOVL", 'ERICB.ST', "INVEB.ST", "\^OMXSPI" );

my $rrd = '/var/rrd_db/stocks';
# define location of images
my $img = '/home/peter/public_html/stocks';

my $cur_time = time();                # set current time
my $end_time = $cur_time ;     # set end

#$value=int(rand(1000));

foreach (@symbols) {
        @quote = getonequote ("$_"); # Get a quote for a single symbol
        #print"@quote\n";
        $quote[0]=&trim($quote[0]);
        print "Ticker:$quote[0]\tName:$quote[1]\tLast_price:$quote[2]\t50d_Mov_Avg:$quote[35]\n";
        if ($quote[35] eq "N/A") {
                $quote[35]=0;
        }

        #&graph (ticket,description,price,movingavarage-50-days)
        &graph ($quote[0],$quote[1],$quote[2],$quote[35]);
}

sub graph
{

        # if rrdtool database doesn't exist, create it
        if (! -e "$rrd/stock$_[0].rrd")
        {
                print "creating rrd database for stock $_[0]...\n";
                RRDs::create "$rrd/stock$_[0].rrd",
                        "-s 86400", #1 day = 24h/day*60min/h*60sec/min= 86400 sec/day
                        "DS:price:GAUGE:172800:U:U", #Heartbeat 2 days = 172800 sec
                         "RRA:AVERAGE:0.5:1:365", #1 year 365 days
                         "RRA:AVERAGE:0.5:1:3650", #10 years 10*365=3650
                        "DS:movingavarage:GAUGE:172800:U:U", #Heartbeat 2 days = 172800 sec
                         "RRA:AVERAGE:0.5:1:365", #1 year 365 days
                         "RRA:AVERAGE:0.5:1:3650"; #10 years 10*365=3650

        }
        if ($ERROR = RRDs::error) { print "$0: failed to create $_[0] database file: $ERROR\n"; }

        # insert values into rrd
        #print "Insert:$_[2]:$_[3] into $rrd/mail$_[0].rrd\n";
        RRDs::update "$rrd/stock$_[0].rrd",
                        "-t", "price:movingavarage",
                        "N:$_[2]:$_[3]";
        if ($ERROR = RRDs::error) { print "$0: failed to insert $_[0] data into rrd: $ERROR\n"; }

        #print "$_[0], \"year\", $_[2], $start_time_year, $_[3], $_[1]\n";
        #SWEDA.ST, "year", 75.75, 1193058922, 127.669, SWEDBANK -A-
        if ($_[3] != 0 ) {
                &CreateGraph($_[0], "year", $start_time_year, "Aktie:$_[1] Ticker:$_[0]");
        } else {
                &CreateGraphLight($_[0], "year", $start_time_year, "Aktie:$_[1] Ticker:$_[0]");
        }
        #&CreateGraph($_[0], "decade", $start_time_decade, $_[1]);

}

sub CreateGraph
{
# creates graph
# inputs: $_[0]: ticketname
#         $_[1]: interval (ie, day, week, month, year)
#         $_[2]: starttime
#         $_[3]: Full name

        RRDs::graph "$img/stocks$_[0]-$_[1].png",
#                "-s -1$_[1]",
                "-t $_[3]",
#                "--lazy",
                "--start=$_[2]",
                "--end=$end_time",
                "-h", "500", "-w", "1100",
                "-a", "PNG",
                "-v It-Slav.Net",
                "--slope-mode",
                "DEF:price=$rrd/stock$_[0].rrd:price:AVERAGE",
                "LINE2:price#FF0000:Kurs:",
                "DEF:movingavarage=$rrd/stock$_[0].rrd:movingavarage:AVERAGE",
                "LINE1:movingavarage#0000FF:Medelvarde 50 dagar:",
                "GPRINT:price:MIN:  Min\\: %6.lf",
                "GPRINT:price:MAX:  Max\\: %6.lf",
                "GPRINT:price:AVERAGE: Avarage\\: %6.lf",
                "GPRINT:price:LAST: Last\\: %6.lf $_[0]\\n";
        if ($ERROR = RRDs::error) { print "$0: unable to generate graph $_[0] $_[1] graph: $ERROR\n"; }
}

sub CreateGraphLight
{
# creates graph
# inputs: $_[0]: ticketname
#         $_[1]: interval (ie, day, week, month, year)
#         $_[2]: starttime
#         $_[3]: Full name

        RRDs::graph "$img/stocks$_[0]-$_[1].png",
#                "-s -1$_[1]",
                "-t $_[3]",
#                "--lazy",
                "--start=$_[2]",
                "--end=$end_time",
                "-h", "500", "-w", "1100",
                "-a", "PNG",
                "-v It-Slav.Net",
                "--slope-mode",
                "DEF:price=$rrd/stock$_[0].rrd:price:AVERAGE",
                "LINE2:price#FF0000:Kurs:",
#                "DEF:movingavarage=$rrd/stock$_[0].rrd:movingavarage:AVERAGE",
#                "LINE1:price#0000FF:Movingavarage_50days:",
                "GPRINT:price:MIN:  Min\\: %6.lf",
                "GPRINT:price:MAX:  Max\\: %6.lf",
                "GPRINT:price:AVERAGE: Avarage\\: %6.lf",
                "GPRINT:price:LAST: Last\\: %6.lf $_[0]\\n";
        if ($ERROR = RRDs::error) { print "$0: unable to generate graph $_[0] $_[1] graph: $ERROR\n"; }
}

sub trim($)
{
        my $string = shift;
#       $string =~ s/^\s+//; #Remove left whitespaces
#       $string =~ s/\s+$//; #Remove right whitespaces
        $string =~ s/\^//; #Remove ^
        return $string;
}

In my case the backup script is started on another host then op5 Monitor or Nagios server, so I also will need a way of sending the data from the passive check over the network, the recommended way is to use nsca. Read the theory at http://nagios.sourceforge.net/docs/3_0/addons.html#nsca

In my op5 Monitor system the nsca daemon to recieve nsca information was installed so I only had to start it:

/etc/init.d/nsca start

This is the steps I did to install it on the client:

1. Download nsca from here.

2. Untar and compile nsca

3. Create a ncsa config file i.e. send_nsca.cfg

encryption_method=0

Now the data will be transmitted unencrypted over the network, this might not be what you want. Make sure that the corresponding nsca config file on the Nagios or op5 Monitor host has the same encryption method.

4. Create a passive check for testing.

# service 'Passive check test'

define service{

    use                            default-service

    host_name                      dull

    service_description            Passive check test

    check_command                  check_dummy!3 "No Data from passive check"

    max_check_attempts             1

    active_checks_enabled          0

    check_freshness                1

    freshness_threshold            300

    flap_detection_options         n

    contact_groups                 it-slav_mail,call_it-slav,it-slav_msn

    stalking_options               n

    }

Explanation:

The check_dummy command will be run if no passive check has been recieved within 5 minutes (300 seconds).

4. test

-First test, wait 5 minutes and your service "Passive check test" should be in status UNKNOWN

-Second test, create a file passive_file_test_critical (the separator is TAB):

dull    Passive check test      2       CRITICAL:test critical

run command:

send_nsca -H nagios_host  -c  send_nsca.cfg < passive_check_data_critical

and the status should change to CRITICAL

-Third test, create a file passive_check_data_ok (the separator is TAB):

dull    Passive check test      0       OK: test ok

Run the command

send_nsca -H  nagios_host -c  send_nsca.cfg < passive_check_data_ok

And the status should change to OK

Now you can set the status of a Nagios or op5 Monitor service by using commands that can be used in scripts. I will in a later article describe how I use it in my MySQL backup script.

Links:

• NSCA
• Nagios passive check theory
• op5 Monitor
• An article about monitor automysqlbackup with passive checks

Troubleshooting hint:

If it does not work, a good hint is to take a look into nagios.log

This description is op5 Statistics centric, however op5 Statistics is based on Cacti so it will work there as well. The only differences between op5 Statistics and Cacti to this guide is the paths to different files.

Links to the files used is located at the bottom.

1. Create a directory called /opt/statistics/scripts/custom

#mkdir /opt/statistics/scripts/custom

Put the file esxiograph.sh in /opt/statistics/scripts/custom

2. Test the script by manually running the script

./esxiograph.sh io_vm vmware-stockholm demo.op5.com <user> <password>

usage:0.02 read:0.00 write:0.02

It works

3. Import the xml files

console->import templates

Browse to the xml file and choose default rrd settings

Do it with both xml files.

4. Add the graph

console->new graph

Choose your vmware ESX server

Graph template choose "ESX – IO Load per VS" to graph the host vmware ESX server load

or "ESX – IO Load per VM" to graph a particular guest host.

Click Create

Enter guest name (if you pick ESX – IO Load per VM), login and password. Change the title to " -IO load" if it is a guest IO you are graphing otherwise all graphs will have the same name.

Click Create

note: The login and password will be non-encrypted in a op5 Statistcs or Cacti MySQL database, use a read only user!

5. Repeat 4 with all ESX servers and guest host you want to graph.

Hint: to get a list of virtual machines run:

/opt/plugins/check_esx3.pl -H  -u xxxxx -p yyyyyy
 -l runtime -s list

CHECK_ESX3.PL OK - 26/28 VMs up: xxxx.op5.se(UP), yyyyy.op5.se(UP), zzzzz.op5.com(UP)....

6. Create a graph tree

Console->Graph Trees

Click add

Enter a name i.e. vmware ESX

7. Put the graphs in your graph tree with the ESX server at root

console->Graph Management

Enter ESX in search field so you only have yor hraphs containing ESX

click checkbox next to – VS IO Load

Choose action scrolllist "Place on tree (vmware ESX)"

Click Go

Choose [Root]

8. Put the guest machines under your vmware server in the tree

console->Graph Trees

Click add

Enter name "Guest machines"

9. Put the guest graphs under the leaf in the graph tree

console -> Graph Management

Enter ESX in search field

Click the checkbox next to all guests

Choose "Place on tree (vmware ESX)"

Choose "Guest Machines"

10. Now you should have a graph tree that looks similiar to this:

Congratulations, now you can look at some nice vmware I/O graphs.

Links:

• op5 Statistics
• Cacti
• Files used in this guide
• IXX IT-Partner AB
  
• op5 check_esx3.pl

op5 has put alot of resources into reporting because that is the feedback we got from our customers. Here is two examples of reports:

• Availability report
• SLA report

The reports is based on monitoring a friends home file server in Gothenburg, thats why it is not five nines

Here is the script that I’m using to read the temperatures, store the data in a round robin database and finally create the graphs.

#!/usr/bin/perl

# By Peter Andersson peter@it-slav.net

# rrd_tempsensor.pl

use RRDs;

use OW;

# define location of rrdtool databases

my $rrd = '/root/owfs/script/rrd_db';

# define location of images

my $img = '/home/peter/public_html/temps';

# process data for each interface (add/delete as required)

&ProcessSensor(0, "Temperatur i garaget", "127.0.0.1" , "3001" , "10.87507C010800");

&ProcessSensor(1, "Temperatur i PDCn", "op5.mynet",3001, "10.04E060010800");

&ProcessSensor(2, "Temperatur ute", "op5.mynet",3001,"10.DEF05F010800");

sub ProcessSensor

{

# process sensor

# inputs: $_[0]: sensor number (ie, 0/1/2/etc)

#         $_[1]: sensor description

        print "number:$_[0] desc:$_[1], server:$_[2] port:$_[3], id: $_[4]\n";

        my $owserver = "$_[2]:$_[3]";

        unless(OW::init($owserver)) {

            $status = $ERRORS{CRIT};

            $message = "OWServer not running at $owserver\n";

            exit $status;

        }

        # get temperature from sensor

        my $handle = OW::get("$_[4]/temperature");

#       print "handle=$handle\n";

        $handle =~ s/^\s*(.*?)\s*$/$1/;

        ## Check if input is an integer or decimal

        unless (($handle =~ /^-?(?:\d+(?:\.\d*)?|\.\d+)$/) || ($handle =~ /^[+-]?\d+$/))

        {

                print "Not an integer or a decimal\n";

                return($ERRORS{CRITICAL});

        }

        $temp=$handle;

        # remove eol chars

        chomp($temp);

        print "sensor $_[0]: $temp degrees C\n";

        # if rrdtool database doesn't exist, create it

        if (! -e "$rrd/temp$_[0].rrd")

        {

                print "creating rrd database for temp sensor $_[0]...\n";

                RRDs::create "$rrd/temp$_[0].rrd",

                        "-s 300",

                        "DS:temp:GAUGE:600:U:U",

                        "RRA:AVERAGE:0.5:1:2016",

                        "RRA:AVERAGE:0.5:6:1344",

                        "RRA:AVERAGE:0.5:24:2190",

                        "RRA:AVERAGE:0.5:144:3650";

        }

        if ($ERROR = RRDs::error) { print "$0: failed to create $_[0] database file: $ERROR\n"; }

        # check for error code from temp sensor

        if (int $temp eq 85)

        {

                print "failed to read value from sensor $_[0]\n";

        }

        else

        {

                # insert values into rrd

                RRDs::update "$rrd/temp$_[0].rrd",

                        "-t", "temp",

                        "N:$temp";

                if ($ERROR = RRDs::error) { print "$0: failed to insert $_[0] data into rrd: $ERROR\n"; }

        }

        # create graphs for current sensor

        &CreateGraph($_[0], "day", $_[1]);

        &CreateGraph($_[0], "week", $_[1]);

        &CreateGraph($_[0], "month", $_[1]);

        &CreateGraph($_[0], "year", $_[1]);

}

sub CreateGraph

{

# creates graph

# inputs: $_[0]: sensor number (ie, 0/1/2/etc)

#         $_[1]: interval (ie, day, week, month, year)

#         $_[2]: sensor description

        RRDs::graph "$img/temp$_[0]-$_[1].png",

                "-s -1$_[1]",

                "-t $_[2]",

                "-h", "150", "-w", "475",

                "-a", "PNG",

                "-v WWW.It-Slav.Net",

                "--slope-mode",

                "DEF:temp=$rrd/temp$_[0].rrd:temp:AVERAGE",

                "LINE2:temp#FF0000::",

                "CDEF:tempavarage=temp,86400,TREND",

                "LINE1:tempavarage#000088::",

                "GPRINT:temp:MIN:  Min\\: %6.1lf",

                "GPRINT:temp:MAX:  Max\\: %6.1lf",

                "GPRINT:temp:AVERAGE: Snitt\\: %6.1lf",

                "GPRINT:temp:LAST: Nuvarande\\: %6.1lf grader C\\n";

        if ($ERROR = RRDs::error) { print "$0: unable to generate sensor $_[0] $_[1] graph: $ERROR\n"; }

}

Use crontab to run it every 5 minutes:

*/5     *       *       *       *       /root/owfs/script/tempgraph.pl > /dev/null

This a problem that grows and there are many technologies how to fight it.

As I’m the sysadmin of my mailserver it makes it possible to use many approaches. The best way is to find out if it is a spam before it is accepted at the SMTP server. I’m using a Fairly-Secure Anti-SPAM Gateway Using OpenBSD, Postfix, Greylisting, Amavisd-new, SpamAssassin, Razor and DCC and it is very effective.

• The first filter is to check if the receiver of the mail is valid. It might seem like a obvious first filter but in many cases the host that receives mails from internet only forward the mail to an inner mail server and suddenly invalid mails with probably incorrect from address is the receiving organizations problem. With this approach a valid mail but misspelled to address will bounce back to the sender. If it is a spam mail it will be the sending hosts problem how to handle. I graph this and it can be found here. A qualified guess is that more or less all of them are spams.
• The second step is a little bit more complex, the mail is scanned before it is accepted. So if my spam scanner finds that the mail is a spam it will tell the sending mailserver that it-slav.net thinks that this mail is a spam and that it is not accepted. If nothing suspicious is found the mailserver accept the mail and it will be sent to my mailserver. The number of mails that are scanned and a spam is found is graphed and it can be found here.
• A third technology I have used is greylistening, it is very effective but the technologies described above is good enough for me so I’m not using it now. It puts a little more burden on the sending host and the first time a host sends a mail to a new host it will take some extra time.
• A promising technlogy is SPF, the idea is to guarantee that the sending mail comes from the place were it claims to come from, a good description can be found at wikipedia. Spamassassin use SPF. If you want to avoid that your domain can be used as the sender of a spam, add some extra lines to your DNS record.
• To annoy spamsenders a good idea could be to start a tarpit. Send all spammsenders to your tarpitt and enjoy when they use their resources for nothing. It is included in OpenBSD in compination with greylistening.

Graphs

• Rejected
  
• Spam