I have a FON router which allow anyone to connect to Intenet via my network connection. I like the openess and idea behind FON but I do not want to get in trouble if someone do bad things and using my network connection.

One solution is to connect the FON accesspoint to a separate network segment and let all traffic from that network go through TOR, the onion router. 

Another feature is when I want to be anonymous on Internet I plugin my computer to that network segment.

Setup

I use OpenBSD as my firewall and the first step is to download, compile and configure TOR. I downloaded the tarball from https://www.torproject.org and used the normal procedure:

# wget https://www.torproject.org/dist/tor-0.2.2.35.tar.gz

# tar xzvf tor-0.2.2.35.tar.gz 

# cd tor-0.2.2.35                                                                                                                                                                

# ./configure&&make&&make install

Configure pf

I use a specific ethernet interface, fxp0, which will route all traffic into tor. 

--cut from /etc/pf.conf--

#Tor traffic

tor_if ="fxp0"

# Tor's TransPort

trans_port = "9040"

pass in quick on $tor_if inet proto udp to port domain rdr-to 127.0.0.1 port 5300 

pass in quick on $tor_if inet proto { tcp udp } to !($tor_if) rdr-to 127.0.0.1 port $trans_port

--end cut--

Configure tor

# cat /usr/local/etc/tor/torrc

VirtualAddrNetwork 10.192.0.0/10

AutomapHostsOnResolve 1

TransPort 9040

DNSPort 5300

Log notice syslog

#Log debug stderr

RunAsDaemon 1

Note: I know that best practice is to let the tor process run as non root user. That requires read access to /dev/pf and I did not bother to get it working.

Dhcpd config

I assume that a OpenBSD sysadmin knows how to setup dhcpd so I will just show the config addon I did to /etc/dhcpd.conf.

shared-network tor-net {

        option  domain-name "it-slav-tor-net";

        option  domain-name-servers 10.1.2.1;

        option  ntp-servers 10.1.2.1;

        subnet 10.1.2.0 netmask 255.255.255.0 {

                option routers 10.1.2.1;

                range 10.1.2.100 10.1.2.200;

        }

The ip-adress of the fxp0 interface is 10.1.2.1

Start tor

# /usr/local/bin/tor

Jan 10 20:52:48.880 [notice] Tor v0.2.2.35 (git-b04388f9e7546a9f). This is experimental software. Do not rely on it for strong anonymity. (Running on OpenBSD i386)

Jan 10 20:52:48.885 [warn] It's a little hard to tell, but you seem to have Libevent 1.4.0-beta header files, whereas you have linked against Libevent 1.4.14b-stable.  This will probably make Tor crash.

Jan 10 20:52:48.886 [notice] Initialized libevent version 1.4.14b-stable using method kqueue. Good.

Jan 10 20:52:48.886 [notice] Opening Socks listener on 127.0.0.1:9050

Jan 10 20:52:48.887 [notice] Opening Transparent pf/netfilter listener on 127.0.0.1:9040

Jan 10 20:52:48.887 [notice] Opening DNS listener on 127.0.0.1:5300

Final step

Plugin your fon router and enjoy!

Links

The hints to this article was found at:

•  https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/TransparentProxy 

It really annoyed me that the iowait at the cpu on the Dom 0 system was approximatly 50% when the 4 virtual system was doing more or less nothing.

I showed my earlier blogpost about VM virtualization to our sysadmin at op5. He gave me a couple of more hints that really make a big difference, the best part is that they are real simple to implement.

I also noticed that I forgot some parts in the previous article that was obvius to me.

#1 use virtio drivers if possible

This is probably the single most important step to get the most out of the hardware.

When the virtual systems wants to communicate with the hardware it uses drivers, if the drivers are simulating hardware to give a service to the virtual operating system it adds alot of overhed. A simple example:

• The virtual system wants to access the disk
• The virtual system communicates via the driver to the hardware
• The hardware is not real hardware, it is emulated hardware in sofware 
• The software that emulates the hardware sends the I/O to the Dom 0 virtualization engine
• The Virtualization engine verifies that the access is ok, translate it to real hardware call via its driver
• The hardware do the operation and send the result back up via all the steps to the virtual operating system

All this transalations and software calls take alot of resources from the machine.

By using virtual drivers, so that there is no need to emulate hardware, many of the steps can be skipped or done with much less penalty. However not all operating system have virtio drivers.

#2 load the virtio modules in Dom 0

This is a real god hint I got from our sysadmin, load the virtual drivers in the Dom 0 system asweell. According to him, the virtual drivers in the Dom 0 and Dom U:s communicate and makes the I/O much more effective. I have not found any information about this at any place so I was sceptical but I tried it.

# modprobe virtio_balloon
# modprobe virtio_blk
# modprobe virtio_net
# modprobe virtio_pci

The virtual systems need to be rebooted as far as I know before the loaded drivers will have any effect.

At first no change at all, but after few hours I noticed that CPU idle raized and the iowait went from at least 50% down to below 5%.

A nice graph, created by op5 Monitor showing the difference:

I loaded the virtual drivers approximately 4 pm, and it took until 10 pm before the performance boost showed up.

The correct way of making sure that the drivers are loaded directly would be to remake the kernel but I'm lazy so I have just added the module load to /etc/rc.local

Conclusions

It is always important to monitor systems. When doing changes it is not possible know if the changes do have any impact if you do not monitor and follow up how and if the changes does anything.

My former statement that KVM is not mature is more true then I have thought. The knowledge how to tune and maintain KVM systems seems to be like black art where the knowledge is not documented and well spread. I have read the RedHat virtualization and I have not found anything about that the virtual drivers should be loaded on the Dom0 system.

If there is anyone out there that has some documentation or more hints, please send me the link or comment this blogpost.

Links:

• RedHat virtualization doc
• op5 Monitor that monitor and creates nice graphs
• check_cpu_stats Nagios plugin to monitor cpustat

As an it slave, I have the same problems as most datacenters:

• Running out of space because of more and more machines get into the datacenter
• Overheating
• Powerconsumption

Added to the "normal" datacenter problems, I also have to deal with:

• Wife acceptance
• Noice, my neighbours has complained about the noice outside my basement.

To address this issues I will run a consolidation project to get fewer machines by using virtualization. As virtualization engine, KVM is choosed.

This article will describe some pitfalls I run into and how I solved them.

KVM plattform

KVM seems to be the future for virtualization within the Linux area since Xen has done some mistakes.

As I have good experience of running CentOS and has KVM included I decided to use CentOS as the plattform for my virtualization.

Issues

Bridge network

To let the virtual boxes has full access to the network you need create a network bridge, it is rather straightforward and well documented so I will skip this part. I used the documentation on Red Hat customer support. Why KVM requires a bridge instead of a normal NIC is another question

Snapshots

One handy feature with virtualization is that the virtual systems can be one big fat file at the physical host, this gives the possibility to make full backups without turning the system down a.k.a snapshots.

However to get snapshot to work the filetype must be qcow2 and raw is the default format.

It is possible to convert from raw to qcow2 format by running the command:

qemu-img convert -f raw -O qcow2 <virtualhost>.img <virtualhost>.img.qcow2

Remember to turn of your virtual system by running:

virsh shutdown <virtualhost>

After the conversion:

virsh edit <virtualhost>

modify the type and path, i.e.

      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/web.img.qcow2'/>

Start your virtual system:

virsh start <virtualhost>

Take a snapshot with:

qemu-img snapshot -c <snapshotname> <virtualhost>.img.preallc.qcow2

To create a full image of your snapshot you need to compile a more resent version of qemu-img then shipped with CentOS 6.2, I download a later version and compiled it from here, i renamed the new to qemu.img2 and copied it to /usr/local/bin

qemu-img2 convert -p -f qcow2 -O qcow2 -s <snapshotname> <virtualhost>.img.preallc.qcow2 <targetpath>

Beware: This command can use all your resources and may affect all your virtual systems, use nice and ionice to prevent it.

Disk I/O

After installing just a few virtual system I noticed that my physical host got alot of I/O wait, it can be seen by using tools like top.

To monitor this I installed the nagios plugins:

• check disk io, can be found here
• check cpu stats, can be found here
• check_libvirt, developed by op5 and an Howto

However it seems like that though my virtual systems did more or less nothing they caused alot of disk I/O on the physical system.

I did some investigations and read quite many articles, fiddled and tested. The following is what worked for me in my setup.

#1 Change disk scheduler

The disk scheduler can be changed on runtime by modify the file:

/sys/block/sda/queue/scheduler

to see what scheduler you use now:

cat /sys/block/sda/queue/scheduler

Change sda to the device you have.

The scheduler that worked best for me is deadline

echo deadline > /sys/block/sda/queue/scheduler

#2 Mount with noatime

A feature in Unix and other Unixlike systems like Linux is that it normaly stores when a file is accessed. So one read always produce a write and if you are using raid like morroring this get worse, one reade always generate several writes. This creates alot of overhead for a feature soldom used.

So change /etc/fstab so it will mount the file systems with noatime.
An axample:

UUID=a290aa4b-635c-45fa-b144-1fbef90b3735 /                       ext4    defaults,noatime        1 1

#3 Preallocation disk images

A real boost that is hidden in the featureset and not shown in the virtualmachine gui is using preallocation disk images. They cannot be created from the GUI so I install the virtual machine, turn it of and convert it afterwards.

qemu-img convert -f qcow2 -O qcow2 -o preallocation=metadata <virtualhost>.img.qcow2 <virtualhost>.img.preallc.qcow2

Change the path to new image name by editing the virtual machine settings

virsh edit <virtualhost>

Conclusions

In my opinion it seems like KVM is still a little bit immature or at least the tools to handle it. Maybe it would be a good idea to have one linux distro focusing on beeing the best platform for virtualization.

I am convinced that I can get even more bang for the bucks out of my installation if I learn how to tweak it even more, so if you have any hints, do not hesitate to contact me. Preferably as a comment to this blogpost.

Referenses:

• op5 Monitor, a nagios based enterprise monitor tool to monitor my environment
• check_libvirt, a nagios plugin done by op5 to monitor KVM
• check_diskio, a nagios plugin to monitor disk I/O
• check_cpu_stats, a nagios plugin to monitor cpu usage rather detailed
• Another blogpost which helped me
• KVM performance improvements and optimizations – Red Hat presentation

The new access speed is 100 Mbit/s optical fiber, iperf tests show that I can get approx 97 Mbit/s in booth directions.

Changes:

• 100 Mbit/s internet access provided by Ownit
• Updated firewall to OpenBSD 5.0 running on an Astaro appliance
• Updated Webserver to Centos 6.2, virtulized with KVM
• Updated Mailserver to Centos 6.2, virtulized with KVM

Happy new year

• Support for offline maps – no need to have online gmap connection
• Introducing IP traffic “weathermaps” i.e. a user friendly way to present traffic flows, being WAN or LAN traffic
• Display monitor data on external web i.e visualise and distribute the information to greater crowd
• Multiple of  time saving tasks such as multidelete, free text filter, save searches etc.

Having a modern easy to use proactive monitoring system is the core of assuring quality delivery of your enterprise applications to your user. However cool report and dashboards are great, if they require complex skills, many hours to set up and maintain -  the value is quickly gone.

With op5 Monitor v5.5 we introduce a number of features to make the information more presentable and understandable. We have made lots of new micro features to make the everyday change management easier and faster.

Major features in this release:   
•    Support for offline map
    ⁃    Now integrating the open streetmap (OSM) project, making the vizualisation of large networks easier and offline
    •    Support for IP traffic flow visualisation on maps or any background
    •    Publish op5 Monitor data / widget on external web service
    •    Multi tier host & service groups
    ⁃    A host or service group can now be part of another group creating a tree structure of dependencies.
    •    Multidelete – it’s always easy to add a new host or service to a system – now it´s easy to remove as well – saves lots of time!
Usability improvements:
    •    Saved searches
    •    Search on notes and comments and search in scheduling queue
    •    Free text filter
    •    Enhanced Alert history
    •    Authentication fallback

In total we have fixed and added more then 240 items in this release. Please let us know what you think – we focus on Your needs, just tell us!

Best regards
The op5 Team!

More about the new release

We are pleased to announce the official release of OpenBSD 5.0.
This is our 30th release on CD-ROM (and 31th via FTP).  We remain
proud of OpenBSD's record of more than ten years with only two remote
holes in the default install.

As in our previous releases, 5.0 provides significant improvements,
including new features, in nearly all areas of the system:

 - Improved hardware support, including:
    o MSI interrupts for many devices, on those architectures which can
      support them (amd64, i386, sparc64 only so far).
    o A new dma_alloc(9) API makes it easier for kernel code to allocate
      dma-safe memory.  Many drivers (especially network drivers) and
      subsystems (in particular scsi and the buffer cache) were adapted
      to use this.
    o As a result, big-memory support has been enabled on all possible
      architectures.
    o The rather rare bce(4) driver now copies mbufs all the time, to cope
      with the hardware having a 1GB limit.
    o Added hds(4), a driver for Hitachi Modular Storage SCSI devices.
    o Added myx(4), a driver for the Myricom Myri-10G 10GB Ethernet devices.
    o Added dfs(4), a driver for Dynamic Frequency Switching on some macppc
      systems.
    o cardbus(4) and pcmcia(4) support on sgi.
    o Suspend/resume support on Loongson Yeelong laptops.
    o Interrupt handlers for bnx(4), em(4), ix(4) and sis(4) have been
      improved reducing overhead and increasing performance.
    o New acpitoshiba(4) driver providing ACPI support for Toshiba laptops.
    o Added nvt(4), a driver for the W83795G and W83795ADG hardware monitor.
    o Added support to sdhc(4) for the Ricoh 5U823 SD/MMC controller.
    o A new fw_update(1) tool to install and update non-free firmware packages.

 - Generic network stack improvements:
    o Added support for sending Wake on LAN packets using arp(8).
    o Permit turning Wake on LAN support on/off using ifconfig(8).
    o Added Wake on LAN support to xl(4), re(4), and vr(4).
    o Allow ftp-proxy to proxy across rdomains.
    o The IPv4 stack will no longer accept ICMP redirects when
      acting as a router.
    o By default the IPv6 stack will not process ICMP6 redirects.
      rtsol(8) will turn it back if -F is used.
    o Reworked large parts of the dhclient(8) options processing for better
      interoperability.
    o Fixed carp(4) to work in IPv6 only setups.
    o Make it possible to bind(2) to the local network broadcast address
      on datagram and raw sockets.
    o The default multicast reject route is now ignored if the UDP socket
      uses the IP_MULTICAST_IF socket option.
    o Make gre(4) work between systems in the same LAN.
    o Removed the link1 mode special addressing mode on lo(4).
    o New net.inet.tcp.always_keepalive sysctl, effectively enabling
      SO_KEEPALIVE on all TCP sockets.

 - Routing daemons and other userland network improvements:
    o bgpd(8) no longer bumps the rlimits: the rc.d framework respects
      login classes which is a much better solution.
    o Correctly set the network filtersets on reload in bgpd(8).
    o The routing socket is now sending RTM_DESYNC messages if the
       socketbuffer overflows.
    o Allow ospfd(8) to send out LS updates and other messages
      larger than the MTU.
    o Fixed nexthop calculation in ospfd(8) for directly connected P2P links.
    o First bits to support opaque LSA in ospfd(8).  Only basic redistribute
      logic and LSDB handling for now.
    o Creating new interfaces will no longer cause a fatal error in ospf6d(8).
    o ospf6d(8) handles link-state changes better.
    o Better loopback handling in ospf6d(8).
    o No longer install extra multicast routes in ripd(8) and ldpd(8).
    o Make kqueue(2) work with sosplice(9).
    o Enabled sosplice(9) in relayd(8) for TCP.
    o Added support for divert-to which provides some benefits over
      rdr-to in relayd(8).
    o Reload support in relayd(8) has been fixed.
    o Fixed trap sending in snmpd(8).
    o Make ping6(8) compare minimum amount of bytes between what
      was received and what was sent out.
    o Make traceroute(8) with type-of-service setted (-t) display
      a message if the returned packet has a different tos type.
    o Added the socket splicing fields of struct socket to netstat -vP output.
    o tcpbench(1) now uses libevent and supports both TCP and UDP modes.
    o TCP socket buffer sizes can now be displayed using the netstat(1) -B flag.
    o tcpdump(8) can now filter on icmptype and tcpflags.
    o bgplg(8) now supports "show ip bgp peer-as".

 - pf(4) improvements:
    o Make pf(4) reassemble IPv6 fragments.  In the forward case, pf
      refragments the packets with the same maximum size.
    o Allow pf(4) to filter on the rdomain a packet belongs to.
    o Make pf(4) allow userland proxies to establish cross rdomain
      proxy sessions.
    o Added IPv6 ACK prioritization in pf(4).
    o Change 'set skip on <...>' to work with interface groups.
    o pfsync(4) supports IPv6 as network protocol.
    o Switched ftp-proxy(8) over to divert-to instead of rdr-to.
    o Switched tftp-proxy(8) over to divert-to instead of rdr-to.
    o New very low overhead priority queueing implementation for pf(4) used via
      the "prio" keyword.
    o Support for least-states in load balancing pools and tables.
    o Support for weighted round-robin in load balancing pools and tables.

 - SCSI improvements:
    o Most SCSI hardware drivers now use the new iopools infrastructure.
    o scsi(4) devices are now all provided with a unique devid, which
      is displayed during the probe process.
    o ASC/ASCQ error codes and verbiage now in sync with
      http://www.t10.org/lists/asc-num.txt.
    o Progress on iSCSI includes better login, better logout, preliminary
      FSM support in iscsid(8), and improved logging and debug information.
    o uk(4) can now safely and reliably detach an unknown SCSI device.
    o SCSI multipath device and kernel support has been improved.
    o vscsi(4) now ensures output always goes to the correct connection.
    o vscsi(4) connections can now be reset gracefully.
    o scsi(4) devices on fibre channel fabrics no longer inherit the adapter's
      address.

 - Assorted improvements:
    o Kernel randomization speed and quality improved substantially.
    o For additional security, security(8) was rewritten in Perl.
    o Mandoc 1.11.4: Now accepts eqn(7) input (no fancy formatting yet)
      and supports -Tutf8 output (but no utf8 input yet).
    o Removed a variety of OS-compat emulation code, leaving just the Linux
      support.
    o Small improvements to Linux compat (only available on i386).
    o Improved our own pkg-config(1) implementation with extended comparison
      scheme and implementing various new options.
    o The math library, libm, was fully fleshed out to support all C99 required
      parts.  Many bugs for various architectures were fixed along the way.
    o malloc(3) is a lot faster and has a few further security features (more
      randomization, as well as the 'S' flag to enable all paranoia checks).
    o 'make depend' is no longer neccessary in kernel compilation directories
      since the dependencies are calculated automatically.
    o Increased the default size of the buffer cache.
    o kqueue(2) now works on /dev/random and spliced sockets
    o On MBR-based disks, scan through up to 256 extended partition tables
      when looking for an OpenBSD partition table.
    o Added POSIX 2008 fdopendir(3) and openat(2) functions, as well as the
      O_CLOEXEC, O_DIRECTORY, and F_DUPFD_CLOEXEC flags.
    o Improved lint format string checks and added a few other checks.
    o kdump(8) now dumps stat and sockaddr structures, sysctl mib
      strings, and decodes syscall flags and operation bits.
    o Improved kernel pool debug checking.
    o Improved correctness of signals and various syscalls when rthreads
      are in use.
    o Kernel malloc(9) space and stacks moved to non-dma memory.
    o Fixed some shutdown/reboot hangs on NFS clients.
    o UNIX-domain socket paths are now guaranteed to be NUL-terminated.
    o Added support for *wprintf(3), wcs{,n}casecmp(3), and wcsdup(3).
    o NULL is now a (void *).
    o grep(1) now supports a -H option to always print filename headers.
    o Whitelist expiry for spamlogd(8) can now be configured via a -W flag.
    o ls(1) now supports the POSIX -H option to follow symbolic links specified
      on the command line.
    o disklabel(8) now tries the next auto-allocation scheme if the current one
      fails due to insufficient available partitions.
    o bc(1) gained editline(3) support.
    o Many enhancements and new functionality has been added to tmux(1).
    o disklabel(8) supports absolute resizing of partitions in auto-allocated
      labels.
    o newfs(8) accepts k/m/g suffixes for the -S and -s options.

 - Install/Upgrade process changes:
    o Completed support for DUID disk installs, and enabled it fully.
    o Install non-free firmwares from the internet upon first boot, based on a
      question in the installer.
    o svnd(4)-like behaviour became the default for vnd(4) devices.  This is
      what is used to build the media.

 - rc.d(8) framework improvements:
    o rc.d(8) is now also used for the base system daemons.
    o Backward compatible with the historic way of starting daemons.
    o Notify the user by appending (ok) or (failed) in interactive mode.
    o Better diagnostics with the introduction of RC_DEBUG.

 - OpenSSH 5.9:
    o New features:
      - Introduce sandboxing of the pre-auth privsep child using an
        optional sshd_config(5) "UsePrivilegeSeparation=sandbox" mode
        that enables mandatory restrictions on the syscalls the privsep
        child can perform.
      - Add new SHA256-based HMAC transport integrity modes from
        http://www.ietf.org/id/draft-dbider-sha2-mac-for-ssh-02.txt
        These modes are hmac-sha2-256, hmac-sha2-256-96, hmac-sha2-512,
        and hmac-sha2-512-96, and are available by default in ssh(1)
        and sshd(8).
      - The pre-authentication sshd(8) privilege separation slave process
        now logs via a socket shared with the master process, avoiding
        the need to maintain /dev/log inside the chroot.
      - ssh(1) now warns when a server refuses X11 forwarding.
      - sshd_config(5)'s AuthorizedKeysFile now accepts multiple paths,
        separated by whitespace. The undocumented AuthorizedKeysFile2
        option is deprecated (though the default for AuthorizedKeysFile
        includes .ssh/authorized_keys2).
      - sshd_config(5): similarly deprecate UserKnownHostsFile2 and
        GlobalKnownHostsFile2 by making UserKnownHostsFile and
        GlobalKnownHostsFile accept multiple options and default to
        include known_hosts2.
      - sshd_config(5)'s ControlPath option now expands %L to the host
        portion of the destination host name.
      - sshd_config(5) "Host" options now support negated Host matching.
      - sshd_config(5): a new RequestTTY option provides control over
        when a TTY is requested for a connection, similar to the existing
        -t/-tt/-T ssh(1) commandline options.
      - ssh-keygen(1): Add -A option. For each of the key types (rsa1,
        rsa, dsa and ecdsa) for which host keys do not exist, generate
        the host keys with the default key file path, an empty passphrase,
        default bits for the key type, and default comment. This is useful
        for system initialisation scripts.
      - ssh(1): Allow graceful shutdown of multiplexing: request that
        mux server removes its listener socket and refuse future
        multiplexing requests but don't kill existing connections. This
        may be requested using "ssh -O stop ...".
      - ssh-add(1): now accepts keys piped from standard input.
      - Retain key comments when loading v.2 keys. These will be visible
        in "ssh-add -l" and other places. (bz#439)
      - ssh(1) and sshd(8): set IPv6 traffic class from IPQoS (as well as
        IPv4 ToS/DSCP). (bz#1855)
    o The following significant bugs have been fixed in this
      release:
      - sshd(8): allow GSSAPI authentication to detect when a server-side
        failure causes authentication failure and don't count such failures
        against MaxAuthTries. (bz#1244)
      - ssh-keysign(8): now signs hostbased authentication challenges
        correctly using ECDSA keys. (bz#1858)

 - Over 7,200 ports, major robustness and speed improvements in package tools.
 - Many pre-built packages for each architecture:
    o i386: 7008                      o sparc64: 6456
    o alpha: 6046                     o sh: 3721
    o amd64: 6960                     o powerpc: 6691
    o sparc: 3277                     o arm: 2963
    o hppa: 6125                      o vax: 1409
    o mips64: 5689                    o mips64el: 5709

 - Some highlights:
    o Gnome 2.32.2                    o KDE 3.5.10
    o Xfce 4.8.0                      o MySQL 5.1.54
    o PostgreSQL 9.0.5                o Postfix 2.8.4
    o OpenLDAP 2.3.43 and 2.4.25      o Mozilla Firefox 3.5.19, 3.6.18 and 5.0
    o Mozilla Thunderbird 5.0         o GHC 7.0.4
    o LibreOffice 3.4.1.3             o Emacs 21.4, 22.3 and 23.3
    o Vim 7.3.154                     o PHP 5.2.17 and 5.3.6
    o Python 2.4.6, 2.5.4 and 2.7.1   o Ruby 1.8.7.352 and 1.9.2.200
    o Mono 2.10.2                     o Chromium 12.0.742.122
    o Groff 1.21

 - As usual, steady improvements in manual pages and other documentation.
    o Base system and Xenocara manuals are now installed as source code,
      making grep(1) more useful in /usr/share/man/ and /usr/X11R6/man/.
    o If both formatted and source versions of manuals are installed,
      man(1) automatically displays the newer version of each page.

 - The system includes the following major components from outside suppliers:
    o Xenocara (based on X.Org 7.6 with xserver 1.9 + patches,
      freetype 2.4.5, fontconfig 2.8.0, Mesa 7.8.2, xterm 270,
      xkeyboard-config 2.3 and more)
    o Gcc 2.95.3 (+ patches), 3.3.5 (+ patches) and 4.2.1 (+patches)
    o Perl 5.12.2 (+ patches)
    o Our improved and secured version of Apache 1.3, with
      SSL/TLS and DSO support
    o OpenSSL 1.0.0a (+ patches)
    o Sendmail 8.14.5, with libmilter
    o Bind 9.4.2-P2 (+ patches)
    o Lynx 2.8.7rel.2 with HTTPS and IPv6 support (+ patches)
    o Sudo 1.7.2p8
    o Ncurses 5.7
    o Heimdal 0.7.2 (+ patches)
    o Arla 0.35.7
    o Binutils 2.15 (+ patches)
    o Gdb 6.3 (+ patches)

If you'd like to see a list of what has changed between OpenBSD 4.9
and 5.0, look at

        http://www.OpenBSD.org/plus50.html

Even though the list is a summary of the most important changes
made to OpenBSD, it still is a very very long list.

------------------------------------------------------------------------
- SECURITY AND ERRATA --------------------------------------------------

We provide patches for known security threats and other important
issues discovered after each CD release.  As usual, between the
creation of the OpenBSD 5.0 FTP/CD-ROM binaries and the actual 4.9
release date, our team found and fixed some new reliability problems
(note: most are minor and in subsystems that are not enabled by
default).  Our continued research into security means we will find
new security problems -- and we always provide patches as soon as
possible.  Therefore, we advise regular visits to

        http://www.OpenBSD.org/security.html
and
        http://www.OpenBSD.org/errata.html

Security patch announcements are sent to the security-announce@OpenBSD.org
mailing list.  For information on OpenBSD mailing lists, please see:

        http://www.OpenBSD.org/mail.html

------------------------------------------------------------------------
- CD-ROM SALES ---------------------------------------------------------

OpenBSD 5.0 is also available on CD-ROM.  The 3-CD set costs $50 CDN and
is available via mail order and from a number of contacts around the
world.  The set includes a colourful booklet which carefully explains the
installation of OpenBSD.  A new set of cute little stickers is also
included (sorry, but our FTP mirror sites do not support STP, the Sticker
Transfer Protocol).  As an added bonus, the second CD contains an audio
track, a song entitled "What Me Worry?".  MP3 and OGG versions of
the audio track can be found on the first CD.

Lyrics (and an explanation) for the songs may be found at:

    http://www.OpenBSD.org/lyrics.html#50

Profits from CD sales are the primary income source for the OpenBSD
project -- in essence selling these CD-ROM units ensures that OpenBSD
will continue to make another release six months from now.

The OpenBSD 5.0 CD-ROMs are bootable on the following four platforms:

  o i386
  o amd64
  o macppc
  o sparc64

(Other platforms must boot from floppy, network, or other method).

For more information on ordering CD-ROMs, see:

        http://www.OpenBSD.org/orders.html

The above web page lists a number of places where OpenBSD CD-ROMs
can be purchased from.  For our default mail order, go directly to:

        https://https.OpenBSD.org/cgi-bin/order

All of our developers strongly urge you to buy a CD-ROM and support
our future efforts.  Additionally, donations to the project are
highly appreciated, as described in more detail at:

        http://www.OpenBSD.org/goals.html#funding

------------------------------------------------------------------------
- OPENBSD FOUNDATION ---------------------------------------------------

For those unable to make their contributions as straightforward gifts,
the OpenBSD Foundation (http://www.openbsdfoundation.org) is a Canadian
not-for-profit corporation that can accept larger contributions and
issue receipts.  In some situations, their receipt may qualify as a
business expense write-off, so this is certainly a consideration for
some organizations or businesses.  There may also be exposure benefits
since the Foundation may be interested in participating in press releases.
In turn, the Foundation then uses these contributions to assist OpenBSD's
infrastructure needs.  Contact the foundation directors at
directors@openbsdfoundation.org for more information.

------------------------------------------------------------------------
- T-SHIRT SALES --------------------------------------------------------

The OpenBSD distribution companies also sell tshirts and polo shirts.
And our users like them, too.  We have a variety of shirts available,
with the new and old designs, from our web ordering system at, as
described above.

-----------------------------------------------------------------------
- FTP INSTALLS ---------------------------------------------------------

If you choose not to buy an OpenBSD CD-ROM, OpenBSD can be easily
installed via FTP or HTTP downloads.  Typically you need a single
small piece of boot media (e.g., a boot floppy) and then the rest
of the files can be installed from a number of locations, including
directly off the Internet.  Follow this simple set of instructions
to ensure that you find all of the documentation you will need
while performing an install via FTP or HTTP.  With the CD-ROMs,
the necessary documentation is easier to find.

1) Read either of the following two files for a list of ftp/http
   mirrors which provide OpenBSD, then choose one near you:

        http://www.OpenBSD.org/ftp.html
        ftp://ftp.OpenBSD.org/pub/OpenBSD/5.0/ftplist

   As of Nov 1, 2011, the following ftp mirror sites have the 5.0 release:

        ftp://ftp.eu.openbsd.org/pub/OpenBSD/5.0/       Stockholm, Sweden
        ftp://ftp.bytemine.net/pub/OpenBSD/5.0/         Oldenburg, Germany
        ftp://ftp.ch.openbsd.org/pub/OpenBSD/5.0/       Zurich, Switzerland
        ftp://ftp.fr.openbsd.org/pub/OpenBSD/5.0/       Paris, France
        ftp://ftp5.eu.openbsd.org/pub/OpenBSD/5.0/      Vienna, Austria
        ftp://mirror.aarnet.edu.au/pub/OpenBSD/5.0/     Brisbane, Australia
        ftp://ftp.usa.openbsd.org/pub/OpenBSD/5.0/      CO, USA
        ftp://ftp5.usa.openbsd.org/pub/OpenBSD/5.0/     CA, USA
        ftp://obsd.cec.mtu.edu/pub/OpenBSD/5.0/         Michigan, USA

        The release is also available at the master site:

        ftp://ftp.openbsd.org/pub/OpenBSD/5.0/          Alberta, Canada

        However it is strongly suggested you use a mirror.

   Other mirror sites may take a day or two to update.

2) Connect to that ftp mirror site and go into the directory
   pub/OpenBSD/5.0/ which contains these files and directories.
   This is a list of what you will see:

        ANNOUNCEMENT     armish/          mvme68k/         sparc64/
        Changelogs/      ftplist          mvme88k/         src.tar.gz
        HARDWARE         hp300/           packages/        sys.tar.gz
        PACKAGES         hppa/            ports.tar.gz     tools/
        PORTS            i386/            root.mail        vax/
        README           landisk/         sgi/             xenocara.tar.gz
        alpha/           mac68k/          socppc/          zaurus/
        amd64/           macppc/          sparc/

   It is quite likely that you will want at LEAST the following
   files which apply to all the architectures OpenBSD supports.

        README          - generic README
        HARDWARE        - list of hardware we support
        PORTS           - description of our "ports" tree
        PACKAGES        - description of pre-compiled packages
        root.mail       - a copy of root's mail at initial login.
                          (This is really worthwhile reading).

3) Read the README file.  It is short, and a quick read will make
   sure you understand what else you need to fetch.

4) Next, go into the directory that applies to your architecture,
   for example, i386.  This is a list of what you will see:

        INSTALL.i386    cd50.iso        floppyB50.fs    pxeboot*
        INSTALL.linux   cdboot*         floppyC50.fs    xbase50.tgz
        MD5             cdbr*           game50.tgz      xetc50.tgz
        base50.tgz      cdemu50.iso     index.txt       xfont50.tgz
        bsd*            comp50.tgz      install50.iso   xserv50.tgz
        bsd.mp*         etc50.tgz       man50.tgz       xshare50.tgz
        bsd.rd*         floppy50.fs     misc50.tgz

   If you are new to OpenBSD, fetch _at least_ the file INSTALL.i386
   and the appropriate floppy*.fs or install50.iso files.  Consult the
   INSTALL.i386 file if you don't know which of the floppy images
   you need (or simply fetch all of them).

   If you use the install50.iso file (roughly 250MB in size), then you
   do not need the various *.tgz files since they are contained on that
   one-step ISO-format install CD.

5) If you are an expert, follow the instructions in the file called
   README; otherwise, use the more complete instructions in the
   file called INSTALL.i386.  INSTALL.i386 may tell you that you
   need to fetch other files.

6) Just in case, take a peek at:

        http://www.OpenBSD.org/errata.html

   This is the page where we talk about the mistakes we made while
   creating the 5.0 release, or the significant bugs we fixed
   post-release which we think our users should have fixes for.
   Patches and workarounds are clearly described there.

Note: If you end up needing to write a raw floppy using Windows,
      you can use "fdimage.exe" located in the pub/OpenBSD/5.0/tools
      directory to do so.

------------------------------------------------------------------------
- X.ORG FOR MOST ARCHITECTURES -----------------------------------------

X.Org has been integrated more closely into the system.  This release
contains X.Org 7.6.  Most of our architectures ship with X.Org, including
amd64, sparc, sparc64 and macppc.  During installation, you can install
X.Org quite easily.  Be sure to try out xdm(1) and see how we have
customized it for OpenBSD.

------------------------------------------------------------------------
- PORTS TREE -----------------------------------------------------------

The OpenBSD ports tree contains automated instructions for building
third party software.  The software has been verified to build and
run on the various OpenBSD architectures.  The 5.0 ports collection,
including many of the distribution files, is included on the 3-CD
set.  Please see the PORTS file for more information.

Note: some of the most popular ports, e.g., the Apache web server
and several X applications, come standard with OpenBSD.  Also, many
popular ports have been pre-compiled for those who do not desire
to build their own binaries (see BINARY PACKAGES, below).

------------------------------------------------------------------------
- BINARY PACKAGES WE PROVIDE -------------------------------------------

A large number of binary packages are provided.  Please see the PACKAGES
file (ftp://ftp.OpenBSD.org/pub/OpenBSD/5.0/PACKAGES) for more details.

------------------------------------------------------------------------
- SYSTEM SOURCE CODE ---------------------------------------------------

The CD-ROMs contain source code for all the subsystems explained
above, and the README (ftp://ftp.OpenBSD.org/pub/OpenBSD/5.0/README)
file explains how to deal with these source files.  For those who
are doing an FTP install, the source code for all four subsystems
can be found in the pub/OpenBSD/5.0/ directory:

        xenocara.tar.gz     ports.tar.gz   src.tar.gz     sys.tar.gz

------------------------------------------------------------------------
- THANKS ---------------------------------------------------------------

Ports tree and package building by Jasper Lievisse Adriaanse,
Landry Breuil, Michael Erdely, Stuart Henderson, Peter Hessler,
Paul Irofti, Antoine Jacoutot, Robert Nagy, and Christian Weisgerber.
System builds by Theo de Raadt, Mark Kettenis, and Miod Vallat.
X11 builds by Todd Fries and Miod Vallat.  ISO-9660 filesystem
layout by Theo de Raadt.

We would like to thank all of the people who sent in bug reports, bug
fixes, donation cheques, and hardware that we use.  We would also like
to thank those who pre-ordered the 5.0 CD-ROM or bought our previous
CD-ROMs.  Those who did not support us financially have still helped
us with our goal of improving the quality of the software.

Our developers are:

    Alexander Bluhm, Alexander Hall, Alexander Schrijver,
    Alexander Yurchenko, Alexandr Shadchin, Alexandre Ratchov,
    Anil Madhavapeddy, Anthony J. Bentley, Antoine Jacoutot,
    Ariane van der Steldt, Austin Hook, Benoit Lecocq, Bernd Ahlers,
    Bob Beck, Bret Lambert, Charles Longeau, Chris Kuethe,
    Christian Weisgerber, Christiano F. Haesbaert, Claudio Jeker,
    Dale Rahn, Damien Bergamini, Damien Miller, Darren Tucker,
    David Coppa, David Gwynne, David Hill, David Krause, Edd Barrett,
    Eric Faurot, Federico G. Schwindt, Felix Kronlage, Gilles Chehade,
    Giovanni Bechis, Gleydson Soares, Henning Brauer, Ian Darwin,
    Igor Sobrado, Ingo Schwarze, Jacek Masiulaniec, Jakob Schlyter,
    Janne Johansson, Jason George, Jason McIntyre, Jason Meltzer,
    Jasper Lievisse Adriaanse, Jeremy Evans, Jim Razmus II, Joel Sing,
    Joerg Zinke, Jolan Luff, Jonathan Armani, Jonathan Gray,
    Jonathan Matthew, Jordan Hargrave, Joshua Stein,
    Kenneth R Westerback, Kevin Lo, Kevin Steves, Kurt Miller,
    Landry Breuil, Laurent Fanis, Marc Espie, Marco Peereboom,
    Marco Pfatschbacher, Marcus Glocker, Mark Kettenis, Mark Lumsden,
    Mark Uemura, Markus Friedl, Martin Pieuchot, Martynas Venckus,
    Mats O Jansson, Matthew Dempsky, Matthias Kilian, Matthieu Herrb,
    Michael Erdely, Mike Belopuhov, Mike Larkin, Miod Vallat,
    Nayden Markatchev, Nicholas Marriott, Nick Holland, Nigel Taylor,
    Nikolay Sturm, Okan Demirmen, Otto Moerbeek, Owain Ainsworth,
    Paul de Weerd, Paul Irofti, Peter Hessler, Peter Valchev,
    Philip Guenther, Pierre-Emmanuel Andre, Pierre-Yves Ritschard,
    Remi Pointel, Reyk Floeter, Robert Nagy, Ryan Freeman,
    Ryan Thomas McBride, Sasano, Sebastian Reitenbach, Simon Bertrang,
    Stefan Sperling, Stephan A. Rickauer, Steven Mestdagh,
    Stuart Henderson, Takuya Asada, Ted Unangst, Theo de Raadt,
    Thordur I Bjornsson, Tobias Weingartner, Todd C. Miller, Todd Fries,
    Will Maier, William Yodlowsky, Yasuoka Masahiko, Yojiro Uo

I noticed that I did get alot alerts from op5 Monitor complaining about high error rate on the external network. After some investigation I noticed that UDP port 5060 generated approx 1.5 Mbps in traffic and that is more or less maximum my ADSL connection can handle. UDP port 5060 that is SIP.

A nice graph showing the errorrates, generated by op5 Monitor:

I looked into my Asterisk log:

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

Conclusion

Someone from 50.97.142.134 tries to register their SIP device on my Asterisk server, they do it an abnormal high rate.

Reaction

I created a block in my firewall on everything from 50.97.142.134. Unfortunatly it does not help much because it is on the wrong side of the ADSL connection. But I get rid of the handshaking and filled logs.

A whois search showed that the traffic comes from Softlayer in Dallas, so I wrote an email to postmaster@softlayer.com.

Lets see if I get any reaction

I have a long time looked at Telldus Tellstick that can control wall-plug socket recievers and other devices. Finally I got my hands on one that my collegue wanted to sell.

This article will describe how to get it up and running on a x86_64 CentOS 6 box, the procedure would be the same on RHEL or Feodora and similiar on other Linux distributions. Unfortunatly the only distribution that has pre-made packages is Ubuntu so I will describe howto download the source code, compile it and get the amazing Telldus Tellstick up and running.

Kudos to Telldus that has understood the power of OpenSource and opened up the device to let third-part  vendors make solutions based on the device.

Installation

Install packages needed for the compilation

yum install gcc gcc-c++ autoconf automake cmake

Libconfuse is needed: 

wget http://pkgs.repoforge.org/libconfuse/libconfuse-2.6-2.el6.rf.x86_64.rpm

wget http://pkgs.repoforge.org/libconfuse/libconfuse-devel-2.6-2.el6.rf.x86_64.rpm

Install libconfuse:

rpm -Uvh libconfuse-*

warning: libconfuse-2.6-2.el6.rf.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 6b8d79e6: NOKEY

Preparing...                ########################################### [100%]

   1:libconfuse             ########################################### [ 50%]

   2:libconfuse-devel       ########################################### [100%]

Download, configure, compile, and install the Tellstick package from Telldus 

Configure

Useful links:

• Telldus
• Different Telldus Tellsticks
• Supported devices

As a monitoring fanatic, I think that have mirrored disk without monitor the mirror is worse then not have a redundant disk system at all. If you think that you are safe and you are not, is worse then be aware of a risk and calculate with it.

This article will describe how I did monitoring my FreeNAS system mirrored disks status using Nagios or op5 Monitoring. I assume basic knowledge of Nagios or op5 Monitor and that a Nagios agent is installed.

Demarcation

This article will focus on GEOM raid monitoring. Of cource all important services should be monitored, like HTTP, CIFS, SMB, disk space, CPU Load, memory usage and so on. However that is basic monitoring knowledge and there is alot of documentation on the net howto achive that.

Monitor the GEOM mirror

The most interesting part is to monitor the GEOM mirror to make sure that that the raid1 is working. 

After some googleing I found check_geom, downloaded it and put it in /usr/local/libexec/nagios/ where the other plugins are located.

Add a line to /etc/nrpe.cfg

command[check_geom_mirror]=/usr/local/libexec/nagios/check_geom mirror

and test from Nagios/op5 Monitor host:

# ./check_nrpe -H fnas -c check_geom_mirror
OK mirror  - mirror/2TBmirror0 COMPLETE { ad6 , ad4 }

It works!
 

Configure Nagios or op5 Monitor

In /opt/monitor/etc/services.cfg at the op5 Monitor host

# service 'Disk Raid Mirrors'
define service{
    use                            default-service
    host_name                      fnas
    service_description            Disk Raid Mirrors
    check_command                  check_nrpe!check_geom_mirror
    contact_groups                 it-slav_mail,it-slav_jabber
    }

The result

I tested unplugging the SATA cabel to one of the mirrored disk and after a couple of minutes I got an Critical notification that my disk mirror is broken, I plugged SATA cabel and got a Warning:

After a couple of hours the mirror was OK.

Useful links:

• Previous article describing howto install nagios or op5 Monitor agent on FreeNAS
• op5, an Enterprise Monitor product company
• op5 Monitor, Enterprise Monitoring based on opensource
• Nagios, an excellent opensource monitor tool
• FreeNAS, very capable NAS based on FreeBSD

My old NAS built on top of CentOS has some performence issues and the disks are quit old now. The hardware is leftovers that is approx 8 years old and not highend at that time either. The disks age has worried my for a time and I would like to have higher redunduncy with raid1/mirrored disks.

After some research I decided to run my new NAS on FreeNAS, which has all the features I use, like: CIFS, NFS, FTP, SSH, Rsync, Unison, iSCSI and also easy administration and disk management.

This article will describe how I did setup Nagios or op5 Monitoring on my FreeNAS system. I assume basic knowledge of Nagios or op5 Monitor.

Installing nrpe

Nrpe is the agent that nagios and op5 Monitor uses to monitor a system from the inside.

From the webgui add group nagios and user nagios. User nagios should be a meber of group nagios.

 freenas:~# pkg_add -r nrpe2
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/Latest/nrpe2.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/All/perl-5.10.1.tbz... Done.
Removing stale symlinks from /usr/bin...
    Skipping /usr/bin/perl
    Skipping /usr/bin/perl5
Done.
Creating various symlinks in /usr/bin...
    Symlinking /usr/local/bin/perl5.10.1 to /usr/bin/perl
    Symlinking /usr/local/bin/perl5.10.1 to /usr/bin/perl5
Done.
cd: can't cd to /usr/include
Cleaning up /etc/make.conf... Done.
Spamming /etc/make.conf... Done.
Cleaning up /etc/manpath.config... Done.
Spamming /etc/manpath.config... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/All/libiconv-1.13.1_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/All/gettext-0.17_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.3-release/All/nagios-plugins-1.4.14,1.tbz... Done.
You already have a "nagios" group, so I will use it.
You already have a "nagios" user, so I will use it.

**********************************************************************
 

freenas:~# cp /usr/local/etc/nrpe.cfg-sample /etc/nrpe.cfg

I changed the allowed_hosts line and added my op5 Monitor host.

So the nagios agent works.

Links

• op5, an Enterprise Monitor product company
• op5 Monitor, Enterprise Monitoring based on opensource
• Nagios, an excellent opensource monitor tool
• FreeNAS, very capable NAS based on FreeBSD

Before installing, read the release notes

The widget keeps track of how many times the word "moln" appear on idg.se webpage and displays it in a easy view in The Tactical Overview.

Screenshoots from the different modes:

Links:

• op5 Sandbox with a live cloud widget online, registration is necessary
• Ninja, the new and inovative nagios gui
• Widget exchange area

Kudos to op5 support that has created this excellent widget 

It is very handy to have a temprobe attached to network devices like a Wlan router, if OpenWRT is installed it is easy to use 1-wire devices. By using this method it is possible to create wireless temperature probes that uses Wlan.

Install

root@OpenWrt:~# opkg install owserver owfs         
 

Using

root@OpenWrt:~# owserver -u -C --usb_regulartime -p 3001

root@OpenWrt:~# mkdir /mnt/1-wire         


root@OpenWrt:~# owfs -s localhost:3001 -m /mnt/1-wire

root@OpenWrt:/mnt/1-wire/10.87507C010800# ls /mnt/1-wire/
10.87507C010800  81.543A2A000000  alarm            bus.0            settings         simultaneous     statistics       structure        system           uncached

It works!         
 

To make sure that owserver starts when the router is rebooted, add to /etc/rc.local


/usr/bin/owserver -u -C --usb_regulartime -p 3001

Firewall

My op5 Monitor/Nagios system is on the Wan side of the OpeWRT box so I need to open the firewall to port 3001.

Add to /etc/config/firewall

#1-wire ok on Wan
config rule
    option src        wan
    option dest_port    3001
    option target        ACCEPT
    option proto        tcp
 

Result

To gather the data from 1-wire I use op5 Monitor and my Nagios plugin check_1-wiretemp, the result can be seen below

Links

• OpenWrt
• owfs
• op5 Monitor

It is real simple, just add to following to /etc/config/firewall:

#open ssh on wan interface
config rule               
        option src              wan
        option dest_port        22
        option target           ACCEPT    
        option proto            tcp 

And restart the firewall:

#/etc/init.d/firewall restart

After struggeling with my Linksys WRT-160NL to get OpenWRT stable I finally gave up. It ended upp with DD-WRT instead. Unfortunatly DD-WRT do not have the functionality I want like:

-rndis device so I could use my Android phone to connect to Internet

-1-wire device support

-Nagios agent

The solution was to buy another device where OpenWRT support is working better with USB and Gigabit interfaces. After some research I found TP-Link TL-WR1043ND for approx 50€.

Unpacking

The TL-WR1043ND package is good looking and has a printed GPL license included, that is impressive.

The basic installation with TP-LINK installed firware was easy and stright forward with all information you need in a simple "Quick installation Guide". No need for installing software on a windows machine to be able to access you new device. I think Cisco/Linksys has something to learn.

Installing OpenWRT

Find the Firmware upgrade menu and upload this firmware. It worked perfectly well and seems very stable. A positive suprise is that the WLAN seems to reach longer then the WRT 160NL.

Conclusion

The TP-LINK TL-WR1043ND seems to be a good Wlan router that has a good support for OpenWRT, with USB, Gigabit ethernetinterfaces and good Wlan range.

Background

I have a need of GPS tag my digital photos. Unfortunatly a GPS device to my camera costs approx 230 Eur. In my humble opinion I think that is way of, especially considering that with some software in the camera any standard USB GPS reciever would do the job. They cost approx 40 Eur.

I have an Android phone where there is tons of apps that could create a GPS log whenever I want to, however it drains the battery in just a couple of hours. I want to track my routes at least a whole day without the need to charge.

The solution is to buy a GPS logger and after some research I found Ventus GPS Route Logger G730, and it looked like it would work with Linux.

This article will describe my experience making the G730 Ventus GPS Route Logger and Linux.

Using skytraq

According to webpages skytraq can be used to create GPX tracks from the G730 Logger.

Unfortunatly the skytraq software are not in the Ubuntu repos so I had to download and install it manually. It is downloadable from  http://code.google.com/p/skytraq-datalogger/

It is out of the scope for this article how to compile and fullfill the requirements needs.

When the GPS Logger is attached the following will show up in /var/log/messages:

Jan 15 14:13:08 it-slav kernel: [21689.101101] usb 6-2: new full speed USB device using uhci_hcd and address 3
Jan 15 14:13:09 it-slav kernel: [21689.316181] usb 6-2: configuration #1 chosen from 1 choice
Jan 15 14:13:09 it-slav kernel: [21689.319138] pl2303 6-2:1.0: pl2303 converter detected
Jan 15 14:13:09 it-slav kernel: [21689.332192] usb 6-2: pl2303 converter now attached to ttyUSB0

So now we can communicate with the GPS Logger:

peter@it-slav:~/dl/skytraq/skytraq-datalogger-0.5-1$ ./skytraq-datalogger --info
kernel version: 1.4.8 -- ODM version: 1.8.22 -- revision: 2008-10-23
log_wr_ptr:      28626
total sectors:   510
sectors left:    506
max time:        3600 s
min time:        1 s
max distance:    1000 m
min distance:    0 m
max speed:       1000 km/h
min speed:       0 km/h
datalog enable:  1
log fifo mode:   1
AGPS enabled:    0
AGPS data left:  none
baud-rate:       38400 bps

Get the gpx file:

./skytraq-datalogger --dump >20110310.gpx

And now a gpx file named 20110310.gpx is created.

Setup

There is a possibility to conifigure the behvior of the GPS logger with skytraq, however it is possible to configure more parameters in the Ventus G730 so to fully configure I had to install the windows software that come with the logger.

The configuration is rather strange, there is a possibilty to log per time, over a certain speed and distance for the following:

• General
• Walking
• Car
• Bicycle

How the device knows what type of vehicle I use for traveling and if the General settings override the others is undocumented.

Problems

Most of the time when trying to communicate with the logger it gives this message:

peter@it-slav:~/dl/skytraq/skytraq-datalogger-0.5-1$ ./skytraq-datalogger --info
No response from datalogger.

And after a few seconds it works again. Sometimes unplugging the device and connect it again helps. The behavior is very stochastic and some patience will help:)

The logger can be used as a GPS recever and the output is in standard NMEA format that GPSd understand. So if you want to use it together with your favorite GPS software it works. However my experience is that if you want to dump the output from the G730 Logger the GPS output must be turned of:

./skytraq-datalogger --set-output-off

Suggestion

I suggest that Ventus should open the specification howto communicat with the device so third part software could use the device. To hide that type of information do not provide any advantage, rather the opposite.

I wrote an email to them and the correspondence can be seen below. I hope this blog post will make Ventus change their mind.

–cut–

-------- Original Message --------
Subject: Sv: Ventus g730
Date: Mon, 3 Jan 2011 13:25:49 +0800
From: <support@ventusdesign.com>
To: Peter Andersson <peter@it-slav.net>
CC: Ventus <info@ventusdesign.com>

> Hi
>
> I have just bought a Ventus g730 GPS Logger and I have some feedback.
> Before buying the device I made sure that it fullfilled my requriements:
> -Works with linux
> (http://www.petrilopia.net/wordpress/hardware/ventus-gps-route-logger-g730-linux/)
> -Create gpx files
> -Battery that last at least 12 hours
>
> Unfortunately you have have taken a strange decision and do not support
> Linux. However according to several forums the G730 is compatible with
> http://code.google.com/p/skytraq-datalogger/
> After fiddling with the device I found that the skytraq-datalogger
> sometimes works and sometimes does not. And I probably will return the
> device to the store if I cannot get it to work better with an open and
> free operating system.
>
> My recommendations to you are:
> 1-Provide an open specification how to access the data in the device
> 2-Modify the http://code.google.com/p/skytraq-datalogger/ software so it
> works with G730 and provide the code change to the team behind
> skytraq-datalogger, and also provide the software on you website.
>
> By following these simple steps (especially #1) you will sell a-lot of
> more devices to the people that prefer open and freedom.
>
> Best regards
> Peter Andersson
> peter@it-slav.net
 

The dashboard has been modified and improved by the community and an example can be seen at op5 sandbox dashboard and can be downloaded from op5 community.

The original project place is http://dingleberry.me/2010/04/our-new-dashboard/

News in op5 Monitor 5.3

• Avail and SLA reports including performance graphs

   

op5 Monitor 5.3 introduce the possibility to include performance graphs in avail reports. This mean you can, for example, create a response time report and not only get the availability but a graphical representation of the actual response times. This will, for instance, help the user in capacity planing since the graphs may present trends that show when disks are full, response times exceed threshold etc.

• Avail and SLA reports including summary report

The avail and SLA reports can now also include a Summary report. There are a number of different Summary reports available. For example, Top Alert Producers and Most Resent Alerts. This mean the report can display what service is causing the most alerts or at what time the respective services breached warning or critical thresholds hence providing a more fine grained view of the report period.

• Automatic Network scan

The Configuration tool in op5 Monitor have for a long time had the ability to scan a network and discover hosts that are not monitored, This scan can now be scheduled to run each night. When the scan discovers one or more new hosts the user is notified and presented with a link leading to the configuration tool. This mean you can minimize the risk of forgetting to monitor hosts added to your network.

• User interface performance improvements

A lot of effort have been put into improving the overall performance of the user interface. This involve indexes in the database as well as optimizing the database queries. Depending on the environment and what pages are used the user can expect an performance improvement of 50% overall and in some cases, host/service group pages in particular, several times more.

More Information about op5 Monitor 5.3

• Release Note for op5 Monitor 5.3
• Change Logs
• Product Roadmaps
• SandBox Live Demo

The rite of passage into adulthood was smoother than expected, but not
without minor bumps. As with people, those bumps made the code stronger.

Testing would be most welcome.

Noteworthy bugfixes in 1.0.0 vs 0.9.0:
The protocol has been changed so that Merlin now adds a signature at
the start of each packet. If this signature isn’t found, the offending
node is disconnected. This will restore synchronization in case the
stream ever gets garbled. It also means that older merlin daemons (0.9.x)
won’t be able to talk to newer merlins (>= 1.0).

A serious issue has been found and fixed in the event-reading code
where one node might experience crashes after some time of running.
The crash only happened if the read-buffer gets filled twice directly
after each other and a packet is broken in two each time the buffer
gets filled. Subsequent reads would then overwrite other segments of
memory, causing random crashes later on (or sometimes not at all,
depending on the load-order of configuration items; Yes, it was that
weird). This issue has been rather rare. In fact, only one customer
that I know of have seen it, and even they only occasionally (although
with an unacceptably high frequency, such as a few times per week).
The symptoms include daemon and module crashing a few seconds apart
at most, and usually on the same second.

An issue in the module has been corrected. This issue could sometimes
cause Nagios to hang indefinitely when exiting or reloading, and could
at other times lead to Nagios crashing on reload or exit. It appears
this only happened upon a hard restart (in which case everything worked,
although the exit-procedure resulted in a segfault rather than a clean
exit), or when the RESTART_PROGRAM command is submitted to the command
pipe, in which case it blocked execution indefinitely by either stalling
or crashing.

Apart from those two issues, no flaws have been found in either module
or daemon during the month that merlin 0.9.0 has been running at most
of our customers and a lot of users worldwide.

Some minor issues have also been fixed, such as:
import and showlog have been fixed to be sturdier when looking at
broken or borked logs.

A hexlog function is added (although disabled). Users with troubles on
systems where I can’t test are welcome to enable hexlog debugging of
inbound (and outbound) events.

Excessive logging for disconnected or unreachable nodes has been fixed.

The binary backlog api has been fortified. There were no issues in it,
although I thought there was, so several checks now ensure that it will
always remain in good working order.

Several thousand new tests have been added to ensure receiver stability
when facing broken packets.

Some minor memory leaks have been fixed. They were not repetitive and
therefore never endangered the runtime stability, and hence they go
under the ‘minor issue’ list.

Features added in 1.0.0:
One can now set "takeover = no" for a poller node and have the master
node ignore taking over checks for that node.

There’s now more indexes on the database, which will significantly
improve Ninja’s performance. Some query analysis has been done to
verify this and come up with the indexes now added.

Special thanks to the fine folk (yes, it’s autogenerated) for helping
us bring Merlin 1.0.0 out the door by testing, reporting bugs and
submitting patches:
Andreas Ericsson <ae@op5.se>
Ken Menzel <Ken.Menzel@fisglobal.com>
Martin Kamijo <mk@op5.com>
Mattias Bergsten <mattias@westbahr.com>
Ola Sandström <ola.sandstrom@WSPGroup.se>
Per Asberg <perasb@op5.se>
Robin Sonefors <robin.sonefors@op5.com>
Stephan Beal <sgbeal@googlemail.com>

For the full list of contributors, clone the git repository and run
  make thanks

– Andreas Ericsson

As everybody knows, the answer is 42.

Today is the date 101010, translate that binary number to decimal, you get 42.

When I was preparing a presentation about what op5 is doing and our contribution to the community, I went to ideas.nagios.org. When I browsed the list of the biggest issues with Nagios I found out that op5 has packaged and solved them all in op5 Monitor. I encourage everyone to take a peak at the list and judge for them self what platform you want to use for your enterprise monitoring solution.

1. Nagios Clusters

op5 has developed Merlin (Module for Effortless and Redundant and Loadbalanced Infrastructure with Nagios). It provides the possibility to have a redundant or/and loadbalanced solution using Nagios. It also provides a scalable database backend for Ninja and Nagvis, take a peak at Merlin webpage.

2. Performance Graphing

In op5 Monitor op5 has added pnp4nagios that graphs more or less any check that gives a numerical value back.

3. New Status Map

We provide a new statusmap, it is a part of Nagvis project and called Automap. op5 has contributed to Nagvis by adding support for Merlin database backend and GoogleMaps integration

4. Better Interface

op5 has created a new PHP based interface that makes you proud of your monitor solution, called Ninja. Ninja stands for Nagios is now Just Awesome, it looks so good so you can show it to your manager:-) Take a peak at Ninja webpage.

We had a naming competition at op5 my suggestion to call it Yang, Yet Another Nagios Gui, did not get so many votes so the name become Ninja.

5. Configuration

Any one that has tried to configure Nagios by hand has thought bad thoughts. It is cumbersome and in the begining it makes even a skilled Unix admin frustrated. op5 has developed a web based configuration tool named Nacoma. It is included in op5 Monitor.

6. SLA reports

op5 has got alot of feedback from the customers and one issue was better reports, we have created two reports: availability and SLA report. Both of them can be scheduled and looks god compared to the reports in Nagios, they are included in Ninja

 
7. More members in core team

Well, this is something that op5 cannot controll, however, Andreas Ericsson one of op5s core developers are a member og Nagios steering board.

8. SNMP Trap receiver

This is something op5 is working on right now, we have today created a prototype together with a customer and depending of the outcome of that project we will know better when it will be available to the market.

9. UI-Improvment

op5s solution to this issue is Ninja, a better, nicer, scalable, database backend gui to Nagios.

10. Nagios Dashboard

A newTactical Overview with widgets or Nagvis, all included in Ninja

Links

• ideas.nagios.org http://ideas.nagios.org
• Nagios http://www.nagios.org
• op5 http://www.op5.com
• op5 Monitor http://www.op5.com/op5/products/network-monitor
• Ninja http://www.op5.org/community/plugin-inventory/op5-projects/ninja
• Merlin http://www.op5.org/community/plugin-inventory/op5-projects/merlin
• Nagvis http://www.nagvis.org/
• pnp4nagios http://docs.pnp4nagios.org/

I have found one way of still doing geeky things combined with vacation. That is update OpenStreetmap. Take a peak at my updates at:

• http://www.openstreetmap.org/?lat=56.4476895332336&lon=14.0158939361572&zoom=14
• http://www.openstreetmap.org/?lat=56.4213931560516&lon=14.0284895896912&zoom=14
• http://www.openstreetmap.org/?lat=60.1418289542198&lon=15.4323390126228&zoom=17

Openstreetmap is wikipedia for maps, and like wikipedia it is dependent of that people will update with correct data. One thing that fascinates me is that openstreetmap is extremly detailed in some areas. Probably a geek lives nearby.

When the earthquake hit Haiti, OpenStreetmaps was used by rescuers because all other maps was inaccurate. Alot of volunteers helped keeping the maps updated.

If you want to participate in the project, the process is simple:

1. Get an GPS reciever that can generate gpx tracks, I use my Android
2. Turn on the GPS and walk, run or bicycle around and gather GPS tracks
3. Upload the gpx to openstreetmap.org
4. Tag the roads, buildings and whatever you found

 To get started, read the beginners guide at Openstreetmap.

Finally it has arrived, Android 2.2 Froyo to HTC Desire. When I installed it I run into several problems and this is my guide how I solved it. It was a rather frustration experience and took a long while to solve. I ended up with a phone where almost everything was gone and I have to reinstall all my apps and configurations again.

I have not rooted my phone or changed anything that HTC or Google allow, so no rooted phone. I’m probably a poweruser but there is no obvious reason why I ended up with all these problems.

During this process I tried to find any information at HTC support site but I could not find anything that helped me. The best source of information is to use different user forums. My opinion is that HTC support web is crap.

This guide shows how I solved the problems I run into, I guess there is better way of doing it because this was a cumbersome and painfull process. I do not take any responsiblity if you end up with a bricked phone or loss of data.

Happy reading!

Installing, first try

Roumors on a swedish site, swedroids.se claimed that Froyo was out. My Desire had not given me the message that an update was available automatically as it should. Probably it just do not check with a high enough frequency to detect it.

I did a manual check and yipii, there was a system update available.

I choosed to do the update over the air, meaning that I used the phones update function.

After downloading and installing approximatly 30 minutes, the phone just showed the HTC logo and after that rebooted over and over again.

My conclusion was that the update has failed.

Recover, first try

I pulled out the battery and put it back again, pushed vol- and power and keept vol- pushed. Then I get into a menu where I could do some choices, I picked recover.

Now the phone managed to get passed the HTC logo and started up. It gave i progress bar starting from 0%and counting up to 92%, after a while it showed an error message "Sorry! Process system is not responding" with 2 options "wait" or "force restart" (or similar). Independent of which I pick or none, after a while the phone restarts with the same progress bar and error message. 

Install, second try

At this moment a couple of hours has passed and I was getting real frustrated. Google and swedroid forums showed me that alot of people seemed to have the same problems as I do.

I find the following guide that seemed to helped alot of people:

1. Download the Android 2.2 firmware for the Desire http://liveimage.htc.com/OTA_Bravo_Froyo_HTC_WWE_2.09.405.8-1.21.405.2_release4dua3bcrut4wrv30.zip

2. Rename the file update.zip and copy it to your microSD card via USB. I used an USB 3G dongle with an microSD reader.

3. Power down your Desire

4. Hold down the “Volume Down” button as you power the phone back on.

5. A screen should appear showing your phone’s system searching for various files. Scroll down to “recovery” and press the “Power” button.

6. When you see the triangle with an exclamation point symbol, press the “Power” and “Volume Up” buttons at the same time.

7. From the menu that appears, select “Apply sdcard:update.zip.”

8. When the screen displays “Install from sdcard complete” select “reboot system now” and wait for the phone to power back up.

It looked very promising, with text and progressbars showing that an installation was taking place. After a while the phone restarted and stoped with a triangle and exclamation mark. I tried it again and again and again…. With the same result.

Calling for help

At this point I get real desperate so I went to HTC supportweb and wrote an email describing my problems. After 1½ days I have  got an email that told me to send the phone to repair. 

Then I tried to call HTC support, they answered very fast but the support only told me to send the phone to the reseller to get it fixed. No trouble shooting at all and no questions what I have done.

Because I have important data, like logins to mail account, sip PBX, monitor solutions, facebook account and so on I decided that I would not send in the phone until I have waited a couple of days to see if something new shows up on different forums.

Install, twentieth something try

At this point I realized that all my settings will be lost I did a clean cache and a factory reset and did a new try with manual installation and this time it looked all right. The phone started up normally and asked for email accounts, facebook credentials and so on.

All my personal settings and data was gone but the phone seemed to work. I started to download a app and the phone rebooted again, sic. Everytime I installed a new app it rebooted and the app did not install.

But I remembered that I read that this was a common problem I searched the forums and did some Googling. Some people has solved this issue by do a "factory data reset" from menu->settings->SD & phone storage->Factory data reset.

So I tried that and everything was gone again, I entered the credentials to my accounts and did some setup. And tried to install a new app, and YES it did not reboot.

Conclusions

This behavior by a product that I have not hacked in any way is not acceptable. I work at a software company and know that there might be bugs but these bugs was so big and costed me so much time that it should have been tested better.

Even worse is that HTC support website give no hints what-so-ever how to solve these issues I had and HTC telephone support did not even try to help me. I think HTC has a lot todo to get a better and more professional support.

In this case enthusiasts and helpfull people that publish their knowledge helped me and probably many others, thanks. Especially swedroid.se which is an excellent site where swedish speaking people can get help with thier expensive Android phones.

I also think that HTC should encourage enthusiasts by be more open and support the possiblity to root the phones. I think that they will make more money in the long run, compare it with Linksys routers and OpenWRT.

Update-Aug 8:th 2010

I was contacted by HTC UK to get more feedback to their support site. It always make me happy when a company listen to the customer and try to improve the customer experience. Hopefully the HTC support web will contain enough info in the future so I could resolve the problems I had by going there. Good work HTC!