I have a FON router which allow anyone to connect to Intenet via my network connection. I like the openess and idea behind FON but I do not want to get in trouble if someone do bad things and using my network connection.

One solution is to connect the FON accesspoint to a separate network segment and let all traffic from that network go through TOR, the onion router. 

Another feature is when I want to be anonymous on Internet I plugin my computer to that network segment.

Setup

I use OpenBSD as my firewall and the first step is to download, compile and configure TOR. I downloaded the tarball from https://www.torproject.org and used the normal procedure:

# wget https://www.torproject.org/dist/tor-0.2.2.35.tar.gz

# tar xzvf tor-0.2.2.35.tar.gz 

# cd tor-0.2.2.35                                                                                                                                                                

# ./configure&&make&&make install

Configure pf

I use a specific ethernet interface, fxp0, which will route all traffic into tor. 

--cut from /etc/pf.conf--

#Tor traffic

tor_if ="fxp0"

# Tor's TransPort

trans_port = "9040"

pass in quick on $tor_if inet proto udp to port domain rdr-to 127.0.0.1 port 5300 

pass in quick on $tor_if inet proto { tcp udp } to !($tor_if) rdr-to 127.0.0.1 port $trans_port

--end cut--

Configure tor

# cat /usr/local/etc/tor/torrc

VirtualAddrNetwork 10.192.0.0/10

AutomapHostsOnResolve 1

TransPort 9040

DNSPort 5300

Log notice syslog

#Log debug stderr

RunAsDaemon 1

Note: I know that best practice is to let the tor process run as non root user. That requires read access to /dev/pf and I did not bother to get it working.

Dhcpd config

I assume that a OpenBSD sysadmin knows how to setup dhcpd so I will just show the config addon I did to /etc/dhcpd.conf.

shared-network tor-net {

        option  domain-name "it-slav-tor-net";

        option  domain-name-servers 10.1.2.1;

        option  ntp-servers 10.1.2.1;

        subnet 10.1.2.0 netmask 255.255.255.0 {

                option routers 10.1.2.1;

                range 10.1.2.100 10.1.2.200;

        }

The ip-adress of the fxp0 interface is 10.1.2.1

Start tor

# /usr/local/bin/tor

Jan 10 20:52:48.880 [notice] Tor v0.2.2.35 (git-b04388f9e7546a9f). This is experimental software. Do not rely on it for strong anonymity. (Running on OpenBSD i386)

Jan 10 20:52:48.885 [warn] It's a little hard to tell, but you seem to have Libevent 1.4.0-beta header files, whereas you have linked against Libevent 1.4.14b-stable.  This will probably make Tor crash.

Jan 10 20:52:48.886 [notice] Initialized libevent version 1.4.14b-stable using method kqueue. Good.

Jan 10 20:52:48.886 [notice] Opening Socks listener on 127.0.0.1:9050

Jan 10 20:52:48.887 [notice] Opening Transparent pf/netfilter listener on 127.0.0.1:9040

Jan 10 20:52:48.887 [notice] Opening DNS listener on 127.0.0.1:5300

Final step

Plugin your fon router and enjoy!

Links

The hints to this article was found at:

•  https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/TransparentProxy 

As an it slave, I have the same problems as most datacenters:

• Running out of space because of more and more machines get into the datacenter
• Overheating
• Powerconsumption

Added to the "normal" datacenter problems, I also have to deal with:

• Wife acceptance
• Noice, my neighbours has complained about the noice outside my basement.

To address this issues I will run a consolidation project to get fewer machines by using virtualization. As virtualization engine, KVM is choosed.

This article will describe some pitfalls I run into and how I solved them.

KVM plattform

KVM seems to be the future for virtualization within the Linux area since Xen has done some mistakes.

As I have good experience of running CentOS and has KVM included I decided to use CentOS as the plattform for my virtualization.

Issues

Bridge network

To let the virtual boxes has full access to the network you need create a network bridge, it is rather straightforward and well documented so I will skip this part. I used the documentation on Red Hat customer support. Why KVM requires a bridge instead of a normal NIC is another question

Snapshots

One handy feature with virtualization is that the virtual systems can be one big fat file at the physical host, this gives the possibility to make full backups without turning the system down a.k.a snapshots.

However to get snapshot to work the filetype must be qcow2 and raw is the default format.

It is possible to convert from raw to qcow2 format by running the command:

qemu-img convert -f raw -O qcow2 <virtualhost>.img <virtualhost>.img.qcow2

Remember to turn of your virtual system by running:

virsh shutdown <virtualhost>

After the conversion:

virsh edit <virtualhost>

modify the type and path, i.e.

      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/web.img.qcow2'/>

Start your virtual system:

virsh start <virtualhost>

Take a snapshot with:

qemu-img snapshot -c <snapshotname> <virtualhost>.img.preallc.qcow2

To create a full image of your snapshot you need to compile a more resent version of qemu-img then shipped with CentOS 6.2, I download a later version and compiled it from here, i renamed the new to qemu.img2 and copied it to /usr/local/bin

qemu-img2 convert -p -f qcow2 -O qcow2 -s <snapshotname> <virtualhost>.img.preallc.qcow2 <targetpath>

Beware: This command can use all your resources and may affect all your virtual systems, use nice and ionice to prevent it.

Disk I/O

After installing just a few virtual system I noticed that my physical host got alot of I/O wait, it can be seen by using tools like top.

To monitor this I installed the nagios plugins:

• check disk io, can be found here
• check cpu stats, can be found here
• check_libvirt, developed by op5 and an Howto

However it seems like that though my virtual systems did more or less nothing they caused alot of disk I/O on the physical system.

I did some investigations and read quite many articles, fiddled and tested. The following is what worked for me in my setup.

#1 Change disk scheduler

The disk scheduler can be changed on runtime by modify the file:

/sys/block/sda/queue/scheduler

to see what scheduler you use now:

cat /sys/block/sda/queue/scheduler

Change sda to the device you have.

The scheduler that worked best for me is deadline

echo deadline > /sys/block/sda/queue/scheduler

#2 Mount with noatime

A feature in Unix and other Unixlike systems like Linux is that it normaly stores when a file is accessed. So one read always produce a write and if you are using raid like morroring this get worse, one reade always generate several writes. This creates alot of overhead for a feature soldom used.

So change /etc/fstab so it will mount the file systems with noatime.
An axample:

UUID=a290aa4b-635c-45fa-b144-1fbef90b3735 /                       ext4    defaults,noatime        1 1

#3 Preallocation disk images

A real boost that is hidden in the featureset and not shown in the virtualmachine gui is using preallocation disk images. They cannot be created from the GUI so I install the virtual machine, turn it of and convert it afterwards.

qemu-img convert -f qcow2 -O qcow2 -o preallocation=metadata <virtualhost>.img.qcow2 <virtualhost>.img.preallc.qcow2

Change the path to new image name by editing the virtual machine settings

virsh edit <virtualhost>

Conclusions

In my opinion it seems like KVM is still a little bit immature or at least the tools to handle it. Maybe it would be a good idea to have one linux distro focusing on beeing the best platform for virtualization.

I am convinced that I can get even more bang for the bucks out of my installation if I learn how to tweak it even more, so if you have any hints, do not hesitate to contact me. Preferably as a comment to this blogpost.

Referenses:

• op5 Monitor, a nagios based enterprise monitor tool to monitor my environment
• check_libvirt, a nagios plugin done by op5 to monitor KVM
• check_diskio, a nagios plugin to monitor disk I/O
• check_cpu_stats, a nagios plugin to monitor cpu usage rather detailed
• Another blogpost which helped me
• KVM performance improvements and optimizations – Red Hat presentation

The new access speed is 100 Mbit/s optical fiber, iperf tests show that I can get approx 97 Mbit/s in booth directions.

Changes:

• 100 Mbit/s internet access provided by Ownit
• Updated firewall to OpenBSD 5.0 running on an Astaro appliance
• Updated Webserver to Centos 6.2, virtulized with KVM
• Updated Mailserver to Centos 6.2, virtulized with KVM

Happy new year

I noticed that I did get alot alerts from op5 Monitor complaining about high error rate on the external network. After some investigation I noticed that UDP port 5060 generated approx 1.5 Mbps in traffic and that is more or less maximum my ADSL connection can handle. UDP port 5060 that is SIP.

A nice graph showing the errorrates, generated by op5 Monitor:

I looked into my Asterisk log:

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

[Sep  2 20:01:35] NOTICE[2459] chan_sip.c: Registration from ‘"3959" <sip:3959@82.182.144.134>’ failed for ’50.97.142.134′ – No matching peer found

Conclusion

Someone from 50.97.142.134 tries to register their SIP device on my Asterisk server, they do it an abnormal high rate.

Reaction

I created a block in my firewall on everything from 50.97.142.134. Unfortunatly it does not help much because it is on the wrong side of the ADSL connection. But I get rid of the handshaking and filled logs.

A whois search showed that the traffic comes from Softlayer in Dallas, so I wrote an email to postmaster@softlayer.com.

Lets see if I get any reaction

I registered on their homepage and after a couple of days I got a snail mail with my credentials.

I configured my FreePBX and calling in worked directly, but not outgoing. After 2 hours of troubleshooting I started to google "phonzo asterisk" and found several people that has the same experience. The reason is that Phonzo does not accepted "Asterisk PBX" as useragent and that is default in Asterisk.

After changing sip.conf

[general]

...

useragent=it-slav PBX

....

It worked!

I do not like unlogical stupidity so I sent an email to the support and the following bizare mail conversation occoured:

I will update this post when new info arrives.

I know for sure that the problems was not on my side, because a friends friend that lives in the same area had the same problem.

This graph shows the packet lost when pinging the closest router:

The more logical name gives date and title of the blog post instead of a cryptic number. Changing from one to another is rather simple, just change it in the wordpress settings, but there is several drawbacks:

• Referers stop working, if someone is refering to your pages they will stop working until someone updates the link
• Search engines will refer to the wrong page until the pages are indexed again and very likely the rank will drop
• WordPress internal references will stop working, so the mainpage links will stop working. This is real stupid and I do not understand why WordPress behaves like this.

The solution is to install the plugin  "Permalinks Moved Permanently" which fixes this problem, make sure that the webserver allows rewrite.

Links

• The plugin used Permalinks Moved Permanently
• A webpage describing howto enable mod_rewrite for Ubuntu Server.