osm.fsffrance.org migration

The osm.fsffrance.org was migrated to the tetaneutral datacenter. It was reconfigured to benefit from a 250GB SSD. The IPv4 assigned to it goes from OVH to tetaneutral (and back) using an IPv6 tunnel.

SSD and ganeti

The version of ganeti on which the FSF France cluster hosting osm.fsffrance.org runs does not allow to select a volume group when adding a disk to an instance. The SSD (available as /dev/sdb1) was added to the volume group used by ganeti but the volume group was forbidden to allocate any space on it:

pvchange  --allocatable n /dev/sdb1

When the time came to add a disk to osm.fsffrance.org, the SSD was made allocatable, and allocation on the SATA disk (available as /dev/sda1) was forbidden:

pvchange  --allocatable y /dev/sdb1
pvchange  --allocatable n /dev/sda1

The disk is then added added using:

gnt-instance modify --disk add:size=250G osm.fsffrance.vm.gnt

And then the SATA is made to be the default disk on which to allocate space again:

pvchange  --allocatable n /dev/sdb1
pvchange  --allocatable y /dev/sda1

.

IPv6 tunnel

The IPv6 bandwidth for tetaneutral does not cost. Instead of maintaining IPv4 tunnels between the nodes of the ganeti cluster located at tetaneutral and OVH, they were converted into IPv6 tunnels by adding:

proto udp6

to each end of the openvpn tunnel. Assigning IPv6 to the ganeti host themselves required different configuration on OVH and tetaneutral.

Public IP on a provider served by a VM on another

The osm.fsffrance.org public IP is assigned to a ganeti node located at OVH. However, the virtual machine is running on a node located at tetaneutral. The incoming traffic is routed to the VM using the following shorewall lines on the OVH ganeti node in the file /etc/shorewall/nat:

91.121.33.240          eth0:240        10.10.1.66      Yes

Where 91.121.33.240 is the osm.fsffrance.org IP and 10.10.1.66 is the private IP assigned to the virtual machine located at tetaneutral. The virtual machine receives the bytes which are routed to it using the IPv6 tunnel mentionned above. However, when the virtual machine sends a reply, the tetaneutral host sends it using the default route of the tetaneutral node. As a consequence, the OVH node never sees the packet and is unable the NAT it properly. On the tetaneutral node, a route is defined by which all outgoing traffic of 10.10.1.66 goes to the tunnel connected to the OVH host.

ip route add default via 10.1.6.9 dev tun6 table 6
ip rule delete from 10.10.1.66 table 6 > /dev/null || true
ip rule add from 10.10.1.66 table 6

The configuration is documented in the
route all VM network traffic to a designated host and IP on the cluster HOWTO.

This entry was posted in FSF, ganeti, openstreetmap. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>