packaging-farm for OpenStack Essex

The Essex OpenStack release is being packaged for Debian GNU/Linux wheezy upcoming release. Each new release of a package is committed to a git repository ( the keystone is an example ) by a member of the OpenStack Debian GNU/Linux release team and uploaded to Debian GNU/Linux : a process that can require a few weeks before it is available in the Debian GNU/Linux worldwide mirror sites. In the meantime, a temporary repository is setup with packaging-farm to host the packages being processed so that they can be used immediately by adding the following line to the source list of packages.

deb http://wheezy.the.re/packaging-farm/openstack/gnulinux/debian wheezy main

packaging-farm support for OpenStack

Each new release of a package is committed to a git repository ( the keystone is an example ) by a member of the OpenStack Debian GNU/Linux release team and uploaded to Debian GNU/Linux : a process that can require a few weeks before it is available in the Debian GNU/Linux worldwide mirror sites. In the meantime, a temporary repository is setup with packaging-farm to host the packages being processed so that they can be used immediately.
Support for xz source archive was implemented in packaging-farm : it is the format used for all OpenStack source archives in Debian GNU/Linux.
Although packaging-farm can be used to build a binary package from a source package, building the source package from the git repository ( keystone for instance ) requires less manual steps and a script was added to the 2.0.20 release of packaging-farm.

instance dedicated to packaging

packaging-farm is available for Debian GNU/Linux squeeze only but is able to create wheezy packages which is the targetted release. A Debian GNU/Linux squeeze AMI is created. A 1G disk image is initialized with:

# kvm-img create -f raw server.img 1G

The squeeze 6.0.5 ISO image is downloaded:

# wget http://cdimage.debian.org/debian-cd/6.0.5/amd64/iso-cd/debian-6.0.5-amd64-netinst.iso

It is booted with kvm:

# kvm -m 256 -cdrom debian-6.0.5-amd64-netinst.iso \
  -drive file=server.img,if=scsi,index=0 -boot d -net nic \
  -net user -vnc :0

And installed interactively with

# xtightvncviewer :0

so that it is all installed in a single primary partition of the disk. The partition is then extracted with:

# kpartx -av server.img
# dd if=/dev/mapper/loop0p1 of=partition.img

The partition is then mounted to get access to the files it contains:

# mount /dev/mapper/loop0p1 /mnt

The kernel and the initrd files are first uploaded to glance with:

# glance add is_public=true name='linux-2.6.32-5-amd64' \
  disk_format=aki container_format=aki < /mnt/boot/vmlinuz-2.6.32-5-amd64
Uploading image 'linux-2.6.32-5-amd64'
Added new image with ID: 42d6e697-28b2-40b7-adcb-7c77b209bf97
#  glance add is_public=true name='initrd-2.6.32-5-amd64' \
  disk_format=ari container_format=ari < /mnt/boot/initrd.img-2.6.32-5-amd64
Uploading image 'initrd-2.6.32-5-amd64'
Added new image with ID: 6bd78e8c-7ada-4e6f-bf5c-158de71c7664

and associated to the file system contained in the partition.img file with:

# glance add is_public=true name='Debian Squeeze 6.0.5' \
  kernel_id=42d6e697-28b2-40b7-adcb-7c77b209bf97 \
  ramdisk_id=6bd78e8c-7ada-4e6f-bf5c-158de71c7664 \
  disk_format=ami container_format=ami \
  < partition.img
Uploading image 'Debian Squeeze 6.0.5'
Added new image with ID: d12a0ca3-88a2-4a20-992e-09e31e26bdda

A new flavor is created to get a 30GB primary disk instead of the 10GB default:

#  nova flavor-create e.1-cpu.30GB-disk.1GB-ram 11 1024 30 1
# nova flavor-show 11
# nova flavor-list
+----+-----------------------------+-----------+------+-----------+------+-------+-------------+
| ID |             Name            | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor |
+----+-----------------------------+-----------+------+-----------+------+-------+-------------+
...
| 11 | e.1-cpu.30GB-disk.1GB-ram   | 1024      | 30   | 0         |      | 1     | 1.0         |
...
+----+-----------------------------+-----------+------+-----------+------+-------+-------------+

An instance is created using the Squeeze 6.0.5 image and the 30GB flavor : the file system from the image is resized to use all of the 30GB.

# nova boot --image d12a0ca3-88a2-4a20-992e-09e31e26bdda \
   --flavor 11 --key_name loic --poll --availability_zone=bm0001 \
  packaging-farm
# ssh -t -A root@bm0001.the.re ssh root@10.145.0.9 df -h /
Filesystem            Size  Used Avail Use% Mounted on
/dev/vda               30G  590M   29G   1% /

packaging-farm installation and configuration

packaging-farm-2.0.20 is installed and a few packages are created with:

# packaging-farm SOURCE=keystone submit ; packaging-farm keystone
# packaging-farm SOURCE=nova submit ; packaging-farm nova
# packaging-farm SOURCE=python-novaclient submit ; packaging-farm python-novaclient

And each can be installed from their individual repositories by adding to the sources.list files:

deb http://wheezy.the.re/packaging-farm/keystone/gnulinux/debian/x86_64/wheezy/src/ ./
deb http://wheezy.the.re/packaging-farm/nova/gnulinux/debian/x86_64/wheezy/src/ ./
deb http://wheezy.the.re/packaging-farm/python-novaclient/gnulinux/debian/x86_64/wheezy/src/ ./

They are all included in a meta-package defined in /etc/packaging-farm/openstack.mk

PACKAGE=openstack
COMPONENT=main
CHILD_PACKAGES = nova keystone python-novaclient
ROLE = meta-package
LIBDIR ?= /usr/lib/packaging-farm
include ${LIBDIR}/Makefile

which is the concatenation of the individual repositories mentionned in the CHILD_PACKAGES variable. It can be used by adding the following line in sources.list:

deb http://wheezy.the.re/packaging-farm/openstack/gnulinux/debian wheezy main

Updating the repositories

Whenever a new package is available from the Debian GNU/Linux OpenStack packaging team git repositories, the following commands can be run on the instance dedicated to packaging:

# for i in nova keystone python-novaclient ; do packaging-farm SOURCE=$i submit ; done
# packaging-farm openstack

To reduce it to a onliner, the following rule is added to /etc/packaging-farm/openstack.mk :

update:
           for package in ${CHILD_PACKAGES} ; do packaging-farm SOURCE=$$package submit ; done

and the update operation can now be reduced to

# packaging-farm --cd openstack update openstack

It will rebuild the packages recursively, checking each of the child package of the openstack meta package for updates.

DNS and proxy

The reverse proxy entry is defined, the wheezy.the host is declared in /etc/puppet/files/hosts for puppet to install on every host and is associated with the private IP of the virtual machine.
The DNS of the.re is edited to add a CNAME so that wheezy becomes a virtual host using the same IP as www.

Getting sources

Rebuilding a package for a debian version greater than -1 requires to reuse exactly the same orig.tar.xz. It can be downloaded from a source repository such as:

deb-src http://ftp.fr.debian.org/debian/ unstable main

which can be added to the sources list with

# cat > /etc/apt/sources.list.d/unstable-src.list <<EOF
deb-src http://ftp.fr.debian.org/debian/ unstable main
EOF

It can then be used by specifying the target release for a source download as follows:

# apt-get source nova/unstable
This entry was posted in Essex, debian, openstack. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>