Uploading Bio++ packages to Debian

What does it take for Bio++ to enter the official repositories of Debian GNU/Linux ? Julien Dutheil and Loïc Dachary worked together to polish the packages and contribute them to Debian. This page is a summary of the topics that were discussed, by mail and on IRC.


Request For Packaging

The first step is to ask for a Debian Developer to package the software. This is called a RFP as in Request For Packaging and it is done by filling a bug report against the wnpp pseudo package. Since Debian packages for Bio++ are already provided the task of the Debian Developer is facilitated.
Even when Debian packages are already available, the odds of seducing a Debian Developer into taking responsibility for the package are small. The quality standards for a package entering Debian are higher than what is implemented by the amateur packager. However, meeting these standards is not a difficult task when using the proper tools.

Bio++ RFP

Bio++ is made of height packages. In theory it would be necessary to create one RFP per package. When a package is uploaded and shows in the new packages queue, a human being will review its content and read the corresponding RFP. As long as all packages are uploaded at the same time, it is likely that the same person will attend all of them at once and won’t object to the single RFP.

The link between a package and the corresponding RFP is done by the following line in debian/changelog

* RFP: Bio++ -- The Bio++ bioinformatics libraries. (Closes: #616373).


Resolving packages dependencies

Using pbuilder ensures that the package can be built on a virgin Debian GNU/Linux installation.
When trying a build on a machine used for other purposes, it is common to forget that it can’t be built unless a specific library is already installed.

	apt-get install pbuilder
	pbuilder --create

Will create a virgin GNU/Linux and archive it in a tar.gz (in /var/cache/pbuilder). It can then be used to build the package (for instance libbpp-core) as follows:

	pbuilder --build libbpp-core_2.0.1.dsc

will copy the package in a chroot extracted from the archive created by pbuilder and try to build it. When finished the result can be found in /var/cache/pbuilder/result.


Packages interdependencies

What should be done for libbpp-seq to build ? It relies on libpp-core which is not standard in Debian and should retrieve it from a previous build. Assuming a web server is installed locally, a Debian repository can be built as follows:

	ln -s /var/cache/pbuilder/result /var/www/result

and a reference to this repository can be added to the pbuilder chroot as follows:

	pbuilder --login  --save-after-login
	echo deb http://localhost/result/ ./  >> /etc/apt/sources.list
	Control-D

From now on, building a package will lookup this repository in addition to the standard ones. Whenever a new package is added to this repository, as a result of a successfull call to pbuilder, the pbuilder chroot must be updated as follows:

	( cd /var/www/result ; dpkg-scanpackages . /dev/null | gzip > Packages.gz ; dpkg-scansources . /dev/null | gzip > Sources.gz )
	pbuilder --update

Julien Dutheil used this technique to ensure that the Build-Depends of each package was accurate.


Packaging farm

In order to review the packages, Loic Dachary created a dedicated virtual machine and installed packaging-farm on it. It essentially does the same as the above with less manual tweaking but the learning curve is steeper. The source packages were downloaded into /var/cache/packaging-farm/sources and extracted as follows

for i in libbpp-core libbpp-seq libbpp-raa libbpp-phyl libbpp-popgen libbpp-qt bppsuite bppphyview ; do mkdir  ; mv *.{dsc,gz}  ; done
for i in libbpp-core libbpp-seq libbpp-raa libbpp-phyl libbpp-popgen libbpp-qt bppsuite bppphyview ; do ( cd   ; dpkg-source -x *.dsc ) ; done

For each of them a Makefile was created with the form:

	PACKAGE=libbpp-core
	DISTRIBUTIONS=unstable
	ARCHITECTURES=i386 x86_64
	LIBDIR ?= /usr/lib/packaging-farm
	include /Makefile

Using the information from the debian/control files, the dependencies between each package was calculated with:

	packaging-farm depends

and each package was built, both for i386 and amd64, with

	packaging-farm libbpp-seq # which built libbpp-core first because it depends on it
	packaging-farm bppsuite
etc.

The generated packages have been aggretated in a repository that can be used on a Debian GNU/Linux unstable by adding to /etc/apt/sources.list

	deb http://biopp.dachary.org/packaging-farm/bpp/gnulinux/debian unstable main
	deb-src http://biopp.dachary.org/packaging-farm/bpp/gnulinux/debian unstable main

Which allows to install bppsuite with a single apt-get install bppsuite

	apt-get install bppsuite
	Reading package lists... Done
	Building dependency tree
	Reading state information... Done
	The following extra packages will be installed:
	  libbpp-core libbpp-core-dev libbpp-phyl libbpp-phyl-dev libbpp-seq libbpp-seq-dev
	The following NEW packages will be installed:
	  bppsuite libbpp-core libbpp-core-dev
	  libbpp-phyl libbpp-phyl-dev libbpp-seq libbpp-seq-dev
	7 newly installed, 0 to remove
	Need to get 10.5MB of archives.
	After this operation, 43.6MB of additional disk space will be used.
	Do you want to continue [Y/n]? y
	Get:1 http://biopp.dachary.org unstable/main libbpp-core 2.0.1 [589kB]
	Get:2 http://biopp.dachary.org unstable/main libbpp-core-dev 2.0.1 [1,310kB]
	Get:3 http://biopp.dachary.org unstable/main libbpp-seq-dev 2.0.0-1 [1,227kB]
	Get:4 http://biopp.dachary.org unstable/main libbpp-seq 2.0.0-1 [540kB]
	Get:5 http://biopp.dachary.org unstable/main libbpp-phyl-dev 2.0.1 [4,660kB]
	Get:6 http://biopp.dachary.org unstable/main libbpp-phyl 2.0.1 [1,779kB]
	Get:7 http://biopp.dachary.org unstable/main bppsuite 0.6.0-2 [410kB]
	Fetched 10.5MB in 18s (575kB/s)
	Selecting previously deselected package libbpp-core.
	(Reading database ... 330640 files and directories currently installed.)
	Unpacking libbpp-core (from .../libbpp-core_2.0.1_amd64.deb) ...
	Selecting previously deselected package libbpp-core-dev.
	Unpacking libbpp-core-dev (from .../libbpp-core-dev_2.0.1_amd64.deb) ...
	Preparing to replace libbpp-seq-dev 1.7.1 (using .../libbpp-seq-dev_2.0.0-1_amd64.deb) ...
	Unpacking replacement libbpp-seq-dev ...
	Preparing to replace libbpp-seq 1.7.1 (using .../libbpp-seq_2.0.0-1_amd64.deb) ...
	Unpacking replacement libbpp-seq ...
	Preparing to replace libbpp-phyl-dev 1.9.1 (using .../libbpp-phyl-dev_2.0.1_amd64.deb) ...
	Unpacking replacement libbpp-phyl-dev ...
	Preparing to replace libbpp-phyl 1.9.1 (using .../libbpp-phyl_2.0.1_amd64.deb) ...
	Unpacking replacement libbpp-phyl ...
	Selecting previously deselected package bppsuite.
	Unpacking bppsuite (from .../bppsuite_0.6.0-2_amd64.deb) ...
	Processing triggers for install-info ...
	install-info: warning: no info dir entry in `/usr/share/info/ispell.info.gz'
	install-info: warning: no info dir entry in `/usr/share/info/bppsuite.info.gz'
	Setting up libbpp-core (2.0.1) ...
	Setting up libbpp-core-dev (2.0.1) ...
	Setting up libbpp-seq (2.0.0-1) ...
	Setting up libbpp-seq-dev (2.0.0-1) ...
	Setting up libbpp-phyl (2.0.1) ...
	Setting up libbpp-phyl-dev (2.0.1) ...
	Setting up bppsuite (0.6.0-2) ...


unstable and testing

When a new package enters Debian, it is uploaded in the unstable repository. The software it contains is expected to work but the package itself may be partially broken. It is supposed to be fixed as soon as possible. When and if a package builds successfully and no bug is filled against it during a few days, it is automatically migrated to testing. For instance openscenegraph shows that the Testing column is 2.8.3-6 meaning it migrated from the Unstable column. When a stable version of Debian is published, which takes more than a year, it is made from the content of Testing.


Versioning libraries

A library binary package must contain a version number that indicates the revision of the ABI. The general idea is that any package that depends on it must keep running as long as the ABI is stable. For instance, package simgear depends on libopenscenegraph65. When openscenegraph which depends on libopenscenegraph71 is installed, both simgear and openscenegraph can run although they depend on different versions of the ABI. If the libraries had the same name, they could not co-exist.

When a package is in a state of flux, with frequent minor versions being uploaded to unstable, this convention may be relaxed. Users of the unstable repository have a higher tolerance to this kind of policy violation. It is also useful to avoid the proliferation of libraries.

Since Bio++ use CMake, the library version will be created from stanzas such as

SET(BPPQT_VERSION_CURRENT "2")
SET(BPPQT_VERSION_REVISION "0")
SET(BPPQT_VERSION_AGE "0")

which will lead to the file name

/usr/lib/libbpp-qt.so.2.0.0

The Debian GNU/Linux policy imposes that the package name containing a shared library is related to the shared library version according to the following:

objdump -p /usr/lib/libbpp-core.so.2.0.0 | sed -n -e's/^[[:space:]]*SONAME[[:space:]]*//p' | sed -e's/([0-9]).so./-/; s/.so.//'

which suggests the following package name:

libbpp-core2

When ”” is added to the Depends: line of a library, the dpkg-shlibdepends will automatically figure out the libraries packages that need to be added to the Depends line. For instance in libbpp-seq it will create the following debian/libbpp-seq9.substvars file:

shlibs:Depends=libbpp-core2, libc6 (>= 2.1.3), libgcc1 (>= 1:4.1.1), libstdc++6 (>= 4.5)

Note that there is no version constraint on libbpp-core2. Such a constraint could be useful if a version of libbpp-core2 has a known bug and libbpp-seq requires a higher version. In this case the #MINVERS# tag can be used as explained in ImprovedDpkgShlibdeps


Legal

All Bio++ software is released under CeCILL version 2. Although it is not among the list of licenses supported by Debian GNU/Linux is has been discussed for a long time and over a dozen packages have already been accepted. Since CeCILL v2 can be replaced transparently by the GNU GPL according to the section 5.3.4 COMPATIBILITY WITH THE GNU GPL, there is little doubt that it will be refused.
The debian/copyright files for each Bio++ packages have been updated to list the licenses and of all third party authors.


Further work

  • Close the RFP
  • Watch the NEW queue
This entry was posted in Bio++. Bookmark the permalink.

One Response to Uploading Bio++ packages to Debian

  1. Julien Dutheil says:

    This was really a nice adventure! I’m sure this will be most useful for anybody like me with little knowledge in debian packaging.
    for (int i = 0; i < 1000; ++i) cout << "thanks a lot!!!" << endl;

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>