Bio++ packaging

What does it take for Bio++ to enter the official repositories of Debian GNU/Linux ?


Request For Packaging

The first step is to ask for a Debian Developer to package the software. This is called a RFP as in Request For Packaging and it is done by filling a bug report against the wnpp pseudo package. Since Debian packages for Bio++ are already provided the task of the Debian Developer is facilitated.
Even when Debian packages are already available, the odds of seducing a Debian Developer into taking responsibility for the package are small. The quality standards for a package entering Debian are higher than what is implemented by the amateur packager. However, meeting these standards is not a difficult task when using the proper tools.


Resolving packages dependencies

Using pbuilder ensures that the package can be built on a virgin Debian GNU/Linux installation.
When trying a build on a machine used for other purposes, it is common to forget that it can’t be built unless a specific library is already installed.

	apt-get install pbuilder
	pbuilder --create

Will create a virgin GNU/Linux and archive it in a tar.gz (in /var/cache/pbuilder). It can then be used to build the package (for instance libbpp-core) as follows:

	pbuilder --build libbpp-core_2.0.1.dsc

will copy the package in a chroot extracted from the archive created by pbuilder and try to build it. When finished the result can be found in /var/cache/pbuilder/result.


Packages interdependencies

What should be done for libbpp-seq to build ? It relies on libpp-core which is not standard in Debian and should retrieve it from a previous build. Assuming a web server is installed locally, a Debian repository can be built as follows:

	ln -s /var/cache/pbuilder/result /var/www/result

and a reference to this repository can be added to the pbuilder chroot as follows:

	pbuilder --login  --save-after-login
	echo deb http://localhost/result/ ./  >> /etc/apt/sources.list
	Control-D

From now on, building a package will lookup this repository in addition to the standard ones. Whenever a new package is added to this repository, as a result of a successfull call to pbuilder, the pbuilder chroot must be updated as follows:

	( cd /var/www/result ; dpkg-scanpackages . /dev/null | gzip > Packages.gz ; dpkg-scansources . /dev/null | gzip > Sources.gz )
	pbuilder --update

Julien Dutheil used this technique to ensure that the Build-Depends of each package was accurate.


Packaging farm

In order to review the packages, Loic Dachary created a dedicated virtual machine and installed packaging-farm on it. It essentially does the same as the above with less manual tweaking but the learning curve is steeper. The source packages were downloaded into /var/cache/packaging-farm/sources :

	/var/cache/packaging-farm/sources/bppphyview/bppphyview_0.1.0.dsc
	/var/cache/packaging-farm/sources/bppphyview/bppphyview_0.1.0.tar.gz
	/var/cache/packaging-farm/sources/bppsuite/bppsuite_0.6.0-2.dsc
	/var/cache/packaging-farm/sources/bppsuite/bppsuite_0.6.0-2.tar.gz
	/var/cache/packaging-farm/sources/libbpp-core/libbpp-core_2.0.1.dsc
	/var/cache/packaging-farm/sources/libbpp-core/libbpp-core_2.0.1.tar.gz
	/var/cache/packaging-farm/sources/libbpp-phyl/libbpp-phyl_2.0.1.dsc
	/var/cache/packaging-farm/sources/libbpp-phyl/libbpp-phyl_2.0.1.tar.gz
	/var/cache/packaging-farm/sources/libbpp-popgen/libbpp-popgen_2.0.0-1.dsc
	/var/cache/packaging-farm/sources/libbpp-popgen/libbpp-popgen_2.0.0-1.tar.gz
	/var/cache/packaging-farm/sources/libbpp-qt/libbpp-qt_2.0.0.dsc
	/var/cache/packaging-farm/sources/libbpp-qt/libbpp-qt_2.0.0.tar.gz
	/var/cache/packaging-farm/sources/libbpp-raa/libbpp-raa_2.0.0-1.dsc
	/var/cache/packaging-farm/sources/libbpp-raa/libbpp-raa_2.0.0-1.tar.gz
	/var/cache/packaging-farm/sources/libbpp-seq/libbpp-seq_2.0.0-1.dsc
	/var/cache/packaging-farm/sources/libbpp-seq/libbpp-seq_2.0.0-1.tar.gz

For each of them a Makefile was created with the form:

	PACKAGE=libbpp-core
	DISTRIBUTIONS=unstable
	ARCHITECTURES=i386 x86_64
	LIBDIR ?= /usr/lib/packaging-farm
	include /Makefile

Using the information from the debian/control files, the dependencies between each package was calculated with:

	packaging-farm depends

and each package was built, both for i386 and amd64, with

	packaging-farm libbpp-seq # which built libbpp-core first because it depends on it
	packaging-farm bppsuite
etc.

The generated packages have been aggretated in a repository that can be used on a Debian GNU/Linux unstable by adding to /etc/apt/sources.list

	deb http://biopp.dachary.org/packaging-farm/bpp/gnulinux/debian unstable main
	deb-src http://biopp.dachary.org/packaging-farm/bpp/gnulinux/debian unstable main

Which allows to install bppsuite with a single apt-get install bppsuite

	apt-get install bppsuite
	Reading package lists... Done
	Building dependency tree
	Reading state information... Done
	The following extra packages will be installed:
	  libbpp-core libbpp-core-dev libbpp-phyl libbpp-phyl-dev libbpp-seq libbpp-seq-dev
	The following NEW packages will be installed:
	  bppsuite libbpp-core libbpp-core-dev
	  libbpp-phyl libbpp-phyl-dev libbpp-seq libbpp-seq-dev
	7 newly installed, 0 to remove
	Need to get 10.5MB of archives.
	After this operation, 43.6MB of additional disk space will be used.
	Do you want to continue [Y/n]? y
	Get:1 http://biopp.dachary.org unstable/main libbpp-core 2.0.1 [589kB]
	Get:2 http://biopp.dachary.org unstable/main libbpp-core-dev 2.0.1 [1,310kB]
	Get:3 http://biopp.dachary.org unstable/main libbpp-seq-dev 2.0.0-1 [1,227kB]
	Get:4 http://biopp.dachary.org unstable/main libbpp-seq 2.0.0-1 [540kB]
	Get:5 http://biopp.dachary.org unstable/main libbpp-phyl-dev 2.0.1 [4,660kB]
	Get:6 http://biopp.dachary.org unstable/main libbpp-phyl 2.0.1 [1,779kB]
	Get:7 http://biopp.dachary.org unstable/main bppsuite 0.6.0-2 [410kB]
	Fetched 10.5MB in 18s (575kB/s)
	Selecting previously deselected package libbpp-core.
	(Reading database ... 330640 files and directories currently installed.)
	Unpacking libbpp-core (from .../libbpp-core_2.0.1_amd64.deb) ...
	Selecting previously deselected package libbpp-core-dev.
	Unpacking libbpp-core-dev (from .../libbpp-core-dev_2.0.1_amd64.deb) ...
	Preparing to replace libbpp-seq-dev 1.7.1 (using .../libbpp-seq-dev_2.0.0-1_amd64.deb) ...
	Unpacking replacement libbpp-seq-dev ...
	Preparing to replace libbpp-seq 1.7.1 (using .../libbpp-seq_2.0.0-1_amd64.deb) ...
	Unpacking replacement libbpp-seq ...
	Preparing to replace libbpp-phyl-dev 1.9.1 (using .../libbpp-phyl-dev_2.0.1_amd64.deb) ...
	Unpacking replacement libbpp-phyl-dev ...
	Preparing to replace libbpp-phyl 1.9.1 (using .../libbpp-phyl_2.0.1_amd64.deb) ...
	Unpacking replacement libbpp-phyl ...
	Selecting previously deselected package bppsuite.
	Unpacking bppsuite (from .../bppsuite_0.6.0-2_amd64.deb) ...
	Processing triggers for install-info ...
	install-info: warning: no info dir entry in `/usr/share/info/ispell.info.gz'
	install-info: warning: no info dir entry in `/usr/share/info/bppsuite.info.gz'
	Setting up libbpp-core (2.0.1) ...
	Setting up libbpp-core-dev (2.0.1) ...
	Setting up libbpp-seq (2.0.0-1) ...
	Setting up libbpp-seq-dev (2.0.0-1) ...
	Setting up libbpp-phyl (2.0.1) ...
	Setting up libbpp-phyl-dev (2.0.1) ...
	Setting up bppsuite (0.6.0-2) ...


Lintian

lintian examines packages and outputs warnings and errors. When running it on all the .changes files created by packaging-farm we get the following:

lintian *.changes
W: bppphyview source: ancient-standards-version 3.7.2 (current is 3.9.1)
W: bppsuite source: ancient-standards-version 3.7.2 (current is 3.9.1)
W: bppsuite source: native-package-with-dash-version
W: libbpp-core source: debhelper-but-no-misc-depends libbpp-core-dev
W: libbpp-core source: ancient-standards-version 3.7.2 (current is 3.9.1)
W: libbpp-phyl source: debhelper-but-no-misc-depends libbpp-phyl-dev
W: libbpp-phyl source: ancient-standards-version 3.7.2 (current is 3.9.1)
W: libbpp-popgen source: debhelper-but-no-misc-depends libbpp-popgen-dev
W: libbpp-popgen source: ancient-standards-version 3.7.2 (current is 3.9.1)
W: libbpp-popgen source: native-package-with-dash-version
W: libbpp-qt source: debhelper-but-no-misc-depends libbpp-qt-dev
W: libbpp-qt source: ancient-standards-version 3.7.2 (current is 3.9.1)
W: libbpp-raa source: changelog-should-mention-nmu
W: libbpp-raa source: source-nmu-has-incorrect-version-number 2.0.0-1
W: libbpp-raa source: debhelper-but-no-misc-depends libbpp-raa-dev
W: libbpp-raa source: ancient-standards-version 3.7.2 (current is 3.9.1)
W: libbpp-raa source: native-package-with-dash-version
W: libbpp-seq source: debhelper-but-no-misc-depends libbpp-seq-dev
W: libbpp-seq source: ancient-standards-version 3.7.2 (current is 3.9.1)
W: libbpp-seq source: native-package-with-dash-version
W: libbpp-seq source: diff-contains-editor-backup-file debian/.control.swp
W: bppphyview: binary-without-manpage usr/bin/phyview
E: bppphyview: missing-dependency-on-libc needed by ./usr/bin/phyview
W: bppsuite: possible-unindented-list-in-extended-description
E: bppsuite: info-document-missing-dir-section usr/share/info/bppsuite.info.gz
E: bppsuite: info-document-missing-dir-entry usr/share/info/bppsuite.info.gz
W: bppsuite: missing-dependency-on-install-info
W: bppsuite: binary-without-manpage usr/bin/bppancestor
W: bppsuite: binary-without-manpage usr/bin/bppconsense
W: bppsuite: binary-without-manpage usr/bin/bppdist
W: bppsuite: binary-without-manpage usr/bin/bppml
W: bppsuite: binary-without-manpage usr/bin/bpppars
W: bppsuite: binary-without-manpage usr/bin/bppphysamp
W: bppsuite: binary-without-manpage usr/bin/bppreroot
W: bppsuite: binary-without-manpage usr/bin/bppseqgen
W: bppsuite: binary-without-manpage usr/bin/bppseqman
W: bppsuite: binary-without-manpage usr/bin/bpptreedraw
W: libbpp-core: package-name-doesnt-match-sonames libbpp-core2
W: libbpp-phyl: package-name-doesnt-match-sonames libbpp-phyl9
E: libbpp-popgen: no-shlibs-control-file usr/lib/libbpp-popgen.so.6.0.0
E: libbpp-popgen: postinst-must-call-ldconfig usr/lib/libbpp-popgen.so.6.0.0
W: libbpp-popgen: package-name-doesnt-match-sonames libbpp-popgen6
W: libbpp-qt: package-name-doesnt-match-sonames libbpp-qt2
W: libbpp-raa-dev: extended-description-line-too-long
W: libbpp-raa: extended-description-line-too-long
E: libbpp-raa: no-shlibs-control-file usr/lib/libbpp-raa.so.1.1.0
E: libbpp-raa: postinst-must-call-ldconfig usr/lib/libbpp-raa.so.1.1.0
W: libbpp-raa: package-name-doesnt-match-sonames libbpp-raa1
W: libbpp-seq: package-name-doesnt-match-sonames libbpp-seq9

Each error must be addressed. Some warnings may be ignored if they are annoying or impractical to fix but it’s good behaviour to fix them too. The error message is terse and can be expanded using lintian-info as follows:

echo E: libbpp-popgen: no-shlibs-control-file usr/lib/libbpp-popgen.so.6.0.0 | lintian-info
E: libbpp-popgen: no-shlibs-control-file usr/lib/libbpp-popgen.so.6.0.0
N:
N:   Although the package includes a shared library, the package does not
N:   have a shlibs control file. If this is intentional, please override
N:   this error.
N:
N:   Refer to Debian Policy Manual section 8.6 (Dependencies between the
N:   library and other packages - the shlibs system) for details.
N:
N:   Severity: serious, Certainty: possible
N:


unstable and testing

When a new package enters Debian, it is uploaded in the unstable repository. The software it contains is expected to work but the package itself may be partially broken. It is supposed to be fixed as soon as possible. When and if a package builds successfully and no bug is filled against it during a few days, it is automatically migrated to testing. For instance openscenegraph shows that the Testing column is 2.8.3-6 meaning it migrated from the Unstable column. When a stable version of Debian is published, which takes more than a year, it is made from the content of Testing.


Versioning libraries

A library binary package must contain a version number that indicates the revision of the ABI. The general idea is that any package that depends on it must keep running as long as the ABI is stable. For instance, package simgear depends on libopenscenegraph65. When openscenegraph which depends on libopenscenegraph71 is installed, both simgear and openscenegraph can run although they depend on different versions of the ABI. If the libraries had the same name, they could not co-exist.

When a package is in a state of flux, with frequent minor versions being uploaded to unstable, this convention may be relaxed. Users of the unstable repository have a higher tolerance to this kind of policy violation. It is also useful to avoid the proliferation of libraries.

This entry was posted in Bio++. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>