My Opinions

Component repositories are becoming an essential part of the Free Software landscape, and grow at a steady pace. Ensuring their quality is of paramount importance, and is a difficult task. One natural question to ask when performing quality assurance is the following : what are the components p in a given repository R that cannot be properly installed?. Unfortunately, there can be many reasons why an installation may fail, and some of these will only be detected on the particular environment of the final user, no matter what amount of testing one tries to do beforehand.

But there is a full class of reasons, related to package interdependencies, that can be found by performing a static analysis of repositories, with no testing at all. This static analysis, originally designed for GNU/Linux distributions, is of general interested, and we are now introducing it for the Opam ecosystem.

Did you know that you can ask apt-get to invoke an external solver for handling the dependency hell that sometimes plagues your upgrades? Well, just now, again, this feature saved my day, and it is just a matter of typing a short option!

Instead of the standard

apt-get install fetchmail

that proposed an horrendous upgrade path detailed below, I typed

 apt-get --solver aspcud install fetchmail

that returned this easy solution

apt-get --solver aspcud install fetchmail
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Execute external solver... Done
The following extra packages will be installed:
 libc-dev-bin libc6 libc6:i386 libc6-dbg libc6-dev libc6-i386 libc6-i686:i386 locales
Suggested packages:
 fetchmailconf resolvconf glibc-doc glibc-doc:i386 locales:i386
The following packages will be upgraded:
 fetchmail libc-dev-bin libc6 libc6:i386 libc6-dbg libc6-dev libc6-i386 libc6-i686:i386 locales
9 upgraded, 0 newly installed, 0 to remove and 2776 not upgraded.
Need to get 25.9 MB of archives.
After this operation, 10.7 MB of additional disk space will be used.
Do you want to continue Y/n? Y

Just make sure you have aspcud and apt-cudf installed for this to work!

Thanks to the effort of many wonderful people (see at the end of this post), it is now possible to call an external solver from apt-get, if you use at least version 0.9.5, which will be the case in Wheezy.

It's pretty simple, just install apt-cudf :

apt-get install apt-cudf

but please check that you get version 2.9.16~rc1-1 or later of it!

dpkg -l apt-cudf
======================================
ii  apt-cudf  2.9.16~rc1-1   CUDF solver integration for APT

This package will also install one of the available solvers that support CUDF, for example aspcud.

Then you can simply use the --solver option of apt-get to have the dependency solving delegated to aspcud:

apt-get -s --solver aspcud install totem

On my machine, this gives a solution with these characteristics:

49 upgraded, 47 newly installed, 1 downgraded, 9 to remove and 2551 not upgraded.

Which might be more interesting than what the standard internal solver of apt finds, as it changes and removes quite a few more packages

79 upgraded, 98 newly installed, 15 to remove and 2520 not upgraded.

Imagine that you want to find out which packages cannot be installed together in your latest and preferred free software distribution... how would you go about it?

One could try and install together all possible combination of packages, but a free software repository contains several tens of thousands packages, and more than one hundred thousand dependencies and conflicts, so this approach might be interesting only if you really dont know how to best use your account on a Google farm, and you don't care about wasting energy.

One could be a bit smarter, have a look at what has been done in the EDOS and Mancoosi european projects, and program a SAT solver to test these combinations of packages without actually installing them.

Or you could take a completely different approach, and come up with a revolutionary new way of looking at free software repositories.

In a few hours, I am taking a plain to Szeged, to attend the 8th ESEC/FSE conferece, where Jerôme Vouillon is going to show how to extract from free software distributions a co-installability kernel which is much more compact, can be visualised easily, and allows by simple visual inspection to answer this kind of questions in a few seconds.

To give you an example, from the main section of the Ubuntu 10.10 alpha 2 distribution, that contains 7277 packages, one can extract a very simple graph, that easily fits in an A4 page, and which shows all co-installability issues.

Looking at the graph, it is easy to spot immediately a problem with the distribution: you can only install ubuntu-desktop with the pulseaudio audio library, and not with the many other options which are available (like alsa): you can check by yourslef in the picture below

If you know Jerôme, you know that there is also some fantastic piece of software around, and you will be eager to try it... don't wait any longer, you can learn more about COINST by just clicking here.

And by the way, Jerôme's COINST tool got the Distinguished Artifact Award, which will presented at the conference.

It happens that the award is sponsored by Microsoft Research, which posted a nice announcement about it, and we can be happy to see Microsoft Research recognising the importance of the research challenges emerging from Free Software.

I wanted to try the mplayer=2:1.0~rc3++svn20100804-0.1 from the (unofficial) http://www.debian-multimedia.org. Again, I first tried using apt-get (Version: 0.7.25.3), and it proposes me a solution with the following characteristics:

17 upgraded, 21 newly installed, 6 to remove and 648 not upgraded.

You can find the full solution, as well as the data to reproduce this upgrade scenario online at the Mancoosi Debian DUDF repository.

I was not satisfied, because it wanted to remove 6 packages, so I tried aptitude (Version: 0.6.2.1-2), which proposes a solution with these characteristics:

1 packages upgraded, 6 newly installed, 171 to remove and 635 not upgraded.

Here too, you can find the full solution, as well as the data to reproduce this upgrade scenario online at the Mancoosi Debian DUDF repository.

Finally, I resorted to apt-mancoosi (see the Mancoosi software page) which gave me the following solution, which I finally used.

As we have seen in the first post on these issues, upgrading a GNU/Linux machine is not an easy task: this is why Mancoosi is working hard at improving the situation. Here comes another report from the frontline, detailing the epic battle on my machine resulting from the need to install a single new package, akonadi-kde-resource-googledata, to be able to sync my korganizer data with a Google agenda.