No Food for Thought

Optimizing the optimization - Performant Incremental Updates for Packages files

admin Wednesday January 6, 2016

In 2006, Michael Vogt implemented support for PDiff (differential Packages) files in APT to optimize the process of updating Packages. At the time, the Packages file, which was already several MB-s compressed, needed to be downloaded entirely to update package indices. Joerg Jaspert and probably other members of the archive maintenance team implemented support for generating PDiff files on the archive side.

Unfortunately, APT's performance when applying (several) PDiff files was quite poor, sometimes worst than the performance for a non-incremental update, as reported (at least) in ticket 372712 and ticket 376158, which was particularly problematic for testing users - until APT 1.1.7, a nice X-mas present from Julian Andres Klode, who identified the bottleneck and optimized the process.

I haven't had the time to test testing since Jessie's release, but I'm starting to miss it smile
I wish to thank Michael, Anthony Towns, Andreas Barth, Joerg and others who contributed to the initial implementation, as well as jak, who's finalizing this work a decade later with an optimization job even more thankless than the initial implementation.
The next step? Differential updating of packages, with a lowercase "p"... which promises to be even harder to get right.

Finally, I'm using this opportunity to thank APT contributors - particularly its current maintainers mvo and jak - for all of their work. Progress has been slow over the last decade, but the direction is right, and each step is appreciated.

Fully Free

Kune ni povos is seriously freethough not completely humor-free:

  • Free to read,
  • free to copy,
  • free to republish;
  • freely licensed.
  • Free from influenceOriginal content on Kune ni povos is created independently. KNP is entirely funded by its freethinker-in-chief and author, and does not receive any more funding from any corporation, government or think tank, or any other entity, whether private or public., advertisement-free
  • Calorie-free*But also recipe-free
  • Disinformation-free, stupidity-free
  • Bias-free, opinion-free*OK, feel free to disagree on the latter.
  • Powered by a free CMS...
  • ...running on a free OS...
  • ...hosted on a server sharedby a great friend for free