Thoughts on soft­ware packages (#265)

Monolithic kernel or not, a base operating system is seldom equipped for all tasks a user expects to per­form on it. As such, most OS’ provide fa­cil­i­ties to run custom executables on the ma­chine, as well as loosely managing them when run­ning. How­ev­er, soft­ware acquisition is largely thought of as an extra layer on top of the base system, interwoven with it to var­i­ous degrees: a userland’s pack­age manager’s job.

Un­for­tu­nate­ly, fueled by the manic lust for usability by a fabricated usager incompétent, the clas­si­cal pack­age manager maxim (as acted upon by e. g. Advanced Pack­age Tool (APT)⁠¹, Pacman⁠², Portage⁠³, Dandified YUM (DNF)⁠⁴, Zypper⁠⁵) seems to fade in popularity, with large GNU/Linux distributions gravitating towards container bundles (see e. g. Flatpak⁠⁶, Snap⁠⁷, AppImage⁠⁸): whilst their advantages of dependency encapsulation and supposed security benefits are never-endingly chanted by their proponents, in my view they are merely an ad-hoc solution. In­stead of crafting solid soft­ware with thoughtful third-party inclusions and kempt versions of all parts (which would in­clude replacing abandoned upstreams in­stead of freezing them to­geth­er with their reason for abandonement indefinetely), a jumbled mess of any bit that has not evaded the grasp of Git­Hub is crammed into a bloat­ed archive, coated in a generic runtime and made executable to fool an unsuspecting quick glance into believing one is installing a piece of soft­ware worth one’s while.

A cleaner ap­proach which does not use the plethora of unwritten bits found on modern storage devices as a lazy cop-out to understand a pack­age as a miniature virtual computer but nonetheless achieves the desirable prop­er­ty of pack­age atomicity, solves the issue of com­pet­ing li­brary names or versions and even promises reproducible builds is to improve upon the clas­si­cal maxim but not dis­card it (see e. g. Nix⁠⁹, Guix⁠¹⁠⁰). How­ev­er, I feel like these projects get carried away by their founding principles, slowly encroaching on the en­tire system: understandling them­selves as much more than merely a pack­age manager, they be­come the dis­tri­bu­tion and even want to manage user-specific configs (see e. g. guix home⁠¹⁠¹). As such, they fall into the trap of complexity and bloatedness, alienating those who came for a Unix ex­pe­ri­ence.

Notably, Guix’ appeal does not stop at boot-level reproducible systems: it is the one entry on the FSF’s list of endorsed GNU/Linux distributions⁠¹⁠² which I managed to successfully use for a few days. The signficance of the existence of such a dis­tri­bu­tion should not be underestimated, since — yet again un­for­tu­nate­ly — the ideals of free soft­ware increasingly fall by the wayside with proprietary firmware, kernel blobs and spyware for the masses being accepted by many. Combined with GNU’s declining relevance⁠¹⁠³, it might be that Stallman’s four freedoms⁠¹⁠⁴ fail to capture a now rel­e­vant di­men­sion of soft­ware; in my opinion a pertinent one is to re­strict source complexity.
Filling this void, the concept of soft­ware minimalism, famously advocated for by suckless.org e.V., prohibits many of the driving forces be­hind abandoning free soft­ware principles: cut­ting-edge hard­ware pressure, bloat-cope and dependency trends to name a few.

If now Guix has shown that a pack­age manager can be used as a vehicle to preach a message but — for me more importantly — also comfort believers, I sense there to be an open niche for a pack­age manager representing both soft­ware minimalism and soft­ware freedom.

Peculiarly, my proposed niche seems to at first glance be diametrically op­posed to the core func­tion of a pack­age manager: managing complex pack­age hierarchies. After all, many few-man-show projects choose to go with the in-memory installer route of curl shady.org | sudo sh. Leaving security concerns aside (should one have reason to as­sume malice, installation via less exploitable means is on­ly a boon when never run­ning the installed), lack of transparency makes uninstalling a daunt­ing task plus further obfuscates the user’s perception of their own system — contrary to the goals of free soft­ware.
Fur­ther­more, the separation of pro­gram binary and run command (i. e. rc, config file) is blurring evermore in light of suckless-style architectures (see e. g. dmenu⁠¹⁠⁵, dwm⁠¹⁠⁶, XMonad⁠¹⁠⁷, knôtM⁠¹⁠⁸) lifting configurations to the compilation stage, thus lead­ing to a deluge of similar forks — a scenario clas­si­cal pack­age managers are not equipped to sat­is­fac­to­rily handle.

On the topic of architectural changes, the decrease in sheer source volume also makes dropping pre-compilation more attractive: Guix consoles the wary with the promise to on­ly optimise reproducible building by injecting pre-compilations and cryptographically proving identity, whilst Portage bears the bur­den of hour-long compile times. Yet many pack­age managers rely on binary dis­tri­bu­tion of clouded origins, which itself undermines the principles of free soft­ware: claiming to treat people in a free manner is not just about pointing at a licence file on a pro­ject’s homepage and executable poisoning is one of the least arduous vectors a miscreant could dream of.
Suffice it to say, soft­ware freedom isn’t worth a cent with­out the source, which is often unfairly treated like an addendum to the binary and thus forgotten about; never shipped. I think source compilation is paramount and beyond its con­cep­tu­al merits, minimalistic soft­ware does not suffer from excruciating compile times, rendering compilation on delivery viable.

From a deeper architectural view, dependency man­age­ment and the attitude towards dynamic libraries has also changed over the last de­cade: go’s toolchain relies heavily on ad fontes source inclusion [C18⁠¹⁠⁹] and the benefits of dynamic libraries are nowadays hard to make out [D20⁠²⁠⁰]. My few encounters with such system-wide dependencies of late have on­ly been Google’s brotli wrapper for go⁠²⁠¹ (I am unsure if the hassle to rely on cgo is worth the presumed per­for­mance superiority) as well as the never-dying fossils Xlib⁠²⁠² and ncurses⁠²⁠³.
Some research-stage ap­proach­es have gone even further, dissecting the concept of a source pack­age into finely hash-indexed lan­guage-se­man­tic atoms in­stead of collections of files (see e. g. Unison⁠²⁠⁴, Fragnix⁠²⁠⁵).⁠²⁠⁶
I surmise the disappearance of a pressing need to save disk space, evermore longed for moral and security-related transparency paired with the tremendous potentials e. g. dead code elimination, call-specific fusion and measurement-based custom op­ti­mi­za­tion heuristics open up will help the proliferation and maturing of such heavily source-oriented soft­ware de­sign ap­proach­es.

My own involvement in the area of pack­age man­age­ment stems from the realisation that my cur­rent dotfile man­age­ment leaves a lot to be desired: fed up with the myriad of individual files I had to manually manage (a shell rc, an editor rc, color schemes for said editor, and var­i­ous utilities ei­ther writ­ten in shell or C — thus also needing compilation), in 2021 I started to verbatim paste or base64 en­code everything into my shell rc, ef­fec­tive­ly devising an ad-hoc archive format.⁠²⁠⁷ Whilst undoubtably in line with the Unix tradition [KP84⁠²⁠⁸, pp. 97—99: 3.9], it proved to be both fiddly to manage and did not play well with version control. Thus, I began think­ing about a uni­fied framework which enables me to portably and recreatably manage my configurations.
This ongoing effort has cur­rent­ly manifested itself in fleetingly⁠²⁠⁹, a light-weight pack­age manager with the goal to un-intrusively and atomically add func­tion­al­i­ty to a run­ning system, journalling what it did to it. As such, fleetingly need not worry about system-in­te­gral or deeply dependant packages nor packages of great binary size. It represents my at­tempt to fill the above described niche, paying attention to proper licencing, proper attribution, soft­ware freedom and soft­ware minimalism.

• pingback from #271
• pingback from #283