[Dev] [RFC] Redesign and reimplement librerelease and db-update
Joshua Ismael Haase Hernandez
hahj87 at gmail.com
Tue Apr 10 18:53:14 GMT 2012
On Mon, 09 Apr 2012 23:24:06 +0200, mtjm at mtjm.eu (=?utf-8?Q?Micha=C5=82_Mas=C5=82owski?=) wrote:
> fauno and I discussed on #parabola changing how db-update works to
> improve its reliability and usability.
I like the idea, it seems feasible and easy to implement, however I
think we should document the whole process of repo maintaining, because
most people (me included) don't know exactly how it works.
> My proposal is to make librerelease run a script on repo and pipe to
> it the files to upload.
+1 server site script. Just some questions, Can multiple packagers upload
their packages at the same time? How will this be addresed?
This is now addresed by uploading the packages to a «staging» area, but
I do think we do need a way to stop outdated or same package version to
be uploaded.
> The script verifies them and fails if one is incorrectly signed.
>
> Then, when all files are ready, it locks all databases used by them,
> checks if the same version is available for any of them, fails if yes
> (explicitly recommending adding .1 to pkgrel if the new version is
> needed), then updates the packages and unlocks the databases.
>
> librerelease outputs errors from the repo-side script.
>
> If multiple uploads including the same package are done at once, only
> at most one succeeds, others fail, the library uploaded must be the one
> From the successful run.
>
> The server-side script might log its output to the maintenance list.
>
> Locks:
>
> - one for all repos, or
>
> - one per repo and arch, needs correct ordering of their acquisitionn
>
> Benefits:
>
> - no duplicated uploads of a package (these contributed to the great
> pcre breakage affecting glib and grep on mips64el and made many
> corrupted package errors)
>
> - immediate error messages on failed upload, for the user who did the
> upload
>
> - no (theoretical yet, although possible) problems like updating
> xorg-server and drivers, while having only drivers updated due to
> missing xorg-server signature or (not sure if rsync prevents this)
> if the connection is interrupted while uploading xorg-server after
> drivers
I think this require use ABS or another thing as control for deps.
> Some requirements:
>
> - a single (repo, arch, pkgname, epoch, pkgver, pkgrel) tuple refers to
> exactly zero or one released package
>
> - multiple uploads at once work, usually at least one succeeds
>
> - all errors make just the affected upload do nothing (and notify
> appropriate users), they don't affect other, correct, uploads
>
> - no deadlocks; interrupted uploads don't cause other problems
>
> - runs are quick, except for waiting for the files to be uploaded (no
> waiting for files of other users)
>
> Possible issues:
>
> - continuing interrupted uploads
>
> fauno recommended checking lockfile-utils for locking support (writing
> our would make more opportunities for bugs). We need lock-or-fail and
> unlock operations for a single lock or a set of locks.
>
> Any comments? Is it too complex, or too unreliable, are the
> requirements unsatisfiable, or ...?
More information about the Dev
mailing list