[Dev] [RFC] Redesign and reimplement librerelease and db-update

Michał Masłowski mtjm at mtjm.eu
Mon Apr 9 21:24:06 GMT 2012


fauno and I discussed on #parabola changing how db-update works to
improve its reliability and usability.

My proposal is to make librerelease run a script on repo and pipe to
it the files to upload.

The script verifies them and fails if one is incorrectly signed.

Then, when all files are ready, it locks all databases used by them,
checks if the same version is available for any of them, fails if yes
(explicitly recommending adding .1 to pkgrel if the new version is
needed), then updates the packages and unlocks the databases.

librerelease outputs errors from the repo-side script.

If multiple uploads including the same package are done at once, only
at most one succeeds, others fail, the library uploaded must be the one
From the successful run.

The server-side script might log its output to the maintenance list.

Locks:

- one for all repos, or

- one per repo and arch, needs correct ordering of their acquisition

Benefits:

- no duplicated uploads of a package (these contributed to the great
  pcre breakage affecting glib and grep on mips64el and made many
  corrupted package errors)

- immediate error messages on failed upload, for the user who did the
  upload

- no (theoretical yet, although possible) problems like updating
  xorg-server and drivers, while having only drivers updated due to
  missing xorg-server signature or (not sure if rsync prevents this)
  if the connection is interrupted while uploading xorg-server after
  drivers

Some requirements:

- a single (repo, arch, pkgname, epoch, pkgver, pkgrel) tuple refers to
  exactly zero or one released package

- multiple uploads at once work, usually at least one succeeds

- all errors make just the affected upload do nothing (and notify
  appropriate users), they don't affect other, correct, uploads

- no deadlocks; interrupted uploads don't cause other problems

- runs are quick, except for waiting for the files to be uploaded (no
  waiting for files of other users)

Possible issues:

- continuing interrupted uploads

fauno recommended checking lockfile-utils for locking support (writing
our would make more opportunities for bugs).  We need lock-or-fail and
unlock operations for a single lock or a set of locks.

Any comments?  Is it too complex, or too unreliable, are the
requirements unsatisfiable, or ...?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <https://lists.parabola.nu/pipermail/dev/attachments/20120409/34f919ba/attachment.sig>


More information about the Dev mailing list