[Dev] dbscripts plan v2
Luke Shumaker
lukeshu at lukeshu.com
Wed Aug 22 22:41:42 GMT 2018
TL;DR: Over the next 2 or more weeks, there will some changes in the
pacman repos/dbscripts. Please keep an eye out for anything amiss,
and reply in this email thread. This going smoothly depends on
packager feedback if something is wrong.
Hi all,
Back in May there was a ~2-week "sprint" to improve dbscripts.
dbscripts did get a lot better, and while we did hit the true
objective of "improve db-import-pkg and add testing such that we
aren't afraid to touch it", we didn't hit either of the stated
concrete objectives. Let's try again.
Objectives:
1. Have db-import-pkg use db-update/db-remove/db-move, rather than
manipulating the repos directly. We couldn't hit this before
because db-remove was subtly broken, and db-move was not-so-subtly
broken. These were fixed in dbscripts v20180716.
2. Have proper user-separation, instead of having everything run as
repo at . We didn't hit this before because there were so many files
with funny permissions. That list is now much smaller.
Plan:
1. Adjust users/groups/ownership on winston.parabola.nu
a. Create the `repo` group
Change: `groupadd --system repo`
Rationale: This is a supporting change to the later bits of step 1.
Timeframe: already done, 2018-08-22
b. Correct any funny permissions in /srv/repo
Change:
rm -f -- /srv/repo/main/other/kodi-libre/.kodi-libre-17.6-Krypton.tar.gz
chmod 0644 -- /srv/repo/main/other/kodi-libre/addons/krypton/makesums.sh
Rationale:
I have a script in my home directory on winston that looks
for any funny permissions in /srv/repo. The output should
be self-explanatory in what it's looking for:
$ ./find-funny
==> Files with funny types: (expecting d/f/l)
==> Files with funny ownership: (expecting repo:users or lukeshu:users)
==> Directories with funny perms: (expecting 0755/2755/2775)
==> DB files with funny perms: (expecting 0664)
==> Non-DB Files with funny perms: (expecting 0644)
1 -rw------- 1 repo users 14417920 Mar 19 20:14 /srv/repo/main/other/kodi-libre/.kodi-libre-17.6-Krypton.tar.gz.9zxQwS
2 -rwxr-xr-x 1 repo users 175 Oct 1 2017 /srv/repo/main/other/kodi-libre/addons/krypton/makesums.sh
==> Links with funny perms: (expecting 0777)
The file `.kodi-libre-17.6-Krypton.tar.gz.9zxQwS` is a
truncated version of `kodi-libre-17.6-Krypton.tar.gz`.
The commands given should make `./find-funny` come back
clean. We should verify that the permissions are what we
expect before we go recursively changing them.
The files that are owned by 'lukeshu' instead of 'repo' are
from when files were deleted when someone left the project,
and I restored them from a mirror.
Timeframe: Tonight or tomorrow; after 1 final snapshot of the
way thing currently are is taken by beefcake.
c. Switch over to using the `repo` group
Change:
usermod --gid=repo repo
chgrp -R repo /srv/repo/main
find /srv/repo/main -type d -exec chmod 2775 -- {} +
Rationale:
All hackers' user accounts are automatically be members of
the new 'repo' group (per `/etc/parabola-hackers.yml`),
following its creation in step 1.a. Given the `002` umask
that `db-functions` sets, write access to the repos will
now be governed by membership in the `repo` NSS group,
rather than restricted to the `repo` user (access to which
is governed by the `repo` hackers.git group).
This is a good idea (security-wise) anyway; since
`db-functions` runs `chmod g+w` the `.db` files, currently
all local users have write access to the `.db` files.
Packagers *should* then be able to adjust their
`libretools.conf:REPODEST` to be
`ssh://$LIBREUSER@repo.parabola.nu:1863/~/staging/` instead of
`ssh://repo@repo.parabola.nu:1863/~/staging/$LIBREUSER/staging/`.
This will be opt-in for a period, so that we can identify any
issues before rolling the change out to all packagers.
Doing user-separation for human packagers first, before
doing it for db-import, allows us to limit large-scale
breakage in case anything goes wrong, and ensures that a
human is looking at what is going on during the initial
days of it.
Timeframe: Tonight or tomorrow; after 1 final snapshot of the
way thing currently are is taken by beefcake.
2. Use db-{move,update,remove}
Change:
Rework `db-import-pkg` to set up a `db-update`-style staging
directory, rather than manipulating the repos directly. At
this point, `db-import-pkg` is still running as `repo`.
However, set `ProtectSystem=strict`, limit it to a staging &
scratch directory, and have it
ssh localhost DBSCRIPTS_CONFIG=...${UPSTREAM} db-update
to add/remove packages. This will involve adding an ssh key
for repo.
Rationale: Obviously, this is a primary objective. Using
ssh+ProtectSystem allows us to ensure that the objective is met.
Concerns: Testing this will probably involve running an SSH server
in the test suite.
Timeframe: As soon as I can get it done after step 1; a couple of
days.
3. Migrate humans off of repo@
Change: Adjust the default `libretools.conf:REPODEST` to be
`ssh://$LIBREUSER@repo.parabola.nu:1863/~/staging/` instead of
`ssh://repo@repo.parabola.nu:1863/~/staging/$LIBREUSER/staging/`.
Rationale: This will change the user-separated `db-update` from
opt-in (following step 1) to opt-out.
DISRUPTION: This will make the default configuration unsuitable
for packagers whose local username doesn't match their username in
hackers.git. They will need to manually adjust their
`libretools.conf:REPODEST` to have the correct username.
Timeframe: at least a week from now [NB: no sooner than 2018-08-29]
4. Migrate robots off of repo@
Change: Add
`db-import-{packages,community,archlinux32,archlinuxarm}` users,
members of the `repo` group; have `db-import-pkg` run as them.
Rationale: Mission accomplished
Timeframe: at least a week after that [NB: no sooner than 2018-09-05]
In summary:
[-] 1. Set up the `repo` group [eta: 2018-08-23]
[ ] 2. Use db-{move,update,remove} [eta: ???]
[ ] 3. Migrate humans off of repo@ [eta: no sooner than 2018-08-29]
[ ] 4. Migrate robots off of repo@ [eta: no sooner than 2018-09-05]
Comments? Concerns?
--
Happy hacking,
~ Luke Shumaker
More information about the Dev
mailing list