[Dev] Install media releng / I made some changes

[bill-auger]

On 11/06/2017 09:05 PM, Luke Shumaker wrote:
> What the processes are when creating those releases,
> how we test them, and what changes we want to make.
> 

the build process is rather simple now - just clone the parabolaiso repo
and run the build script - the build script has many options to decide
which type of ISO is made - i took care to update the docs and explain
as much as could about the process[1]

"testing" them is entirely manual with eyeballs and mouse-clicks but
automated acceptance testing would be a very nice addition to the
work-flow especially if the ISOs were also built in the server - the
opensuse build service has that feature which would be fun to investigate

as for changes, i have made several already - parabolaiso can now build
separate ISOs for i686 and x86 along with the dual-arch kind
it can also build ISO that can install offline - rather than having
seperate trees for each edition, there is now a single build script can
build any combination of target archs, init system, and WM/DE - i am
working now at extracting the user skeleton and artwork out of the
parabolaiso repo and into a distinct package that can be used also by
the calamares installer - the ISO also boots automatically now whcih it
did not before - not a huge gain for the CLI and GUI ISOs but as for the
Talking ISO intended for blind people, that seems like something that
should never have been over-looked


On 11/06/2017 09:05 PM, Luke Shumaker wrote:
> In my view, there are 3 install classes of install images:
>  - x86 ISOs
>  - iPXE images (we don't currently make these, and it is blocking and
>    update of ParabolaWeb)
>  - ARM images (we don't currently make these, and we totally should)
> 

according to the docs "All ISOs are ready to act as PXE server"[2] - i
have not verified this though - in the past i have used the arch-netboot
iPXE kernel that makes PXE boot possible even for those without PXE
hardware support - i assume that could easily be adapted for parabola -
there is a package in the AUR that builds iPXE compatible images[3][4]
for that use-case

i agree that an ARM installer is a good idea - ARM devices are only
going to get more popular - many come with some OS pre-installed
(typically berry-flavored) but today it is generally non-trivial to
replace it safely


On 11/06/2017 09:05 PM, Luke Shumaker wrote:
> Changes I've made
>  - I've re-added the 'latest' symlink in the '/iso/' folder on repo.
>  - I've merge the '/iso-previous/' folder on repo back in to '/iso/'.
>    There's no need to split them off.

the 'latest' symlinks are for the benefit of mirrors is assume? they not
very useful for user downloads though because the actual files are dated
like: 'parabola-2017.05.28-dual.iso' - i started writing a PHP script to
redirect web browsers based on the dates in the directory names - but it
is not clear if keeping old releases around is at all valuable - as i
remember the 'iso-previous' dir was not there before about a month ago -
they never had links on the web - i think i added that as a stash only
in case i needed to revert a release during development of the new set -
i assumed all along that there should be only the latest release
published at any one time because there were no old releases there at
the time and it seems reasonable not to keep them taking up space on the
mirrors

in the future there will be a minimum of 18 ISOs (total ~21GB) available
on the downloads page at any one time - if these are rebuilt regularly
(say every 6 months or so) they will soon be requiring an absurd amount
of disk space or mirrors for somethings that quite likely no one will
ever want - as i said 18 is the current minimum - the build matrix
supports as many as 12 ISOs for each release variant - there are
currently 3 such variants (CLI, LXDE, and talking) and i think 6 of each
is enough - each new variant (KDE for example) would add 6 more  - im
not saying the old ones must be destroyed but i do think at least they
should be stored in a separate location that will not be sync'ed by
mirrors - perhaps a separate archive mirror but in that case i will add
that it would be far more useful to keep an archive of previous packages
which parabola also does not do currently

for the curious, the maximal build matrix looks like this:

  CLI             OpenRC          i686              net-install
  LXDE       x               x    x86_64       x
  Talking         SystemD         dual-arch         complete


On 11/06/2017 09:05 PM, Luke Shumaker wrote:
>  - Do we put the sourceballs on the install CD?  If not, do we ensure
>    that sourceballs for old packages don't get cleaned up if that
>    package is on a CD we still publish?

im not quite sure what this is asking but there is no source code on the
ISOs - the 'complete' editions for offline install include a full cache
of binary packages though - if you are thinking to release periodic
source snapshots that is a good idea but i suggest that be a separate
ISO in order to keep the size down - the 'dual-complete' ISOs are
already 3.5 GB


On 11/06/2017 09:05 PM, Luke Shumaker wrote:
>  - `archboot` is a testing replacement for `archiso`.  Do we have any
>    plans to ditch `parabolaiso` for a `parabolaboot`?

i though about this quite a lot while adding the new features to
parabolaiso - as i see it now that there is probably less benefit to
taking anything from arch than it is worth (even merging the latest
archiso will be tedious at best) - especially if the upstream is
dropping i686 support there could be more lost than gained - another
tool that looks interesting is 'manjaro-tools' which has native support
for exporting calamares build configs - unfortunately it is not licensed
properly and they have ignored my requests[5][6] so i went forward with
upgrading the capabilities of parabolaiso - as it is now, it works well,
it is easy to use, it does everything we need it to do, and it is KISS
and easy to understand (only about 1000 LOC) and easy adapt to future
use-cases (such as ARM builds) so i dont see any reason to replace it


On 11/06/2017 09:05 PM, Luke Shumaker wrote:
>  - I think it would be really cool if any files not owned by a package
>    or modified from the vanilla version on the install media were
>    managed by [holo][].

as i said earlier i have extracted most such files into a dedicated
'parabola-laf' package - probably the only ones left are the CLI install
scripts which could go in the 'parabolaiso-data' package which the
builds already require - the only thing in that package now are two
blobs downloaded from sourceforge for EFI boot (yes it downloads only
the pre-built blobs - no sources) so really everything in the new
'parabola-laf' package could go in that but for semantics


On 11/06/2017 09:05 PM, Luke Shumaker wrote:
>  - Bill Auger tells me that parabolaiso requires root.  I'd like to
>    understand why.  Can we use unprivileged user/mount namespaces to
>    avoid this on the build server?

i have not tried to work around this but the superficial reasons are
that it operates much like pacstrap - it needs to mount the special
files like /dev /sys /proc onto the chroot and install packages - also
there are obviously many critical files that it writes which inevitably
need to be owned by 0:0 such as /etc/init yes?


On 11/06/2017 09:05 PM, Luke Shumaker wrote:
>  - I think the release process should be more documented, and more
>    automated (FOSS automation scripts are the best form of process
>    documentation :P)

having a build server will open up many possibilities - i have not done
so formally yet but i would like to discuss using the opensuse build
service to build the entire parabola system - i mean ALL packages - take
nothing from arch but PKGBUILDS - OBS manages the entire system and has
nice features such as triggering a cascade of rebuilds automatically
when a lower level dependency is rebuilt - for example: trigger a
rebuild of 'libfoo' and any package that depends on it is automatically
rebuilt and then on and on to any dependents farther on until the entire
system is sane again - never again would a user run a program that
worked yesterday but today cannot find it's dependency simply because
the dependency version was upgraded

as for the ISOs being built on a server - it is much as i described
already - the process is very simple now - it is literally a single
command line - again to note on the other hand that OBS has another
attractive feature that can automatically build ISOs, boot them, and run
a GUI-enabled test suite



[1]:
https://notabug.org/bill-auger/parabolaiso/src/refactor/docs/README.build
[2]:
https://notabug.org/bill-auger/parabolaiso/src/refactor/docs/README.altbootmethods#L85
[3]: https://aur.archlinux.org/packages/ipxe-netboot
[4]: https://www.archlinux.org/releng/netboot/
[5]: https://github.com/manjaro/manjaro-tools/pull/273
[6]: https://github.com/manjaro/iso-profiles/pull/75

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://lists.parabola.nu/pipermail/dev/attachments/20171107/4fb90b4e/attachment.sig>