From bill-auger at peers.community Sun Jun 2 13:57:10 2019 From: bill-auger at peers.community (bill-auger) Date: Sun, 2 Jun 2019 09:57:10 -0400 Subject: [Dev] [PATCH] libremakepkg: fix building packages requring a rw startdir In-Reply-To: <20190529075938.408a8cef@parabola> References: <20190517010356.20049-1-GNUtoo@cyberdimension.org> <20190519043325.4200e0ee@parabola> <20190520130116.51973cff@parabola> <20190520193048.4cf5601f@primarylaptop.localdomain> <20190529101215.GA9405@parabola-pocket.localdomain> <20190529075938.408a8cef@parabola> Message-ID: <20190602095710.52c08009@parabola> i noticed that libremakepkg has options to control mounting ro or rw - maybe that was intended to me the solution all along -w Bind mount a file or directory, read/write -r Bind mount a file or directory, read-only From lukeshu at lukeshu.com Sun Jun 2 14:09:50 2019 From: lukeshu at lukeshu.com (Luke Shumaker) Date: Sun, 02 Jun 2019 10:09:50 -0400 Subject: [Dev] [PATCH] libremakepkg: fix building packages requring a rw startdir In-Reply-To: <20190529075938.408a8cef@parabola> References: <20190517010356.20049-1-GNUtoo@cyberdimension.org> <20190519043325.4200e0ee@parabola> <20190520130116.51973cff@parabola> <20190520193048.4cf5601f@primarylaptop.localdomain> <20190529101215.GA9405@parabola-pocket.localdomain> <20190529075938.408a8cef@parabola> Message-ID: <87zhn0t6xd.wl-lukeshu@lukeshu.com> On Wed, 29 May 2019 07:59:38 -0400, bill-auger wrote: > On Wed, 29 May 2019 12:12:15 +0200 Andreas wrote: > > Could someone explain to me why we want to break this > > behaviour? > > i think only luke would know first hand - he explained the > rationale; but its not clear what particular use case it was > intended to support, or some bug it was intended to fix > > "- We mount the temporary directory containing the extracted > source package files read-only, to be sure that makepkg doesn't > modify the PKGBUILD. This is necessary because --holdver only > disables pkgver() if it's a VCS package." The guarantee I was trying to make is that the sources in the source tarball match what is used to build the package, and can be used to build the package. Many packages from AUR do something silly like pkgver() { date +'%Y%m%d'; } With that, it isn't possible to re-build the same package version, even though you have the sourceball. The pkgver() function should always generate the same version from the same sources; if the version doesn't change, then it doesn't try to edit the PKGBUILD. So this is trying to enforce that it doesn't change *again* after the sourceball is created. The issue is that apparently `makepkg --allsource` doesn't run `pkgver()`. IMO, the correct fix is to modify `download_sources()` to do something to make sure that `pkgver()` gets run. Until that is patched, it could be worked around by running `makepkg -o` before runnig libremakepkg (that's not a bad idea anyway--you should glance over the changes for a VCS package before updating it :) ) As for linux-libre modifying the install script in `$startdir`; it should be doing that in `$srcdir`; and I don't have a big problem with saying "fix the PKGBUILD" in that case. -- Happy hacking, ~ Luke Shumaker From andreas at grapentin.org Sun Jun 2 14:44:25 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Sun, 2 Jun 2019 16:44:25 +0200 Subject: [Dev] [PATCH] libremakepkg: fix building packages requring a rw startdir In-Reply-To: <87zhn0t6xd.wl-lukeshu@lukeshu.com> References: <20190517010356.20049-1-GNUtoo@cyberdimension.org> <20190519043325.4200e0ee@parabola> <20190520130116.51973cff@parabola> <20190520193048.4cf5601f@primarylaptop.localdomain> <20190529101215.GA9405@parabola-pocket.localdomain> <20190529075938.408a8cef@parabola> <87zhn0t6xd.wl-lukeshu@lukeshu.com> Message-ID: <20190602144425.GA28848@parabola-pocket.localdomain> On Sun, Jun 02, 2019 at 10:09:50AM -0400, Luke Shumaker wrote: > The guarantee I was trying to make is that the sources in the source > tarball match what is used to build the package, and can be used to > build the package. Many packages from AUR do something silly like > > pkgver() { date +'%Y%m%d'; } I would argue that the version of the package is ephemereal -- we shouldn't have to make guarantees that, and also shouldn't rely on the fact that a subsequent build of the package produces the same version number. Although I do agree that putting the build date into the pkgver is silly... Best, Andreas -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From nobody at parabola.nu Tue Jun 4 18:06:41 2019 From: nobody at parabola.nu (Parabola Website Notification) Date: Tue, 04 Jun 2019 18:06:41 -0000 Subject: [Dev] Orphan Libre package [icecat] marked out-of-date Message-ID: <20190604180641.1078.1421@winston.parabola.nu> tirifto at posteo.cz wants to notify you that the following packages may be out-of-date: * icecat 60.3.0_gnu1-5 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/icecat/ * icecat 60.3.0_gnu1-6 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/icecat/ * icecat 60.3.0_gnu1-5 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/icecat/ The user provided the following additional text: https://lists.gnu.org/archive/html/bug-gnuzilla/2019-06/msg00000.html From nobody at parabola.nu Wed Jun 5 08:21:05 2019 From: nobody at parabola.nu (Parabola Website Notification) Date: Wed, 05 Jun 2019 08:21:05 -0000 Subject: [Dev] Orphan Libre package [iceweasel] marked out-of-date Message-ID: <20190605082105.1077.28606@winston.parabola.nu> grizzlyuser at protonmail.com wants to notify you that the following packages may be out-of-date: * iceweasel 1:67.0-1.parabola1 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/iceweasel/ The user provided the following additional text: Thank you for maintaining this package! A new version of Firefox is available: https://www.mozilla.org/en-US/firefox/releasenotes/ Arch Linux has already updated their package, and it required only version number change in their PKGBUILD. From andreas at grapentin.org Thu Jun 6 17:17:54 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Thu, 6 Jun 2019 19:17:54 +0200 Subject: [Dev] [PATCH] fix for regression in libremakepkg if SRCDEST is set in /etc/makepkg.conf Message-ID: <20190606171754.GA4133@parabola-pocket.localdomain> libretools-20181004-4 has introduced a regression in libremakepkg where setting the SRCDEST variable in /etc/makepkg.conf will result in the package sources not being available when the build enters the chroot: example from icecat: [...] | ==> Retrieving sources... | -> Downloading icecat-60.3.0-gnu1.tar.bz2... | % Total % Received % Xferd Average Speed Time Time Time Current | Dload Upload Total Spent Left Speed | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 3 seconds. 3 retries left. | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 3 seconds. 2 retries left. | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 3 seconds. 1 retries left. | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: ftp.gnu.org | ==> ERROR: Failure while downloading http://ftp.gnu.org/gnu/gnuzilla/60.3.0/icecat-60.3.0-gnu1.tar.bz2 [...] this behaviour is caused by the current version of the patch: 0001-libremakepkg-fix-building-packages-requring-a-rw-sta.patch to restore the functionality of libremakepkg in environments where SRCDEST is set, I propose the following changeset instead: --- a/src/chroot-tools/libremakepkg +++ b/src/chroot-tools/libremakepkg @@ -124,11 +124,11 @@ build() ( local run_ynet=() local run_nnet=() if $INCHROOT; then - local _run=(sh -c "mount --bind -o ro -- ${startdir at Q} ${startdir at Q} && cd ${startdir at Q} && \$@" --) + local _run=(sh -c "cd ${startdir at Q} && \$@" --) run_ynet=(unshare --mount -- "${_run[@]}") run_nnet=(unshare --mount --net -- "${_run[@]}") else - librechroot_flags+=(-r "$startdir:/startdir") + librechroot_flags+=(-w "$startdir:/startdir") run_ynet=(librechroot "${librechroot_flags[@]}" run) run_nnet=(librechroot "${librechroot_flags[@]}" -N run) fi I also feel like we should not maintain that change as a patch, but instead integrate them into libretools once we have reached consensus on whether we want a writable startdir or not. I don't remember that thread having reached consensus yet. Best, Andreas ~oaken-source -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From eschwartz at archlinux.org Thu Jun 6 17:22:21 2019 From: eschwartz at archlinux.org (Eli Schwartz) Date: Thu, 6 Jun 2019 13:22:21 -0400 Subject: [Dev] [PATCH] libremakepkg: fix building packages requring a rw startdir In-Reply-To: <87zhn0t6xd.wl-lukeshu@lukeshu.com> References: <87zhn0t6xd.wl-lukeshu@lukeshu.com> Message-ID: > The guarantee I was trying to make is that the sources in the source > tarball match what is used to build the package, and can be used to > build the package. Many packages from AUR do something silly like > > pkgver() { date +'%Y%m%d'; } > > With that, it isn't possible to re-build the same package version, > even though you have the sourceball. The pkgver() function should > always generate the same version from the same sources; if the version > doesn't change, then it doesn't try to edit the PKGBUILD. So this is > trying to enforce that it doesn't change *again* after the sourceball > is created. Your solution to the problem is not a solution. Now you have a package with dynamically updated contents, where the pkgver does not match. The package is still totally unreproducible, but for bonus points it is also lying, because a writable PKGBUILD is needed in order for it to figure out its own internal truth. Any AUR package that uses `date +'%Y%m%d'` should be deleted with extreme prejudice, or forcibly orphaned and hostilely taken over by a well-behaved package maintainer. You have my blessing as an AUR administrator to pursue that course of action. I strongly encourage making a Parabola project policy that official Parabola packages are not permitted to use `date +'%Y%m%d'`. > The issue is that apparently `makepkg --allsource` doesn't run > `pkgver()`. IMO, the correct fix is to modify `download_sources()` to > do something to make sure that `pkgver()` gets run. Until that is > patched, it could be worked around by running `makepkg -o` before > runnig libremakepkg (that's not a bad idea anyway--you should glance > over the changes for a VCS package before updating it :) ) That is correct -- allsource is meant to download sources, not check them out, run the prepare function, and then calculate versions base on that. In Arch Linux, the package gets built first, then the source-package is created after. > As for linux-libre modifying the install script in `$startdir`; it > should be doing that in `$srcdir`; and I don't have a big problem with > saying "fix the PKGBUILD" in that case. I suggest you try setting install="$srcdir"/foo.install and see what happens. There is a parabola bug report about the rw startdir, and it's been discussed there why that suggestion won't work. -- Eli Schwartz Bug Wrangler and Trusted User -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1601 bytes Desc: OpenPGP digital signature URL: From nobody at parabola.nu Tue Jun 11 09:40:43 2019 From: nobody at parabola.nu (Parabola Website Notification) Date: Tue, 11 Jun 2019 09:40:43 -0000 Subject: [Dev] Orphan Libre package [linux-libre] marked out-of-date Message-ID: <20190611094043.1080.99077@winston.parabola.nu> riku.viitanen0 at gmail.com wants to notify you that the following packages may be out-of-date: * linux-libre 5.1.6_gnu-1 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/linux-libre/ * linux-libre 5.1.6_gnu-1 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/linux-libre/ * linux-libre 5.1.6_gnu-1 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/linux-libre/ * linux-libre-chromebook 5.1.4_gnu-1 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/linux-libre-chromebook/ * linux-libre-docs 5.1.6_gnu-1 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/linux-libre-docs/ * linux-libre-docs 5.1.6_gnu-1 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/linux-libre-docs/ * linux-libre-docs 5.1.6_gnu-1 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/linux-libre-docs/ * linux-libre-headers 5.1.6_gnu-1 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/linux-libre-headers/ * linux-libre-headers 5.1.6_gnu-1 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/linux-libre-headers/ * linux-libre-headers 5.1.6_gnu-1 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/linux-libre-headers/ The user provided the following additional text: Linux-libre 5.1.8 is available. From bill-auger at peers.community Tue Jun 11 17:41:03 2019 From: bill-auger at peers.community (bill-auger) Date: Tue, 11 Jun 2019 13:41:03 -0400 Subject: [Dev] repo redirector modificatoins Message-ID: <20190611134103.6316a037@parabola> there has been discussion on the IRC recently about improving the redirector behavior in various ways for various reasons; and i wanted to get them documented on the mailing list and open for discussion the first concern was regarding which mirrors to use - the parabola mirror pool is healthier than ever now and it would probably improve the user-experience to stop redirecting to any arch mirrors and always redirect to a parabola mirror - the arch mirrors are frequently out-of-sync and return 404 - also, in my experience most of the arch mirrors that we use are slower than the parabola mirrors the second concern stems from that discussion - it was suggested that we could ping each mirror with a HEAD request to determine whether a specific packages exists before actually redirecting the user to it - as i remember, lukeshu explained that this problem only existed for the arch mirrors due to the crude and blind way that they are selected; but that the - redirector for parabola mirrors was introspective and was unlikely to ever result in a 404 one new concern i would like to raise is regarding packages that are "not built from source" by the parabola standard - that is, specifically those the pull dependencies at build-time that are not itemized in the sources() array - this results in source incomplete packages even if disregarding the binary build deps, anything that is not itemized in the sources() array is something that may become unobtainable in the future (or in the case of pulling the the daily HEAD of a VCS repo, non-trivial to resolve which repo state corresponds to the original source, even if the source are obtainable) - that is aside from the concern that it requires the network to be active in build chroots the proper solution, of course, is to package each of the sources separately, which adds a significant amount of work to the overall maintenance of the distro - that amount is roughly gauged by this value/cost metric: where: value = n_users + importance cost = auditing + packaging + maintenance the costs are not simple to guess, but do become evident by doing the work - auditing, although a relatively high cost, can probably be factored out; because that is entailed by each freedom bug report - value however is somewhat subjective, especially because we make no attempt to determine 'n_users', even proportionately - 'importance' is something that could be determined through some research and discussion; but it could be helpful for determining the fate of such packages like this, to start ranking packages by relative popularity (normalizing against something in 'base', such as 'pacman') we dont want to blacklist anything hastily; but we also dont want to expend precious resources rescuing something that no one actually wants to use - most recently, certain packages such as 'dotnet' and 'lsd' have made this decision difficult the 'awesome-terminal-fonts' package identified in the artistic-freedom bug report #2331, on its own, could probably be blacklisted as having a very low value/cost ratio; but that should also factor in the value/cost ratio of it's dependent 'lsd' package https://labs.parabola.nu/issues/2331 community/lsd 0.15.1-1 Modern ls with a lot of pretty colors and awesome icons does any parabola user want such a thing, for example? - just because the devs may not value it, would not imply that users would not; so that is not a good gauge this third concern is related to the previous one of redirecting to out-of-sync mirrors because that results in multiple requests for the same package; often enough to skew the popularity ranking; so collecting fairly accurate numbers would require the redirector to distinguish when a package was actually downloaded and not merely that it was requested From andreas at grapentin.org Tue Jun 11 18:39:19 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Tue, 11 Jun 2019 20:39:19 +0200 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190611134103.6316a037@parabola> References: <20190611134103.6316a037@parabola> Message-ID: <20190611183919.GA18158@parabola-pocket.localdomain> On Tue, Jun 11, 2019 at 01:41:03PM -0400, bill-auger wrote: > the first concern was regarding which mirrors to use - the > parabola mirror pool is healthier than ever now and it would > probably improve the user-experience to stop redirecting to > any arch mirrors and always redirect to a parabola mirror - the > arch mirrors are frequently out-of-sync and return 404 - also, in > my experience most of the arch mirrors that we use are slower > than the parabola mirrors Is this the case for all regions? I'm happy we talk about improving the repomirror, and I agree that it might be time to stop redirecting to arch mirrors, but only if we can make sure to provide proper global coverage. I'd of course happily volunteer to use the repomirror in the future and help fix any issues that crop up. > the second concern stems from that discussion - it was suggested > that we could ping each mirror with a HEAD request to > determine whether a specific packages exists before actually > redirecting the user to it - as i remember, lukeshu explained > that this problem only existed for the arch mirrors due to the > crude and blind way that they are selected; but that the - > redirector for parabola mirrors was introspective and was > unlikely to ever result in a 404 That sounds like a lot of extra work for something that could just be gone by implementing the change proposed above. I think we should keep this option in mind, but postpone it for now, and see if we still need it. Of course, if we come to the conclusion that we are not yet able to drop the arch mirrors, we can consider this as an alternative way to make the repomirror more reliable. Best, oak -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From andreas at grapentin.org Tue Jun 11 18:45:05 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Tue, 11 Jun 2019 20:45:05 +0200 Subject: [Dev] binary dependencies and blacklisting discussion (Was: repo redirector modificatoins) In-Reply-To: <20190611134103.6316a037@parabola> References: <20190611134103.6316a037@parabola> Message-ID: <20190611184505.GB18158@parabola-pocket.localdomain> On Tue, Jun 11, 2019 at 01:41:03PM -0400, bill-auger wrote: > one new concern i would like to raise is regarding packages that > are "not built from source" by the parabola standard - that is, > specifically those the pull dependencies at build-time that are > not itemized in the sources() array - this results in source > incomplete packages I think many of the java packages that build with maven could fall into this category... It's a bit of a mess, last time I checked. > even if disregarding the binary build deps, anything that is not > itemized in the sources() array is something that may become > unobtainable in the future (or in the case of pulling the the > daily HEAD of a VCS repo, non-trivial to resolve which repo > state corresponds to the original source, even if the source > are obtainable) - that is aside from the concern that it > requires the network to be active in build chroots > > the proper solution, of course, is to package each of the sources > separately, which adds a significant amount of work to the > overall maintenance of the distro - that amount is roughly > gauged by this value/cost metric: > > where: > > value = n_users + importance > cost = auditing + packaging + maintenance > > the costs are not simple to guess, but do become evident by doing > the work - auditing, although a relatively high cost, can > probably be factored out; because that is entailed by each > freedom bug report - value however is somewhat subjective, > especially because we make no attempt to determine 'n_users', > even proportionately - 'importance' is something that could be > determined through some research and discussion; but it could be > helpful for determining the fate of such packages like this, to > start ranking packages by relative popularity (normalizing > against something in 'base', such as 'pacman') > > we dont want to blacklist anything hastily; but we also dont > want to expend precious resources rescuing something that no one > actually wants to use - most recently, certain packages such as > 'dotnet' and 'lsd' have made this decision difficult > > the 'awesome-terminal-fonts' package identified in the > artistic-freedom bug report #2331, on its own, could probably be > blacklisted as having a very low value/cost ratio; but that > should also factor in the value/cost ratio of it's dependent > 'lsd' package > > https://labs.parabola.nu/issues/2331 > > community/lsd 0.15.1-1 > Modern ls with a lot of pretty colors and awesome icons > > does any parabola user want such a thing, for example? - just > because the devs may not value it, would not imply that users > would not; so that is not a good gauge I would suggest to get such a metric of cost vs. value, we could change the way we blacklist packages. Instead of simply removing them from the repositories, we could package a stub, that is just empty and has an .inistall script that says something like this: "this package is known to be nonfree because of REASONS. There is an issue report here at URL, you can vote on this issue to let us know we should be working on that, instead of playing dwarf fortress. Thanks!" That way we could more liberally start blacklisting things, and get immediate feedback from the community. Might even be able to automate parts of that process. -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From andreas at grapentin.org Tue Jun 11 18:46:56 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Tue, 11 Jun 2019 20:46:56 +0200 Subject: [Dev] package popularity tracking (Was: repo redirector modificatoins) In-Reply-To: <20190611134103.6316a037@parabola> References: <20190611134103.6316a037@parabola> Message-ID: <20190611184656.GC18158@parabola-pocket.localdomain> On Tue, Jun 11, 2019 at 01:41:03PM -0400, bill-auger wrote: > this third concern is related to the previous one of > redirecting to out-of-sync mirrors because that results in > multiple requests for the same package; often enough to skew the > popularity ranking; so collecting fairly accurate numbers would > require the redirector to distinguish when a package was > actually downloaded and not merely that it was requested Wouldn't it be possible to track successful downloads directly on the mirrors and accumulate that data? Best, -oak -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From bill-auger at peers.community Tue Jun 11 18:46:46 2019 From: bill-auger at peers.community (bill-auger) Date: Tue, 11 Jun 2019 14:46:46 -0400 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190611183919.GA18158@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> Message-ID: <20190611144646.18369ecc@parabola> On Tue, 11 Jun 2019 20:39:19 +0200 Andreas wrote: > stop redirecting to arch mirrors, but only if we can make sure > to provide proper global coverage. i dont see any sacrifice in dropping arch mirrors - arch mirrors do not contain any parabola packages; so even now, parabola packages can only be acquired from one of the existing parabola mirrors From bill-auger at peers.community Tue Jun 11 18:51:23 2019 From: bill-auger at peers.community (bill-auger) Date: Tue, 11 Jun 2019 14:51:23 -0400 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190611183919.GA18158@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> Message-ID: <20190611145123.5201548a@parabola> i could also add that the pacman2pacman network has the potential to provide global accessibility if that were encouraged more and used by a sufficient number of users From bill-auger at peers.community Tue Jun 11 19:05:12 2019 From: bill-auger at peers.community (bill-auger) Date: Tue, 11 Jun 2019 15:05:12 -0400 Subject: [Dev] package popularity tracking (Was: repo redirector modificatoins) In-Reply-To: <20190611184656.GC18158@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611184656.GC18158@parabola-pocket.localdomain> Message-ID: <20190611150512.36f2bf60@parabola> sure, but that would be either voluntary or an extra requirement to place on mirrors - we dont need a total sample set though - i think most users do use the redirector, as it is the only source enabled in default mirrorlist - if so, it should be able capture statistically significant results, without requiring or imposing anything on mirrors - we have so little information about usage now, it would be difficult to determine even that much it is true though, that any user who prioritizes a specific mirror but still keeps the redirector enabled could skew the results - that is the recommended configuration in the mirrorlist file: either to use the redirector exclusively, or to keep it enabled with a lower priority to a preferred mirror - maybe that recommendation would need to be reversed to suggest the redirector be used only mutually exclusive to any other explicit mirror, in order to gather representative samples From andreas at grapentin.org Tue Jun 11 19:30:44 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Tue, 11 Jun 2019 21:30:44 +0200 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190611144646.18369ecc@parabola> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> Message-ID: <20190611193044.GA29857@parabola-pocket.localdomain> On Tue, Jun 11, 2019 at 02:46:46PM -0400, bill-auger wrote: > On Tue, 11 Jun 2019 20:39:19 +0200 Andreas wrote: > > stop redirecting to arch mirrors, but only if we can make sure > > to provide proper global coverage. > > i dont see any sacrifice in dropping arch mirrors - arch mirrors > do not contain any parabola packages; so even now, parabola > packages can only be acquired from one of the existing parabola > mirrors The sacrifice is bandwidth, and by extension time. If I pacstrap from a remote location, that happens to have an arch mirror nearby and a parabola mirror far away, then I can speed up the process a lot by using the arch mirror for the bulk of the packages that come from upstream. Checking the installed sizes of packages on my system gives: 7800 MiB for core/extra/community 2900 MiB for libce/pcr Assuming equal update frequencies for arch and parabola, which is pretty generous for the sake of this argument, that is still ~70% of transfers that can be considerably sped up by having an arch mirror with a better bandwidth than the nearest parabola mirror. Best, -oak -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From bill-auger at peers.community Tue Jun 11 19:31:29 2019 From: bill-auger at peers.community (bill-auger) Date: Tue, 11 Jun 2019 15:31:29 -0400 Subject: [Dev] binary dependencies and blacklisting discussion (Was: repo redirector modificatoins) In-Reply-To: <20190611184505.GB18158@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611184505.GB18158@parabola-pocket.localdomain> Message-ID: <20190611153129.6706f1e2@parabola> On Tue, 11 Jun 2019 20:45:05 +0200 Andreas wrote: > I think many of the java packages that build with maven could > fall into this category... It's a bit of a mess, last time I > checked. also, pretty much anything golang or javascript, and quite commonly, python - these are not isolated cases anymore - this seems to be a trend that is increasing in popularity; so there should be a consensus on how to handle such packages generally - lukeshu suggests that it doesnt require much discussion; because it is a consequence of the FSDG for the distro to be self-hostable, and other reasons such as GPL CCS compliance On Tue, 11 Jun 2019 20:45:05 +0200 Andreas wrote: > Instead of simply > removing them from the repositories, we could package a stub, > that is just empty and has an .inistall script > > That way we could more liberally start blacklisting things, > and get immediate feedback from the community. Might even be > able to automate parts of that process. that would surely ease the issue while deflecting any accusations that we are too trigger-happy it does assume though, that most users use pacman in a shell and not a GUI - freemor and i have discussed similar modifications (on IRC), regarding a proposed "pac-rollback" feature, that would need to be implemented in tandem with GUI representations in octopi in order to cover all users From andreas at grapentin.org Tue Jun 11 19:39:24 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Tue, 11 Jun 2019 21:39:24 +0200 Subject: [Dev] package popularity tracking (Was: repo redirector modificatoins) In-Reply-To: <20190611150512.36f2bf60@parabola> References: <20190611134103.6316a037@parabola> <20190611184656.GC18158@parabola-pocket.localdomain> <20190611150512.36f2bf60@parabola> Message-ID: <20190611193924.GA8672@parabola-pocket.localdomain> On Tue, Jun 11, 2019 at 03:05:12PM -0400, bill-auger wrote: > sure, but that would be either voluntary or an extra requirement > to place on mirrors - we dont need a total sample set though - i > think most users do use the redirector, as it is the only source > enabled in default mirrorlist - if so, it should be able capture > statistically significant results, without requiring or imposing > anything on mirrors - we have so little information about usage > now, it would be difficult to determine even that much To be fair, the first thing I do on a freshly installed parabola system is change the mirrorlist. simply because the repomirror used te be so unreliable. That being said, I don't think we need to track downloads on all mirrors. Assuming statistically equal spread of requests, tracking successful downloads on *any* mirror would do, at least to determine the relative popularity of packages against each other. Tracking the repomirrors redirects would be useful to get a measure on the overall popularity of the distro, i.e. number of installs that actually use the repomirror, but I'm not sure what that measure would be good for. I propose choosing one or two reliable and well-used mirror we have direct control over, and track successful downloads there, and extrapolate the results. Let the redirector do its thing. > it is true though, that any user who prioritizes a specific > mirror but still keeps the redirector enabled could skew the > results - that is the recommended configuration in the mirrorlist > file: either to use the redirector exclusively, or to keep it enabled > with a lower priority to a preferred mirror - maybe that > recommendation would need to be reversed to suggest the > redirector be used only mutually exclusive to any other explicit > mirror, in order to gather representative samples Putting the redirector at the end of the mirrorlist would make a lot of sense. It would allow users to enable (uncomment) a default mirror they prefer, and pacman would only fall back on the redirector if needed. If we want to continue this train of thought, I would recommend doing so in a new thread :) Best, -oak -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From bill-auger at peers.community Tue Jun 11 19:40:26 2019 From: bill-auger at peers.community (bill-auger) Date: Tue, 11 Jun 2019 15:40:26 -0400 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190611193044.GA29857@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> Message-ID: <20190611154026.4173b35d@parabola> On Tue, 11 Jun 2019 21:30:44 +0200 Andreas wrote: > On Tue, Jun 11, 2019 at 02:46:46PM -0400, bill-auger wrote: > > i dont see any sacrifice in dropping arch mirrors - arch > > parabola packages can only be acquired from one of the > > existing parabola mirrors > > The sacrifice is bandwidth > mirror nearby and a parabola mirror far away understood - we do have that information https://www.parabola.nu/mirrors/ it looks like north america, europ, and asia are covered sufficiently - the out-lying areas being south america, africa, middle-east, and australia - but keep in mind that the arch-bound redirector does not have any logic in it now - it simply selects one of a fixed set of ten indiscriminately; and im not sure if geographic diversity was ever considered as a criteria for being place into that set - those more distant locations may not be served optimally with the current redirector either From andreas at grapentin.org Tue Jun 11 19:47:49 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Tue, 11 Jun 2019 21:47:49 +0200 Subject: [Dev] binary dependencies and blacklisting discussion (Was: repo redirector modificatoins) In-Reply-To: <20190611153129.6706f1e2@parabola> References: <20190611134103.6316a037@parabola> <20190611184505.GB18158@parabola-pocket.localdomain> <20190611153129.6706f1e2@parabola> Message-ID: <20190611194749.GA8058@parabola-pocket.localdomain> On Tue, Jun 11, 2019 at 03:31:29PM -0400, bill-auger wrote: > it does assume though, that most users use pacman in a shell and > not a GUI - freemor and i have discussed similar modifications > (on IRC), regarding a proposed "pac-rollback" feature, that would > need to be implemented in tandem with GUI representations in > octopi in order to cover all users Any pacman frontend that does not prominently display the output of .install scripts to the user during installation is broken by design. This behavior should be reported as a bug, if it exists. Best, -oak -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From andreas at grapentin.org Tue Jun 11 19:51:32 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Tue, 11 Jun 2019 21:51:32 +0200 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190611154026.4173b35d@parabola> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> <20190611154026.4173b35d@parabola> Message-ID: <20190611195132.GB8058@parabola-pocket.localdomain> On Tue, Jun 11, 2019 at 03:40:26PM -0400, bill-auger wrote: > understood - we do have that information > > https://www.parabola.nu/mirrors/ > > it looks like north america, europ, and asia are covered > sufficiently - the out-lying areas being south america, africa, > middle-east, and australia - but keep in mind that the > arch-bound redirector does not have any logic in it now - it > simply selects one of a fixed set of ten indiscriminately; and > im not sure if geographic diversity was ever considered as a > criteria for being place into that set - those more distant > locations may not be served optimally with the current redirector > either Well, arch is pretty good at providing information about their current mirror status. We could choose to make the arch-bound portion of the redirector smarter, and also make the redirector smarter to make sure arch mirrors are only used when needed. I'm sure we could draft up a quick sketch for an appropriate heuristic in a couple of man-hours. Best, -oak -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From bill-auger at peers.community Tue Jun 11 19:55:30 2019 From: bill-auger at peers.community (bill-auger) Date: Tue, 11 Jun 2019 15:55:30 -0400 Subject: [Dev] package popularity tracking (Was: repo redirector modificatoins) In-Reply-To: <20190611193924.GA8672@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611184656.GC18158@parabola-pocket.localdomain> <20190611150512.36f2bf60@parabola> <20190611193924.GA8672@parabola-pocket.localdomain> Message-ID: <20190611155530.0969783e@parabola> On Tue, 11 Jun 2019 21:39:24 +0200 Andreas wrote: > I propose choosing one or two reliable and well-used mirror which are the most used mirrors is currently another unknown - you have one such mirror; so you could be one of the monitors; but how to determine if yours is among the most used On Tue, 11 Jun 2019 21:39:24 +0200 Andreas wrote: > Putting the redirector at the end of the mirrorlist would make > a lot of sense. It would allow users to enable (uncomment) a > default mirror they prefer, and pacman would only fall back on > the redirector if needed. that is already the recommendation - i added that to the mirrorlist some time ago: > # Note: This is not itself a mirror, but a load-balancing redirector. > # It automatically redirects to an approriate Arch or Parabola mirror. > # It works best by specifying it a few times, to recover from 404s. > # This is the most reliable and recommended option. > # Moving a nearby mirror from the list below to above these can often > # afford a speed boost; but keeping these enabled always is most reliable. > Server = https://redirector.parabola.nu/$repo/os/$arch > Server = https://redirector.parabola.nu/$repo/os/$arch > Server = https://redirector.parabola.nu/$repo/os/$arch > Server = https://redirector.parabola.nu/$repo/os/$arch > Server = https://redirector.parabola.nu/$repo/os/$arch i was only suggesting that it could skew the results; because the redirect or would only see requests from such a user sporadically, only for the most fresh packages, and only for brief periods while their preferred mirror is out-of sync - it would put a slight weight on packages that have higher churn rates, while they are less than a sync interval old - maybe thats negligible From bill-auger at peers.community Tue Jun 11 19:58:20 2019 From: bill-auger at peers.community (bill-auger) Date: Tue, 11 Jun 2019 15:58:20 -0400 Subject: [Dev] binary dependencies and blacklisting discussion (Was: repo redirector modificatoins) In-Reply-To: <20190611194749.GA8058@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611184505.GB18158@parabola-pocket.localdomain> <20190611153129.6706f1e2@parabola> <20190611194749.GA8058@parabola-pocket.localdomain> Message-ID: <20190611155820.7e6eb226@parabola> On Tue, 11 Jun 2019 21:47:49 +0200 Andreas wrote: > Any pacman frontend that does not prominently display the > output of .install scripts to the user during installation yea good point - honestly, ive never used it myself From grizzlyuser at protonmail.com Wed Jun 12 07:09:10 2019 From: grizzlyuser at protonmail.com (grizzlyuser) Date: Wed, 12 Jun 2019 07:09:10 +0000 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190611195132.GB8058@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> <20190611154026.4173b35d@parabola> <20190611195132.GB8058@parabola-pocket.localdomain> Message-ID: Please excuse me if I miss the point, but instead of using the redirector, can't pacman logic be modified to solve the problem the redirector tries to solve? For example, to have the second mirrorlist exclusively for fallback Arch Linux mirrors, so the user can choose which of them to fall back? Or maybe some other smarter change? From andreas at grapentin.org Wed Jun 12 09:06:18 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Wed, 12 Jun 2019 11:06:18 +0200 Subject: [Dev] repo redirector modificatoins In-Reply-To: References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> <20190611154026.4173b35d@parabola> <20190611195132.GB8058@parabola-pocket.localdomain> Message-ID: <20190612090618.GA13402@parabola-pocket.localdomain> On Wed, Jun 12, 2019 at 07:09:10AM +0000, grizzlyuser wrote: > Please excuse me if I miss the point, but instead of using the redirector, can't pacman logic be modified to solve the problem the redirector tries to solve? > For example, to have the second mirrorlist exclusively for fallback Arch Linux mirrors, so the user can choose which of them to fall back? Or maybe some other smarter change? We could in theory modify pacman to add arbitrary logic, that is true, but this is a route I would be hesitant to take. We take pacman from upstream arch and only modify it slightly for parabola. (Looking at the PKGBUILD, we already do modify it much more that I would like.) Which means any added functionality would have to be maintained as a set of patches, or in a forked repository. Updating these patches or the fork for a new version of pacman is a lot of work, and prone to errors and conflicts, so I would expect it would make it considerably harder for us to react to upstream updates. This can work, but I think if there is any other way to do it, that would probably be preferable :) Come to think of it, there already *is* another way that works with an unmodified pacman. pacman does not mandate that all repos use a single mirror list. You could configure a system that has a list of arch mirrors in mirrorlist.arch and a set of parabola mirrors in mirrorlist.parabola, and then choose the appropriate mirror list in the `Include' directive in pacman.conf for the respective repository. So with this in mind, we could drop arch mirrors from the redirector, and write a wiki page explaining how one could go about using arch mirrors for core/extra/community packages, for the probably very limited set of our users for whom this would be beneficial. Best, -oak -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From nobody at parabola.nu Wed Jun 12 15:40:59 2019 From: nobody at parabola.nu (Parabola Website Notification) Date: Wed, 12 Jun 2019 15:40:59 -0000 Subject: [Dev] Orphan Libre package [reflector] marked out-of-date Message-ID: <20190612154059.1079.1527@winston.parabola.nu> grizzlyuser at protonmail.com wants to notify you that the following packages may be out-of-date: * reflector 2018-2.parabola1 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/reflector/ * reflector 2018-2.parabola1 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/reflector/ * reflector 2018-2.parabola1 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/reflector/ The user provided the following additional text: Thank you for maintaining this package! A new version is available: https://xyne.archlinux.ca/projects/reflector/ From grizzlyuser at protonmail.com Wed Jun 12 16:50:32 2019 From: grizzlyuser at protonmail.com (grizzlyuser) Date: Wed, 12 Jun 2019 16:50:32 +0000 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190612090618.GA13402@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> <20190611154026.4173b35d@parabola> <20190611195132.GB8058@parabola-pocket.localdomain> <20190612090618.GA13402@parabola-pocket.localdomain> Message-ID: <7v0uR-NCOCsDqvgiJ9AvpIsyh7zF7Djxtfu9IPZaIrnxFT575LVon14Du-J-SRx74M9OB-1H0DOHiAq6Ssfx9aFbDO1kMZI0JqmCW-fq5tE=@protonmail.com> On Wednesday, June 12, 2019 12:06 PM, Andreas Grapentin wrote: > Come to think of it, there already is another way that works with an > unmodified pacman. pacman does not mandate that all repos use a single > mirror list. You could configure a system that has a list of arch > mirrors in mirrorlist.arch and a set of parabola mirrors in > mirrorlist.parabola, and then choose the appropriate mirror list in the > `Include' directive in pacman.conf for the respective repository. This is a good idea. To test it, I tried to configure a fresh install of Parabola this way. Then ran pacman -Syy to download fresh copies of master package databases. And finally tried to install some blacklisted package. Pacman attempted to install it, but said it conflicts with your-freedom. This behavior is different when package databases have been downloaded from Parabola mirrors: pacman complains the target is not found. From andreas at grapentin.org Wed Jun 12 17:26:27 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Wed, 12 Jun 2019 19:26:27 +0200 Subject: [Dev] repo redirector modificatoins In-Reply-To: <7v0uR-NCOCsDqvgiJ9AvpIsyh7zF7Djxtfu9IPZaIrnxFT575LVon14Du-J-SRx74M9OB-1H0DOHiAq6Ssfx9aFbDO1kMZI0JqmCW-fq5tE=@protonmail.com> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> <20190611154026.4173b35d@parabola> <20190611195132.GB8058@parabola-pocket.localdomain> <20190612090618.GA13402@parabola-pocket.localdomain> <7v0uR-NCOCsDqvgiJ9AvpIsyh7zF7Djxtfu9IPZaIrnxFT575LVon14Du-J-SRx74M9OB-1H0DOHiAq6Ssfx9aFbDO1kMZI0JqmCW-fq5tE=@protonmail.com> Message-ID: <20190612172627.GA18655@parabola-pocket.localdomain> On Wed, Jun 12, 2019 at 04:50:32PM +0000, grizzlyuser wrote: > On Wednesday, June 12, 2019 12:06 PM, Andreas Grapentin wrote: > > Come to think of it, there already is another way that works with an > > unmodified pacman. pacman does not mandate that all repos use a single > > mirror list. You could configure a system that has a list of arch > > mirrors in mirrorlist.arch and a set of parabola mirrors in > > mirrorlist.parabola, and then choose the appropriate mirror list in the > > `Include' directive in pacman.conf for the respective repository. > > This is a good idea. To test it, I tried to configure a fresh install of Parabola this way. Then ran pacman -Syy to download fresh copies of master package databases. And finally tried to install some blacklisted package. Pacman attempted to install it, but said it conflicts with your-freedom. This behavior is different when package databases have been downloaded from Parabola mirrors: pacman complains the target is not found. This is a good point -- why does this not happen when syncing through the redirector? Does that always sync the .db.tar.* files from a parabola mirror on -Sy? -oak -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From grizzlyuser at protonmail.com Wed Jun 12 20:59:01 2019 From: grizzlyuser at protonmail.com (grizzlyuser) Date: Wed, 12 Jun 2019 20:59:01 +0000 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190612172627.GA18655@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> <20190611154026.4173b35d@parabola> <20190611195132.GB8058@parabola-pocket.localdomain> <20190612090618.GA13402@parabola-pocket.localdomain> <7v0uR-NCOCsDqvgiJ9AvpIsyh7zF7Djxtfu9IPZaIrnxFT575LVon14Du-J-SRx74M9OB-1H0DOHiAq6Ssfx9aFbDO1kMZI0JqmCW-fq5tE=@protonmail.com> <20190612172627.GA18655@parabola-pocket.localdomain> Message-ID: On Wednesday, June 12, 2019 8:26 PM, Andreas Grapentin wrote: > This is a good point -- why does this not happen when syncing through > the redirector? Does that always sync the .db.tar.* files from a > parabola mirror on -Sy? I have no access to what's running on redirector, so don't know for sure. I've tried a few times to do 'pacman -Syy amd-ucode electron firefox' (blacklisted packages from [Core], [Community] and [Extra]) with the default pacman mirrorlist where only redirector.parabola.nu is uncommented. After each attempt the result was the same: "target not found" for each package. So it looks like redirector always returns databases from Parabola mirror. From bill-auger at peers.community Thu Jun 13 00:51:23 2019 From: bill-auger at peers.community (bill-auger) Date: Wed, 12 Jun 2019 20:51:23 -0400 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190612172627.GA18655@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> <20190611154026.4173b35d@parabola> <20190611195132.GB8058@parabola-pocket.localdomain> <20190612090618.GA13402@parabola-pocket.localdomain> <7v0uR-NCOCsDqvgiJ9AvpIsyh7zF7Djxtfu9IPZaIrnxFT575LVon14Du-J-SRx74M9OB-1H0DOHiAq6Ssfx9aFbDO1kMZI0JqmCW-fq5tE=@protonmail.com> <20190612172627.GA18655@parabola-pocket.localdomain> Message-ID: <20190612205123.2ebe8635@parabola> On Wed, 12 Jun 2019 19:26:27 +0200 Andreas wrote: > This is a good point -- why does this not happen when syncing > through the redirector? Does that always sync the .db.tar.* > files from a parabola mirror on -Sy? yes, when pacman -Sy hits the redirector,, it always gets the DB directly from parabola From bill-auger at peers.community Thu Jun 13 00:57:33 2019 From: bill-auger at peers.community (bill-auger) Date: Wed, 12 Jun 2019 20:57:33 -0400 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190612090618.GA13402@parabola-pocket.localdomain> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> <20190611154026.4173b35d@parabola> <20190611195132.GB8058@parabola-pocket.localdomain> <20190612090618.GA13402@parabola-pocket.localdomain> Message-ID: <20190612205733.52c0b52a@parabola> On Wed, 12 Jun 2019 11:06:18 +0200 Andreas wrote: > You could configure a system > that has a list of arch mirrors in mirrorlist.arch and a set > of parabola mirrors in mirrorlist.parabola, and then choose > the appropriate mirror list in the `Include' directive in > pacman.conf for the respective repository. > > So with this in mind, we could drop arch mirrors from the > redirector, and write a wiki page explaining how one could go > about using arch mirrors for core/extra/community packages, using tools that already exist is definitely best when possible - there is the 'reflector' package - ive not used it, but its purpose seems to be to routinely ping all mirrors and generate an optimized mirrorlist based on the ping latency - it may even install it automatically; but im not sure From andreas at grapentin.org Thu Jun 13 08:38:04 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Thu, 13 Jun 2019 10:38:04 +0200 Subject: [Dev] repo redirector modificatoins In-Reply-To: <20190612205733.52c0b52a@parabola> References: <20190611134103.6316a037@parabola> <20190611183919.GA18158@parabola-pocket.localdomain> <20190611144646.18369ecc@parabola> <20190611193044.GA29857@parabola-pocket.localdomain> <20190611154026.4173b35d@parabola> <20190611195132.GB8058@parabola-pocket.localdomain> <20190612090618.GA13402@parabola-pocket.localdomain> <20190612205733.52c0b52a@parabola> Message-ID: <20190613083804.GA2659@parabola-pocket.localdomain> On Wed, Jun 12, 2019 at 08:57:33PM -0400, bill-auger wrote: > using tools that already exist is definitely best when possible > - there is the 'reflector' package - ive not used it, but its > purpose seems to be to routinely ping all mirrors and generate > an optimized mirrorlist based on the ping latency - it may even > install it automatically; but im not sure I've used reflector in the past, it's a neat tool. It retrieves a list of mirrors directly from arch, and allows to filter them by sync status and geographical location, and ranks them by latency. You are correct that it can also directly install the generated mirrorlist to the system. -oak -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From nobody at parabola.nu Fri Jun 14 20:34:03 2019 From: nobody at parabola.nu (Parabola Website Notification) Date: Fri, 14 Jun 2019 20:34:03 -0000 Subject: [Dev] Orphan Libre package [cpupower] marked out-of-date Message-ID: <20190614203403.1077.55663@winston.parabola.nu> grizzlyuser at protonmail.com wants to notify you that the following packages may be out-of-date: * cgroup_event_listener 4.20_gnu-2 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/cgroup_event_listener/ * cgroup_event_listener 4.20_gnu-2 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/cgroup_event_listener/ * cgroup_event_listener 4.20_gnu-2 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/cgroup_event_listener/ * cpupower 4.20_gnu-2 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/cpupower/ * cpupower 4.20_gnu-2 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/cpupower/ * cpupower 4.20_gnu-2 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/cpupower/ * gpio-utils 4.20_gnu-2 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/gpio-utils/ * iio-utils 4.20_gnu-2 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/iio-utils/ * libtraceevent 4.20_gnu-2 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/libtraceevent/ * libtraceevent 4.20_gnu-2 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/libtraceevent/ * libtraceevent 4.20_gnu-2 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/libtraceevent/ * linux-libre-tools-meta 4.20_gnu-2 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/linux-libre-tools-meta/ * linux-libre-tools-meta 4.20_gnu-2 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/linux-libre-tools-meta/ * linux-libre-tools-meta 4.20_gnu-2 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/linux-libre-tools-meta/ * perf 4.20_gnu-2 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/perf/ * perf 4.20_gnu-2 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/perf/ * perf 4.20_gnu-2 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/perf/ * tmon 4.20_gnu-2 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/tmon/ * tmon 4.20_gnu-2 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/tmon/ * tmon 4.20_gnu-2 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/tmon/ * turbostat 4.20_gnu-2 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/turbostat/ * turbostat 4.20_gnu-2 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/turbostat/ * usbip 4.20_gnu-2 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/usbip/ * usbip 4.20_gnu-2 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/usbip/ * usbip 4.20_gnu-2 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/usbip/ * x86_energy_perf_policy 4.20_gnu-2 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/x86_energy_perf_policy/ * x86_energy_perf_policy 4.20_gnu-2 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/x86_energy_perf_policy/ The user provided the following additional text: Thank you for maintaining this package! A new version is available From nobody at parabola.nu Sat Jun 15 06:10:29 2019 From: nobody at parabola.nu (Parabola Website Notification) Date: Sat, 15 Jun 2019 06:10:29 -0000 Subject: [Dev] Orphan Libre package [linux-libre-api-headers] marked out-of-date Message-ID: <20190615061029.1080.97845@winston.parabola.nu> grizzlyuser at protonmail.com wants to notify you that the following packages may be out-of-date: * linux-libre-api-headers 4.17.11_gnu-1 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/linux-libre-api-headers/ * linux-libre-api-headers 4.17.11_gnu-1 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/linux-libre-api-headers/ * linux-libre-api-headers 4.17.11_gnu-1 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/linux-libre-api-headers/ The user provided the following additional text: Thank you for maintaining this package! A new version is available: https://linux-libre.fsfla.org/pub/linux-libre/releases/ From grizzlyuser at protonmail.com Thu Jun 20 05:46:55 2019 From: grizzlyuser at protonmail.com (grizzlyuser) Date: Thu, 20 Jun 2019 05:46:55 +0000 Subject: [Dev] Removal of unblacklisted packages from [libre]? Message-ID: <-_geR3Ub23uej_gFilwMjpCt46J4zsQMNhvsGOoYrs0gFo2UFK7jQ6fSF5qyYyUd1YoUovjTiWPhb6hPEevriAYRx_oUeQtX8bY-W4uQbB8=@protonmail.com> Two packages got recently removed from the blacklist [1], which makes me wonder: is there any established process of removal of such packages from [libre] repo? At this moment, when I issue, for example: # pacman -Syu kio-extras It tries to install libre/kio-extras. The newer version of this package from [extra] can be installed by: # pacman -Syu extra/kio-extras But on subsequent system upgrades pacman returns a warning, that the local package is newer than in libre. As I understand, since [libre] is higher in pacman.conf, the package from it has higher priority. This can be problematic in case if it's available in both [libre] and Arch repo, and is a dependency of another package being installed (dolphin depends on kio-extras). Then it's not convenient to get the most recent version of the dependency installed from Arch repo. Something like this is needed to be done instead of just the latter command: # pacman -S extra/kio-extras --asdeps # pacman -S dolphin [1]: https://git.parabola.nu/blacklist.git From bill-auger at peers.community Thu Jun 20 06:29:21 2019 From: bill-auger at peers.community (bill-auger) Date: Thu, 20 Jun 2019 02:29:21 -0400 Subject: [Dev] Removal of unblacklisted packages from [libre]? In-Reply-To: <-_geR3Ub23uej_gFilwMjpCt46J4zsQMNhvsGOoYrs0gFo2UFK7jQ6fSF5qyYyUd1YoUovjTiWPhb6hPEevriAYRx_oUeQtX8bY-W4uQbB8=@protonmail.com> References: <-_geR3Ub23uej_gFilwMjpCt46J4zsQMNhvsGOoYrs0gFo2UFK7jQ6fSF5qyYyUd1YoUovjTiWPhb6hPEevriAYRx_oUeQtX8bY-W4uQbB8=@protonmail.com> Message-ID: <20190620022921.3b334b04@parabola> there is no explicit procedure for removing an entry from the blacklist; but there is a procedure fora adding one https://git.parabola.nu/blacklist.git/tree/README#n33 the procedure for removing one, would be simply un-doing what was done to add it; including removing the libre replacement package and updating the original corresponding bug report https://labs.parabola.nu/issues/1167 From nobody at parabola.nu Fri Jun 21 04:45:36 2019 From: nobody at parabola.nu (Parabola Website Notification) Date: Fri, 21 Jun 2019 04:45:36 -0000 Subject: [Dev] Orphan Libre package [virt-install] marked out-of-date Message-ID: <20190621044536.1080.77581@winston.parabola.nu> grizzlyuser at protonmail.com wants to notify you that the following packages may be out-of-date: * virt-install 2.1.0-2.par1 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/virt-install/ * virt-install 2.0.0-3.par1 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/virt-install/ * virt-install 2.1.0-2.par1 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/virt-install/ The user provided the following additional text: Thank you for maintaining this package! A new version has been released: https://virt-manager.org/download/ From andreas at grapentin.org Tue Jun 25 07:55:16 2019 From: andreas at grapentin.org (Andreas Grapentin) Date: Tue, 25 Jun 2019 09:55:16 +0200 Subject: [Dev] [PATCH] fix for regression in libremakepkg if SRCDEST is set in /etc/makepkg.conf In-Reply-To: <20190606171754.GA4133@parabola-pocket.localdomain> References: <20190606171754.GA4133@parabola-pocket.localdomain> Message-ID: <20190625075516.GA3873@parabola-pocket.localdomain> Since nobody cared to raise any issues with the patch below, I've commited it and rebuilt libretools below. -oak On Thu, Jun 06, 2019 at 07:17:54PM +0200, Andreas Grapentin wrote: > > libretools-20181004-4 has introduced a regression in libremakepkg where > setting the SRCDEST variable in /etc/makepkg.conf will result in the > package sources not being available when the build enters the chroot: > > example from icecat: > > [...] > | ==> Retrieving sources... > | -> Downloading icecat-60.3.0-gnu1.tar.bz2... > | % Total % Received % Xferd Average Speed Time Time Time Current > | Dload Upload Total Spent Left Speed > | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 3 seconds. 3 retries left. > | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 3 seconds. 2 retries left. > | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Transient problem: timeout Will retry in 3 seconds. 1 retries left. > | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: ftp.gnu.org > | ==> ERROR: Failure while downloading http://ftp.gnu.org/gnu/gnuzilla/60.3.0/icecat-60.3.0-gnu1.tar.bz2 > [...] > > this behaviour is caused by the current version of the patch: > 0001-libremakepkg-fix-building-packages-requring-a-rw-sta.patch > to restore the functionality of libremakepkg in environments where > SRCDEST is set, I propose the following changeset instead: > > --- a/src/chroot-tools/libremakepkg > +++ b/src/chroot-tools/libremakepkg > @@ -124,11 +124,11 @@ build() ( > local run_ynet=() > local run_nnet=() > if $INCHROOT; then > - local _run=(sh -c "mount --bind -o ro -- ${startdir at Q} ${startdir at Q} && cd ${startdir at Q} && \$@" --) > + local _run=(sh -c "cd ${startdir at Q} && \$@" --) > run_ynet=(unshare --mount -- "${_run[@]}") > run_nnet=(unshare --mount --net -- "${_run[@]}") > else > - librechroot_flags+=(-r "$startdir:/startdir") > + librechroot_flags+=(-w "$startdir:/startdir") > run_ynet=(librechroot "${librechroot_flags[@]}" run) > run_nnet=(librechroot "${librechroot_flags[@]}" -N run) > fi > > I also feel like we should not maintain that change as a patch, but > instead integrate them into libretools once we have reached consensus on > whether we want a writable startdir or not. I don't remember that thread > having reached consensus yet. > > Best, > Andreas > > ~oaken-source > > -- > > ------------------------------------------------------------------------------ > my GPG Public Key: https://files.grapentin.org/.gpg/public.key > ------------------------------------------------------------------------------ > _______________________________________________ > Dev mailing list > Dev at lists.parabola.nu > https://lists.parabola.nu/mailman/listinfo/dev -- ------------------------------------------------------------------------------ my GPG Public Key: https://files.grapentin.org/.gpg/public.key ------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From nobody at parabola.nu Thu Jun 27 14:25:29 2019 From: nobody at parabola.nu (Parabola Website Notification) Date: Thu, 27 Jun 2019 14:25:29 -0000 Subject: [Dev] Orphan Libre package [qutebrowser] marked out-of-date Message-ID: <20190627142529.1080.10295@winston.parabola.nu> theova at member.fsf.org wants to notify you that the following packages may be out-of-date: * qutebrowser 1.6.2-1.par1 [libre] (armv7h): https://parabolagnulinux.org/packages/libre/armv7h/qutebrowser/ * qutebrowser 1.6.2-1.par1 [libre] (i686): https://parabolagnulinux.org/packages/libre/i686/qutebrowser/ * qutebrowser 1.6.2-1.par1 [libre] (x86_64): https://parabolagnulinux.org/packages/libre/x86_64/qutebrowser/ The user provided the following additional text: Version v1.6.3 is out as of Jun 18, 2019. PKGBUILD needs only a change of version and checksum.