[Dev] [dbscripts] [PATCH 5/6] Cache already built packages

[Luke Shumaker]

On Tue, 03 Apr 2018 16:10:58 -0400,
Luke Shumaker wrote:
> 
> From: Pierre Schmitz <pierre at archlinux.de>
> 
> (cherry picked from commit 59d495b55f52253518fe1a184627e7259034f978)
> ---
>  test/Makefile        |  6 ++++--
>  test/lib/common.bash | 25 +++++++++++++++++++------
>  2 files changed, 23 insertions(+), 8 deletions(-)
> 
> diff --git a/test/Makefile b/test/Makefile
> index d9437bf..5ed3910 100644
> --- a/test/Makefile
> +++ b/test/Makefile
> @@ -1,8 +1,10 @@
> +PACKAGE_CACHE:=$(shell mktemp -d)
> +
>  test:
> -	PATH=$(CURDIR)/../:$(CURDIR)/../cron-jobs/:$(PATH) bats cases
> +	PATH=$(CURDIR)/../:$(CURDIR)/../cron-jobs/:$(PATH) PACKAGE_CACHE=$(PACKAGE_CACHE) bats cases
>  
>  test-coverage:
> -	PATH=$(CURDIR)/../:$(CURDIR)/../cron-jobs/:$(PATH) kcov \
> +	PATH=$(CURDIR)/../:$(CURDIR)/../cron-jobs/:$(PATH) PACKAGE_CACHE=$(PACKAGE_CACHE) kcov \
>  		--include-path=$(CURDIR)/../ \
>  		--exclude-path=$(CURDIR)/../test,$(CURDIR)/../cron-jobs/makepkg.conf,$(CURDIR)/../config \
>  		$(COVERAGE_DIR) \
> diff --git a/test/lib/common.bash b/test/lib/common.bash
> index c61c948..7704098 100644
> --- a/test/lib/common.bash
> +++ b/test/lib/common.bash
> @@ -8,22 +8,30 @@ die() {
>  	exit 1
>  }
>  
> +__getCheckSum() {
> +	local result=($(sha1sum $1))
> +	echo ${result[0]}
> +}
> +
>  __buildPackage() {
>  	local arch=$1
>  	local pkgver
>  	local pkgname
>  	local a
>  	local p
> +	local checkSum
> +
> +	if [[ -n ${PACKAGE_CACHE} ]]; then
> +		checkSum=$(__getCheckSum PKGBUILD)
> +			# TODO: Be more specific
> +			if cp -av ${PACKAGE_CACHE}/${checkSum}/*-${arch}${PKGEXT}{,.sig} .; then
> +				return 0
> +			fi
> +	fi
>  
>  	pkgname=($(. PKGBUILD; echo "${pkgname[@]}"))
>  	pkgver=$(. PKGBUILD; get_full_version)
>  
> -	for p in "${pkgname[@]}"; do
> -		if [ -f "${p}-${pkgver}-${arch}"${PKGEXT} ]; then
> -			return 0
> -		fi
> -	done
> -
>  	if [ "${arch}" == 'any' ]; then
>  		sudo librechroot -n "dbscripts@${arch}" make
>  	else
> @@ -36,6 +44,11 @@ __buildPackage() {
>  			gpg --detach-sign --no-armor --use-agent "$file"
>  		done
>  	done
> +
> +	if [[ -n ${PACKAGE_CACHE} ]]; then
> +		mkdir -p ${PACKAGE_CACHE}/${checkSum}
> +		cp -av *-${arch}${PKGEXT}{,.sig} ${PACKAGE_CACHE}/${checkSum}/
> +	fi

This glob is overly permissive, and copies too much.  releasePackage
calls __buildPackage from
"${TMP}/svn-packages-copy/${pkgbase}/trunk/".  This means that when we
update a package, after the updated version is built, both v1 and v2
exist in that trunk/ directory.  Then, this copies both the v1 and v2
.pkg.tar.gz files to the cache directory for v2.

That is: On subsequent runs (e.i. with a cache hit) of __buildPackage
for v2 of that package, it will look like makepkg magically built both
v1 and v2 of the package, instead of just v2.

This isn't really a problem right now because further down in
releasePackage, when we copy the makepkg outputs to $STAGING, that
glob is sufficiently strict and only copies the v2 outputs.

But this could easily accidentially lead to weird breakage with a
seemingly innocuous change assumes that everything that __buildPackage
spits out is good (*ahem* 76f95dd139094dc2985b6733f23fcb098f07c64e).

>  }
>  
>  setup() {
> -- 
> 2.16.2

-- 
Happy hacking,
~ Luke Shumaker