[Dev] [RFC] Adding DLAGENTS for git and other VCSs to makepkg.conf
Luke T.Shumaker
lukeshu at sbcglobal.net
Thu Apr 12 05:43:55 GMT 2012
Abstract
Given Parabola's policy that sources be downloaded prior to running
a PKGBUILD file's build() function, we are creating multiple
boiler-plate SRCBUILD files that all check out a specific commit
from a version control system (VCS), and generate a tarball from it.
Given that most VCSs are able to identify a commit just as
permanently as a URL, or more so, it makes sense to to add a
URL-handler ("DLAGENT") for URLs with a VCS SCM scheme, such as
"git://".
Permanence of availability concerns
One of the primary objections to getting files directly from an VCS
has been that there must be a permanence of availability, a user
must have a guarantee that the source he is downloading is the
source originally used to build the package.
If we give the DLAGENT the ability to specify a commit ID as part
of the URL (not traditionally done by VCS tools, more below), the
commit is just as unlikely to change as any other URL is.
Further, distributed VCSs (DCVS) typically have cryprographically
secure hash IDs, for example
https://github.com/nothingmuch/git-svn-abandon.git#42014bf555fdac67c8c82ed69c4733ab9dba0c8a
is cryptographically assured to always refer to the same code.
URL schemes for VCS tools
VCS tools frequently use "pseudo-URLs". This necessitates using a
wrapper program to (trivially) map between a URL and a VCS "remote
repository", as well as getting the appropriate commit from the
repository.
Following are definitions for mappings for common VCS
tools.another prot
Git:
git://${authority}/${path}[#${refspec}]
git+${proto}://${authority}/${path}[#${refspec}]
where
* ${proto} is the scheme of the protocol being used to access
the repository, if not git, sush as "ssh" or "http".
* ${authority} is the "[userinfo@]host[:port]" string we all know
so well, defined in RFC 3986.
* ${path} is the path-part after the authority
* ${refspec} is the git refspec of the commit. For our
purposes, this is a commit hash, but it could be a branch or
tag.
SVN:
svn+${proto}://${authority}/${path}[#${revnum}]
where
* ${revnum} is the revision number of the commit we are after
Implications
This would require adding a few lines to the DLAGENTS array in
/etc/makepkg.conf, as well as requiring that a few wrapper scripts
be installed.
Other ideas
I have some more ideas about decent caching/tar'ing of commit trees,
but I'll have to tinker first.
Questions
What do you think of this idea?
If you like it, should the wrapper scripts be packaged with pacman,
or be a separate package?
Sample implementation
The following implementation is from an unrelated project I was
working on that required pacman-like functionality. It is
unsuitable for use by `makepkg`, however, displays the concept
well.
case "$scheme" in
http|https|ftp)
wget -c "$url"
bsdtar -xf "${url##*/}"
;;
svn+*)
host="`echo "$url"|sed -r 's at .*://([^:/]*)[:/].*@\1@'`"
data="`grep -F "$host" -- "$FRCUTILDIR/etc/svn-auth"`"
if [ -n "$data" ]; then
data="`echo "$data"|sed -r 's/\s+/\t/g'|cut -f 2-`"
set -- $data
auth="--username $1 --password $2"
fi
mkdir -p "$FRCUTILDIR/tmp/svn-config"
svn co $auth --config-dir "$FRCUTILDIR/tmp/svn-config" "${url#svn+}" "$package"
;;
git+*|git)
nurl=${url#git+}
base=${nurl%%#*}
frag=${nurl#*#}
frc msg2 "Pulling from GIT repository..."
if [ ! -d "$package" ]; then
git clone "$base" "$package"
else
cd "$package" && {
git pull "$base"
cd ..
}
fi
if [ -n "$frag" ]; then
frc msg2 "Checking out proper commit/tag/branch..."
cd "$package" && {
git checkout "$frag"
cd ..
}
fi
;;
esac
More information about the Dev
mailing list