[PATCH] cygport/lib/src_prep.cygpart: use checksum files with packages

[PATCH] cygport/lib/src_prep.cygpart: use checksum files with packages

[Brian Inglis]

From: "Brian Inglis" <Brian.Inglis@SystematicSW.ab.ca>

Some package upstreams offer only checksums, for example .sha512sum, .sha256sum,
for verification rather than gpg signatures, for example .asc, .sig, .sign, etc;
use these checksum files when provided in a similar manner to gpg signatures;
these files are often provided with fixed names which may be renamed on download
to unique values using cygport URI fragment support like #/$NAME-VERSION.sha...sum;
use coreutils cksum as it supports all modern and legacy checksums and formats.

define __sum_verify() after __gpg_verify();
add to readonly function definition list
unpack(): skip files matching *.*sum
__src_prep():
define file types or prefixes in variable sum_exts;
in src files loop after __gpg_verify():
match file checksum type and call __sum_verify()

Signed-off-by: Brian Inglis <Brian.Inglis@SystematicSW.ab.ca>
---
 lib/src_prep.cygpart |   56 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 55 insertions(+), 1 deletion(-)

--- lib/src_prep.cygpart	2024-01-15 05:09:23.000000000 -0700
+++ lib/src_prep.cygpart	2024-04-30 11:41:01.218878400 -0600
@@ -88,6 +88,7 @@ unpack() {
 		# determine correct source decompression command
 		case ${unpack_file_path} in
 			*.asc|*.md5|*.sig|*.sign)  continue ;;
+			*.*sum)			   continue ;;
 			*.tar.lrz)
 				check_prog_req lrzuntar lrzip
 				unpack_cmd="lrzuntar"
@@ -200,6 +201,43 @@ __gpg_verify() {
 	fi
 }
 
+__sum_verify() {
+	local _file=${1#${DISTDIR}/};
+	local _filedesc=${2};
+	local _filetype=${3};
+	local _sum=${3%sum};
+
+	if ! check_prog cksum
+	then
+		# display notice only once
+		if ! defined _cksum_not_found_
+		then
+			inform "cksum must be installed in order to check checksums.";
+			_cksum_not_found_=1
+		fi
+
+		return 0;
+	fi
+
+	# {b2,b2b}{,sum} -> blake2b; ck{,sum} -> crc; {,sum} -> bsd
+	[ -z "${_sum}" ]	&& _sum=${_sum:-bsd}
+	[ "b2" = "${_sum}" ]	&& _sum=blake2b
+	[ "b2b" = "${_sum}" ]	&& _sum=blake2b
+	[ "ck" = "${_sum}" ]	&& _sum=crc
+
+	if defined DISTDIR && [ -d ${DISTDIR} ] && [ -f ${DISTDIR}/${_file} ]
+	then
+		cd ${DISTDIR}
+		inform "${_filedesc} ${_filetype} checksum verification follows:";
+		if [ "${_sum}" = "crc" ] || [ "${_sum}" = "bsd" ] || [ "${_sum}" = "sysv" ]
+		then
+		    cksum -a ${_sum} ${_file%.${_filetype}} || true;
+		else
+		    cksum -a ${_sum} -c ${_file} || true;
+		fi
+	fi
+}
+
 __mkdirs() {
 	cd ${top};
 	mkdir -p ${srcdir} ${origsrcdir} ${B} ${D} ${T} ${configdir} ${logdir} ${distdir} ${patchdir} ${spkgdir};
@@ -298,6 +336,10 @@ __src_prep() {
 	local src_pkg;
 	local tar_patch;
 	local n=1;
+	local sum_exts="sha512 sha384 sha256 sha224 b2 b2b blake2b sm3 sha1 md5 ck crc bsd sysv";
+	# prefer newer stronger keys for faster lookup
+	# blake2b bsd crc md5 sha1 sha224 sha256 sha384 sha512 sm3 sysv
+	# {b2,b2b}{,sum} -> blake2b; ck{,sum} -> crc; {,sum} -> bsd
 
 	cd ${top};
 
@@ -328,6 +370,18 @@ __src_prep() {
 				__gpg_verify ${src_pkg} "SOURCE $((n++))" ${sigext};
 			fi
 		done
+		for sigext in ${sum_exts} ''	# final entry is BSD .sum -> ''
+		do
+			if [ "${src_pkg}" != "${src_pkg%.${sigext}sum}" ]
+			then
+				__sum_verify ${src_pkg} "SOURCE $((n++))" "${sigext}sum";
+				break;
+			elif [ "${src_pkg}" != "${src_pkg%.${sigext}}" ]  # fail if '' unless *.
+			then
+				__sum_verify ${src_pkg} "SOURCE $((n++))" "${sigext}";
+				break;
+			fi
+		done
 	done
 
 	for src_patch in ${_src_orig_patches}
@@ -510,4 +564,4 @@ __src_prep() {
 }
 
 readonly -f __cpio_gz_extract __gem_extract __srpm_extract unpack \
-            __gpg_verify __mkdirs cygpatch __src_prep
+            __gpg_verify __sum_verify __mkdirs cygpatch __src_prep

Re: [PATCH] cygport/lib/src_prep.cygpart: use checksum files with packages

[ASSI]

Brian Inglis via Cygwin-apps writes:
> Some package upstreams offer only checksums, for example .sha512sum, .sha256sum,
> for verification rather than gpg signatures, for example .asc, .sig, .sign, etc;
> use these checksum files when provided in a similar manner to gpg signatures;
> these files are often provided with fixed names which may be renamed on download
> to unique values using cygport URI fragment support like #/$NAME-VERSION.sha...sum;
> use coreutils cksum as it supports all modern and legacy checksums and formats.

https://repo.or.cz/cygport/rpm-style.git/commitdiff/c956092ce8d90230b812fb05ad2b4da13df1e36d


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [PATCH] cygport/lib/src_prep.cygpart: use checksum files with packages

[Brian Inglis]

On 2024-04-30 23:32, ASSI via Cygwin-apps wrote:
> Brian Inglis via Cygwin-apps writes:
>> Some package upstreams offer only checksums, for example .sha512sum, .sha256sum,
>> for verification rather than gpg signatures, for example .asc, .sig, .sign, etc;
>> use these checksum files when provided in a similar manner to gpg signatures;
>> these files are often provided with fixed names which may be renamed on download
>> to unique values using cygport URI fragment support like #/$NAME-VERSION.sha...sum;
>> use coreutils cksum as it supports all modern and legacy checksums and formats.

> https://repo.or.cz/cygport/rpm-style.git/commitdiff/c956092ce8d90230b812fb05ad2b4da13df1e36d

Two similar independent implementations mean it would be a good idea to add the 
feature!

Mine preferred cksum as being the most general approach, while not worrying or 
knowing too much about ancient sums, although your implementation is better, 
that is, works properly on those.

Mine also preferred sha*sum file types, while still allowing prefixes only 
without sum, not enumerating them all in the unpack() case, and respecting the 
cksum crc default.

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

Re: [PATCH] cygport/lib/src_prep.cygpart: use checksum files with packages

[Jon Turney]

On 01/05/2024 17:48, Brian Inglis via Cygwin-apps wrote:
> On 2024-04-30 23:32, ASSI via Cygwin-apps wrote:
>> Brian Inglis via Cygwin-apps writes:
>>> Some package upstreams offer only checksums, for example .sha512sum, 
>>> .sha256sum,
>>> for verification rather than gpg signatures, for example .asc, .sig, 
>>> .sign, etc;
>>> use these checksum files when provided in a similar manner to gpg 
>>> signatures;
>>> these files are often provided with fixed names which may be renamed 
>>> on download
>>> to unique values using cygport URI fragment support like 
>>> #/$NAME-VERSION.sha...sum;
>>> use coreutils cksum as it supports all modern and legacy checksums 
>>> and formats.
> 
>> https://repo.or.cz/cygport/rpm-style.git/commitdiff/c956092ce8d90230b812fb05ad2b4da13df1e36d
> 
> Two similar independent implementations mean it would be a good idea to 
> add the feature!
> 
> Mine preferred cksum as being the most general approach, while not 
> worrying or knowing too much about ancient sums, although your 
> implementation is better, that is, works properly on those.
> 
> Mine also preferred sha*sum file types, while still allowing prefixes 
> only without sum, not enumerating them all in the unpack() case, and 
> respecting the cksum crc default.

I guess this makes sense as a part of the fetch operation, in thsose 
cases where upstream provides signatures or checksums.


But as briefly discussed in [1], independently of that it would also be 
a good idea for cygport to specify it's own checksum file, which is 
incorporated into the source package, and verified at build prep time.

(Since this would protect against such screw ups, help with build 
reproducibility, and defend against supply chain attacks on upstream)

[1] https://cygwin.com/pipermail/cygwin-apps/2024-March/043540.html

Re: [PATCH] cygport/lib/src_prep.cygpart: use checksum files with packages

[Brian Inglis]

On 2024-05-06 09:52, Jon Turney via Cygwin-apps wrote:
> On 01/05/2024 17:48, Brian Inglis via Cygwin-apps wrote:
>> On 2024-04-30 23:32, ASSI via Cygwin-apps wrote:
>>> Brian Inglis via Cygwin-apps writes:
>>>> Some package upstreams offer only checksums, for example .sha512sum, 
>>>> .sha256sum, for verification rather than gpg signatures, for example
>>>> .asc, .sig, .sign, etc;
>>>> use these checksum files when provided in a similar manner to gpg signatures;
>>>> these files are often provided with fixed names which may be renamed
>>>> on download to unique values using cygport URI fragment support like 
>>>> #/$NAME-VERSION.sha...sum;
>>>> use coreutils cksum as it supports all modern and legacy checksums and formats.
>>
>>> https://repo.or.cz/cygport/rpm-style.git/commitdiff/c956092ce8d90230b812fb05ad2b4da13df1e36d
>>
>> Two similar independent implementations mean it would be a good idea to add 
>> the feature!
>>
>> Mine preferred cksum as being the most general approach, while not worrying or 
>> knowing too much about ancient sums, although your implementation is better, 
>> that is, works properly on those.
>>
>> Mine also preferred sha*sum file types, while still allowing prefixes only 
>> without sum, not enumerating them all in the unpack() case, and respecting the 
>> cksum crc default.
> 
> I guess this makes sense as a part of the fetch operation, in those cases where 
> upstream provides signatures or checksums.

I will retry incorporating Achim's approach so hopefully we can both retire our 
local cygport patches.

I would also appreciate other comments or feedback to my reply to Achim's NAK on 
my patch for `gpgv` replacing `gnupg2 --verify`?

> But as briefly discussed in [1], independently of that it would also be a good 
> idea for cygport to specify it's own checksum file, which is incorporated into 
> the source package, and verified at build prep time.

As in Fedora RPM package `sources` BSD-style sum prefix, for example (one line):

https://src.fedoraproject.org/rpms/bash-completion/blob/rawhide/f/sources
SHA512 (bash-completion-2.13.0.tar.xz) = 
7c65fea599a25c2c9d6ef300a9cc2d5fbabd0bcc9e09fe32bb706d3398936f40501171f03280f042465bc0d9aca4b1b53c2c13a99bbdfb6fe916767a267158af

or also in the source package for cygport and each source file included, as in 
Debian dsc, for example:

https://deb.debian.org/debian/pool/main/b/bash-completion/bash-completion_2.13.0-1.dsc
Checksums-Sha1:
  0c045cc06b57bbe8945bc6c4ea8f2b52f1285903 484155 bash-completion_2.13.0.orig.tar.gz
  66f10d161e71c0725a61d5bde1c6b89f9bdb61e3 17840 
bash-completion_2.13.0-1.debian.tar.xz
Checksums-Sha256:
  6c1cc04bb506e7ba6bd7bb3c7c6f6ad2b46e6198e86666ef4c88139597250601 484155 
bash-completion_2.13.0.orig.tar.gz
  d2de6c33d14843da64e4b20e6330c14079b2c73f04c9b4c544d6435930003a67 17840 
bash-completion_2.13.0-1.debian.tar.xz
Files:
  93527b12850a781744e3f335f904bdf1 484155 bash-completion_2.13.0.orig.tar.gz
  a831ae35940daf95016fce1b655955a1 17840 bash-completion_2.13.0-1.debian.tar.xz

> (Since this would protect against such screw ups, help with build 
> reproducibility, and defend against supply chain attacks on upstream)
> 
> [1] https://cygwin.com/pipermail/cygwin-apps/2024-March/043540.html

Coreutils `cksum` does BSD-style checksums, although I would prefer sha256 sums 
for brevity and consistency with setup.ini, and base64 encoding rather than hex 
to shorten the checksum representation, in recent coreutils.

We all have SSH keys, which I also have as a GPG key, could we also use them for 
signing source packages?
Calm could validate ours and checksums, and re-sign with Cygwin key, which setup 
could validate.
Could osslsigncode have any application here?

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry