From: Christian Franke <Christian.Franke@t-online.de>
To: cygwin-apps@cygwin.com
Subject: Re: [PATCH cygport] Add repro-check command
Date: Fri, 1 Mar 2024 20:16:04 +0100 [thread overview]
Message-ID: <1afbda62-e471-ce8b-46b0-3a2cc46dc0b0@t-online.de> (raw)
In-Reply-To: <44c79198-ebee-6711-9251-58fd0078ec10@t-online.de>
[-- Attachment #1: Type: text/plain, Size: 1231 bytes --]
Christian Franke wrote:
> This could be used to check whether a package is possibly
> reproducible. Then it could make sense to add a reasonable
> SOURCE_DATE_EPOCH value to the cygport file.
>
> Example:
>
> $ export SOURCE_DATE_EPOCH=$(date +%s)
>
> $ cygport project.cygport all repro-check
> ...
> *** Info: Build reproducibility test succeeded
>
> $ TZ=UTC cygport project.cygport repro-check
> ...
> *** Info: Build reproducibility test succeeded
>
> $ unset SOURCE_DATE_EPOCH
>
> $ cygport project.cygport repro-check
> ...
> *** ERROR: Build reproducibility test failed
>
An enhanced version of the patch is attached. The build and diff could
now be run also individually and the diff report includes individual
files from the packages.
As a side effect, this enables another use case: Check whether changes
to cygport only change the expected files.
$ cygport project.cygport all repro-check
...
*** Info: Rebuild produced identical packages
$ editor project.cygport
... Change some comments ...
$ cygport project.cygport all repro-diff
...
Differing files found:
! dist/project/project-1.2-3-src.tar.xz
! spkg/project-1.2-3.src/project.cygport
*** ERROR: Rebuild differs from original
--
Regards,
Christian
[-- Attachment #2: 0001-Add-repro-build-repro-diff-und-repro-check-commands.patch --]
[-- Type: text/plain, Size: 5097 bytes --]
From 152a21dfad4c786cff1712e6aa1c33f2db0b6a75 Mon Sep 17 00:00:00 2001
From: Christian Franke <christian.franke@t-online.de>
Date: Fri, 1 Mar 2024 19:46:50 +0100
Subject: [PATCH] Add repro-build, repro-diff und repro-check commands
These commands check for reproducibility of distribution packages.
The repro-build command unpacks the source package from the dist
directory to the temp directory and performs a nested rebuild of
the packages there.
The repro-diff command compares original and rebuild packages.
If different, a report about individual differing files in dist,
inst and spkg directories is printed.
The repro-check command combines both commands.
---
README | 3 +++
bin/cygport.in | 17 +++++++++++++
lib/help.cygpart | 3 +++
lib/pkg_pkg.cygpart | 58 ++++++++++++++++++++++++++++++++++++++++++++-
4 files changed, 80 insertions(+), 1 deletion(-)
diff --git a/README b/README
index fd16df6b..3c9e4d4a 100644
--- a/README
+++ b/README
@@ -163,6 +163,9 @@ Other COMMANDs are meant primarily for maintainers:
diff - write a patch file capturing changes to source in the working directory
stage - as upload, but don't request processing of uploaded packages
announce - compose and send a package announcement
+ repro-build - rebuild from created source package to temp directory
+ repro-diff - check whether packages from original and rebuild differ
+ repro-check - run repro-build and repro-diff
The standard arguments --help or --version may also be passed to cygport.
diff --git a/bin/cygport.in b/bin/cygport.in
index 5fc89eaf..a2c2b5a3 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -29,6 +29,10 @@ set -e;
#
################################################################################
+# Preserve original environment for repro-check command
+declare -r _cygport_orig_env=$(export)
+declare -r _cygport_orig_pwd=$(pwd)
+
# for regexes, sort, etc.
export LC_COLLATE=C
@@ -784,6 +788,19 @@ do
test ${PIPESTATUS[0]} -eq 0
_status=$?;
;;
+ repro-build)
+ __pkg_repro_build
+ _status=$?
+ ;;
+ repro-diff)
+ __pkg_repro_diff
+ _status=$?
+ ;;
+ repro-check)
+ __pkg_repro_build && \
+ __pkg_repro_diff
+ _status=$?
+ ;;
help)
__show_help;
exit 0;
diff --git a/lib/help.cygpart b/lib/help.cygpart
index a7f30f7a..d28fd7bb 100644
--- a/lib/help.cygpart
+++ b/lib/help.cygpart
@@ -56,6 +56,9 @@ __show_help() {
finish delete the working directory
all run prep, compile, install and package
all-test run prep, compile, install and package-test
+ repro-build rebuild from created source package to temp directory
+ repro-diff check whether packages from original and rebuild differ
+ repro-check run repro-build and repro-diff
See the included README file for further documentation.
diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart
index 756a687c..3c531c0e 100644
--- a/lib/pkg_pkg.cygpart
+++ b/lib/pkg_pkg.cygpart
@@ -992,6 +992,62 @@ _EOF
fi
}
+__pkg_repro_build() {
+ local srcpkg=${distdir}/${PN}/${PF}-src.tar.${TAR_COMPRESSION_EXT}
+ local t_spkgdir=${T}/${spkgdir##*/}
+ local t_workdir=${t_spkgdir}/${PF}.${ARCH}
+ local t_cygport="cygport ${cygportfile} finish all"
+ local rc
+
+ __stage "Rebuilding"
+ __step "Unpacking ${srcpkg}"
+ [ -f ${srcpkg} ] || error "Packages not built yet"
+ tar xf ${srcpkg} -C ${T} || error "tar xf ${srcpkg} -C ${T} failed"
+
+ __step "Rebuilding in ${t_spkgdir}"
+ echo "${_cygport_orig_env}" > ${T}/.cygport_orig_env
+ echo
+ __step "=== Start: ${t_cygport} ================================="
+
+ # Start nested cygport with original environment in temp directory
+ rc=0
+ env --chdir=${_cygport_orig_pwd} --ignore-environment /bin/bash -c \
+ "source ${T}/.cygport_orig_env && cd ${t_spkgdir} && ${t_cygport}" \
+ || rc=$?
+
+ __step "=== Done: ${t_cygport} (exit $rc) ========================="
+ echo
+ [ $rc = 0 ] || error "Rebuild failed"
+}
+
+__pkg_repro_diff() {
+ local t_spkgdir=${T}/${spkgdir##*/}
+ local t_workdir=${t_spkgdir}/${PF}.${ARCH}
+ local t_srcpkg=${t_workdir}/dist/${PN}/${PF}-src.tar.${TAR_COMPRESSION_EXT}
+ local d
+
+ __stage "Comparing original and rebuild of"
+ inform "Rebuild dir: ${t_spkgdir}"
+ [ -f ${t_srcpkg} ] || error "Packages not rebuilt yet"
+
+ if ! diff -qr --no-dereference ${distdir} ${t_workdir}/dist >/dev/null
+ then
+ echo "Differing files found:"
+ for d in dist inst spkg
+ do
+ LC_MESSAGES=C \
+ diff -qr --no-dereference ${workdir}/${d} ${t_workdir}/${d} \
+ | sed -n -e "s|^Files ${workdir}/\([^ ][^ ]*\) and .* differ\$|! \\1|p" \
+ -e "s|^Only in ${workdir}/\(${d}/[^:]*\): |- \\1/|p" \
+ -e "s|^Only in ${t_workdir}/\(${d}/[^:]*\): |+ \\1/|p"
+ done
+ echo
+ error "Rebuild differs from original"
+ fi
+ echo
+ inform "Rebuild produced identical packages"
+}
+
# protect functions
readonly -f __pkg_binpkg __pkg_diff __gpg_sign __pkg_srcpkg __pkg_dist \
- __squeeze_whitespace __tar
+ __pkg_repro_build __pkg_repro_diff __squeeze_whitespace __tar
--
2.43.0
next prev parent reply other threads:[~2024-03-01 19:16 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-18 15:54 Christian Franke
2024-03-01 19:16 ` Christian Franke [this message]
2024-03-10 17:20 ` Jon Turney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1afbda62-e471-ce8b-46b0-3a2cc46dc0b0@t-online.de \
--to=christian.franke@t-online.de \
--cc=cygwin-apps@cygwin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).