Re: [RFC] External testsuite using binaries

Re: [RFC] External testsuite using binaries

[Mark Wielaard]

Hi Tom,

Sorry for the late response. I saw others (on overseers) already gave
some feedback. I don't think my feedback is really very different, but
here are my 2 cents.

On Thu, 2019-03-07 at 14:51 +0100, Tom de Vries wrote:
> since large files tend to be a problem in git, I've created a
> testsuite
> containing binaries, that can be downloaded from ftp and plugged into
> the dwz sources.
> 
> It contains the following executables:
> ...
> 272M    cc1
> 223M    cc1.dwz-processed
> 16K     dw2-restrict
> 16K     dw2-skip-prologue
> 9,1M    etcd
> 8,2M    etcdctl
> 16K     multidictionary
> 8,0K    py-section-script.debug
> ...
> 
> Is it ok to move some of these (based on a size limit) to dwz.git, or
> do we want to keep these separate?

- I think the binaries shouldn't be in the main git repo.

- For the smaller ones adding .S files might be acceptable in the
  main git repo.

- If you offer them on some ftp site you want some mechanism (hash?)
  to make sure people don't need to redownload these binaries over
  and over again (especially important for the autobuilders).

- So a (separate) git repo seems better. I don't think you really
  need an extension for large binary files unless you intend to
  replace or alter the binaries checked in. If it is just adding
  new binaries I believe git handles that fine as is.

- It IMHO essential that they come with corresponding source
  and build instructions. So they can be reproduced.

- If at all possible I would try to only use small ones of
  a couple of K. I am not sure what multi-megabyte binaries add.

- If multi-megabyte binaries are really essential how would we
  handle the extra memory and disk space needed on the test
  machines? Can skip processing if there isn't enough
  disk/memory available?

- Have you considered extending the dwz.tests/dwz.sh self test instead?
  Build the different .o files with -g(123) -g[no-]strict-dwarf,
  -gdwarf-(2345), -f[no-]debug-types-section, etc. And see if dwz
  handles itself when combining those object files? Or is this for
  catching different issues?

Cheers,

Mark

[RFC] External testsuite using binaries

[Tom de Vries]

[-- Attachment #1: Type: text/plain, Size: 613 bytes --]

Hi,

since large files tend to be a problem in git, I've created a testsuite
containing binaries, that can be downloaded from ftp and plugged into
the dwz sources.

It contains the following executables:
...
272M    cc1
223M    cc1.dwz-processed
16K     dw2-restrict
16K     dw2-skip-prologue
9,1M    etcd
8,2M    etcdctl
16K     multidictionary
8,0K    py-section-script.debug
...

Is it ok to move some of these (based on a size limit) to dwz.git, or do
we want to keep these separate?

Or, is there some large file support I should be using to integrate this
into dwz.git?

Any other comments?

Thanks,
- Tom


[-- Attachment #2: 0001-Add-external-tests-scripts.patch --]
[-- Type: text/x-patch, Size: 2529 bytes --]

Add external-tests scripts

We keep test-cases using binaries rather than sources in a seperate test
directory dwz-external.tests.  The directory currently has a size of ~512MB.

This directory can be installed using the scripts in this patch:
...
$ ./scripts/fetch-external-tests.sh
$ ./scripts/untar-external-tests.sh
$ ls testsuite
dwz-external.tests  dwz.tests  lib  scripts
...

Result with current (md5sum 09cd309b8e720242cfc79d8de75c8563)
dwz-external-tests.tgz:
...
$ cat dwz.sum
Test run by vries on Wed Mar  6 15:56:13 2019
Native configuration is x86_64-pc-linux-gnu

                === dwz tests ===

Schedule of variations:
    unix

Running target unix
Running testsuite/dwz-external.tests/dwz-external-tests.exp ...
FAIL: testsuite/dwz-external.tests/pr24169.sh
FAIL: testsuite/dwz-external.tests/pr24171.sh
FAIL: testsuite/dwz-external.tests/pr24170.sh
FAIL: testsuite/dwz-external.tests/pr24172.sh
FAIL: testsuite/dwz-external.tests/pr24173.sh
FAIL: testsuite/dwz-external.tests/pr24195.sh
FAIL: testsuite/dwz-external.tests/pr24204.sh
Running testsuite/dwz.tests/dwz-tests.exp ...
PASS: testsuite/dwz.tests/low-mem.sh
PASS: testsuite/dwz.tests/multifile.sh
PASS: testsuite/dwz.tests/regular.sh
PASS: testsuite/dwz.tests/too-many-dies.sh
PASS: testsuite/dwz.tests/hardlink.sh
PASS: testsuite/dwz.tests/multifile-name.sh
PASS: testsuite/dwz.tests/multifile-relative.sh
PASS: testsuite/dwz.tests/regular-o.sh

                === dwz Summary ===

nr of expected passes            8
nr of unexpected failures        7
...

---
 scripts/fetch-external-tests.sh | 5 +++++
 scripts/tar-external-tests.sh   | 6 ++++++
 scripts/untar-external-tests.sh | 5 +++++
 3 files changed, 16 insertions(+)

diff --git a/scripts/fetch-external-tests.sh b/scripts/fetch-external-tests.sh
new file mode 100755
index 0000000..6a521ef
--- /dev/null
+++ b/scripts/fetch-external-tests.sh
@@ -0,0 +1,5 @@
+#!/bin/sh
+
+wget \
+    http://ftp.suse.com/pub/people/tdevries/dwz/dwz-external-tests.tgz
+
diff --git a/scripts/tar-external-tests.sh b/scripts/tar-external-tests.sh
new file mode 100755
index 0000000..9fe87fa
--- /dev/null
+++ b/scripts/tar-external-tests.sh
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+tar \
+    cvfz \
+    ../dwz-external-tests.tgz \
+    testsuite/dwz-external.tests
diff --git a/scripts/untar-external-tests.sh b/scripts/untar-external-tests.sh
new file mode 100755
index 0000000..7193725
--- /dev/null
+++ b/scripts/untar-external-tests.sh
@@ -0,0 +1,5 @@
+#!/bin/sh
+
+tar \
+    xvfz \
+    dwz-external-tests.tgz