Hi, there was an off-list discussion so posting my home-made scripts for testing: - Daily mails of regressions (on selected Fedora OSes). - Mock based (=chroot); no kvm/xen/virtualization needed. - Primarily gdb (binutils and glibc are also supported). - Primarily any Fedora OS. CentOS/RHEL-{4,5} are supported only for draft testing as their kernels differ for ptrace. RHEL-6 should be OK. - Only x86_64 and i686 (by setarch) are supported. - Upstream CVS is used (Fedora/RHEL repos and .src.rpms are offtopic here). Archer GIT is used for the archer-* trees. Some scripts can be missing, I have only a (private) backup of the box, there is no packaged release. It was scripted originally for my personal needs. Run the test: hammock -c archer-jankratochvil-vla -i vla0-f13 -d f13 hammock -c gdbcvs -i clean0-f13 -d f13 hammock -c gdbcvs -i sigill-cleanup0-f13 -d f13 --file ~/t/sigill.patch ~/t/sigill-cleanup.patch hammock -c rhelgdb -d epel5 -i rel24-el5 hammock -c fedoragdb -d f13 -i fedora-f13 --file gdb.spec gdb-bz631158-cxx-this-lookup.patch hammock -c binutilscvs -i binutils-f13 -d f13 --target `cat ~/redhat/my/targets` hammock -c fedoragdb --cd rawhide -i rel5-f13 -d f13 hammock -c gdbcvs -i servpatched-f14 -d f14 --gdbserver --file ~/t/server.patch experimental: hammock --gdbserver (it errorneously creates several GBs of logs) experimental: hammock --valgrind (first run now, it runs for an hour, still not finished) known bug: --path for builds by alternative GCC ($prefix/bin) IIRC stopped working. And compare results of two runs: diffgdbdir 20100912Build-gdbcvs-f14/ 20100912nokill-f14/ diffgdbdir log 20100912Build-gdbcvs-f14/ 20100912nokill-f14/ diffgdbdir -N 20100817clean-el5/ 20100818rhel27c-el5/ ^^ see even new PASSes (not for nightly regression testing) (Build-* trees are the clean nightly runs.) Daily build size is about 20GB. Results archives since Feb 2010 have 2.4GB. http://dyn.jankratochvil.net/nethome/bin/hammock?view=co = run the testsuite, wait for its results http://dyn.jankratochvil.net/nethome/bin/diffgdb?view=co = intelligent diff of two gdb.sum files http://dyn.jankratochvil.net/nethome/bin/diffgdbdir?view=co = diffgdb for all the build-arches/os-arches. http://pkgs.fedoraproject.org/gitweb/?p=gdb.git;a=blob_plain;f=gdb-orphanripper.c;hb=f14/master = stale processes killer daily stuff: http://dyn.jankratochvil.net/nethome/bin/build?view=co = daily builder http://dyn.jankratochvil.net/nethome/bin/diffgdbday?view=co = daily diffgdbdir regressions results http://dyn.jankratochvil.net/nethome/bin/diffgdbdaymail?view=co = mailer of diffgdbday mock interface: http://dyn.jankratochvil.net/nethome/bin/mockrun?view=co = like mock --shell, requires SUID root http://dyn.jankratochvil.net/nethome/bin/mocksetup?view=co = like mock --chroot setup http://dyn.jankratochvil.net/nethome/bin/mockupdate?view=co = like mock --update with rpm/yum workarounds build to run from shell: http://dyn.jankratochvil.net/nethome/bin/errs12?view=co = like ./configure; make http://dyn.jankratochvil.net/nethome/bin/errs2?view=co = like make Attached some stubs in /etc/cron.daily/. The whole $HOME tree is at: cvs -d :pserver:pserver:@dyn.jankratochvil.net/cvs co nethome Two days output, directory names used for diffgdbdir: drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 05:09 20100914Build-archer-jankratochvil-fedora14-merge-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 05:09 20100914Build-archer-jankratochvil-ifunc-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 05:01 20100914Build-archer-jankratochvil-vla-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 04:54 20100914Build-archer-jankratochvil-watchpoint2-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 05:25 20100914Build-archer-pmuldoon-next-over-throw2-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 05:17 20100914Build-archer-tromey-python-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 04:46 20100914Build-binutilscvs-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 04:31 20100914Build-fedoragdb-f12/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 03:45 20100914Build-fedoragdb-f13/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 04:20 20100914Build-fedoragdb-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 05:42 20100914Build-gdbcvs-f12/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 05:33 20100914Build-gdbcvs-f13/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 14 04:40 20100914Build-gdbcvs-f14/ drwxr-xr-x 5 jkratoch jkratoch 4096 Sep 14 22:17 20100914ada-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:38 20100915Build-archer-jankratochvil-fedora14-merge-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:38 20100915Build-archer-jankratochvil-ifunc-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:30 20100915Build-archer-jankratochvil-vla-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:22 20100915Build-archer-jankratochvil-watchpoint2-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:54 20100915Build-archer-pmuldoon-next-over-throw2-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:46 20100915Build-archer-tromey-python-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:14 20100915Build-binutilscvs-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 04:59 20100915Build-fedoragdb-f12/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 04:14 20100915Build-fedoragdb-f13/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 04:48 20100915Build-fedoragdb-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 06:11 20100915Build-gdbcvs-f12/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 06:02 20100915Build-gdbcvs-f13/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:08 20100915Build-gdbcvs-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 19:43 20100915valgrind0-f14/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 19:44 20100915valgrind1-f14/ ls -l 20100915Build-gdbcvs-f14/ total 24 -rw-r--r-- 1 jkratoch jkratoch 7 Sep 15 04:59 component drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:07 fedora-14-i386/ drwxr-xr-x 4 jkratoch jkratoch 4096 Sep 15 05:08 fedora-14-x86_64/ -rw-r--r-- 1 jkratoch jkratoch 25 Sep 15 04:59 kernel -rw-r--r-- 1 jkratoch jkratoch 74 Sep 15 04:59 log -rw-r--r-- 1 jkratoch jkratoch 6 Sep 15 05:08 time ls -l 20100915Build-gdbcvs-f14/fedora-14-x86_64/ total 1076 drwxr-xr-x 3 jkratoch jkratoch 4096 Sep 15 05:00 build/ -rw-r--r-- 1 jkratoch jkratoch 1085481 Sep 15 05:08 log drwxr-xr-x 2 jkratoch jkratoch 4096 Sep 15 05:08 out/ ls -l 20100915Build-gdbcvs-f14/fedora-14-x86_64/out/ total 16656 -rw-r--r-- 2 jkratoch jkratoch 6443500 Sep 15 05:08 gdb-m32.log -rw-r--r-- 2 jkratoch jkratoch 1173386 Sep 15 05:08 gdb-m32.sum -rw-r--r-- 2 jkratoch jkratoch 8080314 Sep 15 05:08 gdb-m64.log -rw-r--r-- 2 jkratoch jkratoch 1287878 Sep 15 05:08 gdb-m64.sum -rw-r--r-- 1 jkratoch jkratoch 26984 Sep 15 05:00 rpm-qa Regards, Jan