[glibc-buildbot] Add 'glibc-s390x-linux': 'marist-fedora-s390x' to builder_map.

[glibc-buildbot] Add 'glibc-s390x-linux': 'marist-fedora-s390x' to builder_map.

[Mark Wielaard]

Hi,

I setup a buildbot on the marist s390x fedora VM that I like to
add to master.cfg. I think the patch (also on the mjw/s390x branch)
does that. Could it be applied and installed on the buildbot master?

Thanks,

Mark

---
 master.cfg | 1 +
 1 file changed, 1 insertion(+)

diff --git a/master.cfg b/master.cfg
index 78ecf51..88e997a 100644
--- a/master.cfg
+++ b/master.cfg
@@ -22,6 +22,7 @@ builder_map = {
   'glibc-x86_64-linux': 'glibc-ubuntu-trusty-slave2',
   'glibc-i686-linux': 'glibc-ubuntu-trusty-slave2',
   'glibc-power8-linux': 'fedora22-ppc64-1',
+  'glibc-s390x-linux': 'marist-fedora-s390x',
 }
 
 builders = sorted(set(builder_map.iterkeys()))
-- 
2.7.4

Re: [glibc-buildbot] Add 'glibc-s390x-linux': 'marist-fedora-s390x' to builder_map.

[Carlos O'Donell]

On 10/03/2016 06:52 PM, Mark Wielaard wrote:
> Hi,
> 
> I setup a buildbot on the marist s390x fedora VM that I like to
> add to master.cfg. I think the patch (also on the mjw/s390x branch)
> does that. Could it be applied and installed on the buildbot master?

Awesome! Roland would have to do that himself since the master is on
Google infrastructure (for the short term until we move to something
more accessible that we can all help out with).

-- 
Cheers,
Carlos.

Re: [glibc-buildbot] Add 'glibc-s390x-linux': 'marist-fedora-s390x' to builder_map.

[Roland McGrath]

Merged and restarted.  It looks like the slave is connected.
So the next commit to libc should trigger a build there.

Re: [glibc-buildbot] Add 'glibc-s390x-linux': 'marist-fedora-s390x' to builder_map.

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

On Mon, 2016-10-03 at 16:31 -0700, Roland McGrath wrote:
> Merged and restarted.  It looks like the slave is connected.
> So the next commit to libc should trigger a build there.

It did trigger, but then immediately died:
http://130.211.48.148:8080/builders/glibc-s390x-linux/builds/0/steps/annotate/logs/stdio
ERROR - unset/invalid builder name
builder name: glibc-s390x-linux

I believe that is because I forgot to add the new builder to the
BOT_ASSIGNMENT. The attached patch (also on mjw/s390x branch) does that.

Cheers,

Mark

[-- Attachment #2: 0001-scripts-slave-buildbot_selector.py-BOT_ASSIGNMENT-Ad.patch --]
[-- Type: text/x-patch, Size: 798 bytes --]

From e9fadb8090adbab2976f0ef3732155665f176f2e Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mjw@redhat.com>
Date: Tue, 4 Oct 2016 16:11:44 +0200
Subject: [PATCH] scripts/slave/buildbot_selector.py (BOT_ASSIGNMENT): Add
 glibc-s390x-linux.

---
 scripts/slave/buildbot_selector.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/slave/buildbot_selector.py b/scripts/slave/buildbot_selector.py
index 1bb72d0..12bfbb3 100755
--- a/scripts/slave/buildbot_selector.py
+++ b/scripts/slave/buildbot_selector.py
@@ -20,7 +20,8 @@ BOT_ASSIGNMENT = {
     'glibc-power8-linux': bash('glibc-native.sh', [
         '--with-cpu=power8',
         '--enable-lock-elision',
-    ])
+    ]),
+    'glibc-s390x-linux': bash('glibc-native.sh'),
 }
 
 BOT_ADDITIONAL_ENV = {
-- 
1.8.3.1

Re: [glibc-buildbot] Add 'glibc-s390x-linux': 'marist-fedora-s390x' to builder_map.

[Roland McGrath]

Merged.  

Re: [glibc-buildbot] Add 'glibc-s390x-linux': 'marist-fedora-s390x' to builder_map.

[Mark Wielaard]

On Tue, 2016-10-04 at 12:01 -0700, Roland McGrath wrote:
> Merged.  

Thanks. And now it triggers builds! It is already on its second one.
But some target steps seem to fail. I am wondering how to fix that so
the build gets green.

First there is the 'annotate' step which seems to fail because make
check isn't zero FAIL:
http://130.211.48.148:8080/builders/glibc-s390x-linux/builds/3/steps/annotate/logs/stdio

Summary of test results:
      4 FAIL
   2478 PASS
     13 UNSUPPORTED
     44 XFAIL
Makefile:353: recipe for target 'tests' failed
make[1]: Leaving directory '/home/mjw/glibc/build/glibc-s390x-linux/build/glibc'
make[1]: *** [tests] Error 1

Where the FAILS are:

FAIL: malloc/tst-malloc-thread-fail
FAIL: malloc/tst-mallocfork2
FAIL: posix/tst-getaddrinfo4
FAIL: timezone/tst-tzset

Can/should I ignore those (how?) or are these suspicious?

Then there is the 'check' target which fails as follows:
http://130.211.48.148:8080/builders/glibc-s390x-linux/builds/3/steps/check/logs/stdio

test -s /home/mjw/glibc/build/glibc-s390x-linux/build/build/rt/librt.so.phdrT
mv -f /home/mjw/glibc/build/glibc-s390x-linux/build/build/rt/librt.so.phdrT /home/mjw/glibc/build/glibc-s390x-linux/build/build/rt/librt.so.phdr
make[1]: *** [tests] Error 1
make[1]: Target 'check' not remade because of errors.
make[1]: Leaving directory '/home/mjw/glibc/build/glibc-s390x-linux/build/glibc'
Makefile:9: recipe for target 'check' failed
make: Leaving directory '/home/mjw/glibc/build/glibc-s390x-linux/build/build'
@@@STEP_FAILURE@@@
make: *** [check] Error 2

I cannot find anything that explains that failure in the stdio log.

Finally the 'check (clobber)' target fails apparently for the same
reason as the 'annotate' one.

Any advise on how to resolve these issues and turn the build green?

Thanks,

Mark

Re: [glibc-buildbot] Add 'glibc-s390x-linux': 'marist-fedora-s390x' to builder_map.

[Joseph Myers]

On Wed, 5 Oct 2016, Mark Wielaard wrote:

> FAIL: malloc/tst-malloc-thread-fail
> FAIL: malloc/tst-mallocfork2
> FAIL: posix/tst-getaddrinfo4
> FAIL: timezone/tst-tzset
> 
> Can/should I ignore those (how?) or are these suspicious?

Those should be investigated just like any other test failures.  You can 
look at <https://sourceware.org/glibc/wiki/Release/2.24> to see if they 
were previously reported in release testing, on this or other 
architectures.  (XFAILing in Makefiles is possible but best done only when 
there is sufficient analysis, likely a bug filed, etc. - and we don't have 
any architecture-specific XFAILs in Makefiles at present.)

> Then there is the 'check' target which fails as follows:
> http://130.211.48.148:8080/builders/glibc-s390x-linux/builds/3/steps/check/logs/stdio
> 
> test -s /home/mjw/glibc/build/glibc-s390x-linux/build/build/rt/librt.so.phdrT
> mv -f /home/mjw/glibc/build/glibc-s390x-linux/build/build/rt/librt.so.phdrT /home/mjw/glibc/build/glibc-s390x-linux/build/build/rt/librt.so.phdr
> make[1]: *** [tests] Error 1
> make[1]: Target 'check' not remade because of errors.
> make[1]: Leaving directory '/home/mjw/glibc/build/glibc-s390x-linux/build/glibc'
> Makefile:9: recipe for target 'check' failed
> make: Leaving directory '/home/mjw/glibc/build/glibc-s390x-linux/build/build'
> @@@STEP_FAILURE@@@
> make: *** [check] Error 2
> 
> I cannot find anything that explains that failure in the stdio log.

<https://sourceware.org/glibc/wiki/Buildbot> links to previous discussion 
of this issue.  As I suggested in 
<https://sourceware.org/ml/libc-alpha/2015-09/msg00215.html>, making your 
buildbot instance run make with -d might help get more information for 
debugging this issue.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [glibc-buildbot] Add 'glibc-s390x-linux': 'marist-fedora-s390x' to builder_map.

[Florian Weimer]

* Mark Wielaard:

> Where the FAILS are:
>
> FAIL: malloc/tst-malloc-thread-fail
> FAIL: malloc/tst-mallocfork2
> FAIL: timezone/tst-tzset

Do we dump the .out files somewhere, like we do for Fedora?

Anyway, the three above are likely due to resource constraints, either
RLIMIT_DATA/RLIMIT_AS, or RLIMIT_NPROC.  The latter could come from a
systemd task limit which is incorrectly enforced due to a kernel bug:
The kernel says the thread is gone while it still counts against the
limit, and those tests will spawn new threads prematurely, eventually
exceeding RLIMIT_NPROC.

timezone/tst-tzset points towards an RLIMIT_DATA/RLIMIT_AS limit,
though.

> FAIL: posix/tst-getaddrinfo4

This could be due to lack of network access.  We could add a configure
option like --disable-network-tests for that.

> Can/should I ignore those (how?) or are these suspicious?

The first three are somewhat suspicious, yes.