From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 4D404388450F for ; Wed, 19 Jun 2024 09:38:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4D404388450F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4D404388450F Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1718789924; cv=none; b=YT7A8eKukY36VBwGHzsGHGtGum96rV5v81zRX97p08kKiv3DEbrri5lPhWHEQK0YOwlILjuWmgCroYc6srhv4Xb1gexLS3vmqCx5/f1WpbK+YeB8oC6zCmR9nmeqPHCcrnM9Z+XxcoKVKCWCI5rfrXyJNgwD2KN4q12xZJCxSYM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1718789924; c=relaxed/simple; bh=RU5VwfZ29YKvgulBdT7kXJi0Yn6kSPdWmqKzeYHFFf4=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=HO4MXwXP05CRCnE3OIA0kel71UqfEr2mkOb8gdNxKapinwZhLTSLGUDkCMrn+OnuT6OoaQcan5py26kTnqFQQ68uMpigyjXDilHfLQqL5dMJTssEVBZzmOYDcK+pSXd/1cVAmPeVsZLokqBfwpfeawZbbQhXqjoDJoX9EfTWNR0= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1718789921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=R4fKXkpE9Kd1sBfFRiwkh04tHA+MwyQs+h6T5Ujoas8=; b=Z7Kz3OPWlMa00t57tOiGWHflEOoIb5ACW7/obXxXDUo8rvqzdl1rD5KCHPwSFW3Ld3OwuB 0MiUf+gJ61qF+3g5KZkwAVWkg/YTXDmdKUemU9312CZRC5droXOXB+kfrohn7LrmTfjYp7 8ATFDYlPS8XAaUDImSuWQKmBSM8sslM= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-136-TeTFiEz-P7WDjbis6IDwfw-1; Wed, 19 Jun 2024 05:38:40 -0400 X-MC-Unique: TeTFiEz-P7WDjbis6IDwfw-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-363542774e1so461125f8f.2 for ; Wed, 19 Jun 2024 02:38:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718789919; x=1719394719; h=mime-version:message-id:date:references:in-reply-to:subject:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=R4fKXkpE9Kd1sBfFRiwkh04tHA+MwyQs+h6T5Ujoas8=; b=g2qLZ1Z6NS6URTwUEyIphxwk/LgzPFy9ubrm1Ol1F7WAU1Bz3rMckLIgHxbQg4zAII FLiGkMspTomqNNJtBxouusQ2dJtqBzQP7fGsDirRTnHSP+foFfLNVrULUmoqk0X0nC7I OAY86koTokYa9g+Nx1uu78zOKKgf7gl8osil3BzULAqQSVfdOejwmFcas2jhgf9Ala93 kwjT1rlnlGa9nBOxRvp0CazlFxTQYNvDJyu/IxtZWZlUm1ODgVqfQ7pk8MOwNu30UkZK S2i+tzM8Uh4AC/jiYHmNjsUdgJIVwdyJKWXuj+Itbpn+k0BePlQRw/nBx66g+/STi5Q6 /LOw== X-Forwarded-Encrypted: i=1; AJvYcCX06BvgfbwNwOdA+d9QFZEZZrxdKKu9X9UIt20G2j3C73n/j04am3w7x7p/8XDFF8XqC0NEHjONZwwDuRnWCKNgdz1an0CFj0w8rg== X-Gm-Message-State: AOJu0Ywqn89SnYw+i27XE5/zyuKyolad47amw/pFi3RWCvnegbhJPT9F m+I07DW7EgUZGtul3eV/qAFUDqg4S3EjaiqdVCHvYG33Nr929ZJMghq/q2V80i8QACrtbcgHLNs urTD4SFdnDrXHQUd+HdqM6UPyn1/FPIlXQC9xg0z7JBsETxuvYeDEbEd6d4o= X-Received: by 2002:a5d:4907:0:b0:362:a1e8:15b with SMTP id ffacd0b85a97d-363177a2272mr1836075f8f.28.1718789918823; Wed, 19 Jun 2024 02:38:38 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGfFXZ/MdId/jJIWIsxedEjAAJDVDIDkiwV553IUmKjwHJKYPMQl46VXqnWuSrsBh0deC7qBQ== X-Received: by 2002:a5d:4907:0:b0:362:a1e8:15b with SMTP id ffacd0b85a97d-363177a2272mr1836056f8f.28.1718789918169; Wed, 19 Jun 2024 02:38:38 -0700 (PDT) Received: from localhost ([31.111.84.186]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-360750ad216sm16628806f8f.57.2024.06.19.02.38.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jun 2024 02:38:37 -0700 (PDT) From: Andrew Burgess To: Mark Wielaard , gdb-testers@sourceware.org Subject: Re: =?utf-8?Q?=E2=98=A0?= Buildbot (Sourceware): binutils-gdb - failed test (failure) test (failure) (master) In-Reply-To: <8348bc95a39c6ddee0458b0973fa3fbc4e0c22a5.camel@klomp.org> References: <20240614114809.203C83882100@sourceware.org> <878qz78ccz.fsf@redhat.com> <8348bc95a39c6ddee0458b0973fa3fbc4e0c22a5.camel@klomp.org> Date: Wed, 19 Jun 2024 10:38:37 +0100 Message-ID: <877cel5l2a.fsf@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Status: No, score=-5.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Mark Wielaard writes: > Hi Andrew, > > On Fri, 2024-06-14 at 22:04 +0100, Andrew Burgess wrote: >> Mark Wielaard writes: >> > It looks like you latest patch caused breakage on i386 with >> > RUNTESTFLAGS=--target_board=native-gdbserver and --target_board=native- >> > extended-gdbserver but a "native" make check-gdb is fine as are all >> > other arches. >> >> I saw the buildbot report for these failures, but thanks for confirming >> that these were triggered by my commit, I've seen a few false positives >> from buildbot occasionally. >> >> The bad news is that it's not that easy to reproduce. I have a i386 VM >> running Debian here on which I've been testing the patch in question, >> and at least for some of the tests in question, everything passes fine >> for me here. >> >> Can you give more details about the setup in which the test is being >> run? > > It is a debian stable i386 install (in a VM on an x86_64 machine). > > Debian GNU/Linux 12 (bookworm) Linux 6.1.0-21-686-pae #1 SMP > PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) GNU C Library (Debian > GLIBC 2.36-9+deb12u7) stable release version 2.36. g++ 12.2.0 GNU > objdump (GNU Binutils for Debian) 2.40 I'm still unable to reproduce this failure. I also am using Debian i386 (12.5) image running within a VM on an x86-64 machine. I've done an update which pulled all the version numbers into line with what you listed above. > > Build with: > https://sourceware.org/cgit/builder/tree/builder/master.cfg#n3362 > $ ../configure --enable-targets=all --disable-sim > $ make -j4 all-gdb all-gdbserver I've tried building with these exact flags too, but I've also done a build with address and undefined behaviour sanitisers in place to see if they would throw anything up. > > A "native" minimal make check passes: > https://sourceware.org/cgit/builder/tree/builder/master.cfg#n3208 > $ make check-gdb 'TESTS= gdb.base/break-always.exp gdb.base/break- > caller-line.exp gdb.base/break-entry.exp gdb.base/break.exp > gdb.base/break-fun-addr.exp gdb.base/break-idempotent.exp > gdb.base/break-include.exp gdb.base/break-inline.exp gdb.base/break- > main-file-remove-fail.exp gdb.base/break-on-linker-gcd-function.exp > gdb.base/breakpoint-in-ro-region.exp gdb.base/breakpoint-shadow.exp > gdb.base/break-probes.exp gdb.gdb/unittest.exp gdb.server/unittest.exp > ' > > === gdb Summary === > > # of expected passes 379 > # of unsupported tests 1 > > But the same with native-gdbserver produces failures: > > $ make check-gdb 'TESTS= gdb.base/break-always.exp gdb.base/break- > caller-line.exp gdb.base/break-entry.exp gdb.base/break.exp > gdb.base/break-fun-addr.exp gdb.base/break-idempotent.exp > gdb.base/break-include.exp gdb.base/break-inline.exp gdb.base/break- > main-file-remove-fail.exp gdb.base/break-on-linker-gcd-function.exp > gdb.base/breakpoint-in-ro-region.exp gdb.base/breakpoint-shadow.exp > gdb.base/break-probes.exp gdb.gdb/unittest.exp gdb.server/unittest.exp > ' RUNTESTFLAGS=--target_board=native-gdbserver > > === gdb Summary === > > # of expected passes 97 > # of unexpected failures 65 > # of unsupported tests 3 > > > This takes a long time since a lot of tests time out. > > The whole gdb.log is attached. > > I think the issue is: > > target remote localhost:2355 > Remote debugging using localhost:2355 > warning: Architecture rejected target-supplied description > Remote connection closed That's super interesting. When buildbot originally reported this issue the warning was not present, instead the connection was just being dropped, like this: Listening on port 2346 target remote localhost:2346 Remote debugging using localhost:2346 Remote connection closed (gdb) continue The program is not being run. (gdb) FAIL: gdb.base/break-main-file-remove-fail.exp: cmdline: runto: run to start This is why I thought there might be a core file to be found; I figured that gdbserver was crashing for some reason, but it appears that the behaviour has changed. > > At least that warning seems new. Agreed, that would certainly cause the problem. > >> Is it possible to get access to the machine in question? I'm >> assuming that gdbserver is crashing and possibly dumping core, so just >> getting the core file might help. Or maybe it's a VM? In which case >> maybe the image is available somewhere that I might reproduce the >> environment locally? > > I don't see any core files. You can certainly get access to the VM, but > it takes a little setup since it is located after a jump host. The > install is a plain Debian i386 image, completely up to date. I'm really at a loss here for how to move forward without debugging this on the exact VM where the issue is showing up. I'm happy to try anything you can suggest, but right now this bug seems to only occur on the buildbot VM. Thanks, Andrew