From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oa1-x2f.google.com (mail-oa1-x2f.google.com [IPv6:2001:4860:4864:20::2f]) by sourceware.org (Postfix) with ESMTPS id 6BB893858415 for ; Mon, 11 Jul 2022 12:01:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 6BB893858415 Received: by mail-oa1-x2f.google.com with SMTP id 586e51a60fabf-1013ecaf7e0so6370348fac.13 for ; Mon, 11 Jul 2022 05:01:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=mpGACS1NC2NzBFcik1IMJFxVwCSHyZL/maSJ5nZWHTs=; b=R/vDnxZLqbcAsRQTfQyKdDeX4SmiHDLUqchw02DIHw5ksBQF+x3phr7k+srr/CsD+Q DdJK0rUwJ5ouSnZUAmAFoE/Ua/rEhRXUwEZ+DKHDmbxkbXimNiNI8/U2I//XxTR0hsZg vQMtTiwZInNtVALFAIgYDE27Wm/xvyKYPTqvlyXIaqDUpgxWIF8LphnGVE2WzZNHAPbL opeFLVynDC9r3hCJSpgl3IFnsVsVASCpwBdUyhpkFXBaidz8V6KcKXHtt0nZ8Blk8TmD jN1Kwkcs/WxuUEP/Qougu3q736FlevJpHl+nbauyzujPfra++fVDJkZlHSCM4xMp3PJ6 5vtA== X-Gm-Message-State: AJIora//Q6C7QPsofxaaA4vovyrMsIw5DVe46x+A7pkTipfTQC8ApEEj DQnlBVOIXrJzdTiN0Cmpdg6zPw== X-Google-Smtp-Source: AGRyM1sRR5XHO5LdC4JNQcqylkiQ9SWTAP8Lh/hS4iCPrVkY0r2CK8+0C4smccWvPQGakU339TXzXg== X-Received: by 2002:a05:6870:c1cb:b0:10b:f2d0:e0bf with SMTP id i11-20020a056870c1cb00b0010bf2d0e0bfmr6757916oad.143.1657540866343; Mon, 11 Jul 2022 05:01:06 -0700 (PDT) Received: from smtpclient.apple ([2804:431:c7cb:5bec:a591:bf2:d8b8:7a84]) by smtp.gmail.com with ESMTPSA id 7-20020a9d0687000000b0061c5233ae54sm467139otx.60.2022.07.11.05.01.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 11 Jul 2022 05:01:05 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\)) Subject: Re: glibc test procedure needed From: Adhemerval Zanella In-Reply-To: <004401d89286$20979e70$61c6db50$@gmail.com> Date: Mon, 11 Jul 2022 09:01:03 -0300 Cc: Libc-help Content-Transfer-Encoding: quoted-printable Message-Id: References: <000e01d89035$cf28e5a0$6d7ab0e0$@gmail.com> <004401d89286$20979e70$61c6db50$@gmail.com> To: sundeep.kokkonda@gmail.com X-Mailer: Apple Mail (2.3696.100.31) X-Spam-Status: No, score=-5.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-help@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-help mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jul 2022 12:01:13 -0000 > On 8 Jul 2022, at 01:49, sundeep.kokkonda@gmail.com wrote: >=20 >=20 >=20 >> -----Original Message----- >> From: Adhemerval Zanella >> Sent: 05 July 2022 18:13 >> To: sundeep.kokkonda@gmail.com >> Cc: Libc-help >> Subject: Re: glibc test procedure needed >>=20 >>=20 >>=20 >>> On 5 Jul 2022, at 03:09, sundeep.kokkonda--- via Libc-alpha > alpha@sourceware.org> wrote: >>>=20 >>> Hello, >>>=20 >>>=20 >>>=20 >>> I need some info on glibc testing. >>>=20 >>>=20 >>>=20 >>> In the Yocto project we're updating glibc frequently and so we need = to >>> ensure the updated glibc is not introduced any issues, by performing >>> the testing. >>>=20 >>> Can you let me know how can I test glibc? Is there any standardized >>> test procedure available to do full test/regression test? >>>=20 >>=20 >> The usual =E2=80=98make check=E2=80=99 should cover all the required = testing, you might need to >> use =E2=80=98run-built-tests=3Dyes=E2=80=99 additional directive if = you are cross-compiling to force >> the make to issues the built tests (for instance on architecture that = support >> multiple ABI, like x86, or to force the use of some emulation like = qemu-user). >>=20 >> We are still fixing up some clunky tests and there are some broken = environment >> (for instance broken syscall filtering) where some tests might = advertise a failure. >=20 > Hello, >=20 > I tried with 'Testing with a cross-compiler' method and test is = aborted when the flag '-Wformat-security' is enabled. When this flag is = removed the test is executed successfully. Is it a known issue? (The = error log make-check.out is attached for reference) > Also, can anyone share the list of known test failures (if any). >=20 >=20 > Thanks, > Sundeep K. >=20 > Since glibc enables -Werror as default, not all compiler flags are = supported without triggering warning that might abort the build. Ideally for each = new warning flag you will first build with =E2=80=94disable-error and check = each error to possible either check if this is a compiler issue or if it requires = patch glibc. For the =E2=80=98make check=E2=80=99 output you can see why might be = happening by checking each file.out file: UNSUPPORTED: assert/tst-assert-c++ UNSUPPORTED: assert/tst-assert-g++ It means that you do not configure with a C++ compiler (it is required = only for testing). FAIL: catgets/test1.cat I am not sure about this one, but it seems that is a locale not being = present in the testing. FAIL: conform/ISO/setjmp.h/conform FAIL: conform/ISO/stdio.h/conform FAIL: conform/ISO/stdlib.h/conform FAIL: conform/ISO/stdlib.h/linknamespace FAIL: conform/ISO11/setjmp.h/conform FAIL: conform/ISO11/stdio.h/conform FAIL: conform/ISO11/stdlib.h/conform FAIL: conform/ISO11/stdlib.h/linknamespace FAIL: conform/ISO11/wchar.h/conform FAIL: conform/ISO99/setjmp.h/conform FAIL: conform/ISO99/stdio.h/conform FAIL: conform/ISO99/stdlib.h/conform FAIL: conform/ISO99/stdlib.h/linknamespace FAIL: conform/ISO99/wchar.h/conform FAIL: conform/POSIX/stdio.h/conform FAIL: conform/POSIX/stdlib.h/conform FAIL: conform/POSIX/stdlib.h/linknamespace FAIL: conform/POSIX2008/arpa/inet.h/conform FAIL: conform/POSIX2008/fcntl.h/conform FAIL: conform/POSIX2008/mqueue.h/conform FAIL: conform/POSIX2008/netdb.h/conform FAIL: conform/POSIX2008/netinet/in.h/conform FAIL: conform/POSIX2008/stdio.h/conform FAIL: conform/POSIX2008/stdlib.h/conform FAIL: conform/POSIX2008/stdlib.h/linknamespace FAIL: conform/POSIX2008/sys/socket.h/conform FAIL: conform/POSIX2008/unistd.h/linknamespace FAIL: conform/POSIX2008/wchar.h/conform FAIL: conform/UNIX98/arpa/inet.h/conform XPASS: conform/UNIX98/ndbm.h/linknamespace FAIL: conform/UNIX98/netdb.h/conform FAIL: conform/UNIX98/netinet/in.h/conform FAIL: conform/UNIX98/stdio.h/conform FAIL: conform/UNIX98/stdlib.h/conform FAIL: conform/UNIX98/sys/socket.h/conform FAIL: conform/UNIX98/unistd.h/conform FAIL: conform/UNIX98/unistd.h/linknamespace FAIL: conform/UNIX98/wchar.h/conform FAIL: conform/XOPEN2K/arpa/inet.h/conform FAIL: conform/XOPEN2K/fcntl.h/conform FAIL: conform/XOPEN2K/mqueue.h/conform XPASS: conform/XOPEN2K/ndbm.h/linknamespace FAIL: conform/XOPEN2K/netdb.h/conform FAIL: conform/XOPEN2K/netinet/in.h/conform FAIL: conform/XOPEN2K/stdio.h/conform FAIL: conform/XOPEN2K/stdlib.h/conform FAIL: conform/XOPEN2K/sys/socket.h/conform FAIL: conform/XOPEN2K/syslog.h/conform FAIL: conform/XOPEN2K/unistd.h/conform FAIL: conform/XOPEN2K/unistd.h/linknamespace FAIL: conform/XOPEN2K/wchar.h/conform FAIL: conform/XOPEN2K8/arpa/inet.h/conform FAIL: conform/XOPEN2K8/fcntl.h/conform FAIL: conform/XOPEN2K8/mqueue.h/conform XPASS: conform/XOPEN2K8/ndbm.h/linknamespace FAIL: conform/XOPEN2K8/netdb.h/conform FAIL: conform/XOPEN2K8/netinet/in.h/conform FAIL: conform/XOPEN2K8/stdio.h/conform FAIL: conform/XOPEN2K8/stdlib.h/conform FAIL: conform/XOPEN2K8/sys/socket.h/conform FAIL: conform/XOPEN2K8/syslog.h/conform FAIL: conform/XOPEN2K8/unistd.h/conform FAIL: conform/XOPEN2K8/unistd.h/linknamespace FAIL: conform/XOPEN2K8/wchar.h/conform FAIL: conform/XPG4/stdio.h/conform FAIL: conform/XPG4/stdlib.h/conform FAIL: conform/XPG4/stdlib.h/linknamespace FAIL: conform/XPG4/unistd.h/conform FAIL: conform/XPG42/arpa/inet.h/conform XPASS: conform/XPG42/ndbm.h/linknamespace FAIL: conform/XPG42/netdb.h/conform FAIL: conform/XPG42/netinet/in.h/conform FAIL: conform/XPG42/stdio.h/conform FAIL: conform/XPG42/stdlib.h/conform FAIL: conform/XPG42/sys/socket.h/conform FAIL: conform/XPG42/unistd.h/conform You will need to check on the *.out generated file to see which symbols = is being triggering this. It might a wrong toolchain build pulling some=20 unexpected symbol. FAIL: elf/circleload1 FAIL: elf/constload1 FAIL: elf/dblload FAIL: elf/dblunload FAIL: elf/lateglobal FAIL: elf/reldep6 FAIL: elf/resolvfail FAIL: elf/tst-bz15311 FAIL: elf/tst-global1 FAIL: elf/tst-tls20 As before, it will require to check the *.out file. FAIL: localedata/sort-test FAIL: localedata/tst-mbswcs6 FAIL: localedata/tst-numeric FAIL: localedata/tst-setlocale2 FAIL: localedata/tst-setlocale3 FAIL: localedata/tst-sscanf FAIL: localedata/tst-strfmon1 I am not sure what is state of locale tests for cross-tests. It might require some fixes here. FAIL: malloc/tst-malloc-thread-fail FAIL: malloc/tst-malloc-thread-fail-malloc-check FAIL: malloc/tst-malloc-thread-fail-malloc-hugetlb1 FAIL: malloc/tst-malloc-thread-fail-malloc-hugetlb2 FAIL: malloc/tst-malloc_info FAIL: malloc/tst-malloc_info-malloc-check FAIL: malloc/tst-malloc_info-malloc-hugetlb1 FAIL: malloc/tst-malloc_info-malloc-hugetlb2 I think these might be due default timeout if too low depending of the target machine. Check the *.out file and you can use a larger default timeout by exporting TIMEOUTFACTOR=3DN environment variable. Ideally we want to setup timeout for tests that are not susceptible to machine speed (although depending of the system load it might still trigger timeouts). FAIL: posix/bug-regex18 FAIL: posix/bug-regex35 FAIL: posix/globtest FAIL: posix/tst-fnmatch4 FAIL: posix/tst-rxspencer FAIL: stdlib/test-bz22786 FAIL: stdlib/tst-strfmon_l FAIL: stdlib/tst-strtod-nan-locale FAIL: stdlib/tst-strtod4 FAIL: stdlib/tst-strtod5 FAIL: stdlib/tst-strtod5i FAIL: stdlib/tst-strtol-locale FAIL: string/bug-strcoll2 FAIL: time/tst-strptime FAIL: timezone/tst-tzset FAIL: wcsmbs/tst-btowc FAIL: wcsmbs/tst-wcstod-nan-locale FAIL: wcsmbs/tst-wcstol-locale I think these are related to locales missing, although I am not sure why exactly. FAIL: string/test-memcpy-large This might also be a timeout issue. FAIL: string/test-strcasecmp FAIL: string/test-strncasecmp FAIL: string/tst-strerror FAIL: string/tst-strsignal These use containers, you will need to check the *.out file to see exactly what is happening.