From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from sonic311-31.consmr.mail.ir2.yahoo.com (sonic311-31.consmr.mail.ir2.yahoo.com [77.238.176.163]) by sourceware.org (Postfix) with ESMTPS id 60FEA3858405 for ; Sat, 22 Jan 2022 21:20:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 60FEA3858405 X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1642886425; bh=jyLFuq9XDN2I4WHaQq+k6SxKIl45YdJJ51elGMDEnxa=; h=X-Sonic-MF:Date:From:Subject:To:From:Subject; b=VjI7kF3gx1gLsTvTnvQlySAtgVf1KU4RCyzr1ECQjNXuOTsqWSRzces9tvbJWEwCFmCAPLP/+kwUnHfvqObucLMJAsYQqKi1e4ohNoBvj3J0C6hFDKkRvRem7mP+7X6mCCMuVV7wVUeeWfK43mEK9q3FIIiu+18VoOTk/r5vFToAnVA6aOL8oxcXeeRxToRnjcc1wE2HVIpy93DNGaN2naXIL5gsh+jeR8+dZVjG8LUnHgmTMGk2MIeFQa5qBvhk05A/aiNLvwAo9NQLEVbDqq3bEP0ckOCni9hzfzU8ClWEXcEggcCdwgC2OmGryD71xtgRBNsWVkexX9o7mgCGMQ== X-YMail-OSG: tVtkMOsVM1k1WVOoUVr59BmX5WbnjHJmb4RBVDZ05lCYpgcovNqdRfZK6jfMaTE rBFipPfdHOEuTan2KKnNzTAN6vDrXqCmb1i8aGQa56t8Wnr6qH5ccHJESlvmUi4rA.0vv8G0V9_A VDmdr5oE.u2aMw2_0RbggoHdvbZpe_._dVmot4Inz_p0SKyWkHej6EbXewox9DuQTRLw6dphEcy_ bj7Dn4ZHKvTXb16fVheCCd00cTEUt4WA9QaqZYhKVimtFQ6B0mIwGrHjq.1ibX04lIje5iEEMZg0 CS8CcjKW1BjWNTL0zSlkvk2gFtgvFbjix11IeJLIAGB6IzR44Egg6KWD2orNuHRNcuqw7rW2Q23V LUZYkFisFCuCYuflD.Y7TLL30c9JSTqG.uSAPCJ9jIiW3P1qAfG1g0.EXJzJ8H5MNFJRWhfwIdfE G5YDtVi6cgiCGQtpFTzzAD7DDmOXJZ.vmvATbolYqunwlUDdQPg2HIrBmadbfbI4ev6ycm3mz1qf ZWKgob2hTyZ5XqC49Kl8sJtN24U6U_2AkxfAN5v.8zYJjPu1Mnjz5KmyNNLCibv45DQRvKIMPcX2 fwOBuY8a4o2fKKx6UdSsedZysUwpzmqOk2GYQpyXWfAxDohsM9xewJHaJOWqfcNCuPkojKgW7WI2 nIqmoqV7eGRPiIWP3s4noJb5gueBZ5M_JhDoaNEklxQRzRRl_vVYA1CgeKdqHsQuCQHZ6P_I518m 0FbSbxyTgYpg4bELmJwGzQR6VvCGpImeIcA9z8S7_yEOB1AeWPwMQDTwhr_Jsf38AiKkHGqjXy44 s2Z5WU99LgcRoaVIix3tag6XBCR6YGWFwjawkcW8ARyRy8blUI_XUxgsbFOcMOEz.JHz4OqUPsPf E9Xdg9dTP1BwAvD3HJbYlWG9oz25S06aTp97Ylkn4n5uuhe1Pbp_KMFlS2Xy9GDBZi7pGN18_Vhi SCw_azvZCjUBXuVGxZhbq65eVkHoeMipu8dSxjp6YldVsvfW72YMXrfDev1dE9CzSwM3xsEM3yTV p1MGbtJD3UNC85cCs4Q8z2tGvPppzjuKRtW9y.F6DoNogILE.93FiBoKr26KRcP_T8KFg.nEUJk6 bVW_szEmVlrP01iovDEcQH9bFEnpB7XkunjFuyGAxp0RAZwA3S4xMoevKMLP7jyGjE637VDscFJC 4pnEHONBsTQ3N2RCqX3glp.NhNCTkkXLIbP7.nJLAyGMCOupUiEu23.1R.cWaWU1mkbK3X77X1aE izHjKw8fcChMfSlJzO3x4YiYEdMsYRL4583_VFRQAoy5s2DJ5cCD7gzl4TSF.9JZTgt1jGHKPu1y GlLs2Gx9igwFz9VbmL1lnZ92iSS0QOG.jGam7vY.gjXmhyYiAFBb2DDayJIJQywmh4GofALs_5B6 gVKdhcgtcMsVV51u81NHIMdV8Jzwpb4JHoXgR8hdlEWiV_mkRLKcmVFjaaYSCwJv1SKRFXYw_Lba VGlj_Ce0Llksv9Jdl14GjDWTYnse.tp05idY7XQPsgeojbW4fBAySR_KoRoNGVA4BGigyHUAw0Dp DMYTkGK1MzeeWqp6fWzPb7pxn3nuogu7bufpjfW9IlPbBAV86HaKhmPF.hXqw744fwoBuTgi59el cwp3h9u1xIPJrOn6RsvUOFBVCcyrUDLU63CtQ0IzBgSJslhwspGwyoOFvADHa5oeS2x_dJrwobae F3bfiRnsiyBdffiPCCU7xOKhmh7jHJkeN578KKlRJU7hJizZd3k4DnBVKhF4glan3zFqvn34sSvD JfkKuMeftM7JLNJYgeYdmRmuJGcXph.y0nORnXJALrUxFaOP81TRJHPb8OxlGeFKzKdBx6UfT8hi .HfBkXnMFwYStsGUuSHMFBCsHa.CD46Gx6A_NJjox0CjvMR66LwbmMZVOkmwcun0cRedxBlvNtMC bwJNN9VFG2Kpv7Z3QybofMKeba1szH1ToFr1i1jsj1M0HpZbtV6ctxmoarAaA7RaNLIj5GFUBW4G ztUQjCbVPxL1in5pFB8uwNKIRusHSCAMQINKMLrZJhd0jY6kcJ1Gy8UuQ1c_fv4svOxzcHv0dZHC QqN8zNulWE_irLWWRUW_lTbx90j97_kakWj35sw87CIgMA4sJ3kV6cXewwp.URY5lcuVVKlpr8M3 60z8_LYkdVGR9MPg2NiV_QLCx7XOdPoqAzvZqs__str0h9smU6R4ShippSnS0dVdkHSlXryW0F3I muku9ow_VnmRP_PSKrgA8_YtgZIalwCrhQvKH09pCtD4m8y_DihISfQk- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.ir2.yahoo.com with HTTP; Sat, 22 Jan 2022 21:20:25 +0000 Received: by kubenode518.mail-prod1.omega.ir2.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID c05348b20c4e37b5d0f5abadd6dddb11; Sat, 22 Jan 2022 21:20:21 +0000 (UTC) Message-ID: <2bc01390-3a5b-fd2d-822c-c4cd9a96f744@yahoo.de> Date: Sat, 22 Jan 2022 22:20:20 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 From: "R. Diez" Subject: Re: Question about autoreconf to regenerate configuration files Cc: Newlib , Matthew Joyce , joel@rtems.org References: <26fff1a3-7743-e0ce-f7a6-70b39a030118@embedded-brains.de> <41deefa7-d1ae-18f2-6a8e-25e002d85ed6@yahoo.de> Content-Language: en-GB To: Mike Frysinger In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Mailer: WebService/1.1.19615 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, KAM_SHORT, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: newlib@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Newlib mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jan 2022 21:20:29 -0000 Hallo Mike: Newlib's build system has been a major pain and I would like to take this opportunity to learn more about it, and if that also helps with the other toolchain components, it's a plus point. > [...] > autotools (autoreconf really) doesn't run in parallel, so every subdir > with a configure script needs a separate serialized run of all the tools. > newlib has many many of these (arguably, too many). > > on my quad core 4.2GHz AMD that is otherwise idle ... > > $ time (cd newlib && autoreconf) > real 5m22.170s > user 3m13.709s > sys 0m12.332s My system is not really a very fast machine for development purposes: - A laptop with an Intel Core i5-8265U CPU @ 1.60GHz (4 cores + hyperthreading) - 1TB consumer-grade SSD Crucial MX500 - Ubuntu 20.04 My result is: $ ( cd newlib && time autoreconf --force --verbose ) real 3m47,685s user 1m48,533s sys 0m10,338s That's actually a rather long time. But if I understand correctly, your target is to have just 2 'configure' scripts for newlib and libgloss, which means that the Autoconf regeneration time is going to get cut drastically. Is that right? Is it actually a problem that regenerating the Autoconf files in Newlib takes 5 minutes? Is the Newlib project doing Continuous Integration? If so, it should actually be doing the Autoconf regeneration anyway, in order to test it as well. In any case, would a 5-minute delay there be an issue? This Autoconf-related delay would only apply to sources directly obtained from the Git repository. The Newlib tarball releases would already include the Autoconf files, so no delay there. If somebody, like RTEMS, is tracking Newlib's Git repository directly, that's their choice. But even then, they have options. If they distribute a release with patches etc, they can regenerate the Autoconf files before packaging Newlib for their own release. If the problem lies in the Continuous Integration system, you can cache the Autoconf files: if last time you checked out the same Git commit hash, then you do not need to regenerate the Autotools files again. > again, this isn't "just newlib". newlib is part of the historically combined > toolchain tree/ecosystem. that means you can take binutils, gdb, gcc, newlib, > libgloss, cgen, sim, zlib, etc... and have a single monolithic source tree and > build them all at once. the projects have separated a little bit in that they > have diff git repos, but the top-level dir and a few subdirs are still shared, > and some folks still hand merge them. newlib is part of that ecosystem and as > such, follows its conventions. changing newlib behavior would have a ripple > effect and is why consensus across all of them is desirable. although usually > if you can convince gcc to change, the rest will follow to keep things simple. OK, let me try to understand the details here. I have been using the following makefile for years to build a cross-compiler GCC toolchain: https://github.com/rdiez/JtagDue/blob/master/Toolchain/Makefile That makefile builds GCC in 2 phases: a first, temporary GCC without a C library, and the final GCC with Newlib. With that makefile, Newlib is configured, built and installed separately from GCC, so Newlib's build system does not matter at all in this scenario. I think that is the same strategy as the OpenWrt toolchain makefile: https://git.openwrt.org/ toolchain/Makefile This way, you can use another libc. I believe that OpenWrt tends to use Musl instead of glibc. The other way to build such a cross-compiler toolchain is to merge Newlib's sources with GCC's etc. That is called a "combined tree" and is mentioned here: https://gcc.gnu.org/wiki/Building_Cross_Toolchains_with_gcc The instructions on that page appear outdated and do not seem complete. If you are keeping the Newlib build system compatible with GCC's etc, you must be testing with some commands, or maybe a script, that merges the trees. Could you share those steps? Alternatively, do you know of a web page that describes the steps accurately? The instructions on the page I mentioned above talk about GCC's files overwriting any files in Newlib etc. I wonder at what level that is supposed to happen. For example, gcc-10.3.0 has a top-level 'configure' script. Is that supposed to overwrite Newlib's top-level 'configure' script? In the combined tree, GCC must know somehow that it should include the 'newlib' subdirectory. Is that what GCC's 'configure' option '--with-newlib' is supposed to do? I could not find any information about "--with-newlib" in the GCC documentation. To what extent is build system compatibility in this GCC ecosystem a problem? Newlib has had very outdated Autoconf files for quite some time and I presume that it has been working with many different GCC versions over the years. I do not suppose that we maintain a table of build system version compatibility between Newlib and GCC releases, do we? My guess is that GCC's build system is calling 'configure', 'make' etc. inside newlib/, and it does not really matter if the versions are a little different. Or is a lot of the build system really shared? -- rdiez