From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 1521) id 5BA20385DC31; Thu, 17 Mar 2022 01:59:11 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5BA20385DC31 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable From: Michael Frysinger To: newlib-cvs@sourceware.org Subject: [newlib-cygwin] newlib: update build system generation documentation X-Act-Checkin: newlib-cygwin X-Git-Author: Mike Frysinger X-Git-Refname: refs/heads/master X-Git-Oldrev: 96bc16f6b27e9183c293c1a77351ec71f71bc32f X-Git-Newrev: f2471da7db7d49e384007c41759e4e8cb3d8340c Message-Id: <20220317015911.5BA20385DC31@sourceware.org> Date: Thu, 17 Mar 2022 01:59:11 +0000 (GMT) X-BeenThere: newlib-cvs@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Newlib GIT logs List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Mar 2022 01:59:11 -0000 https://sourceware.org/git/gitweb.cgi?p=3Dnewlib-cygwin.git;h=3Df2471da7db7= d49e384007c41759e4e8cb3d8340c commit f2471da7db7d49e384007c41759e4e8cb3d8340c Author: Mike Frysinger Date: Sun Feb 13 21:43:37 2022 -0500 newlib: update build system generation documentation =20 Replace all of the individual autotool steps with a single autoreconf. This simplifies the documentation greatly, and in the current system, only takes ~10 seconds to regenerate everything. =20 Update the developer documentation to cover all the major components of the current build system. Hopefully this is a fairly complete road map to everything. I tried to include everything that I wish I knew when I started hacking on this :P. Diff: --- newlib/HOWTO | 171 ++++++++++++++++++++++++++++++++++++++++++++++++++++++= ++++ newlib/NEWS | 1 + newlib/README | 91 +++++++------------------------ 3 files changed, 191 insertions(+), 72 deletions(-) diff --git a/newlib/HOWTO b/newlib/HOWTO index 668766096..e9af3a408 100644 --- a/newlib/HOWTO +++ b/newlib/HOWTO @@ -110,3 +110,174 @@ Level 4 Full POSIX or Linux compliance. Essentially these are functions that= are present in a standard Linux kernel but are irrelevant to an embedded syste= m. These functions do not form part of the EL/IX API. + +BUILD SYSTEM +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + + Newlib utilizes GNU Autotools (GNU Autoconf & GNU Automake) for its b= uild +system. It can be broken down into two phases: configure & compile (make). +NB: GNU Libtool is not used. Newlib only produces static archives, and GNU +Automake is capable of handling that. + + As noted in the README file, you do not need to worry about which too= ls to +run at any particular point. Simply invoke `autoreconf` and it will do it= all. + + Tool versions are tightly controlled. GNU Autoconf 2.69 exactly must= be +used. Attempts to use any other version, newer or older, will be rejected= . If +the current system does not include that version, it can be downloaded fro= m the +GNU Autoconf website and installed into a local directory tree (like ~/.lo= cal). +GNU Automake 1.15.1 is required as a minimum. Newer versions are allowed = when +running locally, but developers submitting patches must use 1.15.1 exactly. + + The configure phase naturally starts with the configure script. It m= erges +multiple inputs to produce the final script. +* aclocal.m4: Generated by aclocal; do not edit. Copies all the various m4 + macros used by configure from the system install into the local copy. T= his + allows regenerating configure without needing all the same system packag= es. +* autom4te.cache/: Generated by aclocal to cache system search results. S= afe + to delete at any time, but never edit anything in this directory. +* iconv.m4: Generated by libc/iconv/ces/mkdeps.pl; do not edit. Keeps the + set of dynamic iconv synced between the source files and configure. +* acinclude.m4: Historically, a way of sharing configure logic with subdir= s. + At some point, it should all be merged into configure.ac directly. +* configure.ac: The main configure script for supporting all Newlib option= s. +* configure.host: An index of every supported Newlib host and the settings= to + use when compiling for it. +* **/acinclude.m4: Subdirectories may include custom configure tests witho= ut + having to clutter the common configure.ac or acinclude.m4 file. When ad= ding + a new file, it must be manually included -- there is no mechanism to sea= rch + and automatically include every such file in the tree. Look for m4_incl= ude + lines in configure.ac or existing acinclude.m4 files as examples. Care = must + be taken to not run excessive tests for incompatible targets, or to decl= are + generic names or symbols that can collide with other targets. Any such = file + should try to match $host (or similar) settings before running. Settings + from configure.host (e.g. machine_dir or sys_dir) are guaranteed to be + available. When creating names (Makefile variables, defines, etc...), t= ry to + include the respective library and machine or sys name. Do not confuse = these + with the historical top-level acinclude.m4 file! + + The configure script has to take special consideration to the fact th= at it +usually runs with a toolchain that lacks a C library (since Newlib will pr= ovide +it once it's compiled & installed). Newlib uses the uncommon AC_NO_EXECUT= ABLES +macro to tell GNU Autoconf to not automatically require a toolchain that c= an +link executables. As a result, Newlib itself cannot perform any link test= s. +It shouldn't really need to, but configure authors must keep this in mind = and +stick to the appropriate source-level test -- AC_PREPROC for testing behav= ior +of the preprocessor and AC_COMPILE for testing compiler behavior. + + The newlib.h header is created from newlib.hin during the configure p= hase. +The newlib.hin file is managed by autoheader, so do not edit it directly. = Any +symbol declared in configure via AC_DEFINE is automatically included. The= file +declares all the Newlib-specific configure settings and will be installed = as +part of the standard set of headers. Critically, all symbols are namespac= ed +with a leading _ prefix to avoid collisions with user code, and any symbol= s not +namespaced are automatically removed. This is necessary due to some syste= m m4 +macros automatically adding their own defines that we cannot disable. It = also +means that, when using AC_DEFINE, the _ prefix must be manually added. + + A single Makefile is created during the configure phase. This will b= uild +all of Newlib using non-recursive make (i.e., there are no subdirectories = with +their own Makefiles). The single Makefile.am file includes Makefile.inc f= iles +in its immediate subdirectories, and those further include Makefile.inc fi= les +in their subdirectories. This means all variables and targets exist in a +single global flat namespace, so care must be taken to not use generic nam= es +like "SOURCES" in any Makefile.inc file. Instead, use %C%_ on all variabl= es to +get a unique prefix, and GNU Automake will expand it. In order to use one= of +the standard GNU Automake variables (e.g., CLEANFILES), first declare it a= t the +top of the Makefile.am by assigning it (e.g., CLEANFILES =3D), and then al= ways +append to it in Makefile.inc (e.g. CLEANFILES +=3D ...). Again, GNU Autom= ake +will take care of combining all of this logic together to produce a portab= le +Makefile. + + Developers used to changing to a subdirectory and running make comman= ds +to iterate quickly on a smaller section of Newlib might be surprised that = this +does not work -- there is no Makefile in any subdirectory. Instead, make = has +to be run from the top-level, and the full path to the target is specified. +So instead of running `cd subdir && make foo.o`, run `make subdir/foo.o`. = This +applies to any target, not just objects. Some common targets: +make libm.a +make libc.a + + Conditionals are allowed in Makefile.inc files, but using them to eli= de +an entire subdir Makefile.inc should be avoided. Instead, GNU Automake +conditionals should be used only within a single Makefile.inc file to cont= rol +settings specific to that directory. See the documentation section below = for +more information. + + Per-directory compiler settings are allowed. Similar to the syntax u= sed +above, the _%C% suffix can be used. See the Makefile.am file for the curr= ent +set of flags (e.g. CFLAGS, CPPFLAGS, etc...) that are supported. +AM_CFLAGS_%C% =3D ... +libm_a_CFLAGS_%C% =3D ... +libc_a_CFLAGS_%C% =3D ... + + Per-file compiler settings are allowed. Similar to the syntax used +above, the _%C% suffix can be used. See the Makefile.am file for the curr= ent +set of flags (e.g. CFLAGS, CPPFLAGS, etc...) that are supported. +AM_CFLAGS_%C%_moo.c =3D ... +libm_a_CFLAGS_%C%_foo.c =3D ... +libc_a_CFLAGS_%C%_bar.c =3D ... + + An important aspect of the build system is allowing machine-specific +directories to override common implementations. For example, Newlib provi= des +generic C implementations of string & memory functions in libc/string/ that +perform adequately for any target, but many architectures include hand wri= tten +assembly files for smaller & faster execution. Those files live under the +corresponding libc/machine// directory. Both versions are +compiled when building Newlib, but when generating the libc.a archive, the +common object is added first, followed by the machine-specific one. This = works +because the object names follow a consistent convention: libm_a-= .o +for the math library and libc_a-.o for the C library. If the co= mmon +code stores a function foo in foo.c, then it can be overridden by another = foo.c +or foo.S or foo.s file since they all produce foo.o. For this reason, all +machine Makefile.inc files must be included last after all common Makefile= .inc +files. + + Note that the Makefiles do not use VPATH to search source files across +multiple source directories. VPATH is used to match a single build direct= ory +to the corresponding source directory, but that is it. For more details on +how machine directories override functions in common directories, see the +previous section. + + Keep in mind that machine overrides only work at the file level, not = at +the symbol level. If a common file defines multiple symbols, then the mac= hine +override must define all the same symbols. This is why most common code s= tores +only one function per source file. + + The documentation, both the manual and man pages and header reference= s, +should be consistent regardless of what configure options are used (includ= ing +the target system). This makes it easy to get a complete documentation bu= ild +at all times. If inclusion of chapters depended on specific configure opt= ions, +it would be difficult to find the set of options necessary to produce the +complete manual, if it were even possible in the first place (as some conf= igure +options are incompatible). Since documentation chapters & APIs are declar= ed in +the respective directories and not in a centralized location, it is import= ant +to not use any GNU Automake conditional to control whether Makefile.inc fi= les +are included. Documentation that covers optional APIs should note that it= is +not guaranteed to be included in every build of Newlib for every target. = The +level of detail is left up to the author's discretion. + + Newlib automatically builds itself for all multilib configurations th= at +the active toolchain supports. This logic is provided by common code from= the +combined toolchain source tree, but bolts itself into the GNU Autoconf and +Automake files. The critical parts to keep in mind: +* ../config/multi.m4: AM_ENABLE_MULTILIB adds the --{en,dis}able-multilib + options to configure for users, and takes care of calling ../config-ml.in + for the Makefile in the current directory. While not applicable to Newl= ib + anymore (since Newlib only produces a single Makefile now), this does not + automatically run for any subdir Makefiles. Instead, it must be run man= ually + in AC_CONFIG_FILES's commands argument. +* ../config-ml.in: Rewrites the specified Makefile to include the various + multilib variables and the multi-do command. Notably, this only makes t= he + multi-do rule available, it does not cause it to be used anywhere. +* ../multilib.am: Manually included in Newlib's Makefile.am. This adds ca= lls + to multi-do for the common important rules: all (i.e. compiling), instal= l, + clean, etc... +* FLAGS_TO_PASS: Newlib's Makefile.am declares all the variables to pass to + each multilib configuration via the multi-do rule. While the multi-do r= ule + includes many common flags automatically, Newlib passes along a few extra + that are unique to it. + + The testsuite is not integrated at all currently. It is not really we= ll +tested in general. The README file has more information. Hopefully someo= ne +someday will step up to integrate it into the existing framework. diff --git a/newlib/NEWS b/newlib/NEWS index aaed47aae..c8e5a7cd3 100644 --- a/newlib/NEWS +++ b/newlib/NEWS @@ -2,6 +2,7 @@ =20 - remove i?86-pc-linux-gnu support - remove decstation & sunos support +- build system internals heavily rewritten & updated =20 *** Major changes in newlib version 4.2.0: =20 diff --git a/newlib/README b/newlib/README index aa48b246f..97890b9d2 100644 --- a/newlib/README +++ b/newlib/README @@ -21,15 +21,13 @@ Unpacking and Installation -- quick overview =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D =20 When you unpack the newlib-4.2.0.tar.gz file, you'll find a directory -called `newlib-4.2.0', which contains: - -COPYING config/ install-sh* mpw-configure -COPYING.LIB config-ml.in libgloss/ mpw-install -COPYING.NEWLIB config.guess* mkinstalldirs* newlib/ -CYGNUS config.sub* move-if-change* symlink-tree* -ChangeLog configure* mpw-README texinfo/ -Makefile.in configure.in mpw-build.in -README etc/ mpw-config.in +called `newlib-4.2.0', which contains many files. Interesting ones: + COPYING* - License files for the sources + README - A common overview of all GNU development projects + configure - The build script for configuring the source tree + Makefile* - Inputs used by configure to generate the Makefile + libgloss/ - The libgloss project + newlib/ - The newlib project =20 To build NEWLIB, you must follow the instructions in the section entitled "Compiling NEWLIB". @@ -46,7 +44,7 @@ More Documentation =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 Newlib documentation is available on the net via: - http://sourceware.org/newlib/docs.html + https://sourceware.org/newlib/docs.html =20 All the documentation for NEWLIB comes as part of the machine-readable distribution. The documentation is written in Texinfo format, which is @@ -77,7 +75,6 @@ format. On its own, TeX cannot read, much less typeset a= Texinfo file. `newlib-VERSION-NUMBER/texinfo' directory. =20 =20 - Compiling NEWLIB =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 @@ -511,70 +508,20 @@ run the testsuite. Regenerating Configuration Files =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D =20 -At times you will need to make changes to configure.ac and Makefile.am fil= es. -This will mean that configure and Makefile.in files will need to be -regenerated. - -At the top level of newlib is the file: acinclude.m4. This file contains -the definition of the NEWLIB_CONFIGURE macro which is used by all configur= e.ac -files in newlib. You will notice that each directory in newlib containing -a configure.ac file also contains an aclocal.m4 file. This file is -generated by issuing: aclocal -I${relative_path_to_toplevel_newlib_dir} --I${relative_path_to_toplevel_src_dir} -The first relative directory is to access acinclude.m4. The second relati= ve -directory is to access libtool information in the top-level src directory. - -For example, to regenerate aclocal.m4 in newlib/libc/machine/arm: - - aclocal -I ../../.. -I ../../../.. - -Note that if the top level acinclude.m4 is altered, every aclocal.m4 file = -in newlib should be regenerated. - -If the aclocal.m4 file is regenerated due to a change in acinclude.m4 or -if a configure.ac file is modified, the corresponding configure file in th= e=20 -directory must be regenerated using autoconf. No parameters are necessary. -In the previous example, we would issue: - - autoconf - -from the newlib/libc/machine/arm directory. - -If you have regenerated a configure file or if you have modified a Makefil= e.am -file, you will need to regenerate the appropriate Makefile.in file(s). -For newlib, automake is a bit trickier. - -Makefile.in files are generated from the nearest directory up the chain -which contains a configure.ac file. In most cases, this is the same -directory containing configure.ac, but there are exceptions. -For example, the newlib/libc directory has a number of -subdirectories that do not contain their own configure.ac files (e.g. stdi= o). -For these directories, you must issue the automake command from newlib/libc -which is the nearest parent directory that contains a configure.ac. -When you issue the automake command, you specify the subdirectory for -the Makefile.in you are regenerating. For example: - - automake stdio/Makefile stdlib/Makefile - -Note how multiple Makefile.in files can be created in the same step. You -would not specify machine/Makefile or sys/Makefile in the previous example -because both of these subdirectories contain their own configure.ac files. -One would change to each of these subdirectories and in turn issue: - - automake Makefile +At times you will need to make changes to configure.ac, Makefile.am and +Makefile.inc files. This will mean that configure and Makefile.in files w= ill +need to be regenerated. The easiest way to do so is by using the autoreco= nf +tool in the newlib directory. =20 -Let's say you create a new machine directory XXXX off of newlib/libc/machi= ne. -After creating a new configure.ac and Makefile.am file, you would issue: + autoreconf =20 - aclocal -I ../../.. - autoconf - automake Makefile +This will run a number of autotool programs for you. To see the individual +steps, add the -v option. =20 -from newlib/libc/machine/XXXX + autoreconf -v =20 -It is strongly advised that you use an adequate version of autotools. -For this latest release, the following were used: autoconf 2.69, aclocal 1= .15.1, -and automake 1.15.1. +It is strongly advised that you use an adequate version of autotools. For= this +latest release, the following were used: autoconf 2.69 and automake 1.15.1. =20 Reporting Bugs =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D @@ -587,4 +534,4 @@ Since NEWLIB supports many different configurations, it= is important that you be precise about this. =20 Archives of the newlib mailing list are on-line, see - http://sourceware.org/ml/newlib/ + https://sourceware.org/ml/newlib/