[Bug other/106429] New: Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[Bug other/106429] New: Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[martin.vahi at softf1 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

            Bug ID: 106429
           Summary: Building GCC is Inhibited on old Linux Distributions
                    due to the use of "#!/bin/bash"
           Product: gcc
           Version: 11.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: other
          Assignee: unassigned at gcc dot gnu.org
          Reporter: martin.vahi at softf1 dot com
  Target Milestone: ---

Some newer version of C++_compiler is
needed to build applications that use some 
newer C++ standard. By default old Linux distributions have
too old GCC versions, so a newer version of GCC needs to be
built before the applications that use newer C++ standard
can be built. Old Linux distributions are needed for
old hardware, because newer Linux distributions
do not always contain all the drivers for old hardware.
For example, in the case of openSUSE Linux, drivers 
for old hardware are literally removed from the 
package repositories of newer versions of the openSUSE Linux.
The paths at a "dependency graph" are:

    the_use_of_old_hardware
    --depends-on->
    the_availability_of_drivers_for_that_old_hardware
    --depends-on->
    the_use_of_old_operating_system_distributions
    --depends-on->
    possibility_to_backport_applications_to_those_old_distributions

    apps_that_use_newer_Cpp_standard 
    --depend-on->
    some_newer_Cpp_compiler === GCC 
    --depends-during-buildtime-on->
    availability_of_some_newer_version_of_Bash_on_PATH
    --depends-on->
    GCC_code_changes_that_are_described_in_this_bug_report

To build a newer version of GCC, some
newer version of Bash is at least sometimes required, because
the building of GCC, for example, version 11.2.0, includes 
the execution of some Bash scripts. The default Bash 
implementation tends to reside at the path "/bin/bash", but
one needs the newer version of the Bash on PATH only during 
the building of the GCC. The classical solution in that
scenario is 

    export PATH="/path/to/the/newer/Bash/version/bin:$PATH"

which means that the new version of the Bash gets found before
the system default "/bin/bash" is found. However, if the GCC 
build related Bash scripts start with

    #!/bin/bash

in stead of 

    #!/usr/bin/env bash

then that kind of PATH based Bash executable override
does not work. The suggested and tested (GCC v. 11.2.0)
workaround is to use grep to find all Bash files that
start with the "#!/bin/bash" and replace the 
"#!/bin/bash" with the "#!/usr/bin/env bash", except the file

    gcc/contrib/repro_fail

because, for some nice amount of luck, leaving that one 
to its "#!/bin/bash -eu" did not break anything.
The version of Bash that turned out to be too old 
for building the GCC v.11.2.0 was 

    4.3.48(1)-release (x86_64-suse-linux-gnu)

and the version of Bash that was sufficiently new for building
GCC v.11.2.0 was Bash v.5.1

Thank You for reading this bug_report/comment.

[Bug other/106429] Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |INVALID
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
The current list is:
contrib/bench-stringop:#!/bin/bash
contrib/compare-all-tests:#! /bin/bash
contrib/dg-cmp-results.sh:#!/bin/bash
contrib/reghunt/bin/gcc-svn-patchlist:#! /bin/bash
contrib/reghunt/bin/gcc-svn-update:#! /bin/bash
contrib/reghunt/bin/gcc-svn-update-fix:#! /bin/bash
contrib/reghunt/bin/reg-hunt:#! /bin/bash
contrib/reghunt/bin/reg-test:#! /bin/bash
contrib/reghunt/date_based/reg_periodic:#! /bin/bash
contrib/reghunt/date_based/reg_search:#! /bin/bash
contrib/repro_fail:#!/bin/bash -eu
gcc/dbgcnt.def:#!/bin/bash
gcc/doc/install.texi:% CONFIG_SHELL=/opt/freeware/bin/bash
gcc/testsuite/gcc.test-framework/gen_directive_tests:#! /bin/bash
libffi/merge.sh:#!/bin/bash
libgo/go/cmd/go/testdata/script/mod_get_pseudo_other_branch.txt:#!/bin/bash
libgo/go/cmd/go/testdata/script/mod_get_pseudo_prefix.txt:#!/bin/bash
libgo/go/cmd/go/testdata/script/mod_retract_pseudo_base.txt:#!/bin/bash
libgo/go/os/user/lookup_unix_test.go:root:x:0:0:root:/root:/bin/bash
libsanitizer/merge.sh:#!/bin/bash
libstdc++-v3/scripts/run_doxygen:#!/bin/bash
maintainer-scripts/generate_libstdcxx_web_docs:#!/bin/bash
maintainer-scripts/update_web_docs_libstdcxx_git:#!/bin/bash

None are used during the build or are not used unless you are run them directly
(or indirectly building the doxygen for libstdc++ which is not done by
default).

gcc/dbgcnt.def is inside a comment so it does not matter.

[Bug other/106429] Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
GCC 11.2.0 had the same as what I found on the trunk `git grep "/bin/bash"
releases/gcc-11.2.0`):
releases/gcc-11.2.0:contrib/bench-stringop:#!/bin/bash
releases/gcc-11.2.0:contrib/compare-all-tests:#! /bin/bash
releases/gcc-11.2.0:contrib/dg-cmp-results.sh:#!/bin/bash
releases/gcc-11.2.0:contrib/reghunt/bin/gcc-svn-patchlist:#! /bin/bash
releases/gcc-11.2.0:contrib/reghunt/bin/gcc-svn-update:#! /bin/bash
releases/gcc-11.2.0:contrib/reghunt/bin/gcc-svn-update-fix:#! /bin/bash
releases/gcc-11.2.0:contrib/reghunt/bin/reg-hunt:#! /bin/bash
releases/gcc-11.2.0:contrib/reghunt/bin/reg-test:#! /bin/bash
releases/gcc-11.2.0:contrib/reghunt/date_based/reg_periodic:#! /bin/bash
releases/gcc-11.2.0:contrib/reghunt/date_based/reg_search:#! /bin/bash
releases/gcc-11.2.0:contrib/repro_fail:#!/bin/bash -eu
releases/gcc-11.2.0:gcc/dbgcnt.def:#!/bin/bash
releases/gcc-11.2.0:gcc/testsuite/gcc.test-framework/gen_directive_tests:#!
/bin/bash
releases/gcc-11.2.0:libgo/go/cmd/go/mkalldocs.sh:#!/bin/bash
releases/gcc-11.2.0:libgo/go/cmd/go/testdata/script/mod_get_pseudo_other_branch.txt:#!/bin/bash
releases/gcc-11.2.0:libgo/go/cmd/go/testdata/script/mod_get_pseudo_prefix.txt:#!/bin/bash
releases/gcc-
releases/gcc-11.2.0:libsanitizer/merge.sh:#!/bin/bash
releases/gcc-11.2.0:libstdc++-v3/scripts/run_doxygen:#!/bin/bash
releases/gcc-11.2.0:maintainer-scripts/generate_libstdcxx_web_docs:#!/bin/bash
releases/gcc-11.2.0:maintainer-scripts/update_web_docs_libstdcxx_git:#!/bin/bash

So you need to supply more information on what you needed to change.

[Bug other/106429] Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Also almost all of the GCC build scripts depend on just POSIX sh rather than
bash. It might be the case you need to set CONFIG_SHELL instead as defined as
part of the autoconf .

[Bug other/106429] Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #4 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Martin Vahi from comment #0)
> To build a newer version of GCC, some
> newer version of Bash is at least sometimes required, because

Why?

Your "depencency graph" doesn't actually explain anything about the problem
that you say exists. Where is a newer Bash required?

> the building of GCC, for example, version 11.2.0, includes 
> the execution of some Bash scripts.

I don't think that's true. Even if it is, why is a newer Bash required?

> workaround is to use grep to find all Bash files that
> start with the "#!/bin/bash" and replace the 
> "#!/bin/bash" with the "#!/usr/bin/env bash", except the file
> 
>     gcc/contrib/repro_fail
> 
> because, for some nice amount of luck, leaving that one 
> to its "#!/bin/bash -eu" did not break anything.

This is not luck. That script is not used when building GCC. Most of them
aren't used when building GCC, as Andrew said.

> The version of Bash that turned out to be too old 
> for building the GCC v.11.2.0 was 
> 
>     4.3.48(1)-release (x86_64-suse-linux-gnu)

That's not really that old, why can't it be used?

It's newer than the bash-4.2.46-35.el7_9.x86_64 version in RHEL 7 and CentOS 7,
and they can build GCC without problems.

[Bug other/106429] Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[martin.vahi at softf1 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #5 from Martin Vahi <martin.vahi at softf1 dot com> ---
Created attachment 53385
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=53385&action=edit
Console Output of the Bash Version Related GCC Build Failure

    ...
    gmake[1]: Entering directory
'/opt/andmekettad/magnet_465GiB_01/home/ts3b/tmp_/test/demobuild/autogenerated_tmp/
 GCC_v_11_2_0/gcc/x86_64-linux-gnu/libgcc'
    /bin/bash: -/: invalid option
    Usage:  /bin/bash [GNU long option] [option] ...
            /bin/bash [GNU long option] [option] script-file ...
    GNU long options:
            --debug
            --debugger
            --dump-po-strings
    ...

[Bug other/106429] Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[martin.vahi at softf1 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #6 from Martin Vahi <martin.vahi at softf1 dot com> ---
Created attachment 53386
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=53386&action=edit
List of changes that allow temporary use of an alternative Bash binary.

Too old Bash version: 

    GNU bash, version 4.3.48(1)-release (x86_64-suse-linux-gnu)

Bash version that was tested to work after applying the changes to the GCC
code:

    GNU bash, version 5.1.0(1)-release (x86_64-pc-linux-gnu)

[Bug other/106429] Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #7 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Martin Vahi from comment #5)
> Created attachment 53385 [details]

This is a failure while "make clean".

And your patch does not even touch anything related to libgcc and "make clean".

So this is a different issue from what you reported.

[Bug other/106429] Building GCC is Inhibited on old Linux Distributions due to the use of "#!/bin/bash"

[martin.vahi at softf1 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #8 from Martin Vahi <martin.vahi at softf1 dot com> ---
Thank You for the answers/replies.
As of 2022_07_30 I am not familiar with the GCC source code, nor do I know the
GCC build system, but as someone, who can do configure-make-make_install, I
have the following observations:

    observation_0)
        The build fails with Bash 4.3.48, 
        as described at the console output at 
        one of the attached text-files.

    observation_1)
        If I use Bash 5.1.0 with the GCC code changes
        that are described at another attachment/txt/diff file,
        then the build completes without any errors that
        I would easily notice on console.

It's up to GCC developers to figure out, how things _should_ or _should_not_
be, but that's how I fixed it for myself and it would probably help others like
me, if they didn't have to re-do the text changes in the files listed at the
attached diff-file. Probably people smarter than me can figure out, which of
the changes are actually necessary or may be the smarter people, possibly You,
can come up with some different kind of code-change that just throws out the
use of Bash features that are present at the 5.1.0 and missing from the 4.3.48,
but there is the "risk" that at some point someone comes up with some code that
still uses some features that are specific to some newer version of Bash and
some people might use even older Bash versions than the 4.3.48.

The GCC code and build scripts that I used has been packaged to ~1.2GiB (~8GiB
after unpacking wrapped xz)

2022_07_24_created_GCC_bugreport_106429_optional_appendix_01_SHA256_68f6aa8ec04f4adbd31c89e35597b2dbbfce9718ec0b801c4a40ac2dd8ad1767.tar

which MIGHT be downloadable from 

    (Folder view.)
    https://1fichier.com/dir/uiRZKGsU

    (Direct link to tar-file.)
    https://1fichier.com/?qe2en1n9bh6ixl6669pi

Inside the tar-file the 

    ./demobuild/autogenerated_tmp/GCC_v_11_2_0/gcc

includes all files as they were/are right after the failed build. 

Actually the use case that I described here for custom-building GCC is just one
of the 2 use-cases that I have had. Initially I wrote the 

    ./demobuild/try_to_compile_it.bash

for building Linux kernel for the Windows Linux layer, the WSL. I know that the
way I use Git at
   
https://github.com/martinvahi/mmmv_notes/tree/master/mmmv_notes/usage_demos/WSL2_Microsoft_Kernel_installation_notes_t1/WSL1_2_WSL2_upgrade_option_02_hard_and_errorprone/src/GCC/mmmv_repackaging_GCC_v_11_2_0_WSL_AMD64_t1_v02.tar.xz.file_pieces

is plain horrible and that "no decent person does that", but I like to organize
projects into folders and that's how I could shove the whole GCC building
related part to my Git repository while keeping the number of files at my
repository relatively small.

Thank You for the answers/replies and thank You for reading my comment.

[Bug other/106429] make clean in libgcc fails

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|INVALID                     |---
             Status|RESOLVED                    |UNCONFIRMED
            Summary|Building GCC is Inhibited   |make clean in libgcc fails
                   |on old Linux Distributions  |
                   |due to the use of           |
                   |"#!/bin/bash"               |

--- Comment #9 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
As I mentioned this is unrelated to bash version.

"make clean" is failing so reopening for that issue.

clean in libgcc/Makefile.in is defined as:
clean:
        -rm -f libgcc_tm.h libgcc.map
        -rm -f libgcc_tm.stamp stmp-ldirs
        -rm -f *$(objext)
        -rm -f *.dep
        -rm -f *.a
        -rm -f libunwind$(SHLIB_EXT)
        -rm -f libgcc_s*
        @$(MULTICLEAN) multi-clean DO=clean

[Bug other/106429] make clean in libgcc fails

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #10 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
It should be noted almost nobody does "make clean" really. Most just remove the
build directory instead.
Is there a reason why you build scripts are doing "make clean"?

[Bug other/106429] make clean in libgcc fails

[martin.vahi at softf1 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #11 from Martin Vahi <martin.vahi at softf1 dot com> ---
(In reply to Andrew Pinski from comment #10)
> It should be noted almost nobody does "make clean" really. Most just remove
> the build directory instead.
> Is there a reason why you build scripts are doing "make clean"?

For me the "make clean" is a standard feature that is like the close button at
the corner of a window. For me, the fact that someone does not bother to use
the close button at the corner of the window is not an excuse, why the button
at the corner of the window should not work as expected. The same with the
"make clean": it is nice to do it automatically just to make sure that I won't
waste my human-life-minutes/hours on some triviality that comes from leaving
old build results laying around. 

Actually, with some projects, I do not remember, which, there was a distinction
between "make clean" and "make clear", where the "make clear" reset the source
tree to the state as it is expected to be after fresh checkout and the "make
clean" was allowed to not delete all build system generated files. For example,
different developers may use different hardware/build_machines, but between the
builds by a single developer on the same machine the build machine hardware
specification does not change, so hardware related information might be
"cached" by the build system. That is to say, the "make clean" might
intentionally not remove hardware related autodetected information and the
"make clear" would also delete hardware related autodetected information. 

Anyways, I understand that the "make clear" part might be a bit esoteric, but
in my view the "make clean" is mainstream enough to be expected to work without
crashing.

Thank You for the answer and thank You for reading my comment :-)

[Bug other/106429] make clean in libgcc fails

[martin.vahi at softf1 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #12 from Martin Vahi <martin.vahi at softf1 dot com> ---
(In reply to Andrew Pinski from comment #9)
> As I mentioned this is unrelated to bash version.
> 
> "make clean" is failing so reopening for that issue.
>... 

I'm not competent enough to refute Your statement, but I do emphasize that for
whatever reason the crash DID NOT HAPPEN with a newer version of Bash.

[Bug other/106429] make clean in libgcc fails

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #13 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Martin Vahi from comment #11)
> For me the "make clean" is a standard feature that is like the close button
> at the corner of a window. For me, the fact that someone does not bother to
> use the close button at the corner of the window is not an excuse, why the
> button at the corner of the window should not work as expected. The same
> with the "make clean": it is nice to do it automatically just to make sure
> that I won't waste my human-life-minutes/hours on some triviality that comes
> from leaving old build results laying around. 

So delete the whole build tree. Why would you want to keep an empty directory
around after running 'make clean'?


> 
> Actually, with some projects, I do not remember, which, there was a
> distinction between "make clean" and "make clear", where the "make clear"
> reset the source tree to the state as it is expected to be after fresh
> checkout and the "make clean" was allowed to not delete all build system
> generated files.

The GCC build should be done outside the source tree, and won't change anything
in the source tree anyway, so that distinction would be pointless. The source
tree is left clean, and the build tree can be removed after building.

I agree it shouldn't crash, but it also doesn't need to be used at all. It's
there because automake always adds a 'clean' target to makefiles, not because
it's useful.

[Bug other/106429] make clean in libgcc fails

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

--- Comment #14 from Jonathan Wakely <redi at gcc dot gnu.org> ---
A failure during 'make clean' certainly doesn't prevent building GCC on older
distributions as you initially claimed. That's just nonsense.

[Bug other/106429] make clean in libgcc fails

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

Eric Gallager <egallager at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |egallager at gcc dot gnu.org

--- Comment #15 from Eric Gallager <egallager at gcc dot gnu.org> ---
I'd just like to note that the "clean" target is one of the standard Makefile
targets that, according to the GNU Coding Standards, all GNU programs should
include in their Makefiles:
https://www.gnu.org/prep/standards/html_node/Standard-Targets.html#Standard-Targets

[Bug other/106429] make clean in libgcc fails

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106429

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |DUPLICATE

--- Comment #16 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Dup of bug 39933.

*** This bug has been marked as a duplicate of bug 39933 ***