Re: stack grow direction wrongly detected

[gs-cygwin.com@gluelogic.com]

On Fri, Mar 05, 2021 at 11:31:04PM +0900, Takashi Yano via Cygwin wrote:
> On Fri, 5 Mar 2021 13:18:38 +0100
> Marco Atzeri wrote:
> > Hi Guys,
> > noted trying to rebuild guile 1.8.8.
> > 
> > The following piece of code in the past
> > was setting SCM_I_GSC_STACK_GROWS_UP=0
> > and now produces SCM_I_GSC_STACK_GROWS_UP=1
> > 
> > I assume some change in the gcc compiler is causing the issue.
> > I presume most of the programs and libraries do not care,
> > but some special one like guile crashes during build for this issue,
> > so be aware.
> > 
> > Regards
> > Marco
> > 
> > 
> > #--------------------------------------------------------------------
> > #
> > # Which way does the stack grow?
> > #
> > # Following code comes from Autoconf 2.61's internal _AC_LIBOBJ_ALLOCA
> > # macro (/usr/share/autoconf/autoconf/functions.m4).  Gnulib has
> > # very similar code, so in future we could look at using that.
> > #
> > # An important detail is that the code involves find_stack_direction
> > # calling _itself_ - which means that find_stack_direction (or at
> > # least the second find_stack_direction() call) cannot be inlined.
> > # If the code could be inlined, that might cause the test to give
> > # an incorrect answer.
> > #--------------------------------------------------------------------
> > 
> > SCM_I_GSC_STACK_GROWS_UP=0
> > AC_RUN_IFELSE([AC_LANG_SOURCE(
> > [AC_INCLUDES_DEFAULT
> > int
> > find_stack_direction ()
> > {
> >    static char *addr = 0;
> >    auto char dummy;
> >    if (addr == 0)
> >      {
> >        addr = &dummy;
> >        return find_stack_direction ();
> >      }
> >    else
> >      return (&dummy > addr) ? 1 : -1;
> > }
> > 
> > int
> > main ()
> > {
> >    return find_stack_direction () < 0;
> > }])],
> >                 [SCM_I_GSC_STACK_GROWS_UP=1],
> >                 [],
> >                 [AC_MSG_WARN(Guessing that stack grows down -- see 
> > scmconfig.h)])
> 
> This seems to be a result of optimization. With gcc v10.2.0,
> the return value of the code is:
> -O0: 1
> -O1: 1
> -O2: 0
> -O3: 1
> -O4: 1
> 
> If find_stack_direction() is implemented as recursive call,
> and auto variable is allocated in the stack every time,
> in the first call, addr is initialized to the first stack
> position, and in the second call, second address of dummy
> is reduced because stack of x86 is reverse direction.
> Therefore (&dummy > addr) ? 1 : -1; returns -1.
> As a result, the return value find_stack_direction() < 0
> is 1. With -O0 or -O1 this implemented as recursive call,
> so the return value is 1.

If the compiler is gcc or clang:

__attribute__(__noinline__)
int
find_stack_direction ()
{
...
}

Cheers, Glenn