toplevel configure hits sed program length limit on HP-UX

toplevel configure hits sed program length limit on HP-UX

[Ralf Wildenhues]

While still working to prove Bob wrong on the fixincludes sed issues,
I found out that current toplevel config.status hits the program length
limit of HP-UX 11.11 sed:

$ ./config.status
config.status: creating Makefile
sed: There are too many commands for the s&@abs_builddir@&/home/rwild/gcc/build-hppa2.0w-hp-hpux11.11&;t t function.

This is due to the large number of commands added to $extrasub in
configure.ac (around 80 in this case); the limit is (autoconf.info):

|                                       HP-UX sed has a limit of 99
|     commands (not counting `:' commands) and 48 labels, which can not
|     be circumvented by using more than one script file.  [...]

It seems to be possible for now to work around it by hacking Autoconf's
status.m4 to omit the 't t' commands:

$ diff -C 2 config.status config.status1
*** config.status       Mon Aug 31 18:41:02 2009
--- config.status       Mon Aug 31 18:40:49 2009
***************
*** 1079,1093 ****
  :t
  /@[a-zA-Z_][a-zA-Z_0-9]*@/!b
! s|@configure_input@|$ac_sed_conf_input|;t t
! s&@top_builddir@&$ac_top_builddir_sub&;t t
! s&@top_build_prefix@&$ac_top_build_prefix&;t t
! s&@srcdir@&$ac_srcdir&;t t
! s&@abs_srcdir@&$ac_abs_srcdir&;t t
! s&@top_srcdir@&$ac_top_srcdir&;t t
! s&@abs_top_srcdir@&$ac_abs_top_srcdir&;t t
! s&@builddir@&$ac_builddir&;t t
! s&@abs_builddir@&$ac_abs_builddir&;t t
! s&@abs_top_builddir@&$ac_abs_top_builddir&;t t
! s&@INSTALL@&$ac_INSTALL&;t t
  $ac_datarootdir_hack
  "
--- 1079,1093 ----
  :t
  /@[a-zA-Z_][a-zA-Z_0-9]*@/!b
! s|@configure_input@|$ac_sed_conf_input|
! s&@top_builddir@&$ac_top_builddir_sub&
! s&@top_build_prefix@&$ac_top_build_prefix&
! s&@srcdir@&$ac_srcdir&
! s&@abs_srcdir@&$ac_abs_srcdir&
! s&@top_srcdir@&$ac_top_srcdir&
! s&@abs_top_srcdir@&$ac_abs_top_srcdir&
! s&@builddir@&$ac_builddir&
! s&@abs_builddir@&$ac_abs_builddir&
! s&@abs_top_builddir@&$ac_abs_top_builddir&
! s&@INSTALL@&$ac_INSTALL&
  $ac_datarootdir_hack
  "


but in the long run (or short, if you add another couple of directories)
we might have to either
- require a better sed,
- split the script in two inside Autoconf (if $extrasub is nonempty),
- allow for extra sed scripts here.

Should I write a patch for omitting the 't t'?

Cheers,
Ralf

Re: toplevel configure hits sed program length limit on HP-UX

[Paolo Bonzini]

On 08/31/2009 08:54 PM, Ralf Wildenhues wrote:
> While still working to prove Bob wrong on the fixincludes sed issues,

Bob?

> - require a better sed,
> - split the script in two inside Autoconf (if $extrasub is nonempty),
> - allow for extra sed scripts here.

Like a new variable $ac_presub, that is executed as

   sed "$ac_presub" | sed "$extrasub; other config.status substitutions"

?  I think this is what I like the most.  Does the following plan make 
sense to you as Autoconf maintainer? (CCing autoconf@gnu.org).

The patch should be easy, so we can get ac_presub in 2.65 and upgrade 
GCC from 2.64 to 2.65 as soon as it gets out.  The rationale is that 
anyway 2.65 is going to have only minor bug fixes besides the two Erlang 
patches, some of them too invasive to place them in override.m4.

Regatding the timing of Autoconf 2.65 the DJGPP issue is pending, but it 
can probably be fixed only in DJGPP rather than in autoconf, so it 
wouldn't prevent releasing Autoconf 2.65 (in fact, having the 
possibility to mention it in the release notes would be a reason to 
release 2.65 sooner).

Paolo

Re: toplevel configure hits sed program length limit on HP-UX

[Ralf Wildenhues]

* Paolo Bonzini wrote on Mon, Aug 31, 2009 at 09:06:20PM CEST:
> On 08/31/2009 08:54 PM, Ralf Wildenhues wrote:
> >While still working to prove Bob wrong on the fixincludes sed issues,
> 
> Bob?

Bruce; sorry about that, Bruce!

> >- require a better sed,
> >- split the script in two inside Autoconf (if $extrasub is nonempty),
> >- allow for extra sed scripts here.
> 
> Like a new variable $ac_presub, that is executed as
> 
>   sed "$ac_presub" | sed "$extrasub; other config.status substitutions"
> 
> ?  I think this is what I like the most.

Yes, but it is still not extensible.  What if GCC needs three or more
scripts in the future?  Autoconf cannot split scripts, they might be
intertwined, or they might come from different third-party macros.
Granted, this whole concept is a bit flawed, and too low-level.

In fact, I think this API shouldn't be even more encouraged.  It doesn't
really fix things in an elegant way, and it doesn't help for other
pending issues in the GCC tree (such as the multilib fixups that aren't
applied in all cases; report coming up).

> Does the following plan
> make sense to you as Autoconf maintainer? (CCing autoconf@gnu.org).

Well maybe, but I'm not sure whether we want another rush, nor why we
need to decide this alongside with the extrasub issue.  override.m4
is also about relaxing release order requirements, no?

Cheers,
Ralf

Re: toplevel configure hits sed program length limit on HP-UX

[Jeff Law]

On 08/31/09 12:54, Ralf Wildenhues wrote:
> While still working to prove Bob wrong on the fixincludes sed issues,
> I found out that current toplevel config.status hits the program length
> limit of HP-UX 11.11 sed:
>
> $ ./config.status
> config.status: creating Makefile
> sed: There are too many commands for the s&@abs_builddir@&/home/rwild/gcc/build-hppa2.0w-hp-hpux11.11&;t t function.
>
> This is due to the large number of commands added to $extrasub in
> configure.ac (around 80 in this case); the limit is (autoconf.info):
>    
It's been the case for eons that various configury bits have blown out 
the hpux-sed.  I vaguely recall hacks in Cygnus's trees to split big sed 
scripts into smaller pieces -- but with  my hpux machines gathering 
layers of dust, I haven't kept a copy of the script.
jeff

Re: toplevel configure hits sed program length limit on HP-UX

[Paolo Bonzini]

> In fact, I think this API shouldn't be even more encouraged.  It doesn't
> really fix things in an elegant way, and it doesn't help for other
> pending issues in the GCC tree (such as the multilib fixups that aren't
> applied in all cases; report coming up).

I agree.  However, I did not have any alternative idea when I started 
using extrasub, and do not have one now. :-(

> Well maybe, but I'm not sure whether we want another rush, nor why we
> need to decide this alongside with the extrasub issue.  override.m4
> is also about relaxing release order requirements, no?

Yes, though I'd rather not put anything more than 15-20 lines long in 
override.m4.

Paolo

Re: toplevel configure hits sed program length limit on HP-UX

[Ralf Wildenhues]

Hello Jeff,

* Jeff Law wrote on Mon, Aug 31, 2009 at 10:00:57PM CEST:
> On 08/31/09 12:54, Ralf Wildenhues wrote:
> >I found out that current toplevel config.status hits the program length
> >limit of HP-UX 11.11 sed:

> >This is due to the large number of commands added to $extrasub in
> >configure.ac (around 80 in this case); the limit is (autoconf.info):
> It's been the case for eons that various configury bits have blown
> out the hpux-sed.

Configury bits of GCC and src projects, or of Autoconf or other
Autoconf-using projects?

>  I vaguely recall hacks in Cygnus's trees to split
> big sed scripts into smaller pieces -- but with  my hpux machines
> gathering layers of dust, I haven't kept a copy of the script.

Autoconf used to use sed for the bulk of the substitutions (it uses awk
now, which allows for better complexity), and it used to split its sed
scripts into chunks fit for HP-UX sed.

Cheers,
Ralf

Re: toplevel configure hits sed program length limit on HP-UX

[Ralf Wildenhues]

* Paolo Bonzini wrote on Mon, Aug 31, 2009 at 10:00:01PM CEST:
> >In fact, I think this API shouldn't be even more encouraged.  It doesn't
> >really fix things in an elegant way, and it doesn't help for other
> >pending issues in the GCC tree (such as the multilib fixups that aren't
> >applied in all cases; report coming up).
> 
> I agree.  However, I did not have any alternative idea when I
> started using extrasub, and do not have one now. :-(

Just use
  AC_CONFIG_COMMANDS([Makefile], [commands to fix up Makefile])

and do whatever $extrasub did yourself in those commands.