gdb and ancient GNU autotools

gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]

Hi,

In gdb source tree in config/override.m4 is hardcoded autoconf version

m4_if(m4_defn([m4_PACKAGE_VERSION]), [2.64],

That version has been released almost SIXTEEN YEARS AGO !!!
https://savannah.gnu.org/news/?id=5867

Literally NONE Linux distribution currently provides that version so in
case of any necessity to fix something in autoconf it is NOT POSSIBLE to
regenerate GNU autotools files.

The same situation is with binutils ang gcc.

Someone has been thinking about start update all that stuff to be able to
use the latest GNU autotools and/or start moving to cmake or meson (IMO
mesoon is better) arch/OS one by one (still living for other not migrated
archs/OSsess autotools until last arch/OS still in use will be not migrated
or abandoned)?🤔

kloczek
-- 
Tomasz Kłoczko | LinkedIn: *http://lnkd.in/FXPWxH <http://lnkd.in/FXPWxH>*

Re: gdb and ancient GNU autotools

[Sam James]

Tomasz Kłoczko via Gdb <gdb@sourceware.org> writes:

> Hi,
>
> In gdb source tree in config/override.m4 is hardcoded autoconf version
>
> m4_if(m4_defn([m4_PACKAGE_VERSION]), [2.64],
>
> That version has been released almost SIXTEEN YEARS AGO !!!
> https://savannah.gnu.org/news/?id=5867

That specific line is adding a workaround for a bug in GNU Autoconf
2.64. It does not require 2.64.

However, yes, _GCC_AUTOCONF_VERSION is indeed later set to 2.69 and
will bail if *that* version isn't used. It would be worth updating
to autoconf-2.72 but it needs to be coordinated across binutils, gdb,
and gcc.

>
> Literally NONE Linux distribution currently provides that version so in
> case of any necessity to fix something in autoconf it is NOT POSSIBLE to
> regenerate GNU autotools files.
>

This is wrong, per above.

> The same situation is with binutils ang gcc.

Yes, they share the toplevel build system.

Re: gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 9429 bytes --]

On Sat, 24 Feb 2024 at 16:53, Sam James <sam@gentoo.org> wrote:
[..]

> > Literally NONE Linux distribution currently provides that version so in
> > case of any necessity to fix something in autoconf it is NOT POSSIBLE to
> > regenerate GNU autotools files.
> >
>
> This is wrong, per above.
>

Did you try to make a simple experiment to clone for example gdb repo and
execute "autoreconf -fiv"?

I've done that with autoconf 2.72, automake 1.16 and libtool 2.4.7 and here
is the result:
```
+ cd gdb-14.1
+ autoreconf -fiv
autoreconf: export WARNINGS=
autoreconf: Entering directory '.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: configure.ac: not using Intltool
autoreconf: configure.ac: not using Gtkdoc
autoreconf: running: /usr/bin/autoconf --force
configure.ac:1424: warning: The macro 'AC_CHECK_TOOL_PREFIX' is obsolete.
configure.ac:1424: You should run autoupdate.
./lib/autoconf/programs.m4:184: AC_CHECK_TOOL_PREFIX is expanded from...
config/acx.m4:391: ACX_PROG_GNAT is expanded from...
configure.ac:1424: the top level
configure.ac:1683: warning: The macro 'AC_TRY_COMPILE' is obsolete.
configure.ac:1683: You should run autoupdate.
./lib/autoconf/general.m4:2845: AC_TRY_COMPILE is expanded from...
lib/m4sugar/m4sh.m4:697: AS_IF is expanded from...
./lib/autoconf/general.m4:2821: _AC_COMPILE_IFELSE is expanded from...
./lib/autoconf/general.m4:2837: AC_COMPILE_IFELSE is expanded from...
configure.ac:1683: the top level
configure.ac:1701: warning: The macro 'AC_TRY_COMPILE' is obsolete.
configure.ac:1701: You should run autoupdate.
./lib/autoconf/general.m4:2845: AC_TRY_COMPILE is expanded from...
lib/m4sugar/m4sh.m4:697: AS_IF is expanded from...
./lib/autoconf/general.m4:2821: _AC_COMPILE_IFELSE is expanded from...
./lib/autoconf/general.m4:2837: AC_COMPILE_IFELSE is expanded from...
configure.ac:1701: the top level
configure.ac:1718: warning: The macro 'AC_TRY_COMPILE' is obsolete.
configure.ac:1718: You should run autoupdate.
./lib/autoconf/general.m4:2845: AC_TRY_COMPILE is expanded from...
lib/m4sugar/m4sh.m4:697: AS_IF is expanded from...
./lib/autoconf/general.m4:2821: _AC_COMPILE_IFELSE is expanded from...
./lib/autoconf/general.m4:2837: AC_COMPILE_IFELSE is expanded from...
configure.ac:1718: the top level
configure.ac:1888: warning: The macro 'AC_TRY_LINK' is obsolete.
configure.ac:1888: You should run autoupdate.
./lib/autoconf/general.m4:2918: AC_TRY_LINK is expanded from...
config/isl.m4:98: ISL_CHECK_VERSION is expanded from...
configure.ac:1888: the top level
autoreconf: configure.ac: not using Autoheader
autoreconf: configure.ac: not using Automake
autoreconf: './config.sub' is updated
autoreconf: './config.guess' is updated
autoreconf: './install-sh' is updated
autoreconf: Leaving directory '.'
+ /usr/bin/mkdir full minimal
+ cd full
+ ASMFLAGS='-m64 -mtune=generic -fasynchronous-unwind-tables
-fstack-clash-protection -fcf-protection -O2 -g -grecord-gcc-switches -pipe
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer
-mno-omit-leaf-frame-pointer -fdata-sections -ffunction-sections
-fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-flto=auto -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3
-Wp,-D_GLIBCXX_ASSERTIONS -Wall -Werror=format-security'
+ CFLAGS='-m64 -mtune=generic -fasynchronous-unwind-tables
-fstack-clash-protection -fcf-protection -O2 -g -grecord-gcc-switches -pipe
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer
-mno-omit-leaf-frame-pointer -fdata-sections -ffunction-sections
-fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-flto=auto -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3
-Wp,-D_GLIBCXX_ASSERTIONS -Wall -Werror=format-security '
+ CXXFLAGS='-m64 -mtune=generic -fasynchronous-unwind-tables
-fstack-clash-protection -fcf-protection -O2 -g -grecord-gcc-switches -pipe
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer
-mno-omit-leaf-frame-pointer -fdata-sections -ffunction-sections
-fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-flto=auto -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3
-Wp,-D_GLIBCXX_ASSERTIONS -Wall -Werror=format-security
-DDNF_DEBUGINFO_INSTALL -DNEED_DETACH_SIGSTOP'
+ FFLAGS='-m64 -mtune=generic -fasynchronous-unwind-tables
-fstack-clash-protection -fcf-protection -O2 -g -grecord-gcc-switches -pipe
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer
-mno-omit-leaf-frame-pointer -fdata-sections -ffunction-sections
-fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-flto=auto -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3
-Wp,-D_GLIBCXX_ASSERTIONS -Wall -Werror=format-security
-I/usr/lib64/gfortran/modules'
+ FCFLAGS='-m64 -mtune=generic -fasynchronous-unwind-tables
-fstack-clash-protection -fcf-protection -O2 -g -grecord-gcc-switches -pipe
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer
-mno-omit-leaf-frame-pointer -fdata-sections -ffunction-sections
-fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-flto=auto -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3
-Wp,-D_GLIBCXX_ASSERTIONS -Wall -Werror=format-security
-I/usr/lib64/gfortran/modules'
+ LDFLAGS='-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--gc-sections
-Wl,--as-needed -Wl,--build-id=sha1 -Wl,-z,now
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-z,pack-relative-relocs
-flto=auto -fuse-linker-plugin  -DDNF_DEBUGINFO_INSTALL
-DNEED_DETACH_SIGSTOP'
+ RUSTFLAGS='-C codegen-units=1 -C debuginfo=2 -C opt-level=2 -C
link-arg=-Wl,--as-needed -C link-arg=-Wl,--build-id=sha1 -C
link-arg=-Wl,-z,now -C
link-arg=-specs=/usr/lib/rpm/redhat/redhat-hardened-ld -C
link-arg=-Wl,-z,pack-relative-relocs -C link-arg=-Wl,-z,relro -C
link-arg=-flto=auto --cap-lints=warn'
+ VALAFLAGS=-g
+ CC=/usr/bin/gcc
+ CXX=/usr/bin/g++
+ FC=/usr/bin/gfortran
+ AR=/usr/bin/gcc-ar
+ NM=/usr/bin/gcc-nm
+ RANLIB=/usr/bin/gcc-ranlib
+ export ASMFLAGS CFLAGS CXXFLAGS FFLAGS FCFLAGS LDFLAGS VALAFLAGS CC CXX
FC AR NM RANLIB RUSTFLAGS VALAFLAGS
+ ../configure --bindir=/usr/bin --build=x86_64-redhat-linux
--datadir=/usr/share --disable-dependency-tracking --exec-prefix=/usr
--host=x86_64-redhat-linux --includedir=/usr/include
--infodir=/usr/share/info --libdir=/usr/lib64 --libexecdir=/usr/libexec
--localstatedir=/var --mandir=/usr/share/man --prefix=/usr
--program-prefix= --runstatedir=/run --sbindir=/usr/sbin
--sharedstatedir=/var/lib --sysconfdir=/etc --disable-libmcheck
--disable-rpath --disable-sim --enable-64-bit-bfd --enable-build-with-cxx
--enable-targets=s390-linux-gnu,powerpc-linux-gnu,arm-linux-gnu,aarch64-linux-gnu
--enable-tui --enable-unit-tests --sysconfdir=/etc
'--with-auto-load-dir=$debugdir:$datadir/auto-load/usr/share/gdb/auto-load'
'--with-auto-load-safe-path=$debugdir:$datadir/auto-load/usr/share/gdb/auto-load'
--with-babeltrace --with-debuginfod --with-expat
--with-gdb-datadir=/usr/share/gdb --with-lzma --without-guile
--without-libexpat-prefix --without-libunwind --without-mpfr
--without-stage1-ldflags --with-python=/usr/bin/python3
--with-rpm=librpm.so.10 --with-separate-debug-dir=/usr/lib/debug
--with-system-gdbinit=/etc/gdbinit --with-system-readline
--with-system-zlib --with-xxhash --disable-werror --disable-inprocess-agent
--with-intel-pt
checking build system type... x86_64-redhat-linux-gnu
checking host system type... x86_64-redhat-linux-gnu
checking target system type... x86_64-redhat-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln works... yes
checking whether ln -s works... yes
checking for a sed that does not truncate output... /usr/bin/sed
checking for gawk... gawk
checking for gdbserver support... yes
checking for x86_64-redhat-linux-gcc... /usr/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether /usr/bin/gcc accepts -g... yes
checking for /usr/bin/gcc option to enable C11 features... none needed
checking whether the compiler supports GNU C++... yes
checking whether /usr/bin/g++ accepts -g... yes
checking for /usr/bin/g++ option to enable C++11 features... none needed
checking whether g++ accepts -static-libstdc++ -static-libgcc... no
checking for x86_64-redhat-linux-gnatbind... no
checking for gnatbind... no
checking for x86_64-redhat-linux-gnatmake... no
checking for gnatmake... no
checking whether compiler driver understands Ada and is recent enough... no
checking for x86_64-redhat-linux-gdc... no
checking for gdc... no
checking whether the D compiler works... no
checking how to compare bootstrapped objects... cmp --ignore-initial=16
$$f1 $$f2
checking for objdir... .libs
../configure: line 9178: syntax error near unexpected token `,'
../configure: line 9178: `  _au_m4_changequote(,)cat confdefs.h - <<_ACEOF
>conftest.$ac_ext'
```

And autoconf 2.69 has been released (sic!) ONLY TWELVE years ago ..

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Re: gdb and ancient GNU autotools

[Sam James]

Tomasz Kłoczko <kloczko.tomasz@gmail.com> writes:

> On Sat, 24 Feb 2024 at 16:53, Sam James <sam@gentoo.org> wrote:
> [..]
>
>  > Literally NONE Linux distribution currently provides that version so in
>  > case of any necessity to fix something in autoconf it is NOT POSSIBLE to
>  > regenerate GNU autotools files.
>  >
>
>  This is wrong, per above.
>
> Did you try to make a simple experiment to clone for example gdb repo and execute "autoreconf -fiv"?

You misread what I said. I was saying that it does not need
autoconf-2.64.

>
> I've done that with autoconf 2.72, automake 1.16 and libtool 2.4.7 and here is the result:
> [...]>
> And autoconf 2.69 has been released (sic!) ONLY TWELVE years ago ..
>

Again, I actually agree with needing to move to 2.72. But you've misread
my previous message.

Also note that while autoconf-2.69 is old, it took a long time for
autoconf-2.7{0,1} to be released, so it's not quite a fair comparison.

Re: gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 2787 bytes --]

On Sat, 24 Feb 2024 at 17:22, Sam James <sam@gentoo.org> wrote:

> Again, I actually agree with needing to move to 2.72. But you've misread
> my previous message.
>

Yep. My fault.
Sorry.

Also note that while autoconf-2.69 is old, it took a long time for
> autoconf-2.7{0,1} to be released, so it's not quite a fair comparison.
>

Latest one is 2.72 .. released +3 months ago (which in the context of GNU
projects still is cadence below Palnck time).

Generally the issue is that maintaining anything placed on gnu.org is like
dropping something into a black hole. For observers from outside of
that hole stuff moves but slower and slower.
This is partially in context of gcc/binutils/gdb as well. Just check any OS
distro source packages and in each of these distros you can find tenths if
not sometimes hundreds of patches taken from random places fixing different
JFDI issues.
Additionally main git repos all of those projects on
https://sourceware.org/git/ are served from http server which (time to
time) throttles the number of requests/s which sometimes blocks
completely automatic downloading of the patches from exact commits over
https:// interface during automated builds.

Why those repos are not moved yet to github or even own instance of the
gitlab to provide OOTB proper cross refs between commits, discussions
MRs/PRs, issue tickets and commit comments for me is kind of a mystery ..
(IMO change that would allow move saved paid man/h resources from
maintaining that platform to something more productive like review tickets)

All tooling around constantly improves allowing speeding up of the
development processes of many projects and sticking with old platforms is
nothing more than just discouraging to even try to touch something trivial.
Sorry for maybe a few bitter words but just please look at LLVM stack.
Thay have a lot of problems but at least from that angle it is
squeaky clean.
(just please do not comment above because it is kind of rhetorical part and
focus on gdb subject)

Going back to the subject: someone at least started thinking about starting
doing something to move away from those ancient GNU autotools versions
(probably kicking a few as*ess on integrating properly some
libtool changes) or move to meson/cmake?
IMO move to meson/cmake has that advantage that would allow to stop
worrying about proper build automation features/shape necessary for
gdb/gcc/binutils as those projects have enough robust maintainers.

Someone at least made the strategic decision what should happen in that
area?🤔
And/or who has any partial work done going towards that target?
And/or where are those changes maintained/available?🤔

kloczek
-- 
Tomasz Kłoczko | inkedIn: http://lnkd.in/FXPWxH

Re: gdb and ancient GNU autotools

[Mark Wielaard]

Hi Tomasz,

On Sat, Feb 24, 2024 at 06:30:42PM +0000, Tomasz Kłoczko via Gdb wrote:
> Additionally main git repos all of those projects on
> https://sourceware.org/git/ are served from http server which (time to
> time) throttles the number of requests/s which sometimes blocks
> completely automatic downloading of the patches from exact commits over
> https:// interface during automated builds.

How exactly are you downloading the patches? You really should not
download them through the gitweb interface. Please just use git,
either over https or git. Or if you really want to use a web interface
please use the cgit interface at https://sourceware.org/cgit/ which is
much faster than the gitweb interface. If you do have trouble doing
checkouts for your automated builds and are worried that your
connection is trapped in a fail2ban jail please reach out so we can
investigate. https://sourceware.org/mission.html#organization

Cheers,

Mark

Re: gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 7070 bytes --]

On Sat, 24 Feb 2024 at 19:31, Mark Wielaard <mark@klomp.org> wrote:
[..]

> How exactly are you downloading the patches? You really should not
> download them through the gitweb interface. Please just use git,
> either over https or git.


In both cases content is served by http server.

Or if you really want to use a web interface
> please use the cgit interface at https://sourceware.org/cgit/ which is
> much faster than the gitweb interface. If you do have trouble doing
> checkouts for your automated builds and are worried that your
> connection is trapped in a fail2ban jail please reach out so we can
> investigate. https://sourceware.org/mission.html#organization


That cgit interface content is served by the same web server which is
throttling the rate of the requests.

OK I'll give you the full picture ..to show how deep that rabbit hole is.
FYI: I'm doin that because in rpm spec file is possible to specify
something like below:

Name:           mate-utils
Version:        1.28.0
Release:        2%{?dist}
License:        GPL-3.0-or-later (
https://spdx.org/licenses/GPL-3.0-or-later.htm), LGPL-2.0-or-later (
https://spdx.org/licenses/LGPL-2.0-or-later.html)
URL:            https://mate-desktop.org/
VCS:            https://github.com/mate-desktop/mate-utils/
Source:         https://pub.mate-desktop.org/releases/%(v=%{version}; echo
${v%.*})/%{name}-%{version}.tar.xz
Patch:
 %{VCS}/pull/363.patch#/%{name}-build-mate-dictionary-loadable-module-as-not-versioned-DSO.patch

Or other example:

Name:           gsettings-desktop-schemas
Version:        46.beta
Release:        3%{?dist}
License:        LGPL-2.1-or-later (
https://spdx.org/licenses/LGPL-2.1-or-later.html)
URL:            https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/
VCS:            https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/
Source:
https://download.gnome.org/sources/gsettings-desktop-schemas/%(v=%{version};
echo ${v/.*/})/%{name}-%{version}.tar.xz
Patch:
 %{VCS}/commit/3537f5ab.patch#/%{name}-Update-Georgian-translation.patch
Patch:
 %{VCS}/commit/ea06d947.patch#/%{name}-Update-Turkish-translation.patch
Patch:
 %{VCS}/commit/b3ebfbc4.patch#/%{name}-Update-Ukrainian-translation.patch
Patch:
 %{VCS}/commit/9c7a53ed.patch#/%{name}-Update-Czech-translation.patch
Patch:
 %{VCS}/commit/cb370919.patch#/%{name}-Update-Hebrew-translation.patch
Patch:
 %{VCS}/commit/7c7a2136.patch#/%{name}-Update-Norwegian-Bokm-l-translation.patch
Patch:
 %{VCS}/commit/63869de8.patch#/%{name}-Update-Turkish-translation-1.patch
Patch:
 %{VCS}/commit/7e238e6c.patch#/%{name}-Update-Slovenian-translation.patch
Patch:
 %{VCS}/commit/aa2d88f0.patch#/%{name}-Change-default-clock-format-back-to-24h-for-non-US-l.patch
Patch:
 %{VCS}/commit/9c9d4c89.patch#/%{name}-Update-Persian-translation.patch
Patch:
 %{VCS}/commit/5e674865.patch#/%{name}-Update-Persian-translation-1.patch
Patch:
 %{VCS}/commit/10aa69e8.patch#/%{name}-Update-Ukrainian-translation-1.patch
Patch:
 %{VCS}/commit/57282960.patch#/%{name}-Update-Hebrew-translation=1.patch
Patch:
 %{VCS}/commit/dd6b6341.patch#/%{name}-Update-Indonesian-translation-1.patch
Patch:
 %{VCS}/commit/63264176.patch#/%{name}-Update-Slovenian-translation-1.patch
Patch:
 %{VCS}/commit/f36769de.patch#/%{name}-Update-Turkish-translation-2.patch
Patch:
 %{VCS}/commit/0ddb6985.patch#/%{name}-Update-Galician-translation.patch
Patch:
 %{VCS}/commit/4d67a8a5.patch#/%{name}-Update-Basque-translation.patch
Patch:
 %{VCS}/commit/04ffad13.patch#/%{name}-Update-Occitan-translation.patch

Which allows me easy integrate patches taken from commits added after
release %{version} directly over VCS REST interface.

For example elfutils or dwz quite often after release in some commits are
fixing some important issues and maintainers are not willing to release new
version ASP.

Simple 'rpmbuild -bs -D "_disable_source_fetch 0"
gsettings-desktop-schemas.spec --clean --rmsource --rmspec --nodeps' allows
quickly form src.rpm on the system which has access to the public network
which in next step is sent to the prod build env spawned only to build
single package which is cut off from access to the public network.
If that list of patches downloaded over VCS URLs is long enough it exceeds
the throttled rate and .. some patches are not downloaded.

Web server serving web content of the https://sourceware.org/git/ is ONLY
known to me such a server which has such throttling.
Literally NONE of other places with VCS REST interfaces are using such
fascist settings .. IIRC that throttling is something like 25-30 per MINUTE
which is b*dy easy to exceed.

Some samples of use such technique from my packages:

[tkloczko@pers-jacek SPECS]$ grep Patch:.*VCS *spec -c | sort -t: -nk 2 |
tail -n 5
mdadm.spec:125
groff.spec:129
qt5-qtbase.spec:148
libiscsi.spec:187
libimobiledevice.spec:190

And yet another stat ..

[tkloczko@pers-jacek SPECS]$ ls -1 *spec | wc -l; grep Patch:.*VCS *spec|
wc -l
4829
5829

In other words: avg it is more than one VCS based patch taken out of some
commits *per package* in that +4.8k packages population.

As you see I'm using that on a massive scale. With any other VCS REST
interfaces there is no ANY problem .. except with
https://sourceware.org/git/ and I'm using different VCSess in case of +90%
of all my packages

[tkloczko@pers-jacek SPECS]$ grep ^VCS: *spec| wc -l
4444

.. and that VCS: field sits in each of those specs files to be instantly
used if it is needed.
Above technique glued with some one line generator which generates those
Patch: lines allows me to save TONS of time to keep updated packages
downloading all of what is needed in *seconds* ..
Some packages only to test them in advance are using gitlab/github email
notifications processed over procmail to test build after EACH commit made
by maintainers (+ batch of tests to check that produced packages are still
OK and warn me over zabbix that something is wrong).
Above gives the best possible VISIBILITY of what has been changed in all
those commits as well (you can use for that for example mc to manually
check what has been changed in each commit .. offline).
Instead of chasing bugs using git bisect it is easier to have an instant
signal that after commit <hash> something went wrong.

If you may be thinking about security aspects of downloading all those
patches in the batch mode .. github and gitlab allow you to sign commits or
git tags.
On downloading patches out of commits it is possible to verify
those signatures (it takes one line of POSIX sh script to do that).
Percentage of projects especially on github in recent months which
are using signet commits started going up.

Do I need to mention that cgit from https://sourceware.org/git/ do not
offer such additional functionality and will probably never be able to do
that?🤔

---
Any comments about the main topic?🤔

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Re: gdb and ancient GNU autotools

[Mark Wielaard]

Hi Tomasz,

On Sun, Feb 25, 2024 at 12:22:49AM +0000, Tomasz Kłoczko wrote:
> On Sat, 24 Feb 2024 at 19:31, Mark Wielaard <mark@klomp.org> wrote:
> [..]
> 
> > How exactly are you downloading the patches? You really should not
> > download them through the gitweb interface. Please just use git,
> > either over https or git.
> 
> In both cases content is served by http server.

But they use different backends, the gitweb interface is very
inefficient, the cgit one is much more efficient.

> Or if you really want to use a web interface
> > please use the cgit interface at https://sourceware.org/cgit/ which is
> > much faster than the gitweb interface. If you do have trouble doing
> > checkouts for your automated builds and are worried that your
> > connection is trapped in a fail2ban jail please reach out so we can
> > investigate. https://sourceware.org/mission.html#organization
> 
> That cgit interface content is served by the same web server which is
> throttling the rate of the requests.

I don't believe it is. If you believe you are seeing throttling when
using the cgit interface could you please report, so we can
investigate what is going on?

> OK I'll give you the full picture ..to show how deep that rabbit hole is.
> FYI: I'm doin that because in rpm spec file is possible to specify
> something like below:
> 
> Name:           mate-utils
> Version:        1.28.0
> Release:        2%{?dist}
> License:        GPL-3.0-or-later (
> https://spdx.org/licenses/GPL-3.0-or-later.htm), LGPL-2.0-or-later (
> https://spdx.org/licenses/LGPL-2.0-or-later.html)
> URL:            https://mate-desktop.org/
> VCS:            https://github.com/mate-desktop/mate-utils/
> Source:         https://pub.mate-desktop.org/releases/%(v=%{version}; echo
> ${v%.*})/%{name}-%{version}.tar.xz
> Patch:
>  %{VCS}/pull/363.patch#/%{name}-build-mate-dictionary-loadable-module-as-not-versioned-DSO.patch

Right. Do you have an example that uses sourceware.org?

If you have seen any issues using the gitweb interface at
https://sourceware.org/git/project.git... you might simply want to
replace them with https://sourceware.org/cgit/project/tree/... If you
have trouble with the transformation we can show you a perl script
that does it automatically.

> Which allows me easy integrate patches taken from commits added after
> release %{version} directly over VCS REST interface.
> 
> For example elfutils or dwz quite often after release in some commits are
> fixing some important issues and maintainers are not willing to release new
> version ASP.
> 
> Simple 'rpmbuild -bs -D "_disable_source_fetch 0"
> gsettings-desktop-schemas.spec --clean --rmsource --rmspec --nodeps' allows
> quickly form src.rpm on the system which has access to the public network
> which in next step is sent to the prod build env spawned only to build
> single package which is cut off from access to the public network.
> If that list of patches downloaded over VCS URLs is long enough it exceeds
> the throttled rate and .. some patches are not downloaded.
> 
> Web server serving web content of the https://sourceware.org/git/ is ONLY
> known to me such a server which has such throttling.
> Literally NONE of other places with VCS REST interfaces are using such
> fascist settings .. IIRC that throttling is something like 25-30 per MINUTE
> which is b*dy easy to exceed.

I am not aware of such throttling, except for server load. And as said
above, using the cgit interface would avoid any such server load
issues.

BTW. Please try to be civil, the admins are not fascists. They simply
try to help the community and users make most efficient use of the
resources.

> Some samples of use such technique from my packages:
> 
> [tkloczko@pers-jacek SPECS]$ grep Patch:.*VCS *spec -c | sort -t: -nk 2 |
> tail -n 5
> mdadm.spec:125
> groff.spec:129
> qt5-qtbase.spec:148
> libiscsi.spec:187
> libimobiledevice.spec:190
> 
> And yet another stat ..
> 
> [tkloczko@pers-jacek SPECS]$ ls -1 *spec | wc -l; grep Patch:.*VCS *spec|
> wc -l
> 4829
> 5829
> 
> In other words: avg it is more than one VCS based patch taken out of some
> commits *per package* in that +4.8k packages population.
> 
> As you see I'm using that on a massive scale. With any other VCS REST
> interfaces there is no ANY problem .. except with
> https://sourceware.org/git/ and I'm using different VCSess in case of +90%
> of all my packages
> 
> [tkloczko@pers-jacek SPECS]$ grep ^VCS: *spec| wc -l
> 4444
> 
> .. and that VCS: field sits in each of those specs files to be instantly
> used if it is needed.

Please show some that use sourceware.org, so we can investigate and
see if changing from /git/ to /cgit/ helps your use case.

> If you may be thinking about security aspects of downloading all those
> patches in the batch mode .. github and gitlab allow you to sign commits or
> git tags.

So does sourceware of course. Although not many projects are using
that yet.

> Do I need to mention that cgit from https://sourceware.org/git/ do not
> offer such additional functionality and will probably never be able to do
> that?🤔

gitweb sits at https://sourceware.org/git/ (and the plain git https
backend). cgit sits at https://sourceware.org/cgit/ git signing is
independent from the backend used.

An explanation of different ways to sign your git commits (on
sourceware) can be found here:
https://inbox.sourceware.org/overseers/ZJ3Tihvu6GbOb8%2FR@elastic.org/

Cheers,

Mark

Re: gdb and ancient GNU autotools

[Eli Schwartz]

[-- Attachment #1.1.1: Type: text/plain, Size: 9298 bytes --]

On 2/24/24 7:22 PM, Tomasz Kłoczko wrote:
> On Sat, 24 Feb 2024 at 19:31, Mark Wielaard <mark@klomp.org> wrote:
> [..]
> 
>> How exactly are you downloading the patches? You really should not
>> download them through the gitweb interface. Please just use git,
>> either over https or git.
> 
> 
> In both cases content is served by http server.
> 
> Or if you really want to use a web interface
>> please use the cgit interface at https://sourceware.org/cgit/ which is
>> much faster than the gitweb interface. If you do have trouble doing
>> checkouts for your automated builds and are worried that your
>> connection is trapped in a fail2ban jail please reach out so we can
>> investigate. https://sourceware.org/mission.html#organization
> 
> 
> That cgit interface content is served by the same web server which is
> throttling the rate of the requests.
> 
> OK I'll give you the full picture ..to show how deep that rabbit hole is.
> FYI: I'm doin that because in rpm spec file is possible to specify
> something like below:
> 
> Name:           mate-utils
> Version:        1.28.0
> Release:        2%{?dist}
> License:        GPL-3.0-or-later (
> https://spdx.org/licenses/GPL-3.0-or-later.htm), LGPL-2.0-or-later (
> https://spdx.org/licenses/LGPL-2.0-or-later.html)
> URL:            https://mate-desktop.org/
> VCS:            https://github.com/mate-desktop/mate-utils/
> Source:         https://pub.mate-desktop.org/releases/%(v=%{version}; echo
> ${v%.*})/%{name}-%{version}.tar.xz
> Patch:
>  %{VCS}/pull/363.patch#/%{name}-build-mate-dictionary-loadable-module-as-not-versioned-DSO.patch
> 
> Or other example:
> 
> Name:           gsettings-desktop-schemas
> Version:        46.beta
> Release:        3%{?dist}
> License:        LGPL-2.1-or-later (
> https://spdx.org/licenses/LGPL-2.1-or-later.html)
> URL:            https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/
> VCS:            https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/
> Source:
> https://download.gnome.org/sources/gsettings-desktop-schemas/%(v=%{version};
> echo ${v/.*/})/%{name}-%{version}.tar.xz
> Patch:
>  %{VCS}/commit/3537f5ab.patch#/%{name}-Update-Georgian-translation.patch
> Patch:
>  %{VCS}/commit/ea06d947.patch#/%{name}-Update-Turkish-translation.patch
> Patch:
>  %{VCS}/commit/b3ebfbc4.patch#/%{name}-Update-Ukrainian-translation.patch
> Patch:
>  %{VCS}/commit/9c7a53ed.patch#/%{name}-Update-Czech-translation.patch
> Patch:
>  %{VCS}/commit/cb370919.patch#/%{name}-Update-Hebrew-translation.patch
> Patch:
>  %{VCS}/commit/7c7a2136.patch#/%{name}-Update-Norwegian-Bokm-l-translation.patch
> Patch:
>  %{VCS}/commit/63869de8.patch#/%{name}-Update-Turkish-translation-1.patch
> Patch:
>  %{VCS}/commit/7e238e6c.patch#/%{name}-Update-Slovenian-translation.patch
> Patch:
>  %{VCS}/commit/aa2d88f0.patch#/%{name}-Change-default-clock-format-back-to-24h-for-non-US-l.patch
> Patch:
>  %{VCS}/commit/9c9d4c89.patch#/%{name}-Update-Persian-translation.patch
> Patch:
>  %{VCS}/commit/5e674865.patch#/%{name}-Update-Persian-translation-1.patch
> Patch:
>  %{VCS}/commit/10aa69e8.patch#/%{name}-Update-Ukrainian-translation-1.patch
> Patch:
>  %{VCS}/commit/57282960.patch#/%{name}-Update-Hebrew-translation=1.patch
> Patch:
>  %{VCS}/commit/dd6b6341.patch#/%{name}-Update-Indonesian-translation-1.patch
> Patch:
>  %{VCS}/commit/63264176.patch#/%{name}-Update-Slovenian-translation-1.patch
> Patch:
>  %{VCS}/commit/f36769de.patch#/%{name}-Update-Turkish-translation-2.patch
> Patch:
>  %{VCS}/commit/0ddb6985.patch#/%{name}-Update-Galician-translation.patch
> Patch:
>  %{VCS}/commit/4d67a8a5.patch#/%{name}-Update-Basque-translation.patch
> Patch:
>  %{VCS}/commit/04ffad13.patch#/%{name}-Update-Occitan-translation.patch
> 
> Which allows me easy integrate patches taken from commits added after
> release %{version} directly over VCS REST interface.
> 
> For example elfutils or dwz quite often after release in some commits are
> fixing some important issues and maintainers are not willing to release new
> version ASP.
> 
> Simple 'rpmbuild -bs -D "_disable_source_fetch 0"
> gsettings-desktop-schemas.spec --clean --rmsource --rmspec --nodeps' allows
> quickly form src.rpm on the system which has access to the public network
> which in next step is sent to the prod build env spawned only to build
> single package which is cut off from access to the public network.
> If that list of patches downloaded over VCS URLs is long enough it exceeds
> the throttled rate and .. some patches are not downloaded.
> 
> Web server serving web content of the https://sourceware.org/git/ is ONLY
> known to me such a server which has such throttling.
> Literally NONE of other places with VCS REST interfaces are using such
> fascist settings .. IIRC that throttling is something like 25-30 per MINUTE


Can we not, please? The GNU project is not being fascist, regardless of
any throttling settings.


> which is b*dy easy to exceed.


Although really you should not be repeating this work. Why does every
other distro not have the problem you are having? Most likely because
they have spent the last few decades trying to figure out how to make it
NOT be a problem, and eventually came up with an intriguing technology
called:

downloading it once.

And saving it.

- Alongside the build recipe (Arch, Debian, Fedora, Gentoo, Void)
- in a "lookaside cache" / distro mirror site (Fedora, Gentoo, Void)
- in /var/cache/PM-source-distfiles (Arch, Gentoo, Void)

You should consider doing the same. It's not that complicated, there is
already rpm tooling to do it! Provably so since Fedora does it.



> Above technique glued with some one line generator which generates those
> Patch: lines allows me to save TONS of time to keep updated packages
> downloading all of what is needed in *seconds* ..
> Some packages only to test them in advance are using gitlab/github email
> notifications processed over procmail to test build after EACH commit made
> by maintainers (+ batch of tests to check that produced packages are still
> OK and warn me over zabbix that something is wrong).
> Above gives the best possible VISIBILITY of what has been changed in all
> those commits as well (you can use for that for example mc to manually
> check what has been changed in each commit .. offline).
> Instead of chasing bugs using git bisect it is easier to have an instant
> signal that after commit <hash> something went wrong.


... so you've invented a new Linux distro called DDoS Linux?

Are we supposed to clap now?


> If you may be thinking about security aspects of downloading all those
> patches in the batch mode .. github and gitlab allow you to sign commits or
> git tags.
> On downloading patches out of commits it is possible to verify
> those signatures (it takes one line of POSIX sh script to do that).
> Percentage of projects especially on github in recent months which
> are using signet commits started going up.


Okay, now this I am genuinely fascinated by and would like to hear more
about.

- This is not about github or gitlab, it's just a native git feature.
  gitweb and cgit support it too.

- Verifying a signature on a patch file should be impossible, since:
  a) the original signature is not signing a patch file
  b) patch files frequently change their bytes as a reaction to:
    - git version running on the server to produce the patch
    - size of git repo changing the hash abbrev minimum

- git signed commits, even if verified, contain a cryptographic strength
  of: sha1. In general, distros have been moving to the use of sha256 or
  blake2 due to well publicized weaknesses in sha1 that indicate that
  even if there is no known attack that affects git specifically, the
  time is coming and we won't necessarily know when it is here.


> Do I need to mention that cgit from https://sourceware.org/git/ do not
> offer such additional functionality and will probably never be able to do
> that?🤔
> 
> ---
> Any comments about the main topic?🤔


The main topic is a rant about how GDB but actually the several projects
sharing the same build system, do NOT have the resources right at this
second to update the Autoconf in use, from:
- autoconf 2.69 (the latest version as of December 2020)

to:
- autoconf 2.72 (the latest version as of February 2024)


Given that it is apparently challenging to find the resources to perform
a relatively minor facelift to an existing build system, I am quite
shocked that you think there are resources to throw out the entire build
system and rewrite it from ground zero in meson.

However, GDB is Free / Libre and Open Source Software, and in the finest
spirit of FLOSS, I would like to remind you, that:

patches

are welcome

to rewrite binutils, gcc, and gdb in meson.

Aside: the Meson WrapDB would also welcome contribution of such a
rewrite, even a partial one, as it provides a useful functionality to
people that need to depend on binutils libraries. Might be a good place
to test a work-in-progress port, even. ;)


Thank you! And have a very nice day. :)


-- 
Eli Schwartz

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 18399 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: gdb and ancient GNU autotools

[Mark Wielaard]

Hi,

On Sun, Feb 25, 2024 at 03:05:54AM -0500, Eli Schwartz wrote:
> On 2/24/24 7:22 PM, Tomasz Kłoczko wrote:
> Although really you should not be repeating this work. Why does every
> other distro not have the problem you are having? Most likely because
> they have spent the last few decades trying to figure out how to make it
> NOT be a problem, and eventually came up with an intriguing technology
> called:
> 
> downloading it once.
> 
> And saving it.
> 
> - Alongside the build recipe (Arch, Debian, Fedora, Gentoo, Void)
> - in a "lookaside cache" / distro mirror site (Fedora, Gentoo, Void)
> - in /var/cache/PM-source-distfiles (Arch, Gentoo, Void)
> 
> You should consider doing the same. It's not that complicated, there is
> already rpm tooling to do it! Provably so since Fedora does it.

I think it should in theory work, if it really doesn't, then we should
figure out why. I assume it is because the original poster is using
the gitweb interface over https at https://sourcware.org/git/ to
download individual files or diffs, which is known to not be the most
efficient way of fetching the sources. Instead of using the git https
interface. If so, it should be solved by using actually using the git
https backends or switching to the cgit interface at
https://sourceware.org/cgit/

But yes, having a cache certainly helps.

Note that all of sourceware repos are also in the
https://www.softwareheritage.org/ project. And that if you like a
forge like interface you can also use the sourcehut mirrors at
https://sr.ht/~sourceware/ It might be another good fallback if you do
have issues with the main sourceware git servers.

> > Above technique glued with some one line generator which generates those
> > Patch: lines allows me to save TONS of time to keep updated packages
> > downloading all of what is needed in *seconds* ..
> > Some packages only to test them in advance are using gitlab/github email
> > notifications processed over procmail to test build after EACH commit made
> > by maintainers (+ batch of tests to check that produced packages are still
> > OK and warn me over zabbix that something is wrong).
> > Above gives the best possible VISIBILITY of what has been changed in all
> > those commits as well (you can use for that for example mc to manually
> > check what has been changed in each commit .. offline).
> > Instead of chasing bugs using git bisect it is easier to have an instant
> > signal that after commit <hash> something went wrong.
> 
> ... so you've invented a new Linux distro called DDoS Linux?
> 
> Are we supposed to clap now?

Please don't be sarcastic. We do the exact same thing with our CI at
https://builder.sourceware.org/ which gets new commits all the time
sometimes fetching code for 20 builders at a time in parallel from
different machines. So it is known to work. We aren't DDoSing
ourselves. So we just have to figure out why it doesn't seem to work
for Tomasz.

Cheers,

Mark

Re: gdb and ancient GNU autotools

[Eli Schwartz]

[-- Attachment #1.1.1: Type: text/plain, Size: 2257 bytes --]

On 2/25/24 5:40 AM, Mark Wielaard wrote:
> Please don't be sarcastic. We do the exact same thing with our CI at
> https://builder.sourceware.org/ which gets new commits all the time
> sometimes fetching code for 20 builders at a time in parallel from
> different machines. So it is known to work. We aren't DDoSing
> ourselves. So we just have to figure out why it doesn't seem to work
> for Tomasz.

I think it's not entirely unreasonable for a project to internally place
its own demands on its own resources, with the knowledge of how much it
has in the way of resources...

especially when, per the buildbot logs, you are using persistent git
clones and fetching new changes on every buildbot run.

This is rather different from someone saying that he is building all
your projects on every commit when you didn't ask for that to be done,
and is NOT using git, and as a result of not using git, is getting
throttled by mysterious intricacies of gitweb vs cgit.

In fact, if I'm reading Tomasz correctly, in order to do this as an rpm
specfile build, he has automation hooked up to scan for every commit
landing on the binutils repo, then appending that commit as a gitweb
downloaded patchfile to the `Patch:` section of the spec.

So for example for the binutils-gdb case, that currently stands at 1626
separate gitweb downloaded patch files, and as soon as an email
notification comes in that the gdb repo has gotten a 1627th commit since
the latest tag, all those 1626 gitweb formatted patch files are going to
be downloaded again from scratch, except it is 1627 now.

This is why caching is not just:

> But yes, having a cache certainly helps.

Rather, it is "having a cache is basic politeness unto others".

It doesn't matter whether it works, and it doesn't matter whether
avoiding gitweb specifically and using cgit can make it better. It is
simply rude to do this altogether. Use a cache, or use git clones, or
best of all, use both -- just like builder.sourceware.org does.


Refusing to use a cache is called executing a DDoS campaign.

I am not being sarcastic.... okay, the "clapping" thing was a bit
sarcastic. But the DDoS Linux bit was meant in absolute seriousness.


-- 
Eli Schwartz

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 18399 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 2073 bytes --]

On Sun, 25 Feb 2024 at 21:19, Eli Schwartz <eschwartz93@gmail.com> wrote:
[..]

> I think it's not entirely unreasonable for a project to internally place
> its own demands on its own resources, with the knowledge of how much it
> has in the way of resources...
>

For some reason LITERALLY only ONE git repo is treating that as DEMAND.
No, I'm pointing to an INCORRECT/UNREALISTIC/causing some real issues on
client side web server settings.

> But yes, having a cache certainly helps.
>
> Rather, it is "having a cache is basic politeness unto others"


So you suggest that I should start caching ..

[tkloczko@pers-jacek SPECS]$ grep VCS:.*https://sourceware.org/git *spec |
wc -l
8

of 4444 git repos because some wannabe-webmaster added a throttling module
in the web server setup when NO ONE ELSE ANYWHERE is using such settings?
Really?🤔
If "yes" I have no more questions.

[..]

> Refusing to use a cache is called executing a DDoS campaign.
>

In DDoS acronym first D stands for DISTRIBUTED.
I'm using a SINGLE source IP.
Reaching the throttling level is not equal to DENIAL of service. So that is
the second D.
3rd .. do you really want to name ~0.5 req/s as realistic/justificable
setup?
How many people here are using web server reqs throttling and how many
amongst them are using so LOW values?
NONE?🤔
.. THREE misunderstandings/mistakes in ONE sentence.

Thank you for suggestions of using some public buildbots on which I have no
physical control.
Even if I'll be using those buildbots (which I'm not going to waste my time
because I have now WORKING build infra) execution of the same build
procedure will cause the SAME RESULT. Do you see that now?🤔

Nevertheless I have enough HW resources.
This off-topic part has NOTHING to do with using throttling settings in ONE
cgit web frontend is YET another mistake/misunderstanding.
As the old saying goes .. "the road to hell is paved with good advice".

One more time: please focus on gdb and issues related to its build
automation.

kloczek

Re: gdb and ancient GNU autotools

[Andreas Schwab]

I think you should demand your money back.

Re: gdb and ancient GNU autotools

[Mark Wielaard]

Hi Tomasz,

On Sun, Feb 25, 2024 at 09:50:19PM +0000, Tomasz Kłoczko wrote:
> On Sun, 25 Feb 2024 at 21:19, Eli Schwartz <eschwartz93@gmail.com> wrote:
> So you suggest that I should start caching ..
> 
> [tkloczko@pers-jacek SPECS]$ grep VCS:.*https://sourceware.org/git *spec |
> wc -l
> 8
> 
> of 4444 git repos because some wannabe-webmaster added a throttling module
> in the web server setup when NO ONE ELSE ANYWHERE is using such settings?

Please stop being so disrespectful towards people who are trying to
help you. You are complaining about being throttled and if you really
are, then we would like to resolve that. Because we don't want normal
users to be throttled. We can give you some suggestions on how to
prevent that if you clearly explain which resources on sourceware.org
you are fetching.

You seem to be using the gitweb interface to fetching files or
diffs. If so, and because gitweb is known to be not very resource
efficient, you really should switch to using cgit.

But we are really speculating here. It isn't really clear what you are
doing and what issues you are seeing.

Offlist you said that you didn't really got throttled because of those
8 packages you have that are hosted on sourceware none of them need
many patches in the first place. So it isn't clear to me if we are
actually discussing a real issue or if you are just worried that you
could have an issue in theory.

Cheers,

Mark

Re: gdb and ancient GNU autotools

[Eli Schwartz]

[-- Attachment #1.1.1: Type: text/plain, Size: 1771 bytes --]

On 2/25/24 4:50 PM, Tomasz Kłoczko wrote:
> On Sun, 25 Feb 2024 at 21:19, Eli Schwartz <eschwartz93@gmail.com> wrote:
> [..]
> 
>> I think it's not entirely unreasonable for a project to internally place
>> its own demands on its own resources, with the knowledge of how much it
>> has in the way of resources...
>>
> 
> For some reason LITERALLY only ONE git repo is treating that as DEMAND.
> No, I'm pointing to an INCORRECT/UNREALISTIC/causing some real issues on
> client side web server settings.
> 
>> But yes, having a cache certainly helps.
>>
>> Rather, it is "having a cache is basic politeness unto others"
> 
> 
> So you suggest that I should start caching ..
> 
> [tkloczko@pers-jacek SPECS]$ grep VCS:.*https://sourceware.org/git *spec |
> wc -l
> 8
> 
> of 4444 git repos because some wannabe-webmaster added a throttling module
> in the web server setup when NO ONE ELSE ANYWHERE is using such settings?
> Really?🤔
> If "yes" I have no more questions.


No.

I'm suggesting you cache all 4444 of them. When your cache starts to get
near the limits of its disk space, start evicting the least recently
used ones.

This is standard caching technology.

Thanks.


> Thank you for suggestions of using some public buildbots on which I have no
> physical control.
> Even if I'll be using those buildbots (which I'm not going to waste my time
> because I have now WORKING build infra) execution of the same build
> procedure will cause the SAME RESULT. Do you see that now?🤔


I truly have no idea why you think anyone has suggested you acquire an
account on builder.sourceware.org, as that has never even tangentially
been suggested.


Perhaps you should re-read the thread?


-- 
Eli Schwartz

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 18399 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 1809 bytes --]

On Sun, 25 Feb 2024 at 23:32, Mark Wielaard <mark@klomp.org> wrote:

>
> Offlist you said that you didn't really got throttled because of those
> 8 packages you have that are hosted on sourceware none of them need
> many patches in the first place. So it isn't clear to me if we are
> actually discussing a real issue or if you are just worried that you
> could have an issue in theory.
>

So .. more than a decade of not updating gdb build automation to be able
use it with latest GNU autotools tooling is NOT THE REAL ISSUE?🤔
Really?
And .. sincerely thank you for your "argumentum ad hominem".

I found that gdb autoconf has some issue with ncurses detection when
ncurses source code is configured before compiling in some exact way.
I'm able to fix that but *I'm not able to test the result of that fix*
(before possible sending PR) because *I'm not able to regenerate all gdb
build automation .. because for more than decade that gdb automation update
has been postponed by any cost*.
Yes, ncurses code has changed in the last decade and now provides for
example full UTF-8 wide characters support.
In the last 3-4 years it offers as well a simpler source code setup which
so far is not widely used in any leading Linux distro.

Indirectly I've offered my help with updating that build automation as I'm
dealing with ac/am/lt *almost 30 years*.
Issue is that so far I don't see any WILL to work on that .. as it is not a
task for a single person and working on that alone would be WASTE OF TIME.
I have access only to Linux/x86_64 and Solari/OpenSolaris x86 and gdb
ac/am/lt automation needs to be correctly working on many asch and
operating systems.

Are you able to help me with ABOVE issue?🤔

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Re: gdb and ancient GNU autotools

[Eli Schwartz]

[-- Attachment #1.1.1: Type: text/plain, Size: 2246 bytes --]

On 2/25/24 7:29 PM, Tomasz Kłoczko wrote:
> On Sun, 25 Feb 2024 at 23:32, Mark Wielaard <mark@klomp.org> wrote:
> 
>>
>> Offlist you said that you didn't really got throttled because of those
>> 8 packages you have that are hosted on sourceware none of them need
>> many patches in the first place. So it isn't clear to me if we are
>> actually discussing a real issue or if you are just worried that you
>> could have an issue in theory.
>>
> 
> So .. more than a decade of not updating gdb build automation to be able
> use it with latest GNU autotools tooling is NOT THE REAL ISSUE?🤔
> Really?
> And .. sincerely thank you for your "argumentum ad hominem".


Instead of accusing people of not updating the gdb build automation in
"more than a decade", I would like to invite you to reread the previous
messages in this thread. In particular, it appears you accidentally
skipped over my previous message, in which I said:

----------------------


The main topic is a rant about how GDB but actually the several projects
sharing the same build system, do NOT have the resources right at this
second to update the Autoconf in use, from:
- autoconf 2.69 (the latest version as of December 2020)

to:
- autoconf 2.72 (the latest version as of February 2024)


----------------------


Notice that the currently in use gdb build automation is from 3 and a
bit years ago, not a decade.


> I found that gdb autoconf has some issue with ncurses detection when
> ncurses source code is configured before compiling in some exact way.
> I'm able to fix that but *I'm not able to test the result of that fix*
> (before possible sending PR) because *I'm not able to regenerate all gdb
> build automation .. because for more than decade that gdb automation update
> has been postponed by any cost*.


It hasn't been a decade...

But you can also report a bug and show the patches, then ask if someone
else can help you test it.

I've never seen a project with a complex build system bootstrap that
wasn't both:

- understanding of the challenge it imposes on contributors

- extremely willing to help contributors test their patches when the
  bootstrap defeated them


-- 
Eli Schwartz

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 18399 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 213 bytes --]

On Mon, 26 Feb 2024 at 00:46, Eli Schwartz <eschwartz93@gmail.com> wrote:
[..]

> Instead of accusing people of not updating the gdb build automation in
> "more than a decade",
>

Accusing?🤔

kloczek

Re: gdb and ancient GNU autotools

[Andrew Pinski]

On Sat, Feb 24, 2024 at 8:29 AM Tomasz Kłoczko via Gdb
<gdb@sourceware.org> wrote:
>
> Hi,
>
> In gdb source tree in config/override.m4 is hardcoded autoconf version
>
> m4_if(m4_defn([m4_PACKAGE_VERSION]), [2.64],
>
> That version has been released almost SIXTEEN YEARS AGO !!!
> https://savannah.gnu.org/news/?id=5867

Wait what version of gdb are you using here because config/override.m4 contains
dnl Ensure exactly this Autoconf version is used
m4_ifndef([_GCC_AUTOCONF_VERSION],
  [m4_define([_GCC_AUTOCONF_VERSION], [2.69])])

Yes there is a test for a workaround of 2.64 for AVR/RTEMS which might
be able to be removed though. Maybe that is what you are looking at
and misreading the check there.

Note gdb, binutils, and GCC all updated to 2.69 back in 2018.
gdb/binutils commit d0ac1c44885daf68f631befa37e3f3bad318fbbf
gcc commit r9-3776-g22e052725189a4

Yes they have not updated to 2.70 or 2.72 yet. But with a complex
build system as GCC/binutils/gdb, why update to something if it is
working.

Thanks,
Andrew Pinski



>
> Literally NONE Linux distribution currently provides that version so in
> case of any necessity to fix something in autoconf it is NOT POSSIBLE to
> regenerate GNU autotools files.
>
> The same situation is with binutils ang gcc.
>
> Someone has been thinking about start update all that stuff to be able to
> use the latest GNU autotools and/or start moving to cmake or meson (IMO
> mesoon is better) arch/OS one by one (still living for other not migrated
> archs/OSsess autotools until last arch/OS still in use will be not migrated
> or abandoned)?🤔
>
> kloczek
> --
> Tomasz Kłoczko | LinkedIn: *http://lnkd.in/FXPWxH <http://lnkd.in/FXPWxH>*

Re: gdb and ancient GNU autotools

[Mark Wielaard]

On Mon, Feb 26, 2024 at 12:29:38AM +0000, Tomasz Kłoczko wrote:
> On Sun, 25 Feb 2024 at 23:32, Mark Wielaard <mark@klomp.org> wrote:
> > Offlist you said that you didn't really got throttled because of those
> > 8 packages you have that are hosted on sourceware none of them need
> > many patches in the first place. So it isn't clear to me if we are
> > actually discussing a real issue or if you are just worried that you
> > could have an issue in theory.
> 
> So .. more than a decade of not updating gdb build automation to be able
> use it with latest GNU autotools tooling is NOT THE REAL ISSUE?🤔
> Really?
> And .. sincerely thank you for your "argumentum ad hominem".

We are clearly talking past each other. If you do have issues with the
Sourceware infrastructure I am happy to work with you to resolve
them. But it seems we are unable to understand each other. Sorry it
didn't work out.

Re: gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 519 bytes --]

On Mon, 26 Feb 2024 at 11:44, Mark Wielaard <mark@klomp.org> wrote:
[..]

> We are clearly talking past each other. If you do have issues with the
> Sourceware infrastructure I am happy to work with you to resolve
> them. But it seems we are unable to understand each other. Sorry it
> didn't work out.
>

Yep .. that is true looks like a lack of understanding that the topic is
"gdb build automation state" and NOT about Sourceware  infra.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Re: gdb and ancient GNU autotools

[Tom Tromey]

>>>>> "Tomasz" == Tomasz Kłoczko via Gdb <gdb@sourceware.org> writes:

Tomasz> Why those repos are not moved yet to github or even own instance of the
Tomasz> gitlab to provide OOTB proper cross refs between commits, discussions
Tomasz> MRs/PRs, issue tickets and commit comments for me is kind of a mystery ..

The reason is that GNU has a policy against this.  You may disagree with
it but it shouldn't be a mystery.  I agree that better tooling here
would be desirable.  However, my view is that it's not going to happen.

Tomasz> Going back to the subject: someone at least started thinking about starting
Tomasz> doing something to move away from those ancient GNU autotools versions
Tomasz> (probably kicking a few as*ess on integrating properly some
Tomasz> libtool changes) or move to meson/cmake?

It's a colossal job with a modest payoff.  It's also difficult
politically as you must get all the relevant groups on-board before
starting.  I personally have other windmills at which to tilt.

Tom

Re: gdb and ancient GNU autotools

[Tom Tromey]

>>>>> "Andrew" == Andrew Pinski via Gdb <gdb@sourceware.org> writes:

Andrew> Yes they have not updated to 2.70 or 2.72 yet. But with a complex
Andrew> build system as GCC/binutils/gdb, why update to something if it is
Andrew> working.

I don't fully agree with this approach.  There's a benefit to upgrading
periodically, which is that it may ease future upgrades -- including
when one is really needed.  It's kind of like rebasing, where if you let
a branch sit for too long, suddenly you're in trouble.

That said, the upgrades are kind of a pain (I did one of them in the past).
So, it can be hard to motivate to do them; and given the slow pace of
auto* development, there's not often a need to rush into it.

Tom

Re: gdb and ancient GNU autotools

[Simon Marchi]

On 2/24/24 13:30, Tomasz Kłoczko via Gdb wrote:
> Going back to the subject: someone at least started thinking about starting
> doing something to move away from those ancient GNU autotools versions
> (probably kicking a few as*ess on integrating properly some
> libtool changes) or move to meson/cmake?

Good news, I found a concrete of a problem that would be fixed by moving
to a newer autoconf:

https://sourceware.org/bugzilla/show_bug.cgi?id=31417#c1

You can start working on patches to bump the autoconf/automake versions,
I can help with the review.

Simon

Re: gdb and ancient GNU autotools

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 1482 bytes --]

On Tue, 27 Feb 2024, Simon Marchi via Gdb wrote:

> On 2/24/24 13:30, Tomasz Kłoczko via Gdb wrote:
> > Going back to the subject: someone at least started thinking about starting
> > doing something to move away from those ancient GNU autotools versions
> > (probably kicking a few as*ess on integrating properly some
> > libtool changes) or move to meson/cmake?
> 
> Good news, I found a concrete of a problem that would be fixed by moving
> to a newer autoconf:
> 
> https://sourceware.org/bugzilla/show_bug.cgi?id=31417#c1
> 
> You can start working on patches to bump the autoconf/automake versions,
> I can help with the review.

A reminder: mixing a libtool update into an autoconf/automake update would 
be a bad idea, a libtool update is likely a lot harder, since (a) the 
libtool version used is very old (reportedly based on upstream commit 
2c9c38d8a12eb0a2ce7fe9c3862523026c3d5622); (b) there are many local 
patches, probably including some that are not present upstream; (c) 
libtool commit 3334f7ed5851ef1e96b052f2984c4acdbf39e20c will need 
reverting because of usage of --with-sysroot incompatible with how the 
toolchain uses that option.

I did the last autoconf/automake update in GCC, but that was building very 
heavily on your work on that update for binutils-gdb and would have been a 
lot harder without your work to update the shared files and show the way 
for the sort of changes to make to GCC-specific files.

-- 
Joseph S. Myers
josmyers@redhat.com

Re: gdb and ancient GNU autotools

[Tom Tromey]

>>>>> "Joseph" == Joseph Myers via Gdb <gdb@sourceware.org> writes:

Joseph> A reminder: mixing a libtool update into an autoconf/automake update would 
Joseph> be a bad idea, a libtool update is likely a lot harder, since (a) the 
Joseph> libtool version used is very old (reportedly based on upstream commit 
Joseph> 2c9c38d8a12eb0a2ce7fe9c3862523026c3d5622)

I wonder whether we could get rid of libtool.

The host OS landscape is a lot less varied than when it was written.
Many of the problem Unix variants are dead now.  Perhaps libtool doesn't
provide all that much value.

Tom

Re: gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 4098 bytes --]

On Tue, 27 Feb 2024 at 17:33, Joseph Myers <josmyers@redhat.com> wrote:
[..]

> A reminder: mixing a libtool update into an autoconf/automake update would
> be a bad idea, a libtool update is likely a lot harder, since (a) the
> libtool version used is very old (reportedly based on upstream commit
> 2c9c38d8a12eb0a2ce7fe9c3862523026c3d5622); (b) there are many local
> patches, probably including some that are not present upstream; (c)
> libtool commit 3334f7ed5851ef1e96b052f2984c4acdbf39e20c will need
> reverting because of usage of --with-sysroot incompatible with how the
> toolchain uses that option.
>
> I did the last autoconf/automake update in GCC, but that was building very
> heavily on your work on that update for binutils-gdb and would have been a
> lot harder without your work to update the shared files and show the way
> for the sort of changes to make to GCC-specific files.
>

In other words you are 100% AWARE that OOTB libtool *is in dead state*
because it has been abandoned by maintainer(s), and no one wants to
continue that work by forking it from current master (adding fixes ->
release new versions etc), and at the same time gdb maintainer(s) (and
probably gcc and binutils as well) definitely want to stick with GNU
autotools.
You cannot *eat cake and still have it on the plate* ..

Just FTR a few metrics from my +4.83k packages spec files (Fedora IIRC has
now ~21k, Debian 30k .. ish)

[tkloczko@pers-jacek SPECS]$ for i in meson libtool autoconf automake
cmake; do echo "$i = $(grep -c BuildRequires:.$i -l *spec |wc -l)"; done
meson = 436
libtool = 844
autoconf = 68
automake = 1040
cmake = 634

Even if my 4.8k package population produces values which are not
100% accurate, these metrics will be probably only +/- 2-3% different on
sampling them on a bigger population.

In above metrics if something requires automake it uses autoconf as well,
and as you can see:
- The number of people who/projects which abandoned/not choose GNU autoconf
is greater than cmake + meson (634+436 = 1070).
- The number of those which are using libtool (844) is already LOWER than
meson+cmake by ~20%.
- Metric of the meson+cmake is almost as big as the set in which it is used
by autoconf and/or automake (1040+68=1108).

*If no one will try to continue libtool maintenance* atractor anchored in
libtool will be changing those metrics with time in ONLY ONE DIRECTION
which will be ONLY more and more affecting gcc, binutils, gdb trio.

PURE LOGIC says that in this situation you guys (maintainers) have probably
ONLY two solutions/choices:
- someone will take over libtool to recover it from the current coma, and
rescue in longer term gcc, binutils, gdb trio as well.
- you can move to cmake or meson.
Above two possibilities are HARD LOGICAL CONCLUSIONS taken straight from
results of those metrics with which (conclusions) you may not agree but
existence of it is completely independent of what you can add more as
comment (and anyone here as persons with EnoughGood exp in context of build
automation).

As I wrote I can try to help in both possible scenarios but first someone
needs to MAKE THE DECISION about taking over libtool or about starting to
move towards cmake or meson.
I'm 100% sure that many other people may and WILL help with
above (necessary) changes BUT as long as libtool is unmaintained, investing
anyone's time is HIGHLY RISKY that time will be wasted.
In other words in the current state probably that you will stay ALONE and
no one will be trying to help you -> time which you already spend on
maintaining gcc, gdb binutils trion will be more or less kind of LOST .. is
only GROWING with every day of lack of that STRATEGIC decision.

I'm the only messenger .. please don't kill the messenger.

kloczek
PS. I found my ncurses and gdb issue solution but because it is JFDIN I'm
not going to publish it.
In other words my issue has now a solution (which I'm not happy/proud of
what I've done but ItWorks™️)
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Re: gdb and ancient GNU autotools

[Tomasz Kłoczko]

[-- Attachment #1: Type: text/plain, Size: 828 bytes --]

On Tue, 27 Feb 2024 at 20:44, Tomasz Kłoczko <kloczko.tomasz@gmail.com>
wrote:
[..]

> *If no one will try to continue libtool maintenance* atractor anchored in
> libtool will be changing those metrics with time in ONLY ONE DIRECTION
> which will be ONLY more and more affecting gcc, binutils, gdb trio.
>

Only after pressing the "send" button some additional thought bounced
back in my head ..
In gcc, binutils, gdb trio ALREADY is not used OOTB litool and
effectively in those projects source trees effectively are maintained
libtool fork!!!😳
If time on maintaining that tool is already part of those
projects development, why not officially fork it to have 100% control with
less hassles on syncing necessary files across subtrees?🤔

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

Re: gdb and ancient GNU autotools

[Sam James]

Tomasz Kłoczko <kloczko.tomasz@gmail.com> writes:

[resending, sorry, my mail client recently changed bindings.]

> On Tue, 27 Feb 2024 at 17:33, Joseph Myers <josmyers@redhat.com> wrote:
> [..]
>
>  A reminder: mixing a libtool update into an autoconf/automake update would 
>  be a bad idea, a libtool update is likely a lot harder, since (a) the 
>  libtool version used is very old (reportedly based on upstream commit 
>  2c9c38d8a12eb0a2ce7fe9c3862523026c3d5622); (b) there are many local 
>  patches, probably including some that are not present upstream; (c) 
>  libtool commit 3334f7ed5851ef1e96b052f2984c4acdbf39e20c will need 
>  reverting because of usage of --with-sysroot incompatible with how the 
>  toolchain uses that option.
>
>  I did the last autoconf/automake update in GCC, but that was building very 
>  heavily on your work on that update for binutils-gdb and would have been a 
>  lot harder without your work to update the shared files and show the way 
>  for the sort of changes to make to GCC-specific files.
>
> In other words you are 100% AWARE that OOTB libtool is in dead state because it has been abandoned by maintainer(s), and
> no one wants to continue that work by forking it from current master (adding fixes -> release new versions etc), and at

No, the situation with the GNU toolchain's libtool version/soft fork vs
the upstream one is more complicated.

There's some patches which never got submitted upstream, some which need
to be reconciled with upstream changes, some where we really need to
figure out why they were done in the first place, etc.

(Joseph already identified one such deliberate divergence: --with-sysroot.)

It's not as simple as "GNU libtool is unmaintained". In fact, it's got
(another) new maintainer as of last month or so, and a lot of activity
upstream since then. But that's separate to the issue of the libtool
used within the GNU toolchain, although kind of related (active
maintainers might help motivate updating it and sorting through the local changes).


> the same time gdb maintainer(s) (and probably gcc and binutils as well) definitely want to stick with GNU autotools.
> You cannot eat cake and still have it on the plate ..
>
> Just FTR a few metrics from my +4.83k packages spec files (Fedora IIRC has now ~21k, Debian 30k .. ish)
>
> [tkloczko@pers-jacek SPECS]$ for i in meson libtool autoconf automake cmake; do echo "$i = $(grep -c BuildRequires:.$i -l
> *spec |wc -l)"; done
> meson = 436
> libtool = 844
> autoconf = 68
> automake = 1040
> cmake = 634
>
> Even if my 4.8k package population produces values which are not 100% accurate, these metrics will be probably only +/-
> 2-3% different on sampling them on a bigger population.
>
> In above metrics if something requires automake it uses autoconf as well, and as you can see:
> - The number of people who/projects which abandoned/not choose GNU autoconf is greater than cmake + meson (634+436 =
> 1070).
> - The number of those which are using libtool (844) is already LOWER than meson+cmake by ~20%.
> - Metric of the meson+cmake is almost as big as the set in which it is used by autoconf and/or automake (1040+68=1108).
>
> If no one will try to continue libtool maintenance atractor anchored in libtool will be changing those metrics with time
> in ONLY ONE DIRECTION which will be ONLY more and more affecting gcc, binutils, gdb trio.
>
> PURE LOGIC says that in this situation you guys (maintainers) have probably ONLY two solutions/choices:
> - someone will take over libtool to recover it from the current coma, and rescue in longer term gcc, binutils, gdb trio
> as well.
> - you can move to cmake or meson.

Toolchain packages are generally special.

Can you please reconsider your tone? It comes across aggressive to me.

> Above two possibilities are HARD LOGICAL CONCLUSIONS taken straight from results of those metrics with which
> (conclusions) you may not agree but existence of it is completely independent of what you can add more as comment (and
> anyone here as persons with EnoughGood exp in context of build automation).
>
> As I wrote I can try to help in both possible scenarios but first someone needs to MAKE THE DECISION about taking over
> libtool or about starting to move towards cmake or meson.
> I'm 100% sure that many other people may and WILL help with above (necessary) changes BUT as long as libtool is
> unmaintained, investing anyone's time is HIGHLY RISKY that time will be wasted.
> In other words in the current state probably that you will stay ALONE and no one will be trying to help you -> time which
> you already spend on maintaining gcc, gdb binutils trion will be more or less kind of LOST .. is only GROWING with every
> day of lack of that STRATEGIC decision.
>
> I'm the only messenger .. please don't kill the messenger.
>
> kloczek
> PS. I found my ncurses and gdb issue solution but because it is JFDIN I'm not going to publish it.
> In other words my issue has now a solution (which I'm not happy/proud of what I've done but ItWorks™️)