segfault on 32bit cygwin snapshot

segfault on 32bit cygwin snapshot

[Marco Atzeri]

MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)

now fails at startup with

Exception: STATUS_ACCESS_VIOLATION at eip=61083736
eax=00000000 ebx=612F352C ecx=D4BFABF9 edx=00000012 esi=0071C8AC 
edi=00000012
ebp=00000000 esp=Exception: STATUS_ACCESS_VIOLATION at eip=61083736
eax=00000000 ebx=612F352C ecx=D4BFABF9 edx=00000012 esi=0071C8AC 
edi=00000012
ebp=00000000 esp=0071C6F0 program=D:\cygwin32\bin\mc.exe, pid 17880, 
thread main
cs=0023 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame     Function  Args
0071C6F0 program=D:\cygwin32\bin\mc.exe, pid 17880, thread main
cs=0023 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame     Function  Args

not tested yet on 64bit

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

On Sat, 20 Feb 2021 22:01:38 +0100
Marco Atzeri wrote:
> MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)
> 
> now fails at startup with
> 
> Exception: STATUS_ACCESS_VIOLATION at eip=61083736

Thanks for the report. I found the cause of this issue.
I will submit a patch to fix it.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[Marco Atzeri]

On 20.02.2021 23:29, Takashi Yano wrote:
> On Sat, 20 Feb 2021 22:01:38 +0100
> Marco Atzeri wrote:
>> MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)
>>
>> now fails at startup with
>>
>> Exception: STATUS_ACCESS_VIOLATION at eip=61083736
> 
> Thanks for the report. I found the cause of this issue.
> I will submit a patch to fix it.
> 

mc is now fine with 2021-02-22

but I found another issue

  $ /usr/bin/lilypond
GNU LilyPond 2.20.0
Segmentation fault (core dumped)

on 3.1.7 it works fine

stackdump is not very informative

Exception: STATUS_ACCESS_VIOLATION at eip=00000000
eax=FFCD8870 ebx=000no000 ecx=00000000 edx=FF390820 esi=00000000 
edi=00000000
ebp=00B4C1D8 esp=00B4C120 program=D:\cygwin32\bin\lilypond.exe, pid 
44036, thread main
cs=0023 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame     Function  Args
00B4C1D8  00000000 (00000000, 00000000, 00000000, 00000000)
End of stack trace

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

On Sun, 28 Feb 2021 19:48:28 +0100
Marco Atzeri wrote:
> On 20.02.2021 23:29, Takashi Yano wrote:
> > On Sat, 20 Feb 2021 22:01:38 +0100
> > Marco Atzeri wrote:
> >> MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)
> >>
> >> now fails at startup with
> >>
> >> Exception: STATUS_ACCESS_VIOLATION at eip=61083736
> > 
> > Thanks for the report. I found the cause of this issue.
> > I will submit a patch to fix it.
> > 
> 
> mc is now fine with 2021-02-22
> 
> but I found another issue
> 
>   $ /usr/bin/lilypond
> GNU LilyPond 2.20.0
> Segmentation fault (core dumped)
> 
> on 3.1.7 it works fine

I found this problem causes after the commit:

commit 532b91d24e9496c7988b2b1dda7fc0e8b161f782
Author: Corinna Vinschen <corinna@vinschen.de>
Date:   Mon Dec 14 12:29:23 2020 +0100

    Cygwin: Make sure newer apps get uname_x even when loading uname dynamically

    if an application built after API version 334 loads uname dynamically,
    it actually gets the old uname, rather than the new uname_x.  Fix this by
    checking the apps API version in uname and call uname_x instead, if it's
    a newer app.

    Signed-off-by: Corinna Vinschen <corinna@vinschen.de>

Reverting this commit solves the issue.

Corinna, could you please have a look?

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

On Mon, 1 Mar 2021 09:55:46 +0900
Takashi Yano wrote:
> On Sun, 28 Feb 2021 19:48:28 +0100
> Marco Atzeri wrote:
> > On 20.02.2021 23:29, Takashi Yano wrote:
> > > On Sat, 20 Feb 2021 22:01:38 +0100
> > > Marco Atzeri wrote:
> > >> MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)
> > >>
> > >> now fails at startup with
> > >>
> > >> Exception: STATUS_ACCESS_VIOLATION at eip=61083736
> > > 
> > > Thanks for the report. I found the cause of this issue.
> > > I will submit a patch to fix it.
> > > 
> > 
> > mc is now fine with 2021-02-22
> > 
> > but I found another issue
> > 
> >   $ /usr/bin/lilypond
> > GNU LilyPond 2.20.0
> > Segmentation fault (core dumped)
> > 
> > on 3.1.7 it works fine
> 
> I found this problem causes after the commit:
> 
> commit 532b91d24e9496c7988b2b1dda7fc0e8b161f782
> Author: Corinna Vinschen <corinna@vinschen.de>
> Date:   Mon Dec 14 12:29:23 2020 +0100
> 
>     Cygwin: Make sure newer apps get uname_x even when loading uname dynamically
> 
>     if an application built after API version 334 loads uname dynamically,
>     it actually gets the old uname, rather than the new uname_x.  Fix this by
>     checking the apps API version in uname and call uname_x instead, if it's
>     a newer app.
> 
>     Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
> 
> Reverting this commit solves the issue.
> 
> Corinna, could you please have a look?

I looked into this deeper a bit.

Perhaps, lilypond passes old small sized utsname structure to
uname() via old cygguile-17.dll (built on Apr. 2017). This seems
to cause stack destruction.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

On Mon, 1 Mar 2021 17:41:57 +0900
Takashi Yano wrote:
> On Mon, 1 Mar 2021 09:55:46 +0900
> Takashi Yano wrote:
> > On Sun, 28 Feb 2021 19:48:28 +0100
> > Marco Atzeri wrote:
> > > On 20.02.2021 23:29, Takashi Yano wrote:
> > > > On Sat, 20 Feb 2021 22:01:38 +0100
> > > > Marco Atzeri wrote:
> > > >> MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)
> > > >>
> > > >> now fails at startup with
> > > >>
> > > >> Exception: STATUS_ACCESS_VIOLATION at eip=61083736
> > > > 
> > > > Thanks for the report. I found the cause of this issue.
> > > > I will submit a patch to fix it.
> > > > 
> > > 
> > > mc is now fine with 2021-02-22
> > > 
> > > but I found another issue
> > > 
> > >   $ /usr/bin/lilypond
> > > GNU LilyPond 2.20.0
> > > Segmentation fault (core dumped)
> > > 
> > > on 3.1.7 it works fine
> > 
> > I found this problem causes after the commit:
> > 
> > commit 532b91d24e9496c7988b2b1dda7fc0e8b161f782
> > Author: Corinna Vinschen <corinna@vinschen.de>
> > Date:   Mon Dec 14 12:29:23 2020 +0100
> > 
> >     Cygwin: Make sure newer apps get uname_x even when loading uname dynamically
> > 
> >     if an application built after API version 334 loads uname dynamically,
> >     it actually gets the old uname, rather than the new uname_x.  Fix this by
> >     checking the apps API version in uname and call uname_x instead, if it's
> >     a newer app.
> > 
> >     Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
> > 
> > Reverting this commit solves the issue.
> > 
> > Corinna, could you please have a look?
> 
> I looked into this deeper a bit.
> 
> Perhaps, lilypond passes old small sized utsname structure to
> uname() via old cygguile-17.dll (built on Apr. 2017). This seems
> to cause stack destruction.

So, I guess rebuilding and replacing cygguile-17.dll will solve
the issue. However, I am not sure whehter the new binary works
with cygwin 3.1.7. 

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[marco atzeri]

On Mon, Mar 1, 2021 at 12:59 PM Takashi Yano via Cygwin  wrote:
>
> On Mon, 1 Mar 2021 17:41:57 +0900
> Takashi Yano wrote:
> > On Mon, 1 Mar 2021 09:55:46 +0900
> > Takashi Yano wrote:
> > > On Sun, 28 Feb 2021 19:48:28 +0100
> > > Marco Atzeri wrote:
> > > > On 20.02.2021 23:29, Takashi Yano wrote:
> > > > > On Sat, 20 Feb 2021 22:01:38 +0100
> > > > > Marco Atzeri wrote:

> > > > but I found another issue
> > > >
> > > >   $ /usr/bin/lilypond
> > > > GNU LilyPond 2.20.0
> > > > Segmentation fault (core dumped)
> > > >
> > > > on 3.1.7 it works fine
> > >
> > > I found this problem causes after the commit:
> > >
> > > commit 532b91d24e9496c7988b2b1dda7fc0e8b161f782
> > > Author: Corinna Vinschen <corinna@vinschen.de>
> > > Date:   Mon Dec 14 12:29:23 2020 +0100
> > >
> > >     Cygwin: Make sure newer apps get uname_x even when loading uname dynamically
> > >
> > >     if an application built after API version 334 loads uname dynamically,
> > >     it actually gets the old uname, rather than the new uname_x.  Fix this by
> > >     checking the apps API version in uname and call uname_x instead, if it's
> > >     a newer app.
> > >
> > >     Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
> > >
> > > Reverting this commit solves the issue.
> > >
> > > Corinna, could you please have a look?
> >
> > I looked into this deeper a bit.
> >
> > Perhaps, lilypond passes old small sized utsname structure to
> > uname() via old cygguile-17.dll (built on Apr. 2017). This seems
> > to cause stack destruction.
>
> So, I guess rebuilding and replacing cygguile-17.dll will solve
> the issue. However, I am not sure whehter the new binary works
> with cygwin 3.1.7.
>
> --
> Takashi Yano

I can rebuild cygguile-17.dll and see if work also with 3.1.7
but I wonder which other DLL's will suffer the same problem

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

On Mon, 1 Mar 2021 09:55:46 +0900
Takashi Yano wrote:
> On Sun, 28 Feb 2021 19:48:28 +0100
> Marco Atzeri wrote:
> > On 20.02.2021 23:29, Takashi Yano wrote:
> > > On Sat, 20 Feb 2021 22:01:38 +0100
> > > Marco Atzeri wrote:
> > >> MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)
> > >>
> > >> now fails at startup with
> > >>
> > >> Exception: STATUS_ACCESS_VIOLATION at eip=61083736
> > > 
> > > Thanks for the report. I found the cause of this issue.
> > > I will submit a patch to fix it.
> > > 
> > 
> > mc is now fine with 2021-02-22
> > 
> > but I found another issue
> > 
> >   $ /usr/bin/lilypond
> > GNU LilyPond 2.20.0
> > Segmentation fault (core dumped)
> > 
> > on 3.1.7 it works fine
> 
> I found this problem causes after the commit:
> 
> commit 532b91d24e9496c7988b2b1dda7fc0e8b161f782
> Author: Corinna Vinschen <corinna@vinschen.de>
> Date:   Mon Dec 14 12:29:23 2020 +0100
> 
>     Cygwin: Make sure newer apps get uname_x even when loading uname dynamically
> 
>     if an application built after API version 334 loads uname dynamically,
>     it actually gets the old uname, rather than the new uname_x.  Fix this by
>     checking the apps API version in uname and call uname_x instead, if it's
>     a newer app.
> 
>     Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
> 
> Reverting this commit solves the issue.
> 
> Corinna, could you please have a look?

The following check code does not work as expected if
newly build exe file is linked with old dll which calls
uname() as in this case.

  if (CYGWIN_VERSION_CHECK_FOR_UNAME_X)
    return uname_x (in_name);

Any idea?

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

On Mon, 1 Mar 2021 21:25:42 +0900
Takashi Yano wrote:
> On Mon, 1 Mar 2021 09:55:46 +0900
> Takashi Yano wrote:
> > On Sun, 28 Feb 2021 19:48:28 +0100
> > Marco Atzeri wrote:
> > > On 20.02.2021 23:29, Takashi Yano wrote:
> > > > On Sat, 20 Feb 2021 22:01:38 +0100
> > > > Marco Atzeri wrote:
> > > >> MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)
> > > >>
> > > >> now fails at startup with
> > > >>
> > > >> Exception: STATUS_ACCESS_VIOLATION at eip=61083736
> > > > 
> > > > Thanks for the report. I found the cause of this issue.
> > > > I will submit a patch to fix it.
> > > > 
> > > 
> > > mc is now fine with 2021-02-22
> > > 
> > > but I found another issue
> > > 
> > >   $ /usr/bin/lilypond
> > > GNU LilyPond 2.20.0
> > > Segmentation fault (core dumped)
> > > 
> > > on 3.1.7 it works fine
> > 
> > I found this problem causes after the commit:
> > 
> > commit 532b91d24e9496c7988b2b1dda7fc0e8b161f782
> > Author: Corinna Vinschen <corinna@vinschen.de>
> > Date:   Mon Dec 14 12:29:23 2020 +0100
> > 
> >     Cygwin: Make sure newer apps get uname_x even when loading uname dynamically
> > 
> >     if an application built after API version 334 loads uname dynamically,
> >     it actually gets the old uname, rather than the new uname_x.  Fix this by
> >     checking the apps API version in uname and call uname_x instead, if it's
> >     a newer app.
> > 
> >     Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
> > 
> > Reverting this commit solves the issue.
> > 
> > Corinna, could you please have a look?
> 
> The following check code does not work as expected if
> newly build exe file is linked with old dll which calls
> uname() as in this case.
> 
>   if (CYGWIN_VERSION_CHECK_FOR_UNAME_X)
>     return uname_x (in_name);
> 
> Any idea?

Ping Corinna?

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[Corinna Vinschen]

On Mar  2 20:03, Takashi Yano via Cygwin wrote:
> On Mon, 1 Mar 2021 21:25:42 +0900
> Takashi Yano wrote:
> > On Mon, 1 Mar 2021 09:55:46 +0900
> > Takashi Yano wrote:
> > > On Sun, 28 Feb 2021 19:48:28 +0100
> > > Marco Atzeri wrote:
> > > > On 20.02.2021 23:29, Takashi Yano wrote:
> > > > > On Sat, 20 Feb 2021 22:01:38 +0100
> > > > > Marco Atzeri wrote:
> > > > >> MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)
> > > > >>
> > > > >> now fails at startup with
> > > > >>
> > > > >> Exception: STATUS_ACCESS_VIOLATION at eip=61083736
> > > > > 
> > > > > Thanks for the report. I found the cause of this issue.
> > > > > I will submit a patch to fix it.
> > > > > 
> > > > 
> > > > mc is now fine with 2021-02-22
> > > > 
> > > > but I found another issue
> > > > 
> > > >   $ /usr/bin/lilypond
> > > > GNU LilyPond 2.20.0
> > > > Segmentation fault (core dumped)
> > > > 
> > > > on 3.1.7 it works fine
> > > 
> > > I found this problem causes after the commit:
> > > 
> > > commit 532b91d24e9496c7988b2b1dda7fc0e8b161f782
> > > Author: Corinna Vinschen <corinna@vinschen.de>
> > > Date:   Mon Dec 14 12:29:23 2020 +0100
> > > 
> > >     Cygwin: Make sure newer apps get uname_x even when loading uname dynamically
> > > 
> > >     if an application built after API version 334 loads uname dynamically,
> > >     it actually gets the old uname, rather than the new uname_x.  Fix this by
> > >     checking the apps API version in uname and call uname_x instead, if it's
> > >     a newer app.
> > > 
> > >     Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
> > > 
> > > Reverting this commit solves the issue.
> > > 
> > > Corinna, could you please have a look?
> > 
> > The following check code does not work as expected if
> > newly build exe file is linked with old dll which calls
> > uname() as in this case.
> > 
> >   if (CYGWIN_VERSION_CHECK_FOR_UNAME_X)
> >     return uname_x (in_name);
> > 
> > Any idea?
> 
> Ping Corinna?

I have no idea how we could fix that, other than by rebuilding the DLLs
which call uname, too.  We can't check the Cygwin build of all DLLs an
executable is linked to.


Corinna

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

Hi Corinna,

On Tue, 2 Mar 2021 16:48:45 +0100
Corinna Vinschen wrote:
> On Mar  2 20:03, Takashi Yano via Cygwin wrote:
> > On Mon, 1 Mar 2021 21:25:42 +0900
> > Takashi Yano wrote:
> > > On Mon, 1 Mar 2021 09:55:46 +0900
> > > Takashi Yano wrote:
> > > > On Sun, 28 Feb 2021 19:48:28 +0100
> > > > Marco Atzeri wrote:
> > > > > On 20.02.2021 23:29, Takashi Yano wrote:
> > > > > > On Sat, 20 Feb 2021 22:01:38 +0100
> > > > > > Marco Atzeri wrote:
> > > > > >> MC using latest snapshot 32bit 2021-02-19 (and 2021-02-10)
> > > > > >>
> > > > > >> now fails at startup with
> > > > > >>
> > > > > >> Exception: STATUS_ACCESS_VIOLATION at eip=61083736
> > > > > > 
> > > > > > Thanks for the report. I found the cause of this issue.
> > > > > > I will submit a patch to fix it.
> > > > > > 
> > > > > 
> > > > > mc is now fine with 2021-02-22
> > > > > 
> > > > > but I found another issue
> > > > > 
> > > > >   $ /usr/bin/lilypond
> > > > > GNU LilyPond 2.20.0
> > > > > Segmentation fault (core dumped)
> > > > > 
> > > > > on 3.1.7 it works fine
> > > > 
> > > > I found this problem causes after the commit:
> > > > 
> > > > commit 532b91d24e9496c7988b2b1dda7fc0e8b161f782
> > > > Author: Corinna Vinschen <corinna@vinschen.de>
> > > > Date:   Mon Dec 14 12:29:23 2020 +0100
> > > > 
> > > >     Cygwin: Make sure newer apps get uname_x even when loading uname dynamically
> > > > 
> > > >     if an application built after API version 334 loads uname dynamically,
> > > >     it actually gets the old uname, rather than the new uname_x.  Fix this by
> > > >     checking the apps API version in uname and call uname_x instead, if it's
> > > >     a newer app.
> > > > 
> > > >     Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
> > > > 
> > > > Reverting this commit solves the issue.
> > > > 
> > > > Corinna, could you please have a look?
> > > 
> > > The following check code does not work as expected if
> > > newly build exe file is linked with old dll which calls
> > > uname() as in this case.
> > > 
> > >   if (CYGWIN_VERSION_CHECK_FOR_UNAME_X)
> > >     return uname_x (in_name);
> > > 
> > > Any idea?
> > 
> > Ping Corinna?
> 
> I have no idea how we could fix that, other than by rebuilding the DLLs
> which call uname, too.  We can't check the Cygwin build of all DLLs an
> executable is linked to.

I have checked all /usr/bin/*.dll in my cygwin installation and found
following dlls call uname() rather than uname_x().

/usr/bin/cyggcj-16.dll
/usr/bin/cyggstreamer-1.0-0.dll
/usr/bin/cygguile-17.dll
/usr/bin/cygguile-2.0-22.dll
/usr/bin/cyghwloc-15.dll
/usr/bin/cygLLVM-3.8.dll
/usr/bin/cygLLVM-3.9.dll
/usr/bin/cygLLVM-4.0.dll
/usr/bin/cygLLVM-5.0.dll
/usr/bin/cygLLVMSupport-3.7.dll
/usr/bin/cygoctinterp-7.dll
/usr/bin/cygpulsecommon-11.1.dll
/usr/bin/cygQt5Core-5.dll
/usr/bin/cygQt5WebKit-5.dll
/usr/bin/cygQtWebKit-4.dll
/usr/bin/cygslang-2.dll
/usr/bin/cygwebkitgtk-3.0-0.dll
/usr/bin/cygwx_baseu-2.8-0.dll
/usr/bin/libzsh-5.8.dll

I also confirm that the same problem occurs under cygwin snapshot
2021-02-22 if the following code is compiled with Qt5 because
this program is linked to /usr/bin/cygQt5Core-5.dll.

#include <QApplication>
#include <QLabel>
#include <QSysInfo>

int main(int argc, char **argv)
{
    QApplication app(argc, argv);
    QLabel *label = new QLabel(QSysInfo::kernelType());
    label->show();
    return app.exec();
}

Do you think rebuilding all of these (or maybe more) dlls is only
the solution? 

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[Corinna Vinschen]

[Ping Mark Geisert]

On Mar  3 18:56, Takashi Yano via Cygwin wrote:
> Hi Corinna,
> 
> On Tue, 2 Mar 2021 16:48:45 +0100
> Corinna Vinschen wrote:
> > On Mar  2 20:03, Takashi Yano via Cygwin wrote:
> > > > The following check code does not work as expected if
> > > > newly build exe file is linked with old dll which calls
> > > > uname() as in this case.
> > > > 
> > > >   if (CYGWIN_VERSION_CHECK_FOR_UNAME_X)
> > > >     return uname_x (in_name);
> > > > 
> > > > Any idea?
> > > 
> > > Ping Corinna?
> > 
> > I have no idea how we could fix that, other than by rebuilding the DLLs
> > which call uname, too.  We can't check the Cygwin build of all DLLs an
> > executable is linked to.
> 
> I have checked all /usr/bin/*.dll in my cygwin installation and found
> following dlls call uname() rather than uname_x().
> [...]
> Do you think rebuilding all of these (or maybe more) dlls is only
> the solution? 

No, we could also drop the above code snippet.

Here's the problem: When we changed some datatypes in the early 2000s,
the old entry points have been conserved for backward compatibility,
while the new function using the new datatypes got a new name, e. g.,
stat vs. _stat64.

Next, libcygwin.a gets changed so that a newly built executable (using
the new datatypes as defined in the standard headers) calling stat is
redirected to _stat64.

All is well for the next 15+ years or so.

Then we discover that the exact same mechanism fails to work for
uname vs. the new uname_x in python.  What happened?

It turned out that python called uname dynamically Rather then just
calling uname, it calls dlopen();dlsym("uname");

This actually fetches the symbol for uname, not the symbol for uname_x.
The good old mechanism used for ages on standard function, fails as soon
as the caller uses dynamic loading of symbols.  Surprise, surprise!
It was just never taken into consideration that standard libc functions
might be called dynamically, given it usually doesn't make sense.

Given that this affects *all* of these redirected functions, we're in a
bit of a fix.  Fortunately, for all other functions this only affects 32
bit Cygwin, because the 64 bit version never had this backward
compatibility problem.

Therefore, uname vs. uname_x is the only function affecting 64 bit
Cygwin as well, and that's why I added the above crude redirection only
to uname in the first place.

So what we can do is this:

- Either all old DLLs calling uname must be rebuilt.

- Or we remove the above code snippet again and behave like for all
  other redirected functions on 32 bit as well.  Python's os.uname is
  broken, but all the affected DLL sstart working again.

Is there a way around that?  I'm not quite sure, so let's brain storm
a bit, ok?

- One thing we could try is to remove the above code, but add a python
  hack to dlsym instead.  This would let the "old" DLLs work again as
  before and for python we could add a hack to dlsym, along these lines:

    if (CYGWIN_VERSION_CHECK_FOR_UNAME_X
    	&& modulehandle == cygwin1.dll
	&& strcmp (symname, "uname"))
      symname = "uname_x";

Thoughts?  Other ideas?


Thanks,
Corinna

Re: segfault on 32bit cygwin snapshot

[Mark Geisert]

Hi Corinna,

Corinna Vinschen via Cygwin wrote:
> [Ping Mark Geisert]
> 
> On Mar  3 18:56, Takashi Yano via Cygwin wrote:
>> Hi Corinna,
>>
>> On Tue, 2 Mar 2021 16:48:45 +0100
>> Corinna Vinschen wrote:
>>> On Mar  2 20:03, Takashi Yano via Cygwin wrote:
>>>>> The following check code does not work as expected if
>>>>> newly build exe file is linked with old dll which calls
>>>>> uname() as in this case.
>>>>>
>>>>>    if (CYGWIN_VERSION_CHECK_FOR_UNAME_X)
>>>>>      return uname_x (in_name);
>>>>>
>>>>> Any idea?
>>>>
>>>> Ping Corinna?
>>>
>>> I have no idea how we could fix that, other than by rebuilding the DLLs
>>> which call uname, too.  We can't check the Cygwin build of all DLLs an
>>> executable is linked to.
>>
>> I have checked all /usr/bin/*.dll in my cygwin installation and found
>> following dlls call uname() rather than uname_x().
>> [...]
>> Do you think rebuilding all of these (or maybe more) dlls is only
>> the solution?
> 
> No, we could also drop the above code snippet.
> 
> Here's the problem: When we changed some datatypes in the early 2000s,
> the old entry points have been conserved for backward compatibility,
> while the new function using the new datatypes got a new name, e. g.,
> stat vs. _stat64.
> 
> Next, libcygwin.a gets changed so that a newly built executable (using
> the new datatypes as defined in the standard headers) calling stat is
> redirected to _stat64.
> 
> All is well for the next 15+ years or so.
> 
> Then we discover that the exact same mechanism fails to work for
> uname vs. the new uname_x in python.  What happened?
> 
> It turned out that python called uname dynamically Rather then just
> calling uname, it calls dlopen();dlsym("uname");
> 
> This actually fetches the symbol for uname, not the symbol for uname_x.
> The good old mechanism used for ages on standard function, fails as soon
> as the caller uses dynamic loading of symbols.  Surprise, surprise!
> It was just never taken into consideration that standard libc functions
> might be called dynamically, given it usually doesn't make sense.
> 
> Given that this affects *all* of these redirected functions, we're in a
> bit of a fix.  Fortunately, for all other functions this only affects 32
> bit Cygwin, because the 64 bit version never had this backward
> compatibility problem.
> 
> Therefore, uname vs. uname_x is the only function affecting 64 bit
> Cygwin as well, and that's why I added the above crude redirection only
> to uname in the first place.
> 
> So what we can do is this:
> 
> - Either all old DLLs calling uname must be rebuilt.

.. or patched (somehow).  Also, IIUC you're both talking about Cygwin-supplied 
DLLs, not user builds of Cygwin or private DLLs.  Dunno how we'd find every last one.

> - Or we remove the above code snippet again and behave like for all
>    other redirected functions on 32 bit as well.  Python's os.uname is
>    broken, but all the affected DLL sstart working again.

I think you need to keep the above code snippet to handle old exe running with 
current Cygwin DLL, no?

> Is there a way around that?  I'm not quite sure, so let's brain storm
> a bit, ok?
> 
> - One thing we could try is to remove the above code, but add a python
>    hack to dlsym instead.  This would let the "old" DLLs work again as
>    before and for python we could add a hack to dlsym, along these lines:
> 
>      if (CYGWIN_VERSION_CHECK_FOR_UNAME_X
>      	&& modulehandle == cygwin1.dll
> 	&& strcmp (symname, "uname"))
>        symname = "uname_x";
> 
> Thoughts?  Other ideas?

That's a sly fix, but it seems that it would do the job.  That's good!

On a different tack, I was thinking about how run time code could tell the 
difference between the versions of uname() being called.  It can't.  I looked at 
glibc and FreeBSD libc to see if they had to deal with this compatibility issue. 
It seems they don't, or I couldn't locate it if they do.

But FreeBSD has an approach that looked interesting in another way.  They have the 
standard uname() function taking the one usual arg.  But it's just a wrapper on a 
worker function that takes the original arg plus another arg specifying the size 
of the fields in struct utsname.  Paraphrasing, using Cygwin names:
         int uname(struct utsname *uptr)
         {
             return uname_x((int) _UTSNAME_LENGTH, uptr);
         }
They allow the user to override what we call _UTSNAME_LENGTH.  That seems like an 
invitation to exploit so I don't care for that.  But what I was thinking is if we 
make that first arg to uname_x() be a uintptr_t, we could tell (with pretty good 
confidence) at run time inside uname_x() if we were passed a length (from new code 
passing two args) or a ptr-to-buf (from old code just passing one arg).

I am unsure if this is useful.  I am trying to consider the cross-product of exes 
and dlls we care about and whether we can identify which combination we have at 
run time.  I think I need some time with a pad of paper, and maybe a drink.

..mark

Re: segfault on 32bit cygwin snapshot

[Corinna Vinschen]

On Mar  4 01:05, Mark Geisert wrote:
> Corinna Vinschen via Cygwin wrote:
> > Is there a way around that?  I'm not quite sure, so let's brain storm
> > a bit, ok?
> > 
> > - One thing we could try is to remove the above code, but add a python
> >    hack to dlsym instead.  This would let the "old" DLLs work again as
> >    before and for python we could add a hack to dlsym, along these lines:
> > 
> >      if (CYGWIN_VERSION_CHECK_FOR_UNAME_X
> >      	&& modulehandle == cygwin1.dll
> > 	&& strcmp (symname, "uname"))
> >        symname = "uname_x";
> > 
> > Thoughts?  Other ideas?
> 
> That's a sly fix, but it seems that it would do the job.  That's good!
> 
> On a different tack, I was thinking about how run time code could tell the
> difference between the versions of uname() being called.  It can't.  I
> looked at glibc and FreeBSD libc to see if they had to deal with this
> compatibility issue. It seems they don't, or I couldn't locate it if they
> do.
> 
> But FreeBSD has an approach that looked interesting in another way.  They
> have the standard uname() function taking the one usual arg.  But it's just
> a wrapper on a worker function that takes the original arg plus another arg
> specifying the size of the fields in struct utsname.  Paraphrasing, using
> Cygwin names:
>         int uname(struct utsname *uptr)
>         {
>             return uname_x((int) _UTSNAME_LENGTH, uptr);
>         }
> They allow the user to override what we call _UTSNAME_LENGTH.  That seems
> like an invitation to exploit so I don't care for that.  But what I was
> thinking is if we make that first arg to uname_x() be a uintptr_t, we could
> tell (with pretty good confidence) at run time inside uname_x() if we were
> passed a length (from new code passing two args) or a ptr-to-buf (from old
> code just passing one arg).

But uname_x just supports the new implementation, I'm not sure how this
helps us.  We may at least need two wrappers, one is the function called
from new apps, i.e.

  uname () { return uname_worker (sizeof old_utsname, &name); }
  uname_x () { return uname_worker (sizeof utsname, &name); }
  uname_worker { do your worst; }

However, it's not clear how this fixes the actual problem.  We just
don't have a way to know what size the caller expects.

Having version or size info in structs like the Win32 API does in a
couple of cases makes a lot more sense now...


Corinna

Re: segfault on 32bit cygwin snapshot

[Mark Geisert]

Hi Corinna,

Corinna Vinschen via Cygwin wrote:
> On Mar  4 01:05, Mark Geisert wrote:
>> Corinna Vinschen via Cygwin wrote:
>>> Is there a way around that?  I'm not quite sure, so let's brain storm
>>> a bit, ok?
>>>
>>> - One thing we could try is to remove the above code, but add a python
>>>     hack to dlsym instead.  This would let the "old" DLLs work again as
>>>     before and for python we could add a hack to dlsym, along these lines:
>>>
>>>       if (CYGWIN_VERSION_CHECK_FOR_UNAME_X
>>>       	&& modulehandle == cygwin1.dll
>>> 	&& strcmp (symname, "uname"))
>>>         symname = "uname_x";
>>>
>>> Thoughts?  Other ideas?
>>
>> That's a sly fix, but it seems that it would do the job.  That's good!
>>
>> On a different tack, I was thinking about how run time code could tell the
[...somewhat of a dead-end tangent elided...]
> However, it's not clear how this fixes the actual problem.  We just
> don't have a way to know what size the caller expects.
> 
> Having version or size info in structs like the Win32 API does in a
> couple of cases makes a lot more sense now...

Indeed.  I like your dlsym proposal, but I would code the "modulehandle == 
cygwin1.dll" first as it's less likely to be true than the version check.

I think "the above code" you mentioned still needs to be retained to deal with old 
exes calling uname directly and needing the old version.

Shall I write up a patch and submit it?
Thanks & Regards,

..mark

Re: segfault on 32bit cygwin snapshot

[Corinna Vinschen]

On Mar  5 01:18, Mark Geisert wrote:
> Hi Corinna,
> 
> Corinna Vinschen via Cygwin wrote:
> > On Mar  4 01:05, Mark Geisert wrote:
> > > Corinna Vinschen via Cygwin wrote:
> > > > Is there a way around that?  I'm not quite sure, so let's brain storm
> > > > a bit, ok?
> > > > 
> > > > - One thing we could try is to remove the above code, but add a python
> > > >     hack to dlsym instead.  This would let the "old" DLLs work again as
> > > >     before and for python we could add a hack to dlsym, along these lines:
> > > > 
> > > >       if (CYGWIN_VERSION_CHECK_FOR_UNAME_X
> > > >       	&& modulehandle == cygwin1.dll
> > > > 	&& strcmp (symname, "uname"))
> > > >         symname = "uname_x";
> > > > 
> > > > Thoughts?  Other ideas?
> > > 
> > > That's a sly fix, but it seems that it would do the job.  That's good!
> > > 
> > > On a different tack, I was thinking about how run time code could tell the
> [...somewhat of a dead-end tangent elided...]
> > However, it's not clear how this fixes the actual problem.  We just
> > don't have a way to know what size the caller expects.
> > 
> > Having version or size info in structs like the Win32 API does in a
> > couple of cases makes a lot more sense now...
> 
> Indeed.  I like your dlsym proposal, but I would code the "modulehandle ==
> cygwin1.dll" first as it's less likely to be true than the version check.
> 
> I think "the above code" you mentioned still needs to be retained to deal
> with old exes calling uname directly and needing the old version.

To the contrary.  That code was only required to deal with *new*
executables calling uname via the old entry point, because they called
dlopen/dlsym("uname").  Given this is handled in dlsym with my above
proposal, that addition in uname should go away.  After all, it's the
reason for the DLL crashes we're talking about here.


Corinna

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

Hi Corinna,

On Wed, 3 Mar 2021 12:00:25 +0100
Corinna Vinschen wrote:
> [Ping Mark Geisert]
> 
> On Mar  3 18:56, Takashi Yano via Cygwin wrote:
> > Hi Corinna,
> > 
> > On Tue, 2 Mar 2021 16:48:45 +0100
> > Corinna Vinschen wrote:
> > > On Mar  2 20:03, Takashi Yano via Cygwin wrote:
> > > > > The following check code does not work as expected if
> > > > > newly build exe file is linked with old dll which calls
> > > > > uname() as in this case.
> > > > > 
> > > > >   if (CYGWIN_VERSION_CHECK_FOR_UNAME_X)
> > > > >     return uname_x (in_name);
> > > > > 
> > > > > Any idea?
> > > > 
> > > > Ping Corinna?
> > > 
> > > I have no idea how we could fix that, other than by rebuilding the DLLs
> > > which call uname, too.  We can't check the Cygwin build of all DLLs an
> > > executable is linked to.
> > 
> > I have checked all /usr/bin/*.dll in my cygwin installation and found
> > following dlls call uname() rather than uname_x().
> > [...]
> > Do you think rebuilding all of these (or maybe more) dlls is only
> > the solution? 
> 
> No, we could also drop the above code snippet.
> 
> Here's the problem: When we changed some datatypes in the early 2000s,
> the old entry points have been conserved for backward compatibility,
> while the new function using the new datatypes got a new name, e. g.,
> stat vs. _stat64.
> 
> Next, libcygwin.a gets changed so that a newly built executable (using
> the new datatypes as defined in the standard headers) calling stat is
> redirected to _stat64.
> 
> All is well for the next 15+ years or so.
> 
> Then we discover that the exact same mechanism fails to work for
> uname vs. the new uname_x in python.  What happened?
> 
> It turned out that python called uname dynamically Rather then just
> calling uname, it calls dlopen();dlsym("uname");
> 
> This actually fetches the symbol for uname, not the symbol for uname_x.
> The good old mechanism used for ages on standard function, fails as soon
> as the caller uses dynamic loading of symbols.  Surprise, surprise!
> It was just never taken into consideration that standard libc functions
> might be called dynamically, given it usually doesn't make sense.
> 
> Given that this affects *all* of these redirected functions, we're in a
> bit of a fix.  Fortunately, for all other functions this only affects 32
> bit Cygwin, because the 64 bit version never had this backward
> compatibility problem.
> 
> Therefore, uname vs. uname_x is the only function affecting 64 bit
> Cygwin as well, and that's why I added the above crude redirection only
> to uname in the first place.
> 
> So what we can do is this:
> 
> - Either all old DLLs calling uname must be rebuilt.
> 
> - Or we remove the above code snippet again and behave like for all
>   other redirected functions on 32 bit as well.  Python's os.uname is
>   broken, but all the affected DLL sstart working again.
> 
> Is there a way around that?  I'm not quite sure, so let's brain storm
> a bit, ok?
> 
> - One thing we could try is to remove the above code, but add a python
>   hack to dlsym instead.  This would let the "old" DLLs work again as
>   before and for python we could add a hack to dlsym, along these lines:
> 
>     if (CYGWIN_VERSION_CHECK_FOR_UNAME_X
>     	&& modulehandle == cygwin1.dll
> 	&& strcmp (symname, "uname"))
>       symname = "uname_x";
> 
> Thoughts?  Other ideas?

It sounds very reasonable to me to deal with it within dlsym(),
as the problem arises from the use of dlsym(). However, what
happens if newly built .exe is linked to old dll which calls
uname() via dlsym()? I am not sure whether there are such dlls.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[marco atzeri]

On Thu, Mar 4, 2021 at 10:56 AM Takashi Yano via Cygwin  wrote:
>
> Hi Corinna,
>

>
> It sounds very reasonable to me to deal with it within dlsym(),
> as the problem arises from the use of dlsym(). However, what
> happens if newly built .exe is linked to old dll which calls
> uname() via dlsym()? I am not sure whether there are such dlls.
>
> --
> Takashi Yano


I have no problem to patch Python to solve the issue,
but I have not seen evidence of the dlsym mechanism .
But of course I an NOT and expert in this field.

If someone looking to the code can give me some hints,
I will appreciate

Regards
Marco

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

On Thu, 4 Mar 2021 12:11:11 +0100
marco atzeri wrote:
> I have no problem to patch Python to solve the issue,
> but I have not seen evidence of the dlsym mechanism .
> But of course I an NOT and expert in this field.
> 
> If someone looking to the code can give me some hints,
> I will appreciate

I am also not sure where the dlsym() is used in python.
At least, os.uname() works in python 3.8.7 and 2.7.18 in my
environment even without that snippet. It seems that os.uname()
does not use dlsym(). Do I overlook something?

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[Ken Brown]

On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
> On Thu, 4 Mar 2021 12:11:11 +0100
> marco atzeri wrote:
>> I have no problem to patch Python to solve the issue,
>> but I have not seen evidence of the dlsym mechanism .
>> But of course I an NOT and expert in this field.
>>
>> If someone looking to the code can give me some hints,
>> I will appreciate
> 
> I am also not sure where the dlsym() is used in python.
> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
> environment even without that snippet. It seems that os.uname()
> does not use dlsym(). Do I overlook something?

This all started because Mark reported a problem building python 3.8.3:

   https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
   https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html

It's strange that Marco never bumped into the problem.

Ken

Re: segfault on 32bit cygwin snapshot

[Corinna Vinschen]

On Mar  4 10:17, Ken Brown via Cygwin wrote:
> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
> > On Thu, 4 Mar 2021 12:11:11 +0100
> > marco atzeri wrote:
> > > I have no problem to patch Python to solve the issue,
> > > but I have not seen evidence of the dlsym mechanism .
> > > But of course I an NOT and expert in this field.
> > > 
> > > If someone looking to the code can give me some hints,
> > > I will appreciate
> > 
> > I am also not sure where the dlsym() is used in python.
> > At least, os.uname() works in python 3.8.7 and 2.7.18 in my
> > environment even without that snippet. It seems that os.uname()
> > does not use dlsym(). Do I overlook something?
> 
> This all started because Mark reported a problem building python 3.8.3:
> 
>   https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>   https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
> 
> It's strange that Marco never bumped into the problem.

Right.  I was wondering the same when I read Marco's reply.  I thought
this is something in old python versions, but my testing never involved
python.  I just used a simple uname implementation calling uname via
dlopen/dlsym.

Either way, the problem persists.  Newly built executables requesting
uname via dlsym should certainly get the new implementation.


Corinna

Re: segfault on 32bit cygwin snapshot

[Marco Atzeri]

On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
>> On Thu, 4 Mar 2021 12:11:11 +0100
>> marco atzeri wrote:
>>> I have no problem to patch Python to solve the issue,
>>> but I have not seen evidence of the dlsym mechanism .
>>> But of course I an NOT and expert in this field.
>>>
>>> If someone looking to the code can give me some hints,
>>> I will appreciate
>>
>> I am also not sure where the dlsym() is used in python.
>> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
>> environment even without that snippet. It seems that os.uname()
>> does not use dlsym(). Do I overlook something?
> 
> This all started because Mark reported a problem building python 3.8.3:
> 
>    https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>    https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
> 
> It's strange that Marco never bumped into the problem.
> 
> Ken

I never built python using cygwin snapshots as Mark was trying to do,
all my builds were using 3.1.7.

Let me set a separate enviroment for building on latest snapshot

Regards
Marco

Re: segfault on 32bit cygwin snapshot

[Marco Atzeri]

On 04.03.2021 21:17, Marco Atzeri wrote:
> On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
>> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
>>> On Thu, 4 Mar 2021 12:11:11 +0100
>>> marco atzeri wrote:
>>>> I have no problem to patch Python to solve the issue,
>>>> but I have not seen evidence of the dlsym mechanism .
>>>> But of course I an NOT and expert in this field.
>>>>
>>>> If someone looking to the code can give me some hints,
>>>> I will appreciate
>>>
>>> I am also not sure where the dlsym() is used in python.
>>> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
>>> environment even without that snippet. It seems that os.uname()
>>> does not use dlsym(). Do I overlook something?
>>
>> This all started because Mark reported a problem building python 3.8.3:
>>
>>    https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>>    
>> https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
>>
>> It's strange that Marco never bumped into the problem.
>>
>> Ken
> 
> I never built python using cygwin snapshots as Mark was trying to do,
> all my builds were using 3.1.7.
> 
> Let me set a separate enviroment for building on latest snapshot

I can not replicate with latest snapshot

$ uname -svr
CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42

nor in 64bit when building 3.8.8

For what I see the DLL is always using a proper import
from cygwin1.dll

$ objdump -x libpython3.8.dll |grep uname
         2b9de0   2170  uname
         2b9de8   2171  uname_x

the only thing not standard on my build system is a case sensitive
filesystem and mount


Regards
Marco

Re: segfault on 32bit cygwin snapshot

[Mark Geisert]

Marco Atzeri via Cygwin wrote:
> On 04.03.2021 21:17, Marco Atzeri wrote:
>> On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
>>> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
>>>> On Thu, 4 Mar 2021 12:11:11 +0100
>>>> marco atzeri wrote:
>>>>> I have no problem to patch Python to solve the issue,
>>>>> but I have not seen evidence of the dlsym mechanism .
>>>>> But of course I an NOT and expert in this field.
>>>>>
>>>>> If someone looking to the code can give me some hints,
>>>>> I will appreciate
>>>>
>>>> I am also not sure where the dlsym() is used in python.
>>>> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
>>>> environment even without that snippet. It seems that os.uname()
>>>> does not use dlsym(). Do I overlook something?
>>>
>>> This all started because Mark reported a problem building python 3.8.3:
>>>
>>>    https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>>> https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
>>>
>>> It's strange that Marco never bumped into the problem.
>>>
>>> Ken
>>
>> I never built python using cygwin snapshots as Mark was trying to do,
>> all my builds were using 3.1.7.
>>
>> Let me set a separate enviroment for building on latest snapshot
> 
> I can not replicate with latest snapshot
> 
> $ uname -svr
> CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42
> 
> nor in 64bit when building 3.8.8
> 
> For what I see the DLL is always using a proper import
> from cygwin1.dll
> 
> $ objdump -x libpython3.8.dll |grep uname
>          2b9de0   2170  uname
>          2b9de8   2171  uname_x
> 
> the only thing not standard on my build system is a case sensitive
> filesystem and mount

I had concerns that I had somehow corrupted my build environment, and it was 
Marco's successes that convinced me to reinstall 3.1.7 to recover a known-good 
environment.  Then seeing Marco go ahead and release the different Python releases 
(yay!) I didn't investigate any further.

I'm now trying to locate the os.uname usage of dlopen/dlsym again just for the 
record but am having some difficulty.  I'll reply again when I've got it.
Thanks all,

..mark

Re: segfault on 32bit cygwin snapshot

[Corinna Vinschen]

On Mar  5 01:11, Mark Geisert wrote:
> Marco Atzeri via Cygwin wrote:
> > On 04.03.2021 21:17, Marco Atzeri wrote:
> > > On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
> > > > On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
> > > > > On Thu, 4 Mar 2021 12:11:11 +0100
> > > > > marco atzeri wrote:
> > > > > > I have no problem to patch Python to solve the issue,
> > > > > > but I have not seen evidence of the dlsym mechanism .
> > > > > > But of course I an NOT and expert in this field.
> > > > > > 
> > > > > > If someone looking to the code can give me some hints,
> > > > > > I will appreciate
> > > > > 
> > > > > I am also not sure where the dlsym() is used in python.
> > > > > At least, os.uname() works in python 3.8.7 and 2.7.18 in my
> > > > > environment even without that snippet. It seems that os.uname()
> > > > > does not use dlsym(). Do I overlook something?
> > > > 
> > > > This all started because Mark reported a problem building python 3.8.3:
> > > > 
> > > >    https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
> > > > https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
> > > > 
> > > > It's strange that Marco never bumped into the problem.
> > > > 
> > > > Ken
> > > 
> > > I never built python using cygwin snapshots as Mark was trying to do,
> > > all my builds were using 3.1.7.
> > > 
> > > Let me set a separate enviroment for building on latest snapshot
> > 
> > I can not replicate with latest snapshot
> > 
> > $ uname -svr
> > CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42
> > 
> > nor in 64bit when building 3.8.8
> > 
> > For what I see the DLL is always using a proper import
> > from cygwin1.dll
> > 
> > $ objdump -x libpython3.8.dll |grep uname
> >          2b9de0   2170  uname
> >          2b9de8   2171  uname_x
> > 
> > the only thing not standard on my build system is a case sensitive
> > filesystem and mount
> 
> I had concerns that I had somehow corrupted my build environment, and it was
> Marco's successes that convinced me to reinstall 3.1.7 to recover a
> known-good environment.  Then seeing Marco go ahead and release the
> different Python releases (yay!) I didn't investigate any further.
> 
> I'm now trying to locate the os.uname usage of dlopen/dlsym again just for
> the record but am having some difficulty.  I'll reply again when I've got
> it.

Guys,

if it turns out that we fixed a problem that doesn't actually is a
real-world problem, I'm wondering if we shouldn't just revert the Cygwin
patch we're talking about here (commit 532b91d24e9496) and be done with
it.

Special casing dynamic loading of uname just to support some experimental
bordercase doesn't make much sense.  In that case I'm all for "don't do
that"!


Corinna

Re: segfault on 32bit cygwin snapshot

[Marco Atzeri]

On 05.03.2021 15:42, Corinna Vinschen via Cygwin wrote:
> On Mar  5 01:11, Mark Geisert wrote:
>> Marco Atzeri via Cygwin wrote:
>>> On 04.03.2021 21:17, Marco Atzeri wrote:
>>>> On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
>>>>> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
>>>>>> On Thu, 4 Mar 2021 12:11:11 +0100
>>>>>> marco atzeri wrote:
>>>>>>> I have no problem to patch Python to solve the issue,
>>>>>>> but I have not seen evidence of the dlsym mechanism .
>>>>>>> But of course I an NOT and expert in this field.
>>>>>>>
>>>>>>> If someone looking to the code can give me some hints,
>>>>>>> I will appreciate
>>>>>>
>>>>>> I am also not sure where the dlsym() is used in python.
>>>>>> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
>>>>>> environment even without that snippet. It seems that os.uname()
>>>>>> does not use dlsym(). Do I overlook something?
>>>>>
>>>>> This all started because Mark reported a problem building python 3.8.3:
>>>>>
>>>>>     https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>>>>> https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
>>>>>
>>>>> It's strange that Marco never bumped into the problem.
>>>>>
>>>>> Ken
>>>>
>>>> I never built python using cygwin snapshots as Mark was trying to do,
>>>> all my builds were using 3.1.7.
>>>>
>>>> Let me set a separate enviroment for building on latest snapshot
>>>
>>> I can not replicate with latest snapshot
>>>
>>> $ uname -svr
>>> CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42
>>>
>>> nor in 64bit when building 3.8.8
>>>
>>> For what I see the DLL is always using a proper import
>>> from cygwin1.dll
>>>
>>> $ objdump -x libpython3.8.dll |grep uname
>>>           2b9de0   2170  uname
>>>           2b9de8   2171  uname_x
>>>
>>> the only thing not standard on my build system is a case sensitive
>>> filesystem and mount
>>
>> I had concerns that I had somehow corrupted my build environment, and it was
>> Marco's successes that convinced me to reinstall 3.1.7 to recover a
>> known-good environment.  Then seeing Marco go ahead and release the
>> different Python releases (yay!) I didn't investigate any further.
>>
>> I'm now trying to locate the os.uname usage of dlopen/dlsym again just for
>> the record but am having some difficulty.  I'll reply again when I've got
>> it.
> 
> Guys,
> 
> if it turns out that we fixed a problem that doesn't actually is a
> real-world problem, I'm wondering if we shouldn't just revert the Cygwin
> patch we're talking about here (commit 532b91d24e9496) and be done with
> it.
> 
> Special casing dynamic loading of uname just to support some experimental
> bordercase doesn't make much sense.  In that case I'm all for "don't do
> that"!
> 
> 
> Corinna


Python has a lot of problem but not this one for what I can see

$ python3.8 /usr/lib/python3.8/test/pythoninfo.py  | grep uname
os.uname: posix.uname_result(sysname='CYGWIN_NT-10.0-19041-WOW64', 
nodename='LAPTOP-82F08ILC', release='3.1.7-340.i686', 
version='2020-08-22 19:03 UTC', machine='i686')

and similar for the other version

Re: segfault on 32bit cygwin snapshot

[Takashi Yano]

On Fri, 5 Mar 2021 17:30:30 +0100
Marco Atzeri wrote:
> On 05.03.2021 15:42, Corinna Vinschen via Cygwin wrote:
> > On Mar  5 01:11, Mark Geisert wrote:
> >> Marco Atzeri via Cygwin wrote:
> >>> On 04.03.2021 21:17, Marco Atzeri wrote:
> >>>> On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
> >>>>> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
> >>>>>> On Thu, 4 Mar 2021 12:11:11 +0100
> >>>>>> marco atzeri wrote:
> >>>>>>> I have no problem to patch Python to solve the issue,
> >>>>>>> but I have not seen evidence of the dlsym mechanism .
> >>>>>>> But of course I an NOT and expert in this field.
> >>>>>>>
> >>>>>>> If someone looking to the code can give me some hints,
> >>>>>>> I will appreciate
> >>>>>>
> >>>>>> I am also not sure where the dlsym() is used in python.
> >>>>>> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
> >>>>>> environment even without that snippet. It seems that os.uname()
> >>>>>> does not use dlsym(). Do I overlook something?
> >>>>>
> >>>>> This all started because Mark reported a problem building python 3.8.3:
> >>>>>
> >>>>>     https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
> >>>>> https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
> >>>>>
> >>>>> It's strange that Marco never bumped into the problem.
> >>>>>
> >>>>> Ken
> >>>>
> >>>> I never built python using cygwin snapshots as Mark was trying to do,
> >>>> all my builds were using 3.1.7.
> >>>>
> >>>> Let me set a separate enviroment for building on latest snapshot
> >>>
> >>> I can not replicate with latest snapshot
> >>>
> >>> $ uname -svr
> >>> CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42
> >>>
> >>> nor in 64bit when building 3.8.8
> >>>
> >>> For what I see the DLL is always using a proper import
> >>> from cygwin1.dll
> >>>
> >>> $ objdump -x libpython3.8.dll |grep uname
> >>>           2b9de0   2170  uname
> >>>           2b9de8   2171  uname_x
> >>>
> >>> the only thing not standard on my build system is a case sensitive
> >>> filesystem and mount
> >>
> >> I had concerns that I had somehow corrupted my build environment, and it was
> >> Marco's successes that convinced me to reinstall 3.1.7 to recover a
> >> known-good environment.  Then seeing Marco go ahead and release the
> >> different Python releases (yay!) I didn't investigate any further.
> >>
> >> I'm now trying to locate the os.uname usage of dlopen/dlsym again just for
> >> the record but am having some difficulty.  I'll reply again when I've got
> >> it.
> > 
> > Guys,
> > 
> > if it turns out that we fixed a problem that doesn't actually is a
> > real-world problem, I'm wondering if we shouldn't just revert the Cygwin
> > patch we're talking about here (commit 532b91d24e9496) and be done with
> > it.
> > 
> > Special casing dynamic loading of uname just to support some experimental
> > bordercase doesn't make much sense.  In that case I'm all for "don't do
> > that"!
> > 
> > 
> > Corinna
> 
> 
> Python has a lot of problem but not this one for what I can see
> 
> $ python3.8 /usr/lib/python3.8/test/pythoninfo.py  | grep uname
> os.uname: posix.uname_result(sysname='CYGWIN_NT-10.0-19041-WOW64', 
> nodename='LAPTOP-82F08ILC', release='3.1.7-340.i686', 
> version='2020-08-22 19:03 UTC', machine='i686')
> 
> and similar for the other version

I also tried to build python 3.8.7, 3.8.3 and 3.7.9 under cygwin 32bit
and 64bit current git head with the commit 532b91d2 reverted. All trials
resulted in success. Moreover, os.uname() works expectedly.

That is, the problem reported by Mark could not be reproduced.
https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html

Updating something other than cygwin1.dll might fix the issue???

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: segfault on 32bit cygwin snapshot

[Ken Brown]

On 3/5/2021 8:45 PM, Takashi Yano via Cygwin wrote:
> On Fri, 5 Mar 2021 17:30:30 +0100
> Marco Atzeri wrote:
>> On 05.03.2021 15:42, Corinna Vinschen via Cygwin wrote:
>>> On Mar  5 01:11, Mark Geisert wrote:
>>>> Marco Atzeri via Cygwin wrote:
>>>>> On 04.03.2021 21:17, Marco Atzeri wrote:
>>>>>> On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
>>>>>>> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
>>>>>>>> On Thu, 4 Mar 2021 12:11:11 +0100
>>>>>>>> marco atzeri wrote:
>>>>>>>>> I have no problem to patch Python to solve the issue,
>>>>>>>>> but I have not seen evidence of the dlsym mechanism .
>>>>>>>>> But of course I an NOT and expert in this field.
>>>>>>>>>
>>>>>>>>> If someone looking to the code can give me some hints,
>>>>>>>>> I will appreciate
>>>>>>>>
>>>>>>>> I am also not sure where the dlsym() is used in python.
>>>>>>>> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
>>>>>>>> environment even without that snippet. It seems that os.uname()
>>>>>>>> does not use dlsym(). Do I overlook something?
>>>>>>>
>>>>>>> This all started because Mark reported a problem building python 3.8.3:
>>>>>>>
>>>>>>>      https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>>>>>>> https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
>>>>>>>
>>>>>>> It's strange that Marco never bumped into the problem.
>>>>>>>
>>>>>>> Ken
>>>>>>
>>>>>> I never built python using cygwin snapshots as Mark was trying to do,
>>>>>> all my builds were using 3.1.7.
>>>>>>
>>>>>> Let me set a separate enviroment for building on latest snapshot
>>>>>
>>>>> I can not replicate with latest snapshot
>>>>>
>>>>> $ uname -svr
>>>>> CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42
>>>>>
>>>>> nor in 64bit when building 3.8.8
>>>>>
>>>>> For what I see the DLL is always using a proper import
>>>>> from cygwin1.dll
>>>>>
>>>>> $ objdump -x libpython3.8.dll |grep uname
>>>>>            2b9de0   2170  uname
>>>>>            2b9de8   2171  uname_x
>>>>>
>>>>> the only thing not standard on my build system is a case sensitive
>>>>> filesystem and mount
>>>>
>>>> I had concerns that I had somehow corrupted my build environment, and it was
>>>> Marco's successes that convinced me to reinstall 3.1.7 to recover a
>>>> known-good environment.  Then seeing Marco go ahead and release the
>>>> different Python releases (yay!) I didn't investigate any further.
>>>>
>>>> I'm now trying to locate the os.uname usage of dlopen/dlsym again just for
>>>> the record but am having some difficulty.  I'll reply again when I've got
>>>> it.
>>>
>>> Guys,
>>>
>>> if it turns out that we fixed a problem that doesn't actually is a
>>> real-world problem, I'm wondering if we shouldn't just revert the Cygwin
>>> patch we're talking about here (commit 532b91d24e9496) and be done with
>>> it.
>>>
>>> Special casing dynamic loading of uname just to support some experimental
>>> bordercase doesn't make much sense.  In that case I'm all for "don't do
>>> that"!
>>>
>>>
>>> Corinna
>>
>>
>> Python has a lot of problem but not this one for what I can see
>>
>> $ python3.8 /usr/lib/python3.8/test/pythoninfo.py  | grep uname
>> os.uname: posix.uname_result(sysname='CYGWIN_NT-10.0-19041-WOW64',
>> nodename='LAPTOP-82F08ILC', release='3.1.7-340.i686',
>> version='2020-08-22 19:03 UTC', machine='i686')
>>
>> and similar for the other version
> 
> I also tried to build python 3.8.7, 3.8.3 and 3.7.9 under cygwin 32bit
> and 64bit current git head with the commit 532b91d2 reverted. All trials
> resulted in success. Moreover, os.uname() works expectedly.
> 
> That is, the problem reported by Mark could not be reproduced.
> https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
> 
> Updating something other than cygwin1.dll might fix the issue???

I wonder if Mark had a corrupt libcygwin.a installed at the time he encountered 
this problem.  That would explain his observations, except for the claim that 
python was calling uname via dlopen/dlsym.  I guess he's rechecking that claim. 
  Mark, FWIW, I looked at the git history of python's Modules/posixmodule.c, and 
I can't see where they ever did this.

Ken

Re: segfault on 32bit cygwin snapshot

[Mark Geisert]

Ken Brown via Cygwin wrote:
> On 3/5/2021 8:45 PM, Takashi Yano via Cygwin wrote:
>> On Fri, 5 Mar 2021 17:30:30 +0100
>> Marco Atzeri wrote:
>>> On 05.03.2021 15:42, Corinna Vinschen via Cygwin wrote:
>>>> On Mar  5 01:11, Mark Geisert wrote:
>>>>> Marco Atzeri via Cygwin wrote:
>>>>>> On 04.03.2021 21:17, Marco Atzeri wrote:
>>>>>>> On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
>>>>>>>> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
>>>>>>>>> On Thu, 4 Mar 2021 12:11:11 +0100
>>>>>>>>> marco atzeri wrote:
>>>>>>>>>> I have no problem to patch Python to solve the issue,
>>>>>>>>>> but I have not seen evidence of the dlsym mechanism .
>>>>>>>>>> But of course I an NOT and expert in this field.
>>>>>>>>>>
>>>>>>>>>> If someone looking to the code can give me some hints,
>>>>>>>>>> I will appreciate
>>>>>>>>>
>>>>>>>>> I am also not sure where the dlsym() is used in python.
>>>>>>>>> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
>>>>>>>>> environment even without that snippet. It seems that os.uname()
>>>>>>>>> does not use dlsym(). Do I overlook something?
>>>>>>>>
>>>>>>>> This all started because Mark reported a problem building python 3.8.3:
>>>>>>>>
>>>>>>>>      https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>>>>>>>> https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
>>>>>>>>
>>>>>>>> It's strange that Marco never bumped into the problem.
>>>>>>>>
>>>>>>>> Ken
>>>>>>>
>>>>>>> I never built python using cygwin snapshots as Mark was trying to do,
>>>>>>> all my builds were using 3.1.7.
>>>>>>>
>>>>>>> Let me set a separate enviroment for building on latest snapshot
>>>>>>
>>>>>> I can not replicate with latest snapshot
>>>>>>
>>>>>> $ uname -svr
>>>>>> CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42
>>>>>>
>>>>>> nor in 64bit when building 3.8.8
>>>>>>
>>>>>> For what I see the DLL is always using a proper import
>>>>>> from cygwin1.dll
>>>>>>
>>>>>> $ objdump -x libpython3.8.dll |grep uname
>>>>>>            2b9de0   2170  uname
>>>>>>            2b9de8   2171  uname_x
>>>>>>
>>>>>> the only thing not standard on my build system is a case sensitive
>>>>>> filesystem and mount
>>>>>
>>>>> I had concerns that I had somehow corrupted my build environment, and it was
>>>>> Marco's successes that convinced me to reinstall 3.1.7 to recover a
>>>>> known-good environment.  Then seeing Marco go ahead and release the
>>>>> different Python releases (yay!) I didn't investigate any further.
>>>>>
>>>>> I'm now trying to locate the os.uname usage of dlopen/dlsym again just for
>>>>> the record but am having some difficulty.  I'll reply again when I've got
>>>>> it.
>>>>
>>>> Guys,
>>>>
>>>> if it turns out that we fixed a problem that doesn't actually is a
>>>> real-world problem, I'm wondering if we shouldn't just revert the Cygwin
>>>> patch we're talking about here (commit 532b91d24e9496) and be done with
>>>> it.
>>>>
>>>> Special casing dynamic loading of uname just to support some experimental
>>>> bordercase doesn't make much sense.  In that case I'm all for "don't do
>>>> that"!
>>>>
>>>>
>>>> Corinna
>>>
>>>
>>> Python has a lot of problem but not this one for what I can see
>>>
>>> $ python3.8 /usr/lib/python3.8/test/pythoninfo.py  | grep uname
>>> os.uname: posix.uname_result(sysname='CYGWIN_NT-10.0-19041-WOW64',
>>> nodename='LAPTOP-82F08ILC', release='3.1.7-340.i686',
>>> version='2020-08-22 19:03 UTC', machine='i686')
>>>
>>> and similar for the other version
>>
>> I also tried to build python 3.8.7, 3.8.3 and 3.7.9 under cygwin 32bit
>> and 64bit current git head with the commit 532b91d2 reverted. All trials
>> resulted in success. Moreover, os.uname() works expectedly.
>>
>> That is, the problem reported by Mark could not be reproduced.
>> https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>>
>> Updating something other than cygwin1.dll might fix the issue???
> 
> I wonder if Mark had a corrupt libcygwin.a installed at the time he encountered 
> this problem.  That would explain his observations, except for the claim that 
> python was calling uname via dlopen/dlsym.  I guess he's rechecking that claim. 
>   Mark, FWIW, I looked at the git history of python's Modules/posixmodule.c, and I 
> can't see where they ever did this.

I must be mistaken.  I appreciate everyone's help and patience while I tried to 
reproduce what I (thought I) was seeing last December.  I'm going to stand down on 
this topic as I think I'm only adding noise.  Please proceed on Corinna's course.
Thanks & Cheers,

..mark

Re: segfault on 32bit cygwin snapshot

[Mark Geisert]

Hi Corinna,

On Fri, 5 Mar 2021, Corinna Vinschen wrote:
> On Mar  5 01:11, Mark Geisert wrote:
>> Marco Atzeri via Cygwin wrote:
>>> On 04.03.2021 21:17, Marco Atzeri wrote:
>>>> On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
>>>>> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
>>>>>> On Thu, 4 Mar 2021 12:11:11 +0100
>>>>>> marco atzeri wrote:
>>>>>>> I have no problem to patch Python to solve the issue,
>>>>>>> but I have not seen evidence of the dlsym mechanism .
>>>>>>> But of course I an NOT and expert in this field.
>>>>>>>
>>>>>>> If someone looking to the code can give me some hints,
>>>>>>> I will appreciate
>>>>>>
>>>>>> I am also not sure where the dlsym() is used in python.
>>>>>> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
>>>>>> environment even without that snippet. It seems that os.uname()
>>>>>> does not use dlsym(). Do I overlook something?
>>>>>
>>>>> This all started because Mark reported a problem building python 3.8.3:
>>>>>
>>>>>    https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>>>>> https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
>>>>>
>>>>> It's strange that Marco never bumped into the problem.
>>>>>
>>>>> Ken
>>>>
>>>> I never built python using cygwin snapshots as Mark was trying to do,
>>>> all my builds were using 3.1.7.
>>>>
>>>> Let me set a separate enviroment for building on latest snapshot
>>>
>>> I can not replicate with latest snapshot
>>>
>>> $ uname -svr
>>> CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42
>>>
>>> nor in 64bit when building 3.8.8
>>>
>>> For what I see the DLL is always using a proper import
>>> from cygwin1.dll
>>>
>>> $ objdump -x libpython3.8.dll |grep uname
>>>          2b9de0   2170  uname
>>>          2b9de8   2171  uname_x
>>>
>>> the only thing not standard on my build system is a case sensitive
>>> filesystem and mount
>>
>> I had concerns that I had somehow corrupted my build environment, and it was
>> Marco's successes that convinced me to reinstall 3.1.7 to recover a
>> known-good environment.  Then seeing Marco go ahead and release the
>> different Python releases (yay!) I didn't investigate any further.
>>
>> I'm now trying to locate the os.uname usage of dlopen/dlsym again just for
>> the record but am having some difficulty.  I'll reply again when I've got
>> it.
>
> Guys,
>
> if it turns out that we fixed a problem that doesn't actually is a
> real-world problem, I'm wondering if we shouldn't just revert the Cygwin
> patch we're talking about here (commit 532b91d24e9496) and be done with
> it.
>
> Special casing dynamic loading of uname just to support some experimental
> bordercase doesn't make much sense.  In that case I'm all for "don't do
> that"!

That may well be the appropriate endpoint, but please let me dig a little 
further into the recent Python versions.  The fact that they had an 
explicit dlopen/dlsym to get at uname(), but now they don't, troubles me. 
I want to be sure us Cygwin folk aren't in an inadvertent "arms race" with 
the Python devs over the uname API change.  Dunno why this didn't occur 
15+ years ago, but here we are.

I think it was in Python's Modules/posixmodule.c.  They're certainly using 
uname() directly in their most recent builds.  But I believe that wasn't 
always the case, even just a few months ago.  Let me dig for a day or two.
Thanks,

..mark

Re: segfault on 32bit cygwin snapshot

[Mark Geisert]

Following up to myself...

Mark Geisert wrote:
> Hi Corinna,
> 
> On Fri, 5 Mar 2021, Corinna Vinschen wrote:
>> On Mar  5 01:11, Mark Geisert wrote:
>>> Marco Atzeri via Cygwin wrote:
>>>> On 04.03.2021 21:17, Marco Atzeri wrote:
>>>>> On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
>>>>>> On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
>>>>>>> On Thu, 4 Mar 2021 12:11:11 +0100
>>>>>>> marco atzeri wrote:
>>>>>>>> I have no problem to patch Python to solve the issue,
>>>>>>>> but I have not seen evidence of the dlsym mechanism .
>>>>>>>> But of course I an NOT and expert in this field.
>>>>>>>>
>>>>>>>> If someone looking to the code can give me some hints,
>>>>>>>> I will appreciate
>>>>>>>
>>>>>>> I am also not sure where the dlsym() is used in python.
>>>>>>> At least, os.uname() works in python 3.8.7 and 2.7.18 in my
>>>>>>> environment even without that snippet. It seems that os.uname()
>>>>>>> does not use dlsym(). Do I overlook something?
>>>>>>
>>>>>> This all started because Mark reported a problem building python 3.8.3:
>>>>>>
>>>>>>    https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
>>>>>> https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
>>>>>>
>>>>>> It's strange that Marco never bumped into the problem.
>>>>>>
>>>>>> Ken
>>>>>
>>>>> I never built python using cygwin snapshots as Mark was trying to do,
>>>>> all my builds were using 3.1.7.
>>>>>
>>>>> Let me set a separate enviroment for building on latest snapshot
>>>>
>>>> I can not replicate with latest snapshot
>>>>
>>>> $ uname -svr
>>>> CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42
>>>>
>>>> nor in 64bit when building 3.8.8
>>>>
>>>> For what I see the DLL is always using a proper import
>>>> from cygwin1.dll
>>>>
>>>> $ objdump -x libpython3.8.dll |grep uname
>>>>          2b9de0   2170  uname
>>>>          2b9de8   2171  uname_x
>>>>
>>>> the only thing not standard on my build system is a case sensitive
>>>> filesystem and mount
>>>
>>> I had concerns that I had somehow corrupted my build environment, and it was
>>> Marco's successes that convinced me to reinstall 3.1.7 to recover a
>>> known-good environment.  Then seeing Marco go ahead and release the
>>> different Python releases (yay!) I didn't investigate any further.
>>>
>>> I'm now trying to locate the os.uname usage of dlopen/dlsym again just for
>>> the record but am having some difficulty.  I'll reply again when I've got
>>> it.
>>
>> Guys,
>>
>> if it turns out that we fixed a problem that doesn't actually is a
>> real-world problem, I'm wondering if we shouldn't just revert the Cygwin
>> patch we're talking about here (commit 532b91d24e9496) and be done with
>> it.
>>
>> Special casing dynamic loading of uname just to support some experimental
>> bordercase doesn't make much sense.  In that case I'm all for "don't do
>> that"!
> 
> That may well be the appropriate endpoint, but please let me dig a little further 
> into the recent Python versions.  The fact that they had an explicit dlopen/dlsym 
> to get at uname(), but now they don't, troubles me. I want to be sure us Cygwin 
> folk aren't in an inadvertent "arms race" with the Python devs over the uname API 
> change.  Dunno why this didn't occur 15+ years ago, but here we are.
> 
> I think it was in Python's Modules/posixmodule.c.  They're certainly using uname() 
> directly in their most recent builds.  But I believe that wasn't always the case, 
> even just a few months ago.  Let me dig for a day or two.

Not to put too fine a point on it, but I now believe I hit the issue I reported 
due to my own corrupted build environment.  Marco and Ken have both built recent 
Python packages on Cygwin 3.1.7 and proto-3.2.0 respectively, Ken having reverted 
the commit you're talking about (not sure about Marco).  Both had success.

I believe reverting that commit 532b91d24e9496 is the correct action to take.

..mark

Re: segfault on 32bit cygwin snapshot

[Corinna Vinschen]

On Mar  8 03:07, Mark Geisert wrote:
> Following up to myself...
> 
> Mark Geisert wrote:
> > Hi Corinna,
> > 
> > On Fri, 5 Mar 2021, Corinna Vinschen wrote:
> > > On Mar  5 01:11, Mark Geisert wrote:
> > > > Marco Atzeri via Cygwin wrote:
> > > > > On 04.03.2021 21:17, Marco Atzeri wrote:
> > > > > > On 04.03.2021 16:17, Ken Brown via Cygwin wrote:
> > > > > > > On 3/4/2021 6:50 AM, Takashi Yano via Cygwin wrote:
> > > > > > > > On Thu, 4 Mar 2021 12:11:11 +0100
> > > > > > > > marco atzeri wrote:
> > > > > > > > > I have no problem to patch Python to solve the issue,
> > > > > > > > > but I have not seen evidence of the dlsym mechanism .
> > > > > > > > > But of course I an NOT and expert in this field.
> > > > > > > > > 
> > > > > > > > > If someone looking to the code can give me some hints,
> > > > > > > > > I will appreciate
> > > > > > > > 
> > > > > > > > I am also not sure where the dlsym() is used in python.
> > > > > > > > At least, os.uname() works in python 3.8.7 and 2.7.18 in my
> > > > > > > > environment even without that snippet. It seems that os.uname()
> > > > > > > > does not use dlsym(). Do I overlook something?
> > > > > > > 
> > > > > > > This all started because Mark reported a problem building python 3.8.3:
> > > > > > > 
> > > > > > >    https://cygwin.com/pipermail/cygwin-apps/2020-December/040765.html
> > > > > > > https://cygwin.com/pipermail/cygwin-developers/2020-December/012019.html
> > > > > > > 
> > > > > > > It's strange that Marco never bumped into the problem.
> > > > > > > 
> > > > > > > Ken
> > > > > > 
> > > > > > I never built python using cygwin snapshots as Mark was trying to do,
> > > > > > all my builds were using 3.1.7.
> > > > > > 
> > > > > > Let me set a separate enviroment for building on latest snapshot
> > > > > 
> > > > > I can not replicate with latest snapshot
> > > > > 
> > > > > $ uname -svr
> > > > > CYGWIN_NT-10.0-WOW 3.2.0s(0.340/5/3) 2021-03-01 15:42
> > > > > 
> > > > > nor in 64bit when building 3.8.8
> > > > > 
> > > > > For what I see the DLL is always using a proper import
> > > > > from cygwin1.dll
> > > > > 
> > > > > $ objdump -x libpython3.8.dll |grep uname
> > > > >          2b9de0   2170  uname
> > > > >          2b9de8   2171  uname_x
> > > > > 
> > > > > the only thing not standard on my build system is a case sensitive
> > > > > filesystem and mount
> > > > 
> > > > I had concerns that I had somehow corrupted my build environment, and it was
> > > > Marco's successes that convinced me to reinstall 3.1.7 to recover a
> > > > known-good environment.  Then seeing Marco go ahead and release the
> > > > different Python releases (yay!) I didn't investigate any further.
> > > > 
> > > > I'm now trying to locate the os.uname usage of dlopen/dlsym again just for
> > > > the record but am having some difficulty.  I'll reply again when I've got
> > > > it.
> > > 
> > > Guys,
> > > 
> > > if it turns out that we fixed a problem that doesn't actually is a
> > > real-world problem, I'm wondering if we shouldn't just revert the Cygwin
> > > patch we're talking about here (commit 532b91d24e9496) and be done with
> > > it.
> > > 
> > > Special casing dynamic loading of uname just to support some experimental
> > > bordercase doesn't make much sense.  In that case I'm all for "don't do
> > > that"!
> > 
> > That may well be the appropriate endpoint, but please let me dig a
> > little further into the recent Python versions.  The fact that they had
> > an explicit dlopen/dlsym to get at uname(), but now they don't, troubles
> > me. I want to be sure us Cygwin folk aren't in an inadvertent "arms
> > race" with the Python devs over the uname API change.  Dunno why this
> > didn't occur 15+ years ago, but here we are.
> > 
> > I think it was in Python's Modules/posixmodule.c.  They're certainly
> > using uname() directly in their most recent builds.  But I believe that
> > wasn't always the case, even just a few months ago.  Let me dig for a
> > day or two.
> 
> Not to put too fine a point on it, but I now believe I hit the issue I
> reported due to my own corrupted build environment.  Marco and Ken have both
> built recent Python packages on Cygwin 3.1.7 and proto-3.2.0 respectively,
> Ken having reverted the commit you're talking about (not sure about Marco).
> Both had success.
> 
> I believe reverting that commit 532b91d24e9496 is the correct action to take.

Did so this morning.


Thanks,
Corinna

Re: segfault on 32bit cygwin snapshot

[Corinna Vinschen]

On Mar  4 18:05, Takashi Yano via Cygwin wrote:
> Hi Corinna,
> 
> On Wed, 3 Mar 2021 12:00:25 +0100
> Corinna Vinschen wrote:
> > [Ping Mark Geisert]
> > 
> > Is there a way around that?  I'm not quite sure, so let's brain storm
> > a bit, ok?
> > 
> > - One thing we could try is to remove the above code, but add a python
> >   hack to dlsym instead.  This would let the "old" DLLs work again as
> >   before and for python we could add a hack to dlsym, along these lines:
> > 
> >     if (CYGWIN_VERSION_CHECK_FOR_UNAME_X
> >     	&& modulehandle == cygwin1.dll
> > 	&& strcmp (symname, "uname"))
> >       symname = "uname_x";
> > 
> > Thoughts?  Other ideas?
> 
> It sounds very reasonable to me to deal with it within dlsym(),
> as the problem arises from the use of dlsym(). However, what
> happens if newly built .exe is linked to old dll which calls
> uname() via dlsym()? I am not sure whether there are such dlls.

We simply can't fix that, because we don't have a Cygwin-specific
per-DLL information block as we have for EXEs.  There's no way to
workaround that problem.


Corinna