Re: glib2.0 2.64.6-1 (TEST)

Re: glib2.0 2.64.6-1 (TEST)

[Jon Turney]

On 02/07/2023 15:30, Jon Turney wrote:
> 
> The following packages have been uploaded to the Cygwin distribution:
> 
> * libglib2.0_0-2.64.6-1
> * libglib2.0-devel-2.64.6-1
> * libglib2.0-doc-2.64.6-1
> 
[...]
> 
> There are many test-suite failures, however, as far as I can tell, they 
> are not regressions, so this should work as well as it ever did.

At least some of these test failures are due to bugs or shortcomings in 
the Cygwin DLL. In an ideal world, I'd have the time and motivation to 
investigate them all, but here's a brief summary of the few I looked 
into ...

gdatetime: The _NL_CTYPE_OUTDIGITx_MB constants for nl_langinfo aren't 
defined (possibly because we don't have the required information, e.g. 
to produce Persian (as opposed to Arabic) digit characters in the 
fa_IR.utf-8 locale)

test-printf: newlib bug in handling positional format arguments

See https://sourceware.org/pipermail/newlib/2023/020374.html

stream_rw-all: writes to a pipe until it's full, then ends up blocking 
when poll() still indicates it's writeable?

socket, unix-fd: try to send a fd over a socket, not currently implemented.

all the GDbus tests: these seem to get stuck during the handshake for an 
emulated unix-domain socket, somehow (It seems like all the dbus pacakge 
tests fail also, probably in the same way)

Re: glib2.0 2.64.6-1 (TEST)

[Corinna Vinschen]

On Jul  3 15:49, Jon Turney wrote:
> On 02/07/2023 15:30, Jon Turney wrote:
> > 
> > The following packages have been uploaded to the Cygwin distribution:
> > 
> > * libglib2.0_0-2.64.6-1
> > * libglib2.0-devel-2.64.6-1
> > * libglib2.0-doc-2.64.6-1
> > 
> [...]
> > 
> > There are many test-suite failures, however, as far as I can tell, they
> > are not regressions, so this should work as well as it ever did.
> 
> At least some of these test failures are due to bugs or shortcomings in the
> Cygwin DLL. In an ideal world, I'd have the time and motivation to
> investigate them all, but here's a brief summary of the few I looked into
> ...
> 
> gdatetime: The _NL_CTYPE_OUTDIGITx_MB constants for nl_langinfo aren't
> defined (possibly because we don't have the required information, e.g. to
> produce Persian (as opposed to Arabic) digit characters in the fa_IR.utf-8
> locale)

Uhm... we actually have this information.  Only, unfortunately I screwed
up the names:

  _NL_CTYPE_OUTDIGITSx_MB

instead of

  _NL_CTYPE_OUTDIGITx_MB

D'oh.

Do you think we need the wrong names for backward compat?  Theoretically
they should never have been used...

> stream_rw-all: writes to a pipe until it's full, then ends up blocking when
> poll() still indicates it's writeable?

Pipes never worked 100% POSIX-like, unfortunately, but the hang
was supposed to be fixed by introducingh the semaphore select_sem.

Do you have an STC?  Takashi, can you take a look?

> socket, unix-fd: try to send a fd over a socket, not currently implemented.
> 
> all the GDbus tests: these seem to get stuck during the handshake for an
> emulated unix-domain socket, somehow (It seems like all the dbus pacakge
> tests fail also, probably in the same way)

Does an extra call to setsockopt(fd, SOL_SOCKET, SO_PEERCRED, NULL, 0)
fix this, perhaps?


Corinna

Re: glib2.0 2.64.6-1 (TEST)

[Jon Turney]

On 26/07/2023 15:33, Corinna Vinschen wrote:
> On Jul  3 15:49, Jon Turney wrote:
>> On 02/07/2023 15:30, Jon Turney wrote:
>>>
>>> The following packages have been uploaded to the Cygwin distribution:
>>>
>>> * libglib2.0_0-2.64.6-1
>>> * libglib2.0-devel-2.64.6-1
>>> * libglib2.0-doc-2.64.6-1
>>>
>> [...]
>>>
>>> There are many test-suite failures, however, as far as I can tell, they
>>> are not regressions, so this should work as well as it ever did.
>>
>> At least some of these test failures are due to bugs or shortcomings in the
>> Cygwin DLL. In an ideal world, I'd have the time and motivation to
>> investigate them all, but here's a brief summary of the few I looked into
>> ...
>>
>> gdatetime: The _NL_CTYPE_OUTDIGITx_MB constants for nl_langinfo aren't
>> defined (possibly because we don't have the required information, e.g. to
>> produce Persian (as opposed to Arabic) digit characters in the fa_IR.utf-8
>> locale)
> 
> Uhm... we actually have this information.  Only, unfortunately I screwed
> up the names:
> 
>    _NL_CTYPE_OUTDIGITSx_MB
> 
> instead of
> 
>    _NL_CTYPE_OUTDIGITx_MB
> 
> D'oh.
> 
> Do you think we need the wrong names for backward compat?  Theoretically
> they should never have been used...

A brief web search only finds the wrong names in the cygwin source code, 
so yeah, I'd say just fix it.

Re: glib2.0 2.64.6-1 (TEST)

[Jon Turney]

[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]

On 26/07/2023 15:33, Corinna Vinschen wrote:
> On Jul  3 15:49, Jon Turney wrote:
>> On 02/07/2023 15:30, Jon Turney wrote:
>>>
>>> There are many test-suite failures, however, as far as I can tell, they
>>> are not regressions, so this should work as well as it ever did.
>>
>> At least some of these test failures are due to bugs or shortcomings in the
>> Cygwin DLL. In an ideal world, I'd have the time and motivation to
>> investigate them all, but here's a brief summary of the few I looked into
>> ...
> 
>> stream_rw-all: writes to a pipe until it's full, then ends up blocking when
>> poll() still indicates it's writeable?
> 
> Pipes never worked 100% POSIX-like, unfortunately, but the hang
> was supposed to be fixed by introducingh the semaphore select_sem.
> 
> Do you have an STC?  Takashi, can you take a look?

So, I messed up: it's a socketpair, not a pipe.  And I typoed the test 
name, it's actually stream-rw_all

If I'm remembering correctly, it ends up getting stuck at:

https://gitlab.gnome.org/GNOME/glib/-/blob/main/gio/tests/stream-rw_all.c#L179

I think this boils down to just:

- create a socketpair
- while polling if socket is writeable, write 100 zeroes to socket

This should stop when socket is not writeable, but actually just seems 
to block in the write.

Attached is an attempt at that, stripping out the glib stuff, which does 
indeed gets stuck. But maybe I messed up, the expectations of the test 
aren't good...

[-- Attachment #2: socketpair_full.c --]
[-- Type: text/plain, Size: 807 bytes --]

#include <assert.h>
#include <poll.h>
#include <stdio.h>
#include <sys/socket.h>
#include <unistd.h>

int main()
{
  char wbuf[100] = { 0, };
  int out;

  {
    int sv[2];
    int s;

    s = socketpair (AF_UNIX, SOCK_STREAM, 0, sv);
    assert (s == 0);

    out = sv[0];
  }

  size_t in_flight = 0;
  while (1)
    {
      struct pollfd fds[1];
      fds[0].fd = out;
      fds[0].events = POLLOUT;

      int r = poll(fds, 1, 0);
      assert(r >= 0);

      // fd is not ready to write
      if (!(fds[0].revents & POLLOUT))
        break;

      // otherwise, fd is ready to write, implies some data may be written without blocking
      ssize_t s = write (out, wbuf, sizeof wbuf);
      assert (s > 0);
      in_flight += s;
      printf("%zd written, total in_flight %zd\n", s, in_flight);
    }
}

Re: glib2.0 2.64.6-1 (TEST)

[Jon Turney]

On 26/07/2023 15:33, Corinna Vinschen wrote:
> On Jul  3 15:49, Jon Turney wrote:
>> On 02/07/2023 15:30, Jon Turney wrote:
>>>
>>> The following packages have been uploaded to the Cygwin distribution:
>>>
>>> * libglib2.0_0-2.64.6-1
>>> * libglib2.0-devel-2.64.6-1
>>> * libglib2.0-doc-2.64.6-1
>>>
>> [...]
>>>
>>> There are many test-suite failures, however, as far as I can tell, they
>>> are not regressions, so this should work as well as it ever did.
>>
>> At least some of these test failures are due to bugs or shortcomings in the
>> Cygwin DLL. In an ideal world, I'd have the time and motivation to
>> investigate them all, but here's a brief summary of the few I looked into
>> ...
[...]
>>
>> all the GDbus tests: these seem to get stuck during the handshake for an
>> emulated unix-domain socket, somehow (It seems like all the dbus pacakge
>> tests fail also, probably in the same way)
> 
> Does an extra call to setsockopt(fd, SOL_SOCKET, SO_PEERCRED, NULL, 0)
> fix this, perhaps?
> 

I don't quite understand where this call is supposed to go? In the 
client? in the listener?

Isn't SO_PEERCRED only valid with getsockopt()?

Re: glib2.0 2.64.6-1 (TEST)

[Corinna Vinschen]

On Aug  4 18:04, Jon Turney wrote:
> On 26/07/2023 15:33, Corinna Vinschen wrote:
> > On Jul  3 15:49, Jon Turney wrote:
> > > On 02/07/2023 15:30, Jon Turney wrote:
> > > > 
> > > > The following packages have been uploaded to the Cygwin distribution:
> > > > 
> > > > * libglib2.0_0-2.64.6-1
> > > > * libglib2.0-devel-2.64.6-1
> > > > * libglib2.0-doc-2.64.6-1
> > > > 
> > > [...]
> > > > 
> > > > There are many test-suite failures, however, as far as I can tell, they
> > > > are not regressions, so this should work as well as it ever did.
> > > 
> > > At least some of these test failures are due to bugs or shortcomings in the
> > > Cygwin DLL. In an ideal world, I'd have the time and motivation to
> > > investigate them all, but here's a brief summary of the few I looked into
> > > ...
> [...]
> > > 
> > > all the GDbus tests: these seem to get stuck during the handshake for an
> > > emulated unix-domain socket, somehow (It seems like all the dbus pacakge
> > > tests fail also, probably in the same way)
> > 
> > Does an extra call to setsockopt(fd, SOL_SOCKET, SO_PEERCRED, NULL, 0)
> > fix this, perhaps?
> > 
> 
> I don't quite understand where this call is supposed to go? In the client?
> in the listener?

Yes, both.  This switches off the handshake before calling connect or
accept.

> Isn't SO_PEERCRED only valid with getsockopt()?

Not after Christian Franke ported postfix to Cygwin back in 2014,
see commit 697b9afe00c21, especially the new comment in setsockopt.

The whole problem stems from the fact that we need to know that the
partner on the other side is a local Cygwin process and that it's
also an AF_UNIX emulation.  To accomplish that, Egor Duda introduced
the connect/accept handshake in commit 619f7fa0324c0 back in 2001.

The peercred stuff is in fact just some extra info we exchange during
that handshake, so maybe using SO_PEERCRED to switch off the handshake
was a bit weird.

If this is really the problem in your case, too, we should perhaps think
of a new solution to the problem we're trying to fix with the handshake.
I'm open to suggestions.

The unfinished work on fhandler/socket_unix.c using Windows Pipes was
one idea, but it's more complex than anticipated.  Another idea might
be to create a shared memory with a reproducible name, accessible from
both sides and used for the info handshake...


Corinna