[PATCH] Cygwin: pty: Reduce buffer size in get_console_process_id().

[PATCH] Cygwin: pty: Reduce buffer size in get_console_process_id().

[Takashi Yano]

- The buffer used in get_console_process_id(), introduced by commit
  72770148, is too large and ERROR_NOT_ENOUGH_MEMORY occurs in Win7.
  Therefore, the buffer size has been reduced.
---
 winsup/cygwin/fhandler_tty.cc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/winsup/cygwin/fhandler_tty.cc b/winsup/cygwin/fhandler_tty.cc
index c78e996e8..7f0752614 100644
--- a/winsup/cygwin/fhandler_tty.cc
+++ b/winsup/cygwin/fhandler_tty.cc
@@ -65,8 +65,8 @@ static DWORD
 get_console_process_id (DWORD pid, bool match)
 {
   tmp_pathbuf tp;
-  DWORD *list = (DWORD *) tp.w_get ();
-  const DWORD buf_size = NT_MAX_PATH * sizeof (WCHAR) / sizeof (DWORD);
+  DWORD *list = (DWORD *) tp.c_get ();
+  const DWORD buf_size = NT_MAX_PATH / sizeof (DWORD);
 
   DWORD num = GetConsoleProcessList (list, buf_size);
   if (num == 0 || num > buf_size)
-- 
2.30.0

Re: [PATCH] Cygwin: pty: Reduce buffer size in get_console_process_id().

[Corinna Vinschen]

On Jan 20 09:57, Takashi Yano via Cygwin-patches wrote:
> - The buffer used in get_console_process_id(), introduced by commit
>   72770148, is too large and ERROR_NOT_ENOUGH_MEMORY occurs in Win7.

Huh, funny!  Will we ever be happy with just 8192 processes per
console? :)

>   Therefore, the buffer size has been reduced.
> ---
>  winsup/cygwin/fhandler_tty.cc | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Pushed.


THanks,
Corinna

Re: [PATCH] Cygwin: pty: Reduce buffer size in get_console_process_id().

[Takashi Yano]

On Wed, 20 Jan 2021 10:50:24 +0100
Corinna Vinschen wrote:
> On Jan 20 09:57, Takashi Yano via Cygwin-patches wrote:
> > - The buffer used in get_console_process_id(), introduced by commit
> >   72770148, is too large and ERROR_NOT_ENOUGH_MEMORY occurs in Win7.
> 
> Huh, funny!  Will we ever be happy with just 8192 processes per
> console? :)

According to my test, when the buffer size is larger than 15683,
this error occurs. Test environment is Win7 x64. Both inside and
outside of WOW64, the maximum allowed size seems to be the same.

Shall we increase to 15683? :p

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

Re: [PATCH] Cygwin: pty: Reduce buffer size in get_console_process_id().

[Corinna Vinschen]

On Jan 20 19:40, Takashi Yano via Cygwin-patches wrote:
> On Wed, 20 Jan 2021 10:50:24 +0100
> Corinna Vinschen wrote:
> > On Jan 20 09:57, Takashi Yano via Cygwin-patches wrote:
> > > - The buffer used in get_console_process_id(), introduced by commit
> > >   72770148, is too large and ERROR_NOT_ENOUGH_MEMORY occurs in Win7.
> > 
> > Huh, funny!  Will we ever be happy with just 8192 processes per
> > console? :)
> 
> According to my test, when the buffer size is larger than 15683,
> this error occurs. Test environment is Win7 x64. Both inside and
> outside of WOW64, the maximum allowed size seems to be the same.
> 
> Shall we increase to 15683? :p

Ugh, please, no :D

8K processes per console is fine.  The machine will probably be out of
memory before this is a concern.


Corinna