From: Nicholas Williams <nicholas.williams@zaxiom.com>
To: Dan Shelton <dan.f.shelton@gmail.com>
Cc: "cygwin@cygwin.com" <cygwin@cygwin.com>
Subject: Re: Cygwin outputting message to stderr on dofork EAGAIN failure even when Python exception is caught and handled
Date: Tue, 18 Jun 2024 17:22:10 -0500 [thread overview]
Message-ID: <244BB875-5177-402C-B958-FCCB23FEAEEE@zaxiom.com> (raw)
In-Reply-To: <CAAvCNcBg4YQxGvPESnOtFuZFX7L=LQs8k4qJpOZXDqEDSeqWaQ@mail.gmail.com>
Dan,
> On Jun 18, 2024, at 09:44, Dan Shelton via Cygwin <cygwin@cygwin.com> wrote:
>
> On Mon, 17 Jun 2024 at 18:03, Dale Lobb (Sys Admin) via Cygwin
> <cygwin@cygwin.com> wrote:
>>
>> Greetings, Nicholas;
>>
>>> From: Cygwin <cygwin-bounces+dale.lobb=bryanhealth.org@cygwin.com> On Behalf Of Andrey Repin via Cygwin
>>> Sent: Monday, June 17, 2024 2:58 AM
>>> To: Nicholas Williams <nicholas.williams@zaxiom.com>; cygwin@cygwin.com
>>> Cc: Andrey Repin <anrdaemon@yandex.ru>
>>> Subject: EXTERNAL SENDER: Re: Cygwin outputting message to stderr on dofork EAGAIN failure even when Python exception is caught and handled
>>>
>>> Greetings, Nicholas Williams! > We have a Python (installed and run through Cygwin) process running on > Windows Server 2022 that was very, very occasionally failing when subprocess. check_output was called: > 0 [main] python3 28481
>>>
>>> Greetings, Nicholas Williams!
>>>
>>>> We have a Python (installed and run through Cygwin) process running on
>>>> Windows Server 2022 that was very, very occasionally failing when subprocess.check_output was called:
>>>
>>>> 0 [main] python3 28481 dofork: child -1 - forked process 16856 died
>>>> unexpectedly, retry 0, exit code 0xC0000142, errno 11
>>>> …
>>>> subprocess.check_output(["cygpath", "-w", directory.name], encoding="utf-8").strip()
>>>> File "/usr/lib/python3.9/subprocess.py", line 424, in check_output <>
>>>> <>return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, <>
>>>> File "/usr/lib/python3.9/subprocess.py", line 505, in run <>
>>>> <>with Popen(*popenargs, **kwargs) as process: <>
>>>> File "/usr/lib/python3.9/subprocess.py", line 951, in __init__ <>
>>>> <>self._execute_child(args, executable, preexec_fn, close_fds, <>
>>>> File "/usr/lib/python3.9/subprocess.py", line 1754, in _execute_child
>>>> self.pid = _posixsubprocess.fork_exec(
>>>> BlockingIOError: [Errno 11] Resource temporarily unavailable
>>>
>>>> Setting aside for a minute the various reasons this might be happening
>>>> occasionally, which we cannot solve for at this moment, the error number
>>>> (EAGAIN) indicates that you should “try again.” So that’s exactly what we
>>>> did. We added a try/catch to the Python code to catch the BlockingIOError
>>>> and, if and only if the error number is EAGAIN, we try up to two more times.
>>>> This fixed the problem and caused the application to stop quitting. We
>>>> output a warning to our log so that we don’t forget about the problem, but
>>>> the warning only ever appears once, so retrying a single time seems to help.
>>>
>>>> However … even though Python handles the dofork error, turns it into a
>>>> Python exception, and our code catches the Python exception and handles it
>>>> properly, Cygwin (not Python … Cygwin) still outputs a message to stderr
>>>> right before our warning message. This Cygwin error message shows up as an error in our log tracking:
>>>
>>>> 0 [main] python3 15042 dofork: child -1 - forked process 6780 died
>>>> unexpectedly, retry 0, exit code 0xC0000142, errno 11
>>>> 06/16 13:57:53. 87520: WARNING: Retrying command in 2 seconds due to EAGAIN: [the command we’re running]
>
> Does python require fork(), or can it use posix_spawn() in this case?
I imagine this would be question for the CPython development team. While I have a basic understanding of C programming in Python (for the purposes of writing extensions), I do not know why subprocess.Popen is written in this way, and I do not know enough to question their choice here. It uses fork, and for now I have to live with that.
Thanks,
Nick
next prev parent reply other threads:[~2024-06-18 22:22 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-16 18:54 Nicholas Williams
2024-06-17 7:58 ` Andrey Repin
2024-06-17 16:03 ` Dale Lobb (Sys Admin)
2024-06-18 14:44 ` Dan Shelton
2024-06-18 22:22 ` Nicholas Williams [this message]
2024-06-18 14:51 ` Nicholas Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=244BB875-5177-402C-B958-FCCB23FEAEEE@zaxiom.com \
--to=nicholas.williams@zaxiom.com \
--cc=cygwin@cygwin.com \
--cc=dan.f.shelton@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).