From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by sourceware.org (Postfix) with ESMTPS id F333E3858D37 for ; Sun, 17 Mar 2024 19:41:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org F333E3858D37 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org F333E3858D37 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::62b ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710704465; cv=none; b=efOTQBP5ohIpaioIqDgWIdvl/YIcMw2yhMRKpmlKdIiyMkJL80QP07I7yxNjI1JyZ1bJwWl3Ip+ufXNCMPQZ4vl9FcrXtZZ7/2YnDWH9F9Mz2QMx1ftER4fcZC43LUoiwIuB7/8PtoAREX7BcEUF3ic8TUC55Iu6AkR04ufl7wo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710704465; c=relaxed/simple; bh=ISB/WIt4CHx3n5MaH0KWkZO61rclY7oZejNc9NFlntQ=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=EWcInSMrJUHEt2q/OO4gSRYtaz+/TirP9Uasd7i+REftvloo9KeOe8ho0yp+YKA1Qv4MUimDqeIEqredt143JXfItuyO/AqDwpVMigpCozuNjsBN9bDZ4SuDZZAxrDXuaK5oPe0njB3ecwEJ1qVq6fZtQdJvjerzkmvlotltX0g= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-a468616ed70so308724166b.3 for ; Sun, 17 Mar 2024 12:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710704462; x=1711309262; darn=sourceware.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=iYSjcjodp6zcKq0FWQmyPgyuPHCssag+4I7Yloj8Ips=; b=VWeTa/hD6LLyddt9XWT26xlNZrTv5LjRgX1AGLwlFLiGyazirK/W6ZgZayAOciODgi 6FLqdeoU1f8LOIY9oj4OxiNhIkfGi0BmSlSIvHYbZAThRLhD1HFQHbHGuR9kZ8LY01AQ Ke+tx4db5SAW+3tjdYZT2X7K9EjSXmki9i1FdlUnghT1+AAv/Bx6pEB0e5CfrMYe6a+U 3Atn3YRHW1tNqLm2UCw8yEFp/RkYcb0tiHp/+1iq4bh59hm19LdDjWbwZ8Fn/tA/OHr3 nrBSlFSB4DjZF1kPQi8wJNmFgyJrOzjfG67UEXohlJ+D7p18uT/UcH337BPAu2uqJKG7 FYfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710704462; x=1711309262; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iYSjcjodp6zcKq0FWQmyPgyuPHCssag+4I7Yloj8Ips=; b=a/gsQZY0S38jz/tMne535X2Qj/EqIb2RrybfZBoJNQ3BAcIu3q7qWXFBOaoZiHNVtY FeSHzU8JTMKCl2bJi4hDw1KX0ru9Hdx5BWbwsOxAKioXIfuXyEyMjq7cJlcMdh1jrnZ3 +IK6j2rfiw9DDawI678S3xTXC0RvxoP7f6XQqToOHGdZO4Y/gDNOed3gb4tLhJze0ycr yS/Ppt/LqijhxqPqWCOT+iWaI8pA2dJdP+iVCihWi1oF/OLbjANjK+90Up9cGCIjkTJk a+yh5KiimWXRcEFeneUhZWwwJUaUq7WqG6hTw0oYMY9Aq58e6Mq2ugeXVNARMAjMCSLd ZjiQ== X-Forwarded-Encrypted: i=1; AJvYcCWZp/jngNz3EOhnMzN5wQffJxmKyrkJ29dKYLoCMXHRQovABVrTWrFALWqA0SH0N0ZOCJwzhqKbuuniWJlxBWFttJpHOJkAmYKv X-Gm-Message-State: AOJu0Yw1P0KDt4CcT1uurbYYuqOkalfzw1WPOhk4nKOKiqZpXopmq2fP 9noNaJ30GSFay+MYr0sbD3zkKMpgqVLGn2zwOvwdkgAt/Mr1n/zOF/Wyn36sJZYVrk6zxSrd6yu txi6WydpkHm+MOeJljnu1fpbRtZE= X-Google-Smtp-Source: AGHT+IEU3QJIEEXpCC6MDueYjazQIFxWeCYmIO6HnW9YN8YTEJiTmZPowkcp8RoIxCqnMnZbtmn62QWnVNAmUycfoDA= X-Received: by 2002:a17:906:53d1:b0:a46:47bc:580b with SMTP id p17-20020a17090653d100b00a4647bc580bmr7220251ejo.56.1710704462433; Sun, 17 Mar 2024 12:41:02 -0700 (PDT) MIME-Version: 1.0 References: <9831afe6-958a-fbd3-9434-05dd0c9b602a@draigBrady.com> <317fe0e2-8cf9-d4ac-ed56-e6ebcc2baa55@cs.wisc.edu> <8c490a55-598a-adf6-67c2-eb2a6099620a@cs.wisc.edu> <88a67f36-2a56-a838-f763-f55b3073bb50@lando.namek.net> In-Reply-To: <88a67f36-2a56-a838-f763-f55b3073bb50@lando.namek.net> From: Zachary Santer Date: Sun, 17 Mar 2024 15:40:50 -0400 Message-ID: Subject: Re: Examples of concurrent coproc usage? To: Carl Edquist Cc: bug-bash , libc-alpha@sourceware.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-0.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Thu, Mar 14, 2024 at 6:57=E2=80=AFAM Carl Edquist = wrote: > (And in general, latter coproc shells will have stray copies of the user > shell's r/w ends from all previous coprocs.) I didn't know that without MULTIPLE_COPROCS=3D1, bash wouldn't even attempt to keep the fds from earlier coprocs out of later coprocs. > Unexpectedly, the new multi-coproc code seems to close the user shell's > end of a coprocess's pipes, once the coprocess has terminated. When > compiled with MULTIPLE_COPROCS=3D1, this is true even if there is only a > single coproc: > This is a bug. The shell should not automatically close its read pipe to > a coprocess that has terminated -- it should stay open to read the final > output, and the user should be responsible for closing the read end > explicitly. > It also invites trouble if the shell variable that holds the fds gets > removed unexpectedly when the coprocess terminates. (Suddenly the > variable expands to an empty string.) It seems to me that the proper tim= e > to clear the coproc variable (if at all) is after the user has explicitly > closed both of the fds. *Or* else add an option to the coproc keyword to > explicitly close the coproc - which will close both fds and clear the > variable. I agree. This was the discussion in [1], where it sounds like this was the intended behavior. The array that bash originally created to store the coproc fds is removed immediately, but the fds are evidently closed at some later, indeterminate point. So, if you store the coproc fds in a different array than the one bash gave you, you might still be able to read from the read fd for a little while. That sounds suspiciously like a race condition, though. The behavior without MULTIPLE_COPROCS=3D1 might have changed since that discussion. > That's a nice trick with the shell backgrounding all the coprocesses > before connecting the fifos. But yeah, to make subsequent coprocesses yo= u > do still have to close the copy of the user shell's fds that the coproces= s > shell inherits. It sounds like you are doing that (nice!), but in any > case it requires some care, and as these stack up it is really handy to > have something manage it all for you. I absolutely learned more about what I was doing from that conversation with Chet three years ago. > (Perhaps this is where I ask if you are happy with your solution or if yo= u > would like to try out something wildly more flexible...) Admittedly, I am very curious to see your bash coprocess management library. I don't know how you could implement coprocesses outside of bash's coproc keyword without using FIFOs somehow. > Happy coprocessing! :) Thanks for your detailed description of all this. [1] https://lists.gnu.org/archive/html/help-bash/2021-04/msg00136.html