From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by sourceware.org (Postfix) with ESMTPS id 10AA3384AB6F for ; Fri, 19 Apr 2024 09:32:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 10AA3384AB6F Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=draigBrady.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 10AA3384AB6F Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::42e ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713519163; cv=none; b=nHqxvlCgG7VRUlHE28RmN5Wtd1c8ARxrmQEhAlgUwBtAup7+Ko/DsNChc+14kikFr3NwlT712Sueoxk0PQJ8WyJXdEcZa+yS1sR1qeWgxA9bWltC1jUfiozvtWqolBo1tpGhKIKHWIyWiyhkAtl4JxAyQaTfAZysQiimpQ4wd4Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713519163; c=relaxed/simple; bh=YWsMVTYCetOxeWeN+tEdqtiGot6ZvgnCEVHXQa8fui0=; h=DKIM-Signature:Message-ID:Date:MIME-Version:Subject:To:From; b=NRf9+VjzXBoEl5FAiYPY7IBik6+WqbCxA5MKlIgzw0li5xfswLUIkS5AUhzKdN6jqInBMc5M2Z1ZQzkNk3sXsvj3CTLqoQzY8w2NkinAlBSAwzbXmhYwvXw17UFYkTbQZhSMcXHNc2eyGky1wTJ0LE+ReJLw8Cz9Z/AdByoHL+M= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-wr1-x42e.google.com with SMTP id ffacd0b85a97d-345522f7c32so793826f8f.0 for ; Fri, 19 Apr 2024 02:32:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713519151; x=1714123951; darn=sourceware.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=XUmoizh6LAuXLtz3Ul1orSpLPVPjTVGZBM+tw9NSG9Q=; b=fxDTwuGXNfidkyL+j4/lT5RdFU2uj73t8T5ZX+nTu/4lt3jt7asybZDJvev3aAjq5N WkkWkUN1M6dZk+yA32atTrvFxg0p9b6RDk0fbhZKOCrSiUrfc+N2/i6Ed3XdM34B0viI vBgUEmGKr8rDE6Pjo9tSUiAOc2yPuAz2PVMDCD/yatB58xVZ5jlyCS+7BWBNetbyjtYk DqQOCYmmeYGmyPeGwRBSu2ukkBZfMKAgBj78Gm7j8WqmwBW5phXSqJ9btp1yARSblSgL 0iGZ8Rxr492ZhFON56B8z51S+7rqbXknO6gPKrOkf0OGqoFCw3DHzYXxTLL/hAbblpcA RMrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713519151; x=1714123951; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XUmoizh6LAuXLtz3Ul1orSpLPVPjTVGZBM+tw9NSG9Q=; b=h3HAh7FbGz90+Le6t/jFMmgNJ4VVo/7Kif1N6EaA9YrfMTcXtMiA7hFCnCMUPRWlIL qoZDs/Icwbb3gbmhQgp7FMDx1+kXrwm7xClobxs2z2fZaUcUIwx8NUqjTGCGSyLwRy7R RrbomSM+Y976Tp06bgRdxrt+K9RZoC6D3BwrjgLFHBnkTs7QlEBpnZlMUhblpBIjgsKj pxArxtHNM8ZVIxsR9KscUSZuWl6QaqDSYl9inONjFI0148Bc8NVXClDLvhJP52ReOOTu ChQNinGqAO7ugGhlwSp2xZ4hPjUzde5wkECIzsg7NULuc3Orynfl3nS8Qss4mKFW5j76 l/qw== X-Forwarded-Encrypted: i=1; AJvYcCXQ0jXNvikBQoJ7CpmkHpu+zeVhvXlq1A9y+FTr6IeFa2UJq/qJOyRncltmA20eVIzsJjk0Izz5nEbDzMIQjbP5DABb58XY5Pqi X-Gm-Message-State: AOJu0YzghxqBXpIilB8eGGz4oroprm3jx6rBDFj0A6P6zQFx0I4AM75g bMgQHktXhKInW6G4OpwDKOfrXBB9uqYYP0UpsKN1fdDCOpY2hx5R X-Google-Smtp-Source: AGHT+IEmlS/q8kQKMsWj+szVICKyCW0BXrBG0gWHbWpjk1/bC9+r54v88n9zVsJjAKSYeE1i8YcuHg== X-Received: by 2002:adf:e847:0:b0:346:d2c0:7675 with SMTP id d7-20020adfe847000000b00346d2c07675mr1229296wrn.29.1713519150997; Fri, 19 Apr 2024 02:32:30 -0700 (PDT) Received: from [192.168.1.53] (86-44-211-146-dynamic.agg2.lod.rsl-rtd.eircom.net. [86.44.211.146]) by smtp.googlemail.com with ESMTPSA id g30-20020adfa49e000000b00343f662327bsm3991470wrb.77.2024.04.19.02.32.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 19 Apr 2024 02:32:30 -0700 (PDT) Sender: =?UTF-8?Q?P=C3=A1draig_Brady?= Message-ID: Date: Fri, 19 Apr 2024 10:32:28 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Modify buffering of standard streams via environment variables (not LD_PRELOAD)? Content-Language: en-US To: Zachary Santer , Carl Edquist Cc: Kaz Kylheku , libc-alpha@sourceware.org, coreutils@gnu.org References: <9831afe6-958a-fbd3-9434-05dd0c9b602a@draigBrady.com> <317fe0e2-8cf9-d4ac-ed56-e6ebcc2baa55@cs.wisc.edu> <8c490a55-598a-adf6-67c2-eb2a6099620a@cs.wisc.edu> <8d26e5046cc8bf49490e4aa3f6c00b09@kylheku.com> From: =?UTF-8?Q?P=C3=A1draig_Brady?= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_BARRACUDACENTRAL,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=no autolearn_force=no version=3.4.6 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 19/04/2024 01:16, Zachary Santer wrote: > Was "RFE: enable buffering on null-terminated data" > > On Wed, Mar 20, 2024 at 4:54 AM Carl Edquist wrote: >> >> However, if stdbuf's magic env vars are exported in your shell (either by >> doing a trick like 'export $(env -i stdbuf -oL env)', or else more simply >> by first starting a new shell with 'stdbuf -oL bash'), then every command >> in your pipelines will start with the new default line-buffered stdout. >> That way your line-items from build.sh should get passed all the way >> through the pipeline as they are produced. > > Finally had a chance to try to build with 'stdbuf --output=L --error=L > --' in front of the build script, and it caused some crazy problems. I > was building Ada, though, so pretty good chance that part of the build > chain doesn't link against libc at all. > > I got a bunch of > ERROR: ld.so: object '/usr/libexec/coreutils/libstdbuf.so' from > LD_PRELOAD cannot be preloaded: ignored. > > And then it somehow caused compiler errors relating to the size of > what would be pointer types. Cleared out all the build products and > tried again without stdbuf and everything was fine. > >>>From the original thread just within the coreutils email list, "stdbuf > feature request - line buffering but for null-terminated data": > On Tue, Mar 12, 2024 at 12:42 PM Kaz Kylheku wrote: >> >> I would say that if it is implemented, the programs which require >> it should all make provisions to set it up themselves. >> >> stdbuf is a hack/workaround for programs that ignore the >> issue of buffering. Specifically, programs which send information >> to one of the three standard streams, such that the information >> is required in a timely way. Those streams become fully buffered >> when not connected to a terminal. > > I think I've partially come around to this point of view. However, > instead of expecting all sorts of individual programs to implement > their own buffering mode command-line options, could this be handled > with environment variables, but without LD_PRELOAD? I don't know if > libc itself can check for those environment variables and adjust each > program's buffering on its own, but if so, that would be a much > simpler solution. > > You could compare this to the various locale environment variables, > though I think a lot of commands whose behavior differ from locale to > locale do have to implement their own handling of that internally, at > least to some extent. > > This seems like somewhat less of a hack, and if no part of a program > looks for those environment variables, it isn't going to find itself > getting broken by the dynamic linker. It's just not going to change > its buffering. > > Additionally, things that don't link against libc could still honor > these environment variables, if the developers behind them care to put > in the effort. env variables are what I proposed 18 years ago now: https://sourceware.org/bugzilla/show_bug.cgi?id=2457 cheers, Pádraig