From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from hedgehog.birch.relay.mailchannels.net (hedgehog.birch.relay.mailchannels.net [23.83.209.81]) by sourceware.org (Postfix) with ESMTPS id 7D6FE3844036 for ; Thu, 1 Apr 2021 14:48:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 7D6FE3844036 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=nadler.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=drn@nadler.com X-Sender-Id: dreamhost|x-authsender|drn@nadler.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id D357F342D85; Thu, 1 Apr 2021 14:48:13 +0000 (UTC) Received: from pdx1-sub0-mail-a54.g.dreamhost.com (100-96-18-61.trex.outbound.svc.cluster.local [100.96.18.61]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 846C9342C6D; Thu, 1 Apr 2021 14:48:09 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|drn@nadler.com Received: from pdx1-sub0-mail-a54.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.96.18.61 (trex/6.1.1); Thu, 01 Apr 2021 14:48:13 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|drn@nadler.com X-MailChannels-Auth-Id: dreamhost X-Average-Tart: 268d2ac005258447_1617288493673_1350613714 X-MC-Loop-Signature: 1617288493673:3438285900 X-MC-Ingress-Time: 1617288493673 Received: from pdx1-sub0-mail-a54.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a54.g.dreamhost.com (Postfix) with ESMTP id 2D78E7E453; Thu, 1 Apr 2021 14:48:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=nadler.com; h=subject:to :references:from:message-id:date:mime-version:in-reply-to :content-type; s=nadler.com; bh=5wtdSSVQEeSPAkJod7hV9hB1lms=; b= KBLmE8X74a+kEh+71a4XP+Zlre2MMFReBAO2yh9xH0ddWxLCrXhuwwvd0LDQebXB qPzcHKbnNFIezTVuoAZiqk8Dx6inbTdW3GY8o56ni38An03IchGiwXU64ZKS7Wd7 lit471CuOqjrwEO5UE84kmGEGwcJhLwsOqDrbQj6HyA= Received: from [192.168.1.3] (pool-72-74-171-157.bstnma.fios.verizon.net [72.74.171.157]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: drn@nadler.com) by pdx1-sub0-mail-a54.g.dreamhost.com (Postfix) with ESMTPSA id 726E97E455; Thu, 1 Apr 2021 14:48:08 +0000 (UTC) Subject: Re: Some questions on reentrancy, __DYNAMIC_REENT__ and _impure_ptr To: Nick , newlib@sourceware.org References: X-DH-BACKEND: pdx1-sub0-mail-a54 From: Dave Nadler Message-ID: Date: Thu, 1 Apr 2021 10:48:05 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_05, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, HTML_MESSAGE, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: newlib@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Newlib mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Apr 2021 14:48:18 -0000 On 4/1/2021 12:58 AM, Nick wrote: > Hi, > > I've been trying to enable reentrancy of newlib on a home brew kernel=20 > for the x86 platform and have some questions on how various pieces all=20 > fits together. Oy, we can never have enough kernels ;-) I'm not familiar with all the possible permutations. In FreeRTOS, the scheduler simply switches _impure_ptr before each=20 context switch. This is perfectly thread safe given: - the read/write of this ptr is atomic (true on the architectures I=20 know), and - no ISR use of anything in the RTL requiring this (ie no malloc,=20 strtok, etc. in ISR) Here's the code from FreeRTOS: =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 #if ( configUSE_NEWLIB_REENTRANT =3D= =3D 1 ) =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 { =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 /* Switch Newli= b's _impure_ptr variable to point to the _reent =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 structure speci= fic to this task. =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 See the third p= arty link=20 http://www.nadler.com/embedded/newlibAndFreeRTOS.html =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 for additional = information. */ =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 _impure_ptr =3D= &( pxCurrentTCB->xNewLib_reent ); =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 } =C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 #endif /* configUSE_NEWLIB_REENTRA= NT */ I hope that clears up all your questions below! Best Regards, Dave > Implemented __getreent () to return a private copy of struct reent,=20 > and also hard coded __DYNAMIC_REENT__ and GETREENT_PROVIDED in=20 > sys/config.h to rule out any issue of passing in via build CFLAGS or=20 > the CFLAGS in configure.host. Things including errno seem to work but=20 > not totally making sense. > > As many library functions are still accessing the reent structure=20 > using _impure_ptr instead of calling my __getreent () function, for=20 > example, the CHECK_INIT (_REENT, fp) at the beginning of __swsetup_r=20 > (struct _reent *ptr, register FILE * fp). > > Questions: > > 1. Are the library functions expected to still use _impure_ptr instead=20 > of calling __getreent () when both __DYNAMIC_REENT__ and=20 > GETREENT_PROVIDED are hard coded in sys/config.h? > > If so, how do they provide reentrancy? Since _impure_ptr is a global=20 > pointer visible to all threads and threads can easily step on each=20 > other's toes trying to change fields in the reent structure pointed to=20 > by _impure_ptr concurrently. > > If not, what other MACROs or changes should I make so that all the=20 > library functions all use __getreent () instead of _impure_ptr? Is it=20 > okay to set _impure_ptr to a bad value such as NULL in this case, in=20 > order to catch any unintended access? > > 2. in the documentation on https://sourceware.org/newlib/, the=20 > following is mentioned as needed for syscalls stubs to return errno: > > #include > #undef errno > extern int errno; > > If I do include this part, all the syscalls stubs seem to do when they=20 > assign values to errno is setting the global int errno; inside=20 > reent.c. As user code built against the library don=E2=80=99t read out = that=20 > integer but instead calls __(), errno set by syscall stubs can't be=20 > read out by user code. > > If on the other hand I don=E2=80=99t include this part before my syscal= l=20 > stubs, the errno set by them do seem to work as they also set the copy=20 > in reent structures. What might I have missed here? > > 3. There were some old discussions about manually changing _impure_ptr=20 > at each context switch. But I=E2=80=99m wondering about the validity of= such a=20 > method since it seems like a really clumsy maneuver for kernel code at=20 > CPL0 to reach into user space belonging to different binaries to=20 > change a global pointer. What's more, if manually changing _impure_ptr=20 > at each context switch is needed, then what would be the purpose of=20 > __DYNAMIC_REENT__, GETREENT_PROVIDED and implementing a __getreent ()=20 > to get a thread local version? > > 4. Is _global_impure_ptr thread safe? It is a bit concerning as it=20 > seems to be pointing to the same copy of impure_data that some=20 > libraries calls would access, and even if I try to change _impure_ptr=20 > at each context switch, some threads might still be accessing=20 > _global_impure_ptr concurrently? > > 5. There were also old discussions about having to provide mutex for=20 > malloc, is this still the case for newer versions of newlib like 4.10? > > Thanks! > Nick --=20 Dave Nadler, USA East Coast voice (978) 263-0097, drn@nadler.com, Skype Dave.Nadler1