From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from quail.birch.relay.mailchannels.net (quail.birch.relay.mailchannels.net [23.83.209.151]) by sourceware.org (Postfix) with ESMTPS id 9E057385BF99 for ; Fri, 2 Apr 2021 11:34:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 9E057385BF99 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=nadler.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=drn@nadler.com X-Sender-Id: dreamhost|x-authsender|drn@nadler.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 1ED4F6427AF; Fri, 2 Apr 2021 11:34:35 +0000 (UTC) Received: from pdx1-sub0-mail-a54.g.dreamhost.com (100-105-161-106.trex.outbound.svc.cluster.local [100.105.161.106]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id B34E4642620; Fri, 2 Apr 2021 11:34:34 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|drn@nadler.com Received: from pdx1-sub0-mail-a54.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.105.161.106 (trex/6.1.1); Fri, 02 Apr 2021 11:34:35 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|drn@nadler.com X-MailChannels-Auth-Id: dreamhost X-Coil-Dime: 3044c0ef06732d00_1617363275001_2176874189 X-MC-Loop-Signature: 1617363275001:2545386077 X-MC-Ingress-Time: 1617363275001 Received: from pdx1-sub0-mail-a54.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a54.g.dreamhost.com (Postfix) with ESMTP id 6D75A7E467; Fri, 2 Apr 2021 11:34:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=nadler.com; h=subject:to :cc:references:from:message-id:date:mime-version:in-reply-to :content-type; s=nadler.com; bh=SOFBpmgXwhiO3Gk5N17+mEZYCbQ=; b= ZznlbbweU2O369l7jab2uAREE4phvgFRe2M+lXqElMtZyharnqRoTuPuBG/tEJGQ APvO2lm5mxMbAJVu/RqmsklZcaWv1fhRwPl+6UdqoUQZfm5D8txP1LpS++bhPMxB oVzO9NW14oCqT61scf+4cJLHAIGSBZaZSU863OJuRdc= Received: from [192.168.1.3] (pool-72-74-171-157.bstnma.fios.verizon.net [72.74.171.157]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: drn@nadler.com) by pdx1-sub0-mail-a54.g.dreamhost.com (Postfix) with ESMTPSA id ACE317E453; Fri, 2 Apr 2021 11:34:33 +0000 (UTC) Subject: Re: Some questions on reentrancy, __DYNAMIC_REENT__ and _impure_ptr To: Nick Cc: newlib@sourceware.org References: <05ad18dabb56d23d26f1d9c493bcce0c@nicolachel.net> X-DH-BACKEND: pdx1-sub0-mail-a54 From: Dave Nadler Message-ID: <15c0dc57-264d-18b8-5589-87e50288f39f@nadler.com> Date: Fri, 2 Apr 2021 07:34:31 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: <05ad18dabb56d23d26f1d9c493bcce0c@nicolachel.net> Content-Language: en-US X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_00, BODY_8BITS, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, HTML_MESSAGE, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: newlib@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Newlib mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Apr 2021 11:34:39 -0000 Thanks Jeff also for answering the mutex question I missed. One other issue I should have mentioned that becomes more important daily= : If there are multiple processors the simple _impure_ptr mechanism won't=20 work... Good luck and do let us know what you come up with, Best Regards, Dave On 4/1/2021 11:39 PM, Nick wrote: > Thanks Dave, haha we can never have enough of anything =3D) > > It is a bit more challenging for me as AFAIK FreeRTOS is statically=20 > linked so it can easily see and change the _impure_ptr pointer. But in=20 > my case, the kernel is a standalone binary which loads other programs=20 > (that are linked with newlib, elf format) at runtime, so it either has=20 > to parse for that pointer during load, or require special arrangement=20 > in crt0 to report the pointer's location during process init. > > It's feasible, but I'm hoping to get the __DYNAMIC_REENT__ method to=20 > work as it seems quite a bit cleaner. > > Nick > > 2021-04-01 10:48 =E3=81=AB Dave Nadler =E3=81=95=E3=82=93=E3=81=AF=E6=9B= =B8=E3=81=8D=E3=81=BE=E3=81=97=E3=81=9F: >> On 4/1/2021 12:58 AM, Nick wrote: >> >>> Hi, >>> >>> I've been trying to enable reentrancy of newlib on a home brew >>> kernel for the x86 platform and have some questions on how various >>> pieces all fits together. >> >> Oy, we can never have enough kernels ;-) >> >> I'm not familiar with all the possible permutations. >> In FreeRTOS, the scheduler simply switches _impure_ptr before each >> context switch. >> This is perfectly thread safe given: >> - the read/write of this ptr is atomic (true on the architectures I >> know), and >> - no ISR use of anything in the RTL requiring this (ie no malloc, >> strtok, etc. in ISR) >> Here's the code from FreeRTOS: >> >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 #if ( configUSE_NEWLIB_REEN= TRANT =3D=3D 1 ) >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 { >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* = Switch Newlib's _impure_ptr variable to point to the >> _reent >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 str= ucture specific to this task. >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 See= the third party link >> http://www.nadler.com/embedded/newlibAndFreeRTOS.html >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 for= additional information. */ >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 _im= pure_ptr =3D &( pxCurrentTCB->xNewLib_reent ); >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 #endif /* configUSE_NEWLIB_= REENTRANT */ >> >> I hope that clears up all your questions below! >> Best Regards, Dave >> >>> Implemented __getreent () to return a private copy of struct reent, >>> and also hard coded __DYNAMIC_REENT__ and GETREENT_PROVIDED in >>> sys/config.h to rule out any issue of passing in via build CFLAGS or >>> the CFLAGS in configure.host. Things including errno seem to work >>> but not totally making sense. >>> >>> As many library functions are still accessing the reent structure >>> using _impure_ptr instead of calling my __getreent () function, for >>> example, the CHECK_INIT (_REENT, fp) at the beginning of __swsetup_r >>> (struct _reent *ptr, register FILE * fp). >>> >>> Questions: >>> >>> 1. Are the library functions expected to still use _impure_ptr >>> instead of calling __getreent () when both __DYNAMIC_REENT__ and >>> GETREENT_PROVIDED are hard coded in sys/config.h? >>> >>> If so, how do they provide reentrancy? Since _impure_ptr is a global >>> pointer visible to all threads and threads can easily step on each >>> other's toes trying to change fields in the reent structure pointed >>> to by _impure_ptr concurrently. >>> >>> If not, what other MACROs or changes should I make so that all the >>> library functions all use __getreent () instead of _impure_ptr? Is >>> it okay to set _impure_ptr to a bad value such as NULL in this case, >>> in order to catch any unintended access? >>> >>> 2. in the documentation on https://sourceware.org/newlib/, the >>> following is mentioned as needed for syscalls stubs to return errno: >>> >>> >>> #include >>> #undef errno >>> extern int errno; >>> >>> If I do include this part, all the syscalls stubs seem to do when >>> they assign values to errno is setting the global int errno; inside >>> reent.c. As user code built against the library don=E2=80=99t read ou= t >>> that integer but instead calls __(), errno set by syscall stubs >>> can't be read out by user code. >>> >>> If on the other hand I don=E2=80=99t include this part before my sysc= all >>> stubs, the errno set by them do seem to work as they also set the >>> copy in reent structures. What might I have missed here? >>> >>> 3. There were some old discussions about manually changing >>> _impure_ptr at each context switch. But I=E2=80=99m wondering about t= he >>> validity of such a method since it seems like a really clumsy >>> maneuver for kernel code at CPL0 to reach into user space belonging >>> to different binaries to change a global pointer. What's more, if >>> manually changing _impure_ptr at each context switch is needed, then >>> what would be the purpose of __DYNAMIC_REENT__, GETREENT_PROVIDED >>> and implementing a __getreent () to get a thread local version? >>> >>> 4. Is _global_impure_ptr thread safe? It is a bit concerning as it >>> seems to be pointing to the same copy of impure_data that some >>> libraries calls would access, and even if I try to change >>> _impure_ptr at each context switch, some threads might still be >>> accessing _global_impure_ptr concurrently? >>> >>> 5. There were also old discussions about having to provide mutex for >>> malloc, is this still the case for newer versions of newlib like >>> 4.10? >>> >>> Thanks! >>> Nick --=20 Dave Nadler, USA East Coast voice (978) 263-0097, drn@nadler.com, Skype Dave.Nadler1