From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from direct.host-care.com (direct.host-care.com [198.136.54.115]) by sourceware.org (Postfix) with ESMTPS id 393D7385700F for ; Sat, 3 Apr 2021 01:43:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 393D7385700F Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=nicolachel.net Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=cl26@nicolachel.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nicolachel.net; s=default; h=Content-Transfer-Encoding:Content-Type: Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:MIME-Version:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=sJfefyzA5DcMbCy7iR1V5RezhH9NsXd5C2/QcPljnDE=; b=yZfUYDk/eUKSKGPtcwlKCral2g OviLNak790DSMwTutoJ7hfEEX57lbV3kf0lJiZl3i2VJJfp4txOAp08LlZkC3oQDc1UZ7tbUGjDfz kEmpY9HrJgYqSTnR2ao9sS0TA02In5X6i1+3rTTU0BCM/p1R/EVy0c5H1XbUjfrbYra9ZDPLRnpNl aKLWZhi+9BEg3uNGO5IzevTlmbtoG4CVuSijUCgOTjxk5X9tz9c/uTfTFH/9gBXvdrD0Hh1kP454y 7PruulGbzhVv9lfGhA76hO+1Xkx9fbIAhux09vS7wu1CCDIqYVXeibbtY6fUP/etWTmYcKQe2RhHv kJphwhBA==; Received: from [::1] (port=41460 helo=direct.host-care.com) by direct.host-care.com with esmtpa (Exim 4.93) (envelope-from ) id 1lSVKP-0002bv-G5; Fri, 02 Apr 2021 21:43:57 -0400 MIME-Version: 1.0 Date: Fri, 02 Apr 2021 21:43:57 -0400 From: Nick To: Dave Nadler Cc: newlib@sourceware.org Subject: Re: Some questions on reentrancy, __DYNAMIC_REENT__ and _impure_ptr In-Reply-To: <15c0dc57-264d-18b8-5589-87e50288f39f@nadler.com> References: <05ad18dabb56d23d26f1d9c493bcce0c@nicolachel.net> <15c0dc57-264d-18b8-5589-87e50288f39f@nadler.com> Message-ID: <05cca98f0732a6bdbbc4636a1f938fda@nicolachel.net> X-Sender: cl26@nicolachel.net User-Agent: Roundcube Webmail/1.3.15 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - direct.host-care.com X-AntiAbuse: Original Domain - sourceware.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - nicolachel.net X-Get-Message-Sender-Via: direct.host-care.com: authenticated_id: webmaster@nicolachel.net X-Authenticated-Sender: direct.host-care.com: webmaster@nicolachel.net X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00, BODY_8BITS, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: newlib@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Newlib mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Apr 2021 01:44:00 -0000 Thanks Dave for bringing this up, you are totally right swapping _impure_ptr won't work for SMP. Turns out that my build script wasn't clearing the build dir before each build, thus allowing objects compiled with different configs (different implementations of _REENT) to be linked together and caused all the strangeness. After fixing that, __DYNAMIC_REENT__ worked as expected and I can now also set _impure_ptr to NULL in crt0 without issues. The mutex/lock part seems a bit more straightforward than reent. Multiple threads from the same process can malloc/free in a loop without causing any obvious damage to each other now, although I probably need to test more to be sure that the heap itself is still healthy. Thanks Jeff and Dave for helping out! Nick 2021-04-02 07:34 に Dave Nadler さんは書きました: > Thanks Jeff also for answering the mutex question I missed. > One other issue I should have mentioned that becomes more important > daily: > If there are multiple processors the simple _impure_ptr mechanism > won't work... > Good luck and do let us know what you come up with, > Best Regards, Dave > > On 4/1/2021 11:39 PM, Nick wrote: > >> Thanks Dave, haha we can never have enough of anything =) >> >> It is a bit more challenging for me as AFAIK FreeRTOS is statically >> linked so it can easily see and change the _impure_ptr pointer. But >> in my case, the kernel is a standalone binary which loads other >> programs (that are linked with newlib, elf format) at runtime, so it >> either has to parse for that pointer during load, or require special >> arrangement in crt0 to report the pointer's location during process >> init. >> >> It's feasible, but I'm hoping to get the __DYNAMIC_REENT__ method to >> work as it seems quite a bit cleaner. >> >> Nick >> >> 2021-04-01 10:48 に Dave Nadler さんは書きました: >> On 4/1/2021 12:58 AM, Nick wrote: >> >> Hi, >> >> I've been trying to enable reentrancy of newlib on a home brew >> kernel for the x86 platform and have some questions on how various >> pieces all fits together. >> >> Oy, we can never have enough kernels ;-) >> >> I'm not familiar with all the possible permutations. >> In FreeRTOS, the scheduler simply switches _impure_ptr before each >> context switch. >> This is perfectly thread safe given: >> - the read/write of this ptr is atomic (true on the architectures I >> know), and >> - no ISR use of anything in the RTL requiring this (ie no malloc, >> strtok, etc. in ISR) >> Here's the code from FreeRTOS: >> >> #if ( configUSE_NEWLIB_REENTRANT == 1 ) >> { >> /* Switch Newlib's _impure_ptr variable to point to the >> _reent >> structure specific to this task. >> See the third party link >> http://www.nadler.com/embedded/newlibAndFreeRTOS.html >> for additional information. */ >> _impure_ptr = &( pxCurrentTCB->xNewLib_reent ); >> } >> #endif /* configUSE_NEWLIB_REENTRANT */ >> >> I hope that clears up all your questions below! >> Best Regards, Dave >> >> Implemented __getreent () to return a private copy of struct reent, >> and also hard coded __DYNAMIC_REENT__ and GETREENT_PROVIDED in >> sys/config.h to rule out any issue of passing in via build CFLAGS or >> >> the CFLAGS in configure.host. Things including errno seem to work >> but not totally making sense. >> >> As many library functions are still accessing the reent structure >> using _impure_ptr instead of calling my __getreent () function, for >> example, the CHECK_INIT (_REENT, fp) at the beginning of __swsetup_r >> >> (struct _reent *ptr, register FILE * fp). >> >> Questions: >> >> 1. Are the library functions expected to still use _impure_ptr >> instead of calling __getreent () when both __DYNAMIC_REENT__ and >> GETREENT_PROVIDED are hard coded in sys/config.h? >> >> If so, how do they provide reentrancy? Since _impure_ptr is a global >> >> pointer visible to all threads and threads can easily step on each >> other's toes trying to change fields in the reent structure pointed >> to by _impure_ptr concurrently. >> >> If not, what other MACROs or changes should I make so that all the >> library functions all use __getreent () instead of _impure_ptr? Is >> it okay to set _impure_ptr to a bad value such as NULL in this case, >> >> in order to catch any unintended access? >> >> 2. in the documentation on https://sourceware.org/newlib/, the >> following is mentioned as needed for syscalls stubs to return errno: >> >> >> #include >> #undef errno >> extern int errno; >> >> If I do include this part, all the syscalls stubs seem to do when >> they assign values to errno is setting the global int errno; inside >> reent.c. As user code built against the library don’t read out >> that integer but instead calls __(), errno set by syscall stubs >> can't be read out by user code. >> >> If on the other hand I don’t include this part before my syscall >> stubs, the errno set by them do seem to work as they also set the >> copy in reent structures. What might I have missed here? >> >> 3. There were some old discussions about manually changing >> _impure_ptr at each context switch. But I’m wondering about the >> validity of such a method since it seems like a really clumsy >> maneuver for kernel code at CPL0 to reach into user space belonging >> to different binaries to change a global pointer. What's more, if >> manually changing _impure_ptr at each context switch is needed, then >> >> what would be the purpose of __DYNAMIC_REENT__, GETREENT_PROVIDED >> and implementing a __getreent () to get a thread local version? >> >> 4. Is _global_impure_ptr thread safe? It is a bit concerning as it >> seems to be pointing to the same copy of impure_data that some >> libraries calls would access, and even if I try to change >> _impure_ptr at each context switch, some threads might still be >> accessing _global_impure_ptr concurrently? >> >> 5. There were also old discussions about having to provide mutex for >> >> malloc, is this still the case for newer versions of newlib like >> 4.10? >> >> Thanks! >> Nick > > -- > Dave Nadler, USA East Coast voice (978) 263-0097, drn@nadler.com, > Skype > Dave.Nadler1