Re: Some questions on reentrancy, __DYNAMIC_REENT__ and _impure_ptr

[Nick <cl26@nicolachel.net>]

Thanks Dave for bringing this up, you are totally right swapping 
_impure_ptr won't work for SMP.

Turns out that my build script wasn't clearing the build dir before each 
build, thus allowing objects compiled with different configs (different 
implementations of _REENT) to be linked together and caused all the 
strangeness.

After fixing that, __DYNAMIC_REENT__ worked as expected and I can now 
also set _impure_ptr to NULL in crt0 without issues.

The mutex/lock part seems a bit more straightforward than reent. 
Multiple threads from the same process can malloc/free in a loop without 
causing any obvious damage to each other now, although I probably need 
to test more to be sure that the heap itself is still healthy.

Thanks Jeff and Dave for helping out!

Nick

2021-04-02 07:34 に Dave Nadler さんは書きました:
> Thanks Jeff also for answering the mutex question I missed.
> One other issue I should have mentioned that becomes more important
> daily:
> If there are multiple processors the simple _impure_ptr mechanism
> won't work...
> Good luck and do let us know what you come up with,
> Best Regards, Dave
> 
> On 4/1/2021 11:39 PM, Nick wrote:
> 
>> Thanks Dave, haha we can never have enough of anything =)
>> 
>> It is a bit more challenging for me as AFAIK FreeRTOS is statically
>> linked so it can easily see and change the _impure_ptr pointer. But
>> in my case, the kernel is a standalone binary which loads other
>> programs (that are linked with newlib, elf format) at runtime, so it
>> either has to parse for that pointer during load, or require special
>> arrangement in crt0 to report the pointer's location during process
>> init.
>> 
>> It's feasible, but I'm hoping to get the __DYNAMIC_REENT__ method to
>> work as it seems quite a bit cleaner.
>> 
>> Nick
>> 
>> 2021-04-01 10:48 に Dave Nadler さんは書きました:
>> On 4/1/2021 12:58 AM, Nick wrote:
>> 
>> Hi,
>> 
>> I've been trying to enable reentrancy of newlib on a home brew
>> kernel for the x86 platform and have some questions on how various
>> pieces all fits together.
>> 
>> Oy, we can never have enough kernels ;-)
>> 
>> I'm not familiar with all the possible permutations.
>> In FreeRTOS, the scheduler simply switches _impure_ptr before each
>> context switch.
>> This is perfectly thread safe given:
>> - the read/write of this ptr is atomic (true on the architectures I
>> know), and
>> - no ISR use of anything in the RTL requiring this (ie no malloc,
>> strtok, etc. in ISR)
>> Here's the code from FreeRTOS:
>> 
>> #if ( configUSE_NEWLIB_REENTRANT == 1 )
>> {
>> /* Switch Newlib's _impure_ptr variable to point to the
>> _reent
>> structure specific to this task.
>> See the third party link
>> http://www.nadler.com/embedded/newlibAndFreeRTOS.html
>> for additional information. */
>> _impure_ptr = &( pxCurrentTCB->xNewLib_reent );
>> }
>> #endif /* configUSE_NEWLIB_REENTRANT */
>> 
>> I hope that clears up all your questions below!
>> Best Regards, Dave
>> 
>> Implemented __getreent () to return a private copy of struct reent,
>> and also hard coded __DYNAMIC_REENT__ and GETREENT_PROVIDED in
>> sys/config.h to rule out any issue of passing in via build CFLAGS or
>> 
>> the CFLAGS in configure.host. Things including errno seem to work
>> but not totally making sense.
>> 
>> As many library functions are still accessing the reent structure
>> using _impure_ptr instead of calling my __getreent () function, for
>> example, the CHECK_INIT (_REENT, fp) at the beginning of __swsetup_r
>> 
>> (struct _reent *ptr, register FILE * fp).
>> 
>> Questions:
>> 
>> 1. Are the library functions expected to still use _impure_ptr
>> instead of calling __getreent () when both __DYNAMIC_REENT__ and
>> GETREENT_PROVIDED are hard coded in sys/config.h?
>> 
>> If so, how do they provide reentrancy? Since _impure_ptr is a global
>> 
>> pointer visible to all threads and threads can easily step on each
>> other's toes trying to change fields in the reent structure pointed
>> to by _impure_ptr concurrently.
>> 
>> If not, what other MACROs or changes should I make so that all the
>> library functions all use __getreent () instead of _impure_ptr? Is
>> it okay to set _impure_ptr to a bad value such as NULL in this case,
>> 
>> in order to catch any unintended access?
>> 
>> 2. in the documentation on https://sourceware.org/newlib/, the
>> following is mentioned as needed for syscalls stubs to return errno:
>> 
>> 
>> #include <errno.h>
>> #undef errno
>> extern int errno;
>> 
>> If I do include this part, all the syscalls stubs seem to do when
>> they assign values to errno is setting the global int errno; inside
>> reent.c. As user code built against the library don’t read out
>> that integer but instead calls __(), errno set by syscall stubs
>> can't be read out by user code.
>> 
>> If on the other hand I don’t include this part before my syscall
>> stubs, the errno set by them do seem to work as they also set the
>> copy in reent structures. What might I have missed here?
>> 
>> 3. There were some old discussions about manually changing
>> _impure_ptr at each context switch. But I’m wondering about the
>> validity of such a method since it seems like a really clumsy
>> maneuver for kernel code at CPL0 to reach into user space belonging
>> to different binaries to change a global pointer. What's more, if
>> manually changing _impure_ptr at each context switch is needed, then
>> 
>> what would be the purpose of __DYNAMIC_REENT__, GETREENT_PROVIDED
>> and implementing a __getreent () to get a thread local version?
>> 
>> 4. Is _global_impure_ptr thread safe? It is a bit concerning as it
>> seems to be pointing to the same copy of impure_data that some
>> libraries calls would access, and even if I try to change
>> _impure_ptr at each context switch, some threads might still be
>> accessing _global_impure_ptr concurrently?
>> 
>> 5. There were also old discussions about having to provide mutex for
>> 
>> malloc, is this still the case for newer versions of newlib like
>> 4.10?
>> 
>> Thanks!
>> Nick
> 
> --
> Dave Nadler, USA East Coast voice (978) 263-0097, drn@nadler.com,
> Skype
>  Dave.Nadler1