impure_data never reclaimed

impure_data never reclaimed

[Fabian Vogt]

Hi,

I'm using newlib as libc on top of an OS which provides malloc and free.
Threads aren't supported, but programs can be started and stopped. The used
heap is shared between all programs, so leaked memory is lost forever. Thus
it is important that after a clean exit, every allocation was properly freed.

Currently this simple program leaks three allocations though:

#include <stdio.h>
int main()
{
    setbuf(stdout, NULL); // This would be properly freed, avoid distraction
    printf("This is 0.5 as a float: %f\n", 0.5f);
}

Those are from Balloc, which allocates a list also for later use.
FWICT, _reclaim_reent takes care of freeing the list and its contents, but:
- _reclaim_reent is never called for _impure_ptr (== &impure_data)
- _reclaim_reent does nothing if impure_ptr is passed

Is the OS glue code supposed to do something like this in _exit?

    struct _reent *global_reent = _impure_ptr;
    _impure_ptr = NULL;
    _reclaim_reent(global_reent);

With that the leaks are gone, but it seems a bit odd to me.

Thanks,
Fabian

Re: impure_data never reclaimed (unless you add the required cleanup code)

[Dave Nadler]

Hi Fabian - Yes, that is correct, you need that cleanup code.
However, you must also allocate (and clean up) the reentrancy structure 
for each task/thread.
FreeRTOS correctly does this on task/thread creation and cleanup (with 
#define configUSE_NEWLIB_REENTRANT  1).
Hope that helps!
Best Regards, ,Dave

On 12/5/2020 9:27 AM, Fabian Vogt wrote:
> Hi,
>
> I'm using newlib as libc on top of an OS which provides malloc and free.
> Threads aren't supported, but programs can be started and stopped. The used
> heap is shared between all programs, so leaked memory is lost forever. Thus
> it is important that after a clean exit, every allocation was properly freed.
>
> Currently this simple program leaks three allocations though:
>
> #include <stdio.h>
> int main()
> {
>      setbuf(stdout, NULL); // This would be properly freed, avoid distraction
>      printf("This is 0.5 as a float: %f\n", 0.5f);
> }
>
> Those are from Balloc, which allocates a list also for later use.
> FWICT, _reclaim_reent takes care of freeing the list and its contents, but:
> - _reclaim_reent is never called for _impure_ptr (== &impure_data)
> - _reclaim_reent does nothing if impure_ptr is passed
>
> Is the OS glue code supposed to do something like this in _exit?
>
>      struct _reent *global_reent = _impure_ptr;
>      _impure_ptr = NULL;
>      _reclaim_reent(global_reent);
>
> With that the leaks are gone, but it seems a bit odd to me.
>
> Thanks,
> Fabian


-- 
Dave Nadler, USA East Coast voice (978) 263-0097, drn@nadler.com, Skype
  Dave.Nadler1

Re: impure_data never reclaimed (unless you add the required cleanup code)

[Fabian Vogt]

Hi,

Am Samstag, 5. Dezember 2020, 16:48:57 CET schrieb Dave Nadler:
> Hi Fabian - Yes, that is correct, you need that cleanup code.
> However, you must also allocate (and clean up) the reentrancy structure 
> for each task/thread.
> FreeRTOS correctly does this on task/thread creation and cleanup (with 
> #define configUSE_NEWLIB_REENTRANT  1).
> Hope that helps!

While searching for more info about this I found your related thread on this ML
from some time ago, but I think it's not the same situation.

newlib/libc/reent/impure.c has:
static struct _reent __ATTRIBUTE_IMPURE_DATA__ impure_data = _REENT_INIT (impure_data);
struct _reent *__ATTRIBUTE_IMPURE_PTR__ _impure_ptr = &impure_data;

So the struct for the main (without thread support, the only) thread is
allocated statically by newlib.

It would be possible to do it like FreeRTOS and just allocate a new struct for
_impure_ptr for the main thread, but that would leave impure_data unused and
thus waste space and need more code on top.

Cheers,
Fabian

> Best Regards, ,Dave
> 
> On 12/5/2020 9:27 AM, Fabian Vogt wrote:
> > Hi,
> >
> > I'm using newlib as libc on top of an OS which provides malloc and free.
> > Threads aren't supported, but programs can be started and stopped. The used
> > heap is shared between all programs, so leaked memory is lost forever. Thus
> > it is important that after a clean exit, every allocation was properly freed.
> >
> > Currently this simple program leaks three allocations though:
> >
> > #include <stdio.h>
> > int main()
> > {
> >      setbuf(stdout, NULL); // This would be properly freed, avoid distraction
> >      printf("This is 0.5 as a float: %f\n", 0.5f);
> > }
> >
> > Those are from Balloc, which allocates a list also for later use.
> > FWICT, _reclaim_reent takes care of freeing the list and its contents, but:
> > - _reclaim_reent is never called for _impure_ptr (== &impure_data)
> > - _reclaim_reent does nothing if impure_ptr is passed
> >
> > Is the OS glue code supposed to do something like this in _exit?
> >
> >      struct _reent *global_reent = _impure_ptr;
> >      _impure_ptr = NULL;
> >      _reclaim_reent(global_reent);
> >
> > With that the leaks are gone, but it seems a bit odd to me.
> >
> > Thanks,
> > Fabian

Re: impure_data never reclaimed (unless you add the required cleanup code)

[Dave Nadler]

Apologies Fabian but I'm confused.
You say 'operating system', but there is no multi-tasking/threading etc???
In that case, yes, there is exactly one reentrancy structure allocated 
by newlib,
and it can be reused by sequential 'programs'...
That reentrancy structure points to storage allocated to support assorted
functions within newlib, including dtoa (used within printf'ing a double).
Hope that helps,
Best Regards, Dave

On 12/5/2020 10:58 AM, Fabian Vogt wrote:
> Hi,
>
> Am Samstag, 5. Dezember 2020, 16:48:57 CET schrieb Dave Nadler:
>> Hi Fabian - Yes, that is correct, you need that cleanup code.
>> However, you must also allocate (and clean up) the reentrancy structure
>> for each task/thread.
>> FreeRTOS correctly does this on task/thread creation and cleanup (with
>> #define configUSE_NEWLIB_REENTRANT  1).
>> Hope that helps!
> While searching for more info about this I found your related thread on this ML
> from some time ago, but I think it's not the same situation.
>
> newlib/libc/reent/impure.c has:
> static struct _reent __ATTRIBUTE_IMPURE_DATA__ impure_data = _REENT_INIT (impure_data);
> struct _reent *__ATTRIBUTE_IMPURE_PTR__ _impure_ptr = &impure_data;
>
> So the struct for the main (without thread support, the only) thread is
> allocated statically by newlib.
>
> It would be possible to do it like FreeRTOS and just allocate a new struct for
> _impure_ptr for the main thread, but that would leave impure_data unused and
> thus waste space and need more code on top.
>
> Cheers,
> Fabian
>
>> Best Regards, ,Dave
>>
>> On 12/5/2020 9:27 AM, Fabian Vogt wrote:
>>> Hi,
>>>
>>> I'm using newlib as libc on top of an OS which provides malloc and free.
>>> Threads aren't supported, but programs can be started and stopped. The used
>>> heap is shared between all programs, so leaked memory is lost forever. Thus
>>> it is important that after a clean exit, every allocation was properly freed.
>>>
>>> Currently this simple program leaks three allocations though:
>>>
>>> #include <stdio.h>
>>> int main()
>>> {
>>>       setbuf(stdout, NULL); // This would be properly freed, avoid distraction
>>>       printf("This is 0.5 as a float: %f\n", 0.5f);
>>> }
>>>
>>> Those are from Balloc, which allocates a list also for later use.
>>> FWICT, _reclaim_reent takes care of freeing the list and its contents, but:
>>> - _reclaim_reent is never called for _impure_ptr (== &impure_data)
>>> - _reclaim_reent does nothing if impure_ptr is passed
>>>
>>> Is the OS glue code supposed to do something like this in _exit?
>>>
>>>       struct _reent *global_reent = _impure_ptr;
>>>       _impure_ptr = NULL;
>>>       _reclaim_reent(global_reent);
>>>
>>> With that the leaks are gone, but it seems a bit odd to me.
>>>
>>> Thanks,
>>> Fabian


-- 
Dave Nadler, USA East Coast voice (978) 263-0097, drn@nadler.com, Skype
  Dave.Nadler1