Trying to build newlib with -DMALLOC_PROVIDED

Trying to build newlib with -DMALLOC_PROVIDED

[Stephan Mühlstrasser]

Hi,

I'm making my first attempt to build newlib for an ARM bare-metal 
environment from the current master branch. malloc(), free() etc. come 
from the bare-metal library, therefore I'm trying to build newlib with 
-DMALLOC_PROVIDED, using other configurations in newlib/configure.host 
as a blueprint.

I made this modification in newlib/configure.host for my target:

   arm*-*-circleraspi0)
     syscall_dir=syscalls
     newlib_cflags="${newlib_cflags} -DMALLOC_PROVIDED"
     ;;

I can successfully build newlib, and when I try to link a test 
application that just calls printf() I get the following unresolved symbols:

E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-vfprintf.o): 
In function `_vfprintf_r':
vfprintf.c:(.text+0x1fcc): undefined reference to `_free_r'
vfprintf.c:(.text+0x215c): undefined reference to `_free_r'
E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-wsetup.o): 
In function `__swsetup_r':
wsetup.c:(.text+0xcc): undefined reference to `_free_r'
E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-fflush.o): 
In function `__sflush_r':
fflush.c:(.text+0x24c): undefined reference to `_free_r'
E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-findfp.o): 
In function `__sfmoreglue':
findfp.c:(.text+0x130): undefined reference to `_malloc_r'

and also

E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-syscalls.o): 
In function `_sbrk':
syscalls.c:(.text+0x984): undefined reference to `end'

I read various tutorials about porting newlib, and they suggest that it 
should be possible to build without the reentrant versions of the memory 
management functions. But I don't understand how that is supposed to 
work in the source. For example in malloc.h:

extern _PTR malloc _PARAMS ((size_t));
#ifdef __CYGWIN__
#undef _malloc_r
#define _malloc_r(r, s) malloc (s)
#else
extern _PTR _malloc_r _PARAMS ((struct _reent *, size_t));
#endif

This looks like that only for __CYGWIN__ the reentrant variants are 
defined away.

How can I build newlib for an environment that provides the memory 
management functions that are already thread-safe?

Thanks in advance for any suggestions!

Best regards
Stephan

Re: Trying to build newlib with -DMALLOC_PROVIDED

[Stephan Mühlstrasser]

Hello Jeff,

Am 19.04.2017 um 20:27 schrieb Jeff Johnston:
> Stephan,
>
> From newlib/libc/include/reent.h:
>
>    The target may provide the needed syscalls by any of the following:
>
>    1) Define the reentrant versions of the syscalls directly.
>       (eg: _open_r, _close_r, etc.).  Please keep the namespace clean.
>       When you do this, set "syscall_dir" to "syscalls" and add
>       -DREENTRANT_SYSCALLS_PROVIDED to newlib_cflags in configure.host.
>
>    2) Define namespace clean versions of the system calls by prefixing
>       them with '_' (eg: _open, _close, etc.).  Technically, there won't be
>       true reentrancy at the syscall level, but the library will be
> namespace
>       clean.
>       When you do this, set "syscall_dir" to "syscalls" in configure.host.
>
>    3) Define or otherwise provide the regular versions of the syscalls
>       (eg: open, close, etc.).  The library won't be reentrant nor namespace
>       clean, but at least it will work.
>       When you do this, add -DMISSING_SYSCALL_NAMES to newlib_cflags in
>       configure.host.
>
>    4) Define or otherwise provide the regular versions of the syscalls,
>       and do not supply functional interfaces for any of the reentrant
>       calls. With this method, the reentrant syscalls are redefined to
>       directly call the regular system call without the reentrancy argument.
>       When you do this, specify both -DREENTRANT_SYSCALLS_PROVIDED and
>       -DMISSING_SYSCALL_NAMES via newlib_cflags in configure.host and do
>       not specify "syscall_dir".

thanks for the pointer. I had read this, but I hadn't understood that 
"system calls" here also applies to the memory management functions.

I'm sorry, I still don't get it. Could you please explain briefly what I 
need to do in the following context:

- For a start I want to compile newlib without implementing any of the 
system calls now. This should give me the dummy implementations from 
libnosys, right?

- I have reentrant versions of malloc(), free(), realloc(), calloc() in 
a separate library of the target environment.

Thank you
Stephan

> Regards,
>
> -- Jeff J.
>
> On Wed, Apr 19, 2017 at 2:15 PM, Stephan Mühlstrasser
> <stephan.muehlstrasser@web.de <mailto:stephan.muehlstrasser@web.de>> wrote:
>
>     Hi,
>
>     I'm making my first attempt to build newlib for an ARM bare-metal
>     environment from the current master branch. malloc(), free() etc.
>     come from the bare-metal library, therefore I'm trying to build
>     newlib with -DMALLOC_PROVIDED, using other configurations in
>     newlib/configure.host as a blueprint.
>
>     I made this modification in newlib/configure.host for my target:
>
>       arm*-*-circleraspi0)
>         syscall_dir=syscalls
>         newlib_cflags="${newlib_cflags} -DMALLOC_PROVIDED"
>         ;;
>
>     I can successfully build newlib, and when I try to link a test
>     application that just calls printf() I get the following unresolved
>     symbols:
>
>     E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-vfprintf.o):
>     In function `_vfprintf_r':
>     vfprintf.c:(.text+0x1fcc): undefined reference to `_free_r'
>     vfprintf.c:(.text+0x215c): undefined reference to `_free_r'
>     E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-wsetup.o):
>     In function `__swsetup_r':
>     wsetup.c:(.text+0xcc): undefined reference to `_free_r'
>     E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-fflush.o):
>     In function `__sflush_r':
>     fflush.c:(.text+0x24c): undefined reference to `_free_r'
>     E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-findfp.o):
>     In function `__sfmoreglue':
>     findfp.c:(.text+0x130): undefined reference to `_malloc_r'
>
>     and also
>
>     E:/Users/stm/Documents/GIT/newlib-raspi0/arm-none-circleraspi0/lib/libc.a(lib_a-syscalls.o):
>     In function `_sbrk':
>     syscalls.c:(.text+0x984): undefined reference to `end'
>
>     I read various tutorials about porting newlib, and they suggest that
>     it should be possible to build without the reentrant versions of the
>     memory management functions. But I don't understand how that is
>     supposed to work in the source. For example in malloc.h:
>
>     extern _PTR malloc _PARAMS ((size_t));
>     #ifdef __CYGWIN__
>     #undef _malloc_r
>     #define _malloc_r(r, s) malloc (s)
>     #else
>     extern _PTR _malloc_r _PARAMS ((struct _reent *, size_t));
>     #endif
>
>     This looks like that only for __CYGWIN__ the reentrant variants are
>     defined away.
>
>     How can I build newlib for an environment that provides the memory
>     management functions that are already thread-safe?
>
>     Thanks in advance for any suggestions!
>
>     Best regards
>     Stephan
>
>

Re: Trying to build newlib with -DMALLOC_PROVIDED

[Stephan Mühlstrasser]

Am 19.04.2017 um 22:11 schrieb Jeff Johnston:
> ...
>
> The newlib library internally calls the reentrant versions of the
> malloc-family routines (_calloc_r, _free_r, _malloc_r, and _realloc_r).
> You will have to supply these.  I suggest stubs for these that call your
> non-reentrant routines something along the line of the stubs in
> libc/reent directory.

Ah, this wasn't clear to me. I will implement the stubs, and I will see 
how it goes.

Thanks
Stephan