suggestion for GCC (1)

suggestion for GCC (1)

[ali hagigat]

A necessary feature for GCC is to compile C/Assembly programs without
standard libraries for Intel architectures.

Re: suggestion for GCC (1)

[Kai Ruottu]

7.2.2011 11:55, ali hagigat kirjoitti:

> A necessary feature for GCC is to compile C/Assembly programs without
> standard libraries for Intel architectures.

That feature has always existed I think, even producing the executables
for GCC without any kind of target C library during the build has been
possible for the embedded targets like 'i386-aout', 'i386-coff' and
'i386-elf' (for different object formats). The problem maybe being in
that the GCC build needs target headers when producing 'libgcc' for the
target and these plus the target libraries when configuring and
producing 'libstdc++' for the target in the system target like Linux
cases where the native GCC build is the default build type and so
everyone is expected to accept the existence of '/lib', '/usr/include',
'/usr/lib' etc. for the build/host/target systems during the GCC build.

Many crosscompiler builders then cannot grok the need of the target C
library during a GCC build for Linux :o(

Re: suggestion for GCC (1)

[Kai Ruottu]

7.2.2011 12:23, Kai Ruottu kirjoitti:
> 7.2.2011 11:55, ali hagigat kirjoitti:
>
>> A necessary feature for GCC is to compile C/Assembly programs without
>> standard libraries for Intel architectures.
>
> That feature has always existed I think, even producing the executables
> for GCC without any kind of target C library during the build has been
> possible for the embedded targets like 'i386-aout', 'i386-coff' and
> 'i386-elf' (for different object formats).

Oops, of course this will succed for ANY target!  Getting at least
'libgcc' for the embedded targets then has always been possible!

Re: suggestion for GCC (1)

[ali hagigat]

Dear Kai, Thank you so much for the reply.

But I did not understand, can we compile a C/Intel assembly program
(and link) without any standard library?(like libgcc)
or at least libgcc is required? If I link libgcc ONLY on command line,
then I can generate stand alone object files?

On Mon, Feb 7, 2011 at 1:56 PM, Kai Ruottu <kai.ruottu@wippies.com> wrote:
> 7.2.2011 12:23, Kai Ruottu kirjoitti:
>>
>> 7.2.2011 11:55, ali hagigat kirjoitti:
>>
>>> A necessary feature for GCC is to compile C/Assembly programs without
>>> standard libraries for Intel architectures.
>>
>> That feature has always existed I think, even producing the executables
>> for GCC without any kind of target C library during the build has been
>> possible for the embedded targets like 'i386-aout', 'i386-coff' and
>> 'i386-elf' (for different object formats).
>
> Oops, of course this will succed for ANY target!  Getting at least
> 'libgcc' for the embedded targets then has always been possible!
>

Re: suggestion for GCC (1)

[ali hagigat]

Dear Kai,

If I can compile a C/assembly program without linking with standard
libraries by using -nostdlib, so why the manual of gcc says:
-------------------------------------------
Do not use the standard system startup files or libraries when
linking.  No startup files and only the libraries you specify will be
passed to the linker.  The compiler may generate calls to "memcmp",
"memset", "memcpy" and "memmove".
-------------------------------------------
So the compiler still calls "memcmp", "memset", "memcpy" and
"memmove".  While my code does not have them!!
Regards

On Mon, Feb 7, 2011 at 1:56 PM, Kai Ruottu <kai.ruottu@wippies.com> wrote:
> Oops, of course this will succed for ANY target!  Getting at least
> 'libgcc' for the embedded targets then has always been possible!

Re: suggestion for GCC (1)

[Ian Lance Taylor]

ali hagigat <hagigatali@gmail.com> writes:

> A necessary feature for GCC is to compile C/Assembly programs without
> standard libraries for Intel architectures.

This feature can not and will not be implemented.  Some supporting
routines are always required, particularly for gcc extensions like
nested functions and __attribute__ ((cleanup)).

Ian

Re: suggestion for GCC (1)

[Thomas Martitz]

Am 08.02.2011 00:04, schrieb Ian Lance Taylor:
> ali hagigat<hagigatali@gmail.com>  writes:
>
>> A necessary feature for GCC is to compile C/Assembly programs without
>> standard libraries for Intel architectures.
> This feature can not and will not be implemented.  Some supporting
> routines are always required, particularly for gcc extensions like
> nested functions and __attribute__ ((cleanup)).
>
> Ian

IIRC the functions Ali mentioned (mem*) are the only required ones to 
build working binaries without C library (i.e. for bare metal targets). 
We do it this way at Rockbox.

But I wonder why, the mem* functions are trivial to implement in plain C 
so why does one need to provide them?

Best regards.

Re: suggestion for GCC (1)

[Ian Lance Taylor]

Thomas Martitz <thomas.martitz@student.htw-berlin.de> writes:

> Am 08.02.2011 00:04, schrieb Ian Lance Taylor:
>> ali hagigat<hagigatali@gmail.com>  writes:
>>
>>> A necessary feature for GCC is to compile C/Assembly programs without
>>> standard libraries for Intel architectures.
>> This feature can not and will not be implemented.  Some supporting
>> routines are always required, particularly for gcc extensions like
>> nested functions and __attribute__ ((cleanup)).
>
> IIRC the functions Ali mentioned (mem*) are the only required ones to
> build working binaries without C library (i.e. for bare metal
> targets). We do it this way at Rockbox.
>
> But I wonder why, the mem* functions are trivial to implement in plain
> C so why does one need to provide them?

I took ali to be asking to build without any libraries at all, including
libgcc, which is what you get when you use -nostdlib.  That can't work.

I think you are talking about the case where we do use libgcc, but don't
use libc.  In other words: why don't we provide memcpy, etc., in libgcc,
or, rather, just call libgcc-specific routines?  The answer is that we
expect the library provider to have a highly optimized version of those
functions.  That is certainly the case when using glibc on GNU/Linux.
Since the library provider should already have a highly optimized
version, gcc doesn't bother providing one itself.

Of course we could still arrange to provide simple versions in some
additional library which is only linked after libc.  I would not be
opposed to that.  Real C programs, though, almost always call memcpy and
friends themselves, and real C programs always require some level of
supporting code.  The current situation doesn't really bother me.

Ian

Re: suggestion for GCC (1)

[David Brown]

On 08/02/2011 01:44, Ian Lance Taylor wrote:
> Thomas Martitz<thomas.martitz@student.htw-berlin.de>  writes:
>
>> Am 08.02.2011 00:04, schrieb Ian Lance Taylor:
>>> ali hagigat<hagigatali@gmail.com>   writes:
>>>
>>>> A necessary feature for GCC is to compile C/Assembly programs without
>>>> standard libraries for Intel architectures.
>>> This feature can not and will not be implemented.  Some supporting
>>> routines are always required, particularly for gcc extensions like
>>> nested functions and __attribute__ ((cleanup)).
>>
>> IIRC the functions Ali mentioned (mem*) are the only required ones to
>> build working binaries without C library (i.e. for bare metal
>> targets). We do it this way at Rockbox.
>>

Do you not use any sort of libc?  I do embedded programming on a number 
of targets, and while I avoid most of libc (there's seldom any evil 
"malloc" or "printf" calls in my code), some of it is useful.

>> But I wonder why, the mem* functions are trivial to implement in plain
>> C so why does one need to provide them?
>
> I took ali to be asking to build without any libraries at all, including
> libgcc, which is what you get when you use -nostdlib.  That can't work.
>

On most of the targets I use, libgcc is essential for supporting the 
language itself (if your target doesn't have division instructions, for 
example, then the supporting library functions are in libgcc).  My 
understanding is that libgcc contains what is needed for the C language, 
but is not supposed to contain anything from the C standard library.

> I think you are talking about the case where we do use libgcc, but don't
> use libc.  In other words: why don't we provide memcpy, etc., in libgcc,
> or, rather, just call libgcc-specific routines?  The answer is that we
> expect the library provider to have a highly optimized version of those
> functions.  That is certainly the case when using glibc on GNU/Linux.
> Since the library provider should already have a highly optimized
> version, gcc doesn't bother providing one itself.
>

The optimal implementation of something like memcpy is highly dependent 
on the target, so I agree it doesn't make sense to try to put it libgcc. 
  Even for the same architecture, "optimised" will mean different things 
on different platforms.  For a small ARM microcontroller, "optimised" 
will mean "small" - for an ARM netbook, "optimised" will mean "fast". 
The memcpy implementations in the target libc will presumably have an 
appropriate algorithm.

> Of course we could still arrange to provide simple versions in some
> additional library which is only linked after libc.  I would not be
> opposed to that.  Real C programs, though, almost always call memcpy and
> friends themselves, and real C programs always require some level of
> supporting code.  The current situation doesn't really bother me.
>

Doesn't gcc treat functions like memcpy as builtins?  Or is that only if 
you explicitly write __builtin__memcpy?  I would have thought that in 
many cases, a builtin version can be a lot smaller and faster than 
library code, since the compiler often knows about things like alignment 
and sizes and can thus generate better code.

Re: suggestion for GCC (1)

[Ian Lance Taylor]

David Brown <david@westcontrol.com> writes:

>> Of course we could still arrange to provide simple versions in some
>> additional library which is only linked after libc.  I would not be
>> opposed to that.  Real C programs, though, almost always call memcpy and
>> friends themselves, and real C programs always require some level of
>> supporting code.  The current situation doesn't really bother me.
>>
>
> Doesn't gcc treat functions like memcpy as builtins?  Or is that only
> if you explicitly write __builtin__memcpy?  I would have thought that
> in many cases, a builtin version can be a lot smaller and faster than
> library code, since the compiler often knows about things like
> alignment and sizes and can thus generate better code.

The compiler does by default treat functions like memcpy as builtins,
and it's true that when the compiler does know something about alignment
and sizes it can sometimes generate better code.  However, there are
many cases where the compiler does not know anything about the alignment
or size, and then the compiler can't do any better than the library
routine.

Ian

suggestion for GCC (1)

[ali hagigat]

From our previous discussion I only became confused and suspicious
about my previous knowledge i collected from my emails to the mailing
list and raised new questions:

1) What is a built-in function in gcc (and in the manual of gcc)?
Is it a function which its C/Assembly code is inside the code of the
compiler? or Its code is some where on hard disk and inside a library
function? or the code of the built in function is on hard disk but
when we build a program we do not need to link the library of that
function and the compiler automatically finds the code of that
function from a library function on hard disk and adds it to our
program automatically?
I raised different scenarios about what a built in is, unfortunately
the manual of gcc does not explain these facts at all, leaves the
reader completely confused.

2) Ian said in one message that some supporting routine is necessary
for nested functions. What does a nested function mean? It means we
have a C function and then it calls another function inside by its
name?

3) Ian told that the functions like memcpy should be optimized and
they are target dependent and we do not make them as built in. It is
unacceptable in my opinion.
We are talking about a great new feature for gcc, embedded programming
and writing firmwares, not optimization, which gives gcc a great power
and it is a new branch of programming. The fact that the code may be
slower or it takes more memory is a secondary trivial problem in this
case to me.
Besides compiler must be able to translate a code for a specific CPU,
why gcc is dependent to some thing except CPU?!!! Dependent to what?
(To me gcc has a bug which is dependent to some definitions outside
its code, It is a software issue which can be corrected)

4) does gcc have any extension to produce stand alone code in open
source space you may aware of( for a free standing environment without
libraries).

5) Ian said:
"The compiler does by default treat functions like memcpy as builtins,
and ..."
If memcpy is built in why gcc still generates some calls to memcpy
and memcpy should be provided some how as the manual of gcc says?

6) Where is memcpy at last? is it in libgcc, libc or it is built in?

7) Can i link only libgcc with my code in case of -nostdlib? and every
thing is OK? I write libgcc.a on my command line and every thing is
OK?

8) When exactly compiler generates call to mem functions like memcpy,
memcmp, memset, memmove? Would you please write a complete list of the
cases when compiler does that for -nostdlib. Why the manual of gcc
does not talk about such an important subject.

Re: suggestion for GCC (1)

[Ian Lance Taylor]

ali hagigat <hagigatali@gmail.com> writes:

> 1) What is a built-in function in gcc (and in the manual of gcc)?

It is a function which gcc has special knowledge about.  Many of the
builtin functions start with __builtin_, such as
__builtin_return_address.  Others are defined in relevant language
standards, such as memcpy.  Others are widely used and are recognized
when not in strict standard compliance mode, such as alloca.

> Is it a function which its C/Assembly code is inside the code of the
> compiler?

Sometimes.

> or Its code is some where on hard disk and inside a library
> function?

Sometimes.

> or the code of the built in function is on hard disk but
> when we build a program we do not need to link the library of that
> function and the compiler automatically finds the code of that
> function from a library function on hard disk and adds it to our
> program automatically?

Sometimes, if you treat the compiler linking against -lgcc and -lc by
default to be automatically finding the code of the function.

> 2) Ian said in one message that some supporting routine is necessary
> for nested functions. What does a nested function mean? It means we
> have a C function and then it calls another function inside by its
> name?

http://gcc.gnu.org/onlinedocs/gcc/Nested-Functions.html

> 3) Ian told that the functions like memcpy should be optimized and
> they are target dependent and we do not make them as built in. It is
> unacceptable in my opinion.

The memcpy function is a builtin function by the definition above: gcc
knows about it specially.  In fact, memcpy is a special case in that gcc
both recognizes it specially in source code, and gcc also generates
calls to it in some cases.

> We are talking about a great new feature for gcc, embedded programming
> and writing firmwares, not optimization, which gives gcc a great power
> and it is a new branch of programming. The fact that the code may be
> slower or it takes more memory is a secondary trivial problem in this
> case to me.

gcc has supported embedded programming and writing firmware for many
years, that is nothing new.  It is the case that gcc always requires
some supporting code when doing embedded programming.  At the very least
something has to load the program on the board and clear the BSS
section.  That supporting code has to also provide memcpy.

Many people use the newlib and libgloss projects to provide this
supporting code for embedded systems.  See http://sourceware.org/newlib/ .

> Besides compiler must be able to translate a code for a specific CPU,
> why gcc is dependent to some thing except CPU?!!! Dependent to what?
> (To me gcc has a bug which is dependent to some definitions outside
> its code, It is a software issue which can be corrected)

It is not a bug that a compiler requires supporting libraries.  It is
the normal case and it is true of all compilers.  You are confusing the
compiler proper, which is gcc, with a complete development system.  A
complete development system also includes an assembler, a linker,
various libraries, a debugger, etc.  gcc is just a compiler, not a
complete development system.

> 4) does gcc have any extension to produce stand alone code in open
> source space you may aware of( for a free standing environment without
> libraries).

Not all by itself without any support, no.

> 5) Ian said:
> "The compiler does by default treat functions like memcpy as builtins,
> and ..."
> If memcpy is built in why gcc still generates some calls to memcpy
> and memcpy should be provided some how as the manual of gcc says?

Where does the manual say that?

> 6) Where is memcpy at last? is it in libgcc, libc or it is built in?

It in libc and it is builtin.

> 7) Can i link only libgcc with my code in case of -nostdlib? and every
> thing is OK? I write libgcc.a on my command line and every thing is
> OK?

No, you need more than that.

> 8) When exactly compiler generates call to mem functions like memcpy,
> memcmp, memset, memmove? Would you please write a complete list of the
> cases when compiler does that for -nostdlib. Why the manual of gcc
> does not talk about such an important subject.

http://gcc.gnu.org/onlinedocs/gccint/Libgcc.html

If you find places where the documentation should be improved, I would
encourage you to contribute a patch.

Ian

Re: suggestion for GCC (1)

[ali hagigat]

Just type "man gcc" and look for -nostdlib option or I have copied
part of gcc  manual as:

--------------------------------------------
-nostdlib
The compiler may generate calls to memcmp,
memset, memcpy and memmove. These entries are usually resolved by
entries in libc. These entry points should be supplied through some
other mechanism when this option is specified.
--------------------------------------------

As you can see from the manual, compiler still generates some calls to
those functions, so dear Ian why memcpy is built in according to what
you wrote?

> 5) Ian said:
> "The compiler does by default treat functions like memcpy as builtins,
> and ..."
> If memcpy is built in why gcc still generates some calls to memcpy
> and memcpy should be provided some how as the manual of gcc says?

Where does the manual say that?

Re: suggestion for GCC (1)

[ali hagigat]

Are memcmp, memset, and memmove built in?
So a built in function may be in the library or the code I write. right?

I have used cross compilers for micro controllers for years. They
translate C/Assembly code you write and produce a final object file.
they are not dependent on operating system, system calls or force our
code to be linked with some unknown strange libraries which programmer
does not know about them at all.

How they work? Professionals like you can implement the same
functionality for gcc without the weak point of adding mem functions.

It is an idea but I am surprised how you say that it is impossible to
do such a thing.

Besides gcc is dependent to what? operating system or the architecture
of hardware like Intel chipsets, host bridge and I/O bridge in
addition to CPU?

Re: suggestion for GCC (1)

[Ian Lance Taylor]

ali hagigat <hagigatali@gmail.com> writes:

> Are memcmp, memset, and memmove built in?
> So a built in function may be in the library or the code I write. right?

Yes.

> I have used cross compilers for micro controllers for years. They
> translate C/Assembly code you write and produce a final object file.
> they are not dependent on operating system, system calls or force our
> code to be linked with some unknown strange libraries which programmer
> does not know about them at all.
>
> How they work? Professionals like you can implement the same
> functionality for gcc without the weak point of adding mem functions.
>
> It is an idea but I am surprised how you say that it is impossible to
> do such a thing.

I don't think that I said that it is impossible to do such a thing.  If
I did, I misspoke, and I apologize for that if in fact I did do it.

I think that, rather than saying that doing something else is
impossible, I explained why gcc acts the way it does.

> Besides gcc is dependent to what? operating system or the architecture
> of hardware like Intel chipsets, host bridge and I/O bridge in
> addition to CPU?

As you know, gcc is generally independent of the host bridge and I/O
bridge.  It is of course dependent on the CPU itself.  It is dependent
on the target operating system in a few ways.

Ian

Re: suggestion for GCC (1)

[David Brown]

On 09/02/2011 07:03, ali hagigat wrote:
> Are memcmp, memset, and memmove built in?
> So a built in function may be in the library or the code I write. right?
>
> I have used cross compilers for micro controllers for years. They
> translate C/Assembly code you write and produce a final object file.
> they are not dependent on operating system, system calls or force our
> code to be linked with some unknown strange libraries which programmer
> does not know about them at all.
>

You are comparing apples and fruit baskets here.

As Ian says, gcc is just the compiler - it is not a complete development 
toolchain.  What you are referring to here is complete toolchains, 
including the linker, assembler, and libraries (and normally a debugger 
and IDE too).  The programs you generate /are/ linked to "strange" 
libraries - they are libraries that came with the development toolchain. 
  They may also include system call support or dependencies on the 
operating system, depending on the target.

The libraries provided with the toolchain will be dependent on the 
target.  For a simple example, a function like memcpy can be implemented 
in different ways - for a small target, it will be optimised for space. 
  For larger targets, it will be optimised for speed and take into 
account things like memory alignment, cache line size, etc.  This is 
dependent on the complete target - not just the basic CPU architecture. 
  The toolchain supplier will know about these issues, and provide 
appropriate libraries.

gcc itself knows about /some/ of these things, given the correct flags 
(it will probably know about alignment issues, but is unlikely to know 
about cache line sizes).  Thus it will sometimes inline memcpy using the 
builtin code, such as for small copies where the number of bytes is 
fixed at compile time, and it will sometimes call an external function.


There are plenty of complete toolchains based on gcc.  A good source of 
these is www.codesourcery.com - the developers there do a lot of work on 
various embedded gcc targets.  When you download a toolchain from them, 
you get the assembler, linker and libraries included.  There are also 
plenty of other ready-packaged gcc-based toolchains for other 
microcontrollers.

> How they work? Professionals like you can implement the same
> functionality for gcc without the weak point of adding mem functions.
>
> It is an idea but I am surprised how you say that it is impossible to
> do such a thing.
>
> Besides gcc is dependent to what? operating system or the architecture
> of hardware like Intel chipsets, host bridge and I/O bridge in
> addition to CPU?
>

Re: suggestion for GCC (1)

[Enrico Weigelt]

* David Brown <david@westcontrol.com> wrote:

> gcc itself knows about /some/ of these things, given the correct flags 
> (it will probably know about alignment issues, but is unlikely to know 
> about cache line sizes).  Thus it will sometimes inline memcpy using the 
> builtin code, such as for small copies where the number of bytes is 
> fixed at compile time, and it will sometimes call an external function.

Apropos: is there some way to extend these builtins from external
libraries ?

Let's assume I'm writing some library for very common ATDs, eg. lists,
hashtables, etc. And for some specific targets I'd like to do some
heavy optimizations, which should be completely hidden behind the
API (just like done with memcpy, etc). So it would be nice to somehow
plug into the compiler's code generation, but w/o touching the compiler
itself, just giving it more deeper information beyond the usual C code.

An typical use case could be string concatenation, an strcat() that
takes multiple arguments. For example:

    char buffer[1024];
    const char* ptr const = "huh";
    strcatx(buffer, "hello", " world: ", strerror(errno), " blah ", ptr);

This could well be implemented as a macro using __VA_ARGS etc, but there's
still much room for optimization which would require compiler-internal
information. In this particular case we know several things, eg:

    * "hello" and " world: " are really constant strings, so the result
      of their concatenation is known at compile time: "hello world: "
    * strerror() maps a limited number range to fixed strings, to it
      could be a direct array lookup (after proper bounds checking)
    * concetenation of " blah " and ptr also has an known result at
      compile time: " blah huh"
    * sizes of the constant strings is already defined at compile time.

So the same could also be written that ways:

    const char* _errptr;
    int _errlen;
    if ((errno >= ERRNO_MIN_IDX) && (errno <= ERRNO_MAX_IDX))
    {
	_errptr = _errno_str_vector[errno];
	_errlen = _errno_sz_vector[errno];
    }
    else
	_errptr = "Unknown error code";
	_errlen = 18;
    }
    char buffer[1024];
    memcpy(buffer, "hello world: ");
    memcpy(buffer+13, sizeof(buffer)-13, _errptr);
    memcpy(buffer+13+_errlen, sizeof(buffer)-13-_errlen, " blah foo", 9);
    buffer[13+_errlen+9] = 0;


Is there a way to add those things by external libraries, w/o
touching the compiler itself ?

> There are plenty of complete toolchains based on gcc.  A good source of 
> these is www.codesourcery.com - the developers there do a lot of work on 
> various embedded gcc targets.  When you download a toolchain from them, 
> you get the assembler, linker and libraries included.  There are also 
> plenty of other ready-packaged gcc-based toolchains for other 
> microcontrollers.

Maybe crosstool-ng might also be worth looking at.
 

cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: suggestion for GCC (1)

[Ian Lance Taylor]

ali hagigat <hagigatali@gmail.com> writes:

> Just type "man gcc" and look for -nostdlib option or I have copied
> part of gcc  manual as:
>
> --------------------------------------------
> -nostdlib
> The compiler may generate calls to memcmp,
> memset, memcpy and memmove. These entries are usually resolved by
> entries in libc. These entry points should be supplied through some
> other mechanism when this option is specified.
> --------------------------------------------
>
> As you can see from the manual, compiler still generates some calls to
> those functions, so dear Ian why memcpy is built in according to what
> you wrote?

What I wrote, in the message to which you are replying, is that a
builtin function "is a function which gcc has special knowledge about."
That is completely consistent with gcc generating a call to it.  In
fact, by my definition, every function that for which gcc generates
calls (when the function is not mentioned in the source code) is a
builtin function: obviously gcc will only generate a call to a function
which it has special knowledge about.


>> 5) Ian said:
>> "The compiler does by default treat functions like memcpy as builtins,
>> and ..."
>> If memcpy is built in why gcc still generates some calls to memcpy
>> and memcpy should be provided some how as the manual of gcc says?
>
> Where does the manual say that?

Thanks for the pointer.  What the manual says is that if you use the
-nostdlib option you need to provide memcpy and other functions.  I
assume that your other questions have been answered at this point.

Let me repeat that if you don't like gcc's behaviour in some respect,
you should work to change it.  gcc is free software and it improves by
voluntary contributions.

Ian