[ECOS] ARM EABI port / static constructor priority removal

[ECOS] ARM EABI port / static constructor priority removal

[Chris Zimman]

I have a nearly working port of eCos for the Broadcom BCM5890 that also
includes ARM EABI (arm-none-eabi-gcc) support.  The idea is that people
should be able to build eCos with a supported toolchain (aka CodeSourcery)
(and perhaps at some point ADS).  Redboot builds and runs fine under EABI now
(at least on this target, but I suspect it will be OK on other ARM targets as
well).

One of the things that's holding it up is the use of
__attribute__(init_priority(_pri_))) because there appears to be an issue (I
hesitate to call this a bug until it's further understood) where 'ld -r'
combines __init_array entry ordering when creating extras.o  When you link
the resulting target images, the static constructor ordering gets messed up a
little bit, and of consequently things don't work right.

Anyhow -- one of the possible solutions that came out of this was to remove
the requirement for multiple translation unit static constructor ordering.
It's a GNU only extension and generally not considered good behavior.

It's a big change, but that by itself is not a good reason not to do it.  Any
thoughts?

BTW, this ARM tree has a lot of general improvements.  It includes support
for ARM processors that don't have the vector tables at 0x0, has a bunch of
additional ARM9 helper routines, low power sleep in the idle loop, etc.

--Chris

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ARM EABI port / static constructor priority removal

[Andrew Lunn]

> One of the things that's holding it up is the use of
> __attribute__(init_priority(_pri_))) because there appears to be an issue (I
> hesitate to call this a bug until it's further understood) where 'ld -r'
> combines __init_array entry ordering when creating extras.o  When you link
> the resulting target images, the static constructor ordering gets messed up a
> little bit, and of consequently things don't work right.
> 
> Anyhow -- one of the possible solutions that came out of this was to remove
> the requirement for multiple translation unit static constructor ordering.
> It's a GNU only extension and generally not considered good behavior.

Given that this is used in a number of places scattered all over the
code, how do you propose to do this? What is you concept to ensure the
constructors are called in the right order?

    Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] ARM EABI port / static constructor priority removal

[Chris Zimman]

> Given that this is used in a number of places scattered all over the
> code, how do you propose to do this? What is you concept to ensure the
> constructors are called in the right order?

Ordering is preserved within one translation unit by C++ spec, just not
between multiple translation units.

BTW, I think the ARM EABI stuff will get fixed (CodeSourcery is looking at it
right now), I just think this is bad behavior to rely on in general.
It's not part of EABI and was somewhat reluctantly added to EABI GCC.

--Chris

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ARM EABI port / static constructor priority removal

[Andrew Lunn]

On Wed, Mar 26, 2008 at 02:09:31PM -0400, Chris Zimman wrote:
> > Given that this is used in a number of places scattered all over the
> > code, how do you propose to do this? What is you concept to ensure the
> > constructors are called in the right order?
> 
> Ordering is preserved within one translation unit by C++ spec, just not
> between multiple translation units.

This won't work. For example, some watchdog drivers need the I2C
subsystem to be initialized before they can be initialized. The IO
layer must be initialised before libc etc. If you get these in the
wrong order, expect bad things to happen.

In general, it is assumed gcc is used and that the gcc extensions
work. There have been a few attempts to use compilers other than gcc,
but i don't remember anybody ever says they have it working well.

As a side question, how well does function sections work with your
compiler? This is another gcc extension which eCos heavily uses in
order to keep the image size down. Also HAL tables? These also rely on
sorting otherwise the start and end won't be in the right place.

      Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] ARM EABI port / static constructor priority removal

[Chris Zimman]

> This won't work. For example, some watchdog drivers need the I2C
> subsystem to be initialized before they can be initialized. The IO
> layer must be initialised before libc etc. If you get these in the
> wrong order, expect bad things to happen.

I don't understand why you think "this won't work" when "this" is exactly
what the static constructors are doing, except instead of being in one
translation unit, they're in multiple.  Perhaps you're missing what I'm
saying -- I understand that things have to happen in order.  What I'm saying
is that I don't think static constructors are necessarily the best way to
accomplish this.

> In general, it is assumed gcc is used and that the gcc extensions
> work. There have been a few attempts to use compilers other than gcc,
> but i don't remember anybody ever says they have it working well.

It works fine outside of the aforementioned issue.  The 'it requires GCC'
limitation is just a self imposed one.  There's nothing magic going on.

> As a side question, how well does function sections work with your
> compiler? This is another gcc extension which eCos heavily uses in
> order to keep the image size down. Also HAL tables? These also rely on
> sorting otherwise the start and end won't be in the right place.

Everything seems to work fine with the exception of the static constructors.
It shoots off into the flats because some of the ordering is wrong.
I expect that when the __init_array linker issue is resolved, things will
work more or less fine.

See below, built under EABI:

+
RedBoot(tm) bootstrap and debug environment [ROMRAM]
Non-certified release, version v2_0_52 - built 13:33:07, Mar 25 2008

Platform: BCM95890 (ARM9) 
Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc.
Copyright (C) 2003, 2004, 2005, 2006 eCosCentric Limited

RAM: 0x00000000-0x00200000, [0x00022dc0-0x001bd000] available
FLASH: 0x10000000-0x101fffff, 1 x 0x4000 blocks, 2 x 0x2000 blocks, 1 x
0x38000 blocks, 7 x 0x40000 blocks
RedBoot> go 0x40000
EABI constructors invoked
__cxa_atexit called: 0x0004fa00 0x00063eec 0x0006161c
__cxa_atexit called: 0x0004de80 0x00000000 0x0006161c
__cxa_atexit called: 0x0004c4e0 0x00063ed4 0x0006161c
__cxa_atexit called: 0x0005b338 0x000669b8 0x0006161c
__cxa_atexit called: 0x00052d08 0x00063fb4 0x0006161c
__cxa_atexit called: 0x00050114 0x00066990 0x0006161c
__cxa_atexit called: 0x000500e0 0x000669a4 0x0006161c
__cxa_atexit called: 0x0004dd78 0x000643c8 0x0006161c
__cxa_atexit called: 0x00050114 0x00066490 0x0006161c
__cxa_atexit called: 0x000535b8 0x00066a34 0x0006161c
__cxa_atexit called: 0x00050114 0x00066b44 0x0006161c
__cxa_atexit called: 0x000535b8 0x000664a4 0x0006161c
__cxa_atexit called: 0x000535b8 0x000669f0 0x0006161c
Init device '/dev/ser0'
pl011 SERIAL init - dev: a0300000.7
Set output buffer - buf: 6327c len: 128
Set input buffer - buf: 632fc len: 128
Init device '/dev/ser1'
pl011 SERIAL init - dev: a0301000.8
Set output buffer - buf: 6337c len: 128
Set input buffer - buf: 633fc len: 128
Init device '/dev/ttydiag'
Init tty channel: 616d4
Init device '/dev/haldiag'
HAL/diag SERIAL init
Init device '/dev/flash/'
INFO: <code from 0x00040040 -> 0x000602a4, CRC 2fd4>
INFO:<Stress threads test compiled on Mar 26 2008>

[crash]

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ARM EABI port / static constructor priority removal

[Andrew Lunn]

On Wed, Mar 26, 2008 at 02:24:28PM -0400, Chris Zimman wrote:
> > This won't work. For example, some watchdog drivers need the I2C
> > subsystem to be initialized before they can be initialized. The IO
> > layer must be initialised before libc etc. If you get these in the
> > wrong order, expect bad things to happen.
> 
> I don't understand why you think "this won't work" when "this" is exactly
> what the static constructors are doing, except instead of being in one
> translation unit, they're in multiple.  Perhaps you're missing what I'm
> saying -- I understand that things have to happen in order.  What I'm saying
> is that I don't think static constructors are necessarily the best way to
> accomplish this.

So back to my original question, what is your concept for replacing
them with something else. How are you going to ensure things happen in
the right order.

    Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] ARM EABI port / static constructor priority removal

[Chris Zimman]

> So back to my original question, what is your concept for replacing
> them with something else. How are you going to ensure things happen in
> the right order.

One possibility is to put them all in one translation unit.  They are
constructed in the order that they appear, although that's kind of a big
kludge.
Another is to add explicit initialization calls to the various bits to invoke
the constructors at runtime.

--Chris

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ARM EABI port / static constructor priority removal

[Andrew Lunn]

On Wed, Mar 26, 2008 at 02:38:29PM -0400, Chris Zimman wrote:
> > So back to my original question, what is your concept for replacing
> > them with something else. How are you going to ensure things happen in
> > the right order.
> 
> One possibility is to put them all in one translation unit.  They are
> constructed in the order that they appear, although that's kind of a big
> kludge.

Agreed. So lets forget about that.

> Another is to add explicit initialization calls to the various bits to invoke
> the constructors at runtime.

Also ugly. You break the nice packing model. Say i have an out of tree
package, a device driver for the wall clock on my hardware. The
current code allows my code to have a static initializer with priority
that is after I2C is up and running and it is totally independent of
the in tree code. I don't need to modify the in tree code at all. The
linker sorts it all out. Your suggestion would force me to modify the
in tree list of constructors.

The same could be said for application code. My application wants a
static constructor called after the OS is up an running, but before
main() is called. Should i modify the OS to list my application
constructors?

        Andrew


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] ARM EABI port / static constructor priority removal

[Chris Zimman]

> Also ugly. You break the nice packing model. Say i have an out of tree
> package, a device driver for the wall clock on my hardware. The
> current code allows my code to have a static initializer with priority
> that is after I2C is up and running and it is totally independent of
> the in tree code. I don't need to modify the in tree code at all. The
> linker sorts it all out. Your suggestion would force me to modify the
> in tree list of constructors.

With what you're talking about here, you'd have to modify cyg_type.h anyway
if you want to add a new init priority unless you want to keep it private,
which is kind of bogus.  With the current model anyhow, you *have* to be
aware of the ordering in cyg_type.h and known what values are assigned to
which constructors.  That's pretty ugly in and of itself.

There's also no inherent reason you would have to modify in tree code to
accomplish what you're suggesting.  Sections can be created for just that
purpose.  Even enforcing startup ordering rules is pretty straight forward
that way.
 
> The same could be said for application code. My application wants a
> static constructor called after the OS is up an running, but before
> main() is called. Should i modify the OS to list my application
> constructors?

It's not about the use of static constructors in general, it's about relying
on non-portable behavior with respect to using them (construction ordering).

--Chris

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ARM EABI port / static constructor priority removal

[Fabian Scheler]

> So back to my original question, what is your concept for replacing
>  them with something else. How are you going to ensure things happen in
>  the right order.

just my two cents about that topic - the global objects we are talking
about are singletons, right? So what about something like that

Cyg_Scheduler* theScheduler() {
  static Cyg_Scheduler theScheduler;
  return &theScheduler;
}

Ciao, Fabian

btw - I have no problem with the init-priority attribute. I believe
that avoiding compiler dependencies is nearly as hard as to be
processor independent. So, for the most OSes provided as source code
the user is supposed to use a specific compiler and with eCos one is
supposed to use gcc. It is also not possible to compile the linux
kernel using the Microsoft Compiler, is it?

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] ARM EABI port / static constructor priority removal

[Chris Zimman]

> just my two cents about that topic - the global objects we are talking
> about are singletons, right? So what about something like that
> 
> Cyg_Scheduler* theScheduler() {
>   static Cyg_Scheduler theScheduler;
>   return &theScheduler;
> }

That seems like a viable option.

> btw - I have no problem with the init-priority attribute. I believe
> that avoiding compiler dependencies is nearly as hard as to be
> processor independent. So, for the most OSes provided as source code
> the user is supposed to use a specific compiler and with eCos one is
> supposed to use gcc

eCos is by and large CPU independent.  Only a small part of it is processor
specific.  If you *have* to be compiler specific for some reason, fine.  But
I can't see a good reason to be just for the sake of it or because of design
choices that are fixable.

Prior to the ARM EABI work I am doing, you couldn't build CodeSourcery tool
chain either, so if you wanted a supported version of GCC w/ eCos you were
out of luck.  To me, that's a serious drawback for using eCos.  We ran into
internal compiler errors with GCC 3.4.4 and bad behavior in our application
when building with -O2 in some cases.

> It is also not possible to compile the linux kernel using the Microsoft
Compiler, is it?

One thing at a time ;)
 
--Chris

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ARM EABI port / static constructor priority removal

[Jonathan Larmour]

Chris Zimman wrote:
>> Also ugly. You break the nice packing model. Say i have an out of tree
>> package, a device driver for the wall clock on my hardware. The
>> current code allows my code to have a static initializer with priority
>> that is after I2C is up and running and it is totally independent of
>> the in tree code.

And also needs to be run _after_ the kernel code, but potentially before
the libc code (which may reference the RTC), etc.etc. It's just not
feasible while retaining modularity which is far more valuable.

>> I don't need to modify the in tree code at all. The
>> linker sorts it all out. Your suggestion would force me to modify the
>> in tree list of constructors.
> 
> With what you're talking about here, you'd have to modify cyg_type.h anyway

Not really. In cyg_type.h CYG_INIT_APPLICATION is defined, and priorities
60000 up to 65534 are free for application use.

> if you want to add a new init priority unless you want to keep it private,
> which is kind of bogus.  With the current model anyhow, you *have* to be
> aware of the ordering in cyg_type.h and known what values are assigned to
> which constructors.  That's pretty ugly in and of itself.

Arguably not with a reserved range for application use.

> There's also no inherent reason you would have to modify in tree code to
> accomplish what you're suggesting.  Sections can be created for just that
> purpose.  Even enforcing startup ordering rules is pretty straight forward
> that way.

It's potentially feasible to create a section (not .ctors, something
eCos-specific) that contains a list of tuples - a priority number and a
function pointer. But then a) the list needs to be sorted at runtime which
is very bad; and b) it seems very much that we're just swapping one
extension for another.

Although reworking our abstraction to _permit_ this mode of operation
(without requiring it for GCC users) might make eCos more portable to other
compilers by making this sort of thing possible.

>> The same could be said for application code. My application wants a
>> static constructor called after the OS is up an running, but before
>> main() is called. Should i modify the OS to list my application
>> constructors?
> 
> It's not about the use of static constructors in general, it's about relying
> on non-portable behavior with respect to using them (construction ordering).

It's not the best solution, but it may be the least worst.

Jifl
-- 
eCosCentric Limited      http://www.eCosCentric.com/     The eCos experts
 **  Visit us at ESC Silicon Valley <http://www.embedded.com/esc/sv>  **
 **  April 15-17 2008, Booth 3012, San Jose McEnery Convention Center **
Barnwell House, Barnwell Drive, Cambridge, UK.       Tel: +44 1223 245571
Registered in England and Wales: Reg No 4422071.
------["Si fractum non sit, noli id reficere"]------       Opinions==mine

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] ARM EABI port / static constructor priority removal

[Chris Zimman]

> And also needs to be run _after_ the kernel code, but potentially
> before the libc code (which may reference the RTC), etc.etc.  It's just not
> feasible while retaining modularity which is far more valuable.

If it were written in C, there's still a way to do it, and it doesn't require
static constructor init ordering.
 
> Not really. In cyg_type.h CYG_INIT_APPLICATION is defined, and
> priorities
> 60000 up to 65534 are free for application use.
> 
> Arguably not with a reserved range for application use.

What if your code needs to do something in between one of the OS startup
steps?  This statement makes the assumption that the application and kernel
code are really separate, which is sometimes not a very clearly defined line
in eCos.  Presumably, one could argue that you shouldn't be doing anything
outside of the 'allowed' range by definition, but see next section...

> It's potentially feasible to create a section (not .ctors, something
> eCos-specific) that contains a list of tuples - a priority number and a
> function pointer. But then a) the list needs to be sorted at runtime
> which
> is very bad; and b) it seems very much that we're just swapping one
> extension for another.

Or they could be a set of ordered singletons in a startup file or something
equivalent.  Of course, this precludes one from writing a driver outside of
the tree and having it called in between steps.
 
> It's not the best solution, but it may be the least worst.

I guess it depends on how much one considers relying on non-portable compiler
specific behavior bad or not.  It's not part of ARM EABI officially, and had
CodeSourcery not decided to implement it, I would've had to do something
different to get EABI to work.  My point is not that it doesn't work as is,
but rather, that it's a self imposed limitation that there are portable ways
to solve.

If no one else wants to pursue it, I have to leave it as is though.

--Chris

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss