Building RedBoot for SH3 targets with new toolchain

Building RedBoot for SH3 targets with new toolchain

[John Dallaway]

Hi Jifl

There is a problem linking RedBoot for all SH3 targets with the new
sh-elf toolchain based on GCC 4.3.2. For example, linking RAM RedBoot
for sh7708:

> /var/local/releng/build/redboot/install/lib/extras.o: In function `fis_write':
> /var/local/releng/ecos-3.0pre/packages/redboot/current/src/flash.c:1660: undefined reference to `___udivsi3_i4i'
> /var/local/releng/build/redboot/install/lib/extras.o: In function `fis_erase':
> /var/local/releng/ecos-3.0pre/packages/redboot/current/src/flash.c:1710: undefined reference to `___udivsi3_i4i'
> /var/local/releng/build/redboot/install/lib/extras.o: In function `do_flash_init':
> /var/local/releng/ecos-3.0pre/packages/redboot/current/src/flash.c:1920: undefined reference to `___udivsi3_i4i'
> /var/local/releng/build/redboot/install/lib/extras.o: In function `fis_create':
> /var/local/releng/ecos-3.0pre/packages/redboot/current/src/flash.c:1286: undefined reference to `___udivsi3_i4i'
> /var/local/releng/build/redboot/install/lib/libtarget.a(hal_common_hal_if.o): In function `flash_fis_op2':
> /var/local/releng/ecos-3.0pre/packages/hal/common/current/src/hal_if.c:264: undefined reference to `___udivsi3_i4i'
> /var/local/releng/build/redboot/install/lib/libtarget.a(io_flash_flash.o):/var/local/releng/ecos-3.0pre/packages/io/flash/current/src/flash.c:595: more undefined references to `___udivsi3_i4i' follow
> collect2: ld returned 1 exit status

I note that the multi-libbing for sh-elf has changed, but for both old
and new toolchains:

> [jld@cog ~]$ sh-elf-gcc -m3 -print-multi-directory
> m2
> [jld@cog ~]$ sh-elf-gcc -m3 -ml -print-multi-directory
> ml/m2
> [jld@cog ~]$

The m2 and ml/m2 multi-libs exist in both toolchains.

Building RedBoot for target sh4_202_md (an SH4 target) works fine with
the new tools and building RedBoot for SH3 targets works fine with the
old tools.

Any idea what's going wrong here?

John Dallaway

Re: Building RedBoot for SH3 targets with new toolchain

[Jonathan Larmour]

John Dallaway wrote:
> Hi Jifl
> 
> There is a problem linking RedBoot for all SH3 targets with the new
> sh-elf toolchain based on GCC 4.3.2. For example, linking RAM RedBoot
> for sh7708:
> 
>> /var/local/releng/build/redboot/install/lib/extras.o: In function `fis_write':
[snip]
> Building RedBoot for target sh4_202_md (an SH4 target) works fine with
> the new tools and building RedBoot for SH3 targets works fine with the
> old tools.
> 
> Any idea what's going wrong here?

gcc's lib1funcs.asm has that function surrounded by:
#elif defined (__SH3__) || defined (__SH3E__) || defined (__SH4__) ||
defined (__SH4_SINGLE__) || defined (__SH4_SINGLE_ONLY__) || defined
(__SH4_NOFPU__)
/* This code used shld, thus is not suitable for SH1 / SH2.  */

So it's not built in an sh2 multilib. Therefore we will need to build an
sh3 multilib. Bummer. It needs to be added at line 2286 of config.gcc. I'm
rebuilding for linux now, so you can try it out when that's done and I've
packaged it up, but you may want to add this to any todo list you've got to
be sure it's tracked. If that build is ok, I'll have the grief of trying to
rebuild the cygwin version.

Jifl
-- 
eCosCentric Limited      http://www.eCosCentric.com/     The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.       Tel: +44 1223 245571
Registered in England and Wales: Reg No 4422071.
------["Si fractum non sit, noli id reficere"]------       Opinions==mine

Re: Building RedBoot for SH3 targets with new toolchain

[John Dallaway]

Hi Jifl

Jonathan Larmour wrote:

[ snip ]

> So it's not built in an sh2 multilib. Therefore we will need to build an
> sh3 multilib. Bummer. It needs to be added at line 2286 of config.gcc. I'm
> rebuilding for linux now, so you can try it out when that's done and I've
> packaged it up, but you may want to add this to any todo list you've got to
> be sure it's tracked.

Thanks for the rapid response. At least the solution is clear. Of
course, I will be happy to try the new tools out.

You should find some shell scripts on snitch in the base of the
toolchains build directory hierarchy which will help with
building/packaging sourceware toolchains. They will need minor tweaking
for an "sh-elf only" spin.

> If that build is ok, I'll have the grief of trying to
> rebuild the cygwin version.

Similarly, there are scripts on mug for building/packaging sourceware
toolchains in the Cygwin /gnutools directory hierarchy.

I could offer to build the revised sh-elf toolchains here, but it does
seem preferable to use the same hosts for building all the toolchains to
ensure consistency.

John Dallaway

Re: Building RedBoot for SH3 targets with new toolchain

[Jonathan Larmour]

John Dallaway wrote:
> Hi Jifl
> 
> Jonathan Larmour wrote:
> 
> [ snip ]
> 
>> So it's not built in an sh2 multilib. Therefore we will need to build an
>> sh3 multilib. Bummer. It needs to be added at line 2286 of config.gcc. I'm
>> rebuilding for linux now, so you can try it out when that's done and I've
>> packaged it up, but you may want to add this to any todo list you've got to
>> be sure it's tracked.
> 
> Thanks for the rapid response. At least the solution is clear. Of
> course, I will be happy to try the new tools out.
> 
> You should find some shell scripts on snitch in the base of the
> toolchains build directory hierarchy which will help with
> building/packaging sourceware toolchains. They will need minor tweaking
> for an "sh-elf only" spin.

Already seen and tweaked in fact.

>> If that build is ok, I'll have the grief of trying to
>> rebuild the cygwin version.
> 
> Similarly, there are scripts on mug for building/packaging sourceware
> toolchains in the Cygwin /gnutools directory hierarchy.

Hadn't seen those. I'll poke when it comes to that.

> I could offer to build the revised sh-elf toolchains here, but it does
> seem preferable to use the same hosts for building all the toolchains to
> ensure consistency.

You wouldn't have a good glibc version to use for backward compatibility
anyway. (Unless you go and install something old somewhere temporarily).

Anyway, linux build now uploaded to
ftp://ecos.sourceware.org/pub/ecos/gnutools/ecoscentric-gnutools-sh-elf-20090121-sw.i386linux.tar.bz2

If that looks ok, I'll try and get a cygwin build going, but as I'm sure
you would expect, that will take longer.

Jifl
-- 
eCosCentric Limited      http://www.eCosCentric.com/     The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.       Tel: +44 1223 245571
Registered in England and Wales: Reg No 4422071.
------["Si fractum non sit, noli id reficere"]------       Opinions==mine

Re: Building RedBoot for SH3 targets with new toolchain

[John Dallaway]

Hi Jifl

Jonathan Larmour wrote:

> Anyway, linux build now uploaded to
> ftp://ecos.sourceware.org/pub/ecos/gnutools/ecoscentric-gnutools-sh-elf-20090121-sw.i386linux.tar.bz2
> 
> If that looks ok, I'll try and get a cygwin build going, but as I'm sure
> you would expect, that will take longer.

This toolchain certainly seems to have resolved the SH3 build problems I
reported earlier. Thanks. Was adding "m3" to the list of multilibs at
config.gcc:2286 the only change required?

There is a remaining issue with linking the cxxsupp test in particular
(eg target cq7750, default template):

> sh-elf-gcc -L/var/local/build/ecos-new/install/lib -Ttarget.ld -o /var/local/build/ecos-new/install/tests/infra/current/tests/cxxsupp tests/cxxsupp.o -mb -m3 -ggdb -nostdlib -Wl,-static -Wl,--fatal-warnings 
> /var/local/releng/release-3.0/targetbuild/tools/gnutools/sh-elf/bin/../lib/gcc/sh-elf/4.3.2/../../../../sh-elf/bin/ld: error: no memory region specified for loadable section `.eh_frame'
> collect2: ld returned 1 exit status

John Dallaway

Re: Building RedBoot for SH3 targets with new toolchain

[Jonathan Larmour]

John Dallaway wrote:
> Hi Jifl
> 
> Jonathan Larmour wrote:
> 
> 
>>Anyway, linux build now uploaded to
>>ftp://ecos.sourceware.org/pub/ecos/gnutools/ecoscentric-gnutools-sh-elf-20090121-sw.i386linux.tar.bz2
>>
>>If that looks ok, I'll try and get a cygwin build going, but as I'm sure
>>you would expect, that will take longer.
> 
> 
> This toolchain certainly seems to have resolved the SH3 build problems I
> reported earlier. Thanks. Was adding "m3" to the list of multilibs at
> config.gcc:2286 the only change required?

Yes, as per updated patch set, also uploaded.

> There is a remaining issue with linking the cxxsupp test in particular
> (eg target cq7750, default template):
> 
> 
>>sh-elf-gcc -L/var/local/build/ecos-new/install/lib -Ttarget.ld -o /var/local/build/ecos-new/install/tests/infra/current/tests/cxxsupp tests/cxxsupp.o -mb -m3 -ggdb -nostdlib -Wl,-static -Wl,--fatal-warnings 
>>/var/local/releng/release-3.0/targetbuild/tools/gnutools/sh-elf/bin/../lib/gcc/sh-elf/4.3.2/../../../../sh-elf/bin/ld: error: no memory region specified for loadable section `.eh_frame'
>>collect2: ld returned 1 exit status

The sh.ld linker script does have the necessary SECTION_eh_frame define. 
But older SH targets have not been updated to use it. In fact the only one 
that does is the sh4_202_md. Something to note down for after the code freeze.

In a slight change of plan, I think I'll wait until you're finished 
looking at all the targets before starting even the SH cygwin build - 
given the recent report it seems like it may be wise to rebuild all the 
cygwin tools with statically linked GMP and MPFR. I have no desire to 
rebuild them all more than once, so let's wait to see if your experiments 
show up more issues down to the tools.

Jifl
-- 
eCosCentric Limited      http://www.eCosCentric.com/     The eCos experts
Barnwell House, Barnwell Drive, Cambridge, UK.       Tel: +44 1223 245571
Registered in England and Wales: Reg No 4422071.
------["The best things in life aren't things."]------      Opinions==mine

Re: Building RedBoot for SH3 targets with new toolchain

[John Dallaway]

Hi Jifl

Jonathan Larmour wrote:

> The sh.ld linker script does have the necessary SECTION_eh_frame define.
> But older SH targets have not been updated to use it. In fact the only
> one that does is the sh4_202_md. Something to note down for after the
> code freeze.

Hmm, yes. There will be many targets on other architectures that also
need their .ldi files updating.

> In a slight change of plan, I think I'll wait until you're finished
> looking at all the targets before starting even the SH cygwin build -
> given the recent report it seems like it may be wise to rebuild all the
> cygwin tools with statically linked GMP and MPFR. I have no desire to
> rebuild them all more than once, so let's wait to see if your
> experiments show up more issues down to the tools.

You should find that re-spinning on Cygwin takes no more effort than on
Linux. Of course, elapsed time is another matter entirely!

With Cygwin, it is generally preferable to take advantage of shared DLLs
wherever possible to reduce application load times and overall memory
footprint. This is especially true for parallel builds (eg "make -j2").
FYI, the latest Cygwin-hosted tools are also dependent on Cygwin intl
and iconv DLLs for internationalisation.

I am aware of several members of the eCos community that are using the
current Cygwin-hosted toolchains with no reported problems. We also have
the flexibility of releasing updated toolchains independently of our
eCos releases should this prove necessary in the future.

Taking all these things into account, plus the desire to avoid further
delays to the eCos 3.0 release, I think it preferable to stick with the
stock mpfr and gmp DLLs for the Cygwin-hosted toolchains. We must
document the need to install the relevant Cygwin packages though, and
add an entry to the FAQ.

John Dallaway

Re: Building RedBoot for SH3 targets with new toolchain

[Andrew Lunn]

> Taking all these things into account, plus the desire to avoid further
> delays to the eCos 3.0 release, I think it preferable to stick with the
> stock mpfr and gmp DLLs for the Cygwin-hosted toolchains. We must
> document the need to install the relevant Cygwin packages though, and
> add an entry to the FAQ.

Would it be possible to detect at build time if these DLLs are
installed? But a custom build rule with the highest priority in infra,
or HAL which will test if this files exist and stop and print an
useful error message if they are not installed?

The way cygwin is silently failing is not nice...

       Andrew

Re: Building RedBoot for SH3 targets with new toolchain

[John Dallaway]

Hi Andrew

Andrew Lunn wrote:

>> Taking all these things into account, plus the desire to avoid further
>> delays to the eCos 3.0 release, I think it preferable to stick with the
>> stock mpfr and gmp DLLs for the Cygwin-hosted toolchains. We must
>> document the need to install the relevant Cygwin packages though, and
>> add an entry to the FAQ.
> 
> Would it be possible to detect at build time if these DLLs are
> installed? But a custom build rule with the highest priority in infra,
> or HAL which will test if this files exist and stop and print an
> useful error message if they are not installed?
> 
> The way cygwin is silently failing is not nice...

The failure mode is certainly not nice. I expect that the mpfr and gmp
DLLs are loaded explicitly by the compiler using LoadLibrary() and
failure is not being reported at the application level.

We could emit a _warning_ message using a custom rule. An _error_
message would be too strong since older toolchains do not use mpfr and
gmp. However, it seems cleaner to keep such testing out of the build
process and perform the tests in the eCos install script instead when
the relevant toolchains are being installed. Having said that, I
appreciate that experienced eCos developers may not use the install script.

John Dallaway

Re: Building RedBoot for SH3 targets with new toolchain

[Bart Veer]

>>>>> "John" == John Dallaway <john@dallaway.org.uk> writes:

    John> Hi Andrew
    John> Andrew Lunn wrote:

    >>> Taking all these things into account, plus the desire to avoid
    >>> further delays to the eCos 3.0 release, I think it preferable
    >>> to stick with the stock mpfr and gmp DLLs for the
    >>> Cygwin-hosted toolchains. We must document the need to install
    >>> the relevant Cygwin packages though, and add an entry to the
    >>> FAQ.
    >> 
    >> Would it be possible to detect at build time if these DLLs are
    >> installed? But a custom build rule with the highest priority in
    >> infra, or HAL which will test if this files exist and stop and
    >> print an useful error message if they are not installed?
    >> 
    >> The way cygwin is silently failing is not nice...

    John> The failure mode is certainly not nice. I expect that the
    John> mpfr and gmp DLLs are loaded explicitly by the compiler
    John> using LoadLibrary() and failure is not being reported at the
    John> application level.

    John> We could emit a _warning_ message using a custom rule. An
    John> _error_ message would be too strong since older toolchains
    John> do not use mpfr and gmp. However, it seems cleaner to keep
    John> such testing out of the build process and perform the tests
    John> in the eCos install script instead when the relevant
    John> toolchains are being installed. Having said that, I
    John> appreciate that experienced eCos developers may not use the
    John> install script.

Just how much does it save to link mpfr and gmp dynamically instead of
statically? Ditto for intl and iconv? If it only makes a small
difference I would much prefer to keep things robust and use static
links - and as a useful side effect, avoid problems if updated cygwin
versions of these packages have compatibility issues. Or when moving
toolchains between machines with slightly different cygwin setups. And
no doubt there are plenty of other failure scenarios.

I would not mind so much if the users were given sensible error
messages that explained exactly how to resolve the problem, but silent
failures are going to give a really bad impression to anybody just
starting out with eCos. Especially since the chances are high that any
installation instructions will have received, at best, a cursory
glance.

Bart

-- 
Bart Veer                                   eCos Configuration Architect
eCosCentric Limited    The eCos experts      http://www.ecoscentric.com/
Barnwell House, Barnwell Drive, Cambridge, UK.      Tel: +44 1223 245571
Registered in England and Wales: Reg No 4422071.

Re: Building RedBoot for SH3 targets with new toolchain

[John Dallaway]

Hi Bart

Bart Veer wrote:

>>>>>> "John" == John Dallaway <john@dallaway.org.uk> writes:
> 
>     John> Hi Andrew
>     John> Andrew Lunn wrote:
> 
>     >>> Taking all these things into account, plus the desire to avoid
>     >>> further delays to the eCos 3.0 release, I think it preferable
>     >>> to stick with the stock mpfr and gmp DLLs for the
>     >>> Cygwin-hosted toolchains. We must document the need to install
>     >>> the relevant Cygwin packages though, and add an entry to the
>     >>> FAQ.
>     >> 
>     >> Would it be possible to detect at build time if these DLLs are
>     >> installed? But a custom build rule with the highest priority in
>     >> infra, or HAL which will test if this files exist and stop and
>     >> print an useful error message if they are not installed?
>     >> 
>     >> The way cygwin is silently failing is not nice...
> 
>     John> The failure mode is certainly not nice. I expect that the
>     John> mpfr and gmp DLLs are loaded explicitly by the compiler
>     John> using LoadLibrary() and failure is not being reported at the
>     John> application level.
> 
>     John> We could emit a _warning_ message using a custom rule. An
>     John> _error_ message would be too strong since older toolchains
>     John> do not use mpfr and gmp. However, it seems cleaner to keep
>     John> such testing out of the build process and perform the tests
>     John> in the eCos install script instead when the relevant
>     John> toolchains are being installed. Having said that, I
>     John> appreciate that experienced eCos developers may not use the
>     John> install script.
> 
> Just how much does it save to link mpfr and gmp dynamically instead of
> statically? Ditto for intl and iconv?

I cannot quantify the size and performance savings without
experimentation, but I do know that using the DLLs is the right approach
for Windows hosts. If GCC is using LoadLibrary() for the mpfr and gmp
DLLs then the correct fix would be to patch GCC to report the failure.
However, tying such a fix (and the preceding investigation) to the
forthcoming release would not be appropriate at this stage in the
release cycle. Lets keep focussed on the remaining critical issues for
eCos 3.0.

John Dallaway