Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

Hi,
    I have found the ld of the binutils will hang if the output binary
    is too big.
    And here is the crash log that generated:

    ./obj/local/armeabi/objs-debug/WebCore_UC/JavaScriptCore/wtf/ArrayBuffer.o:
    In function `WTF::Vector<WTF::RefPtr<WTF::ArrayBufferView>,
    0u>::expandCapacity(unsigned int)':
    /media/linzj/normal/home/linzj/src/u3/shell-git/core/JavaScriptCore/wtf/Vector.h:793:(.text._ZN3WTF6VectorINS_6RefPtrINS_15ArrayBufferViewEEELj0EE14expandCapacityEj[_ZN3WTF6VectorINS_6RefPtrINS_15ArrayBufferViewEEELj0EE14expandCapacityEj]+0x30):
    relocation truncated to fit: R_ARM_THM_CALL against symbol `unsigned
    int const& std::max<unsigned int>(unsigned int const&, unsigned int
    const&)' defined in .glue_7 section in linker stubs

    To trace what goes wrong, I add an extra print in the function
    elf32_arm_final_link_relocate,

	/* Assumes two's complement.  */
	if (signed_check > reloc_signed_max || signed_check < reloc_signed_min)
      {
        printf ("reloc_signed_min = %ld, reloc_signed_max = %ld, signed_check = %ld, bitsize = %d\n", reloc_signed_min, reloc_signed_max, signed_check, bitsize);
	  overflow = TRUE;

    And here is what comes out:
    reloc_signed_min = -2097152, reloc_signed_max = 2097151,
    signed_check = 2102538, bitsize = 22

    And I find out the stub of the input section is grouped under
    another section.So I tried to half the value of stub_group_size 
    in the function elf32_arm_size_stubs, like following:  

diff --git a/bfd/elf32-arm.c b/bfd/elf32-arm.c
index 6a9e60f..7a216cf 100644
--- a/bfd/elf32-arm.c
+++ b/bfd/elf32-arm.c
@@ -4964,7 +4964,7 @@ elf32_arm_size_stubs (bfd *output_bfd,
         12-byte stubs.  If we exceed that, then we will fail to link.
         The user will have to relink with an explicit group size
         option.  */
-      stub_group_size = 4170000;
+      stub_group_size = 4170000 / 2;
     }
 
   group_sections (htab, stub_group_size, stubs_always_after_branch);
    
    And the crash is vanished immediately.So I think the problem comes
    from this stub_group_size.
    Any idea on this problem?

--
Regards
lin zuojian

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

The following patch works,too.I make the modification referring to the
source of gold.
  return ((thumb2
	   ? Bits<25>::has_overflow32(branch_offset)
	   : Bits<23>::has_overflow32(branch_offset))
It says 25 is okay too.So I try it and it works out.

diff --git a/bfd/elf32-arm.c b/bfd/elf32-arm.c
index 6a9e60f..0c19208 100644
--- a/bfd/elf32-arm.c
+++ b/bfd/elf32-arm.c
@@ -226,7 +226,7 @@ static reloc_howto_type elf32_arm_howto_table_1[] =
   HOWTO (R_ARM_THM_CALL,	/* type */
 	 1,			/* rightshift */
 	 2,			/* size (0 = byte, 1 = short, 2 = long) */
-	 24,			/* bitsize */
+	 25,			/* bitsize */
 	 TRUE,			/* pc_relative */
 	 0,			/* bitpos */
 	 complain_overflow_signed,/* complain_on_overflow */


--
Regards
lin zuojian

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

On Fri, Mar 28, 2014 at 06:18:02PM +0800, lin zuojian wrote:
> The following patch works,too.I make the modification referring to the
> source of gold.
>   return ((thumb2
> 	   ? Bits<25>::has_overflow32(branch_offset)
> 	   : Bits<23>::has_overflow32(branch_offset))
> It says 25 is okay too.So I try it and it works out.
This source is in arm.cc, function
Arm_relocate_functions<big_endian>::thumb_branch_common,
at the end of this function.
> 
> diff --git a/bfd/elf32-arm.c b/bfd/elf32-arm.c
> index 6a9e60f..0c19208 100644
> --- a/bfd/elf32-arm.c
> +++ b/bfd/elf32-arm.c
> @@ -226,7 +226,7 @@ static reloc_howto_type elf32_arm_howto_table_1[] =
>    HOWTO (R_ARM_THM_CALL,	/* type */
>  	 1,			/* rightshift */
>  	 2,			/* size (0 = byte, 1 = short, 2 = long) */
> -	 24,			/* bitsize */
> +	 25,			/* bitsize */
>  	 TRUE,			/* pc_relative */
>  	 0,			/* bitpos */
>  	 complain_overflow_signed,/* complain_on_overflow */
> 
> 
> --
> Regards
> lin zuojian

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[Will Newton]

On 28 March 2014 09:49, lin zuojian <manjian2006@gmail.com> wrote:
> Hi,
>     I have found the ld of the binutils will hang if the output binary
>     is too big.
>     And here is the crash log that generated:
>
>     ./obj/local/armeabi/objs-debug/WebCore_UC/JavaScriptCore/wtf/ArrayBuffer.o:
>     In function `WTF::Vector<WTF::RefPtr<WTF::ArrayBufferView>,
>     0u>::expandCapacity(unsigned int)':
>     /media/linzj/normal/home/linzj/src/u3/shell-git/core/JavaScriptCore/wtf/Vector.h:793:(.text._ZN3WTF6VectorINS_6RefPtrINS_15ArrayBufferViewEEELj0EE14expandCapacityEj[_ZN3WTF6VectorINS_6RefPtrINS_15ArrayBufferViewEEELj0EE14expandCapacityEj]+0x30):
>     relocation truncated to fit: R_ARM_THM_CALL against symbol `unsigned
>     int const& std::max<unsigned int>(unsigned int const&, unsigned int
>     const&)' defined in .glue_7 section in linker stubs
>
>     To trace what goes wrong, I add an extra print in the function
>     elf32_arm_final_link_relocate,
>
>         /* Assumes two's complement.  */
>         if (signed_check > reloc_signed_max || signed_check < reloc_signed_min)
>       {
>         printf ("reloc_signed_min = %ld, reloc_signed_max = %ld, signed_check = %ld, bitsize = %d\n", reloc_signed_min, reloc_signed_max, signed_check, bitsize);
>           overflow = TRUE;
>
>     And here is what comes out:
>     reloc_signed_min = -2097152, reloc_signed_max = 2097151,
>     signed_check = 2102538, bitsize = 22
>
>     And I find out the stub of the input section is grouped under
>     another section.So I tried to half the value of stub_group_size
>     in the function elf32_arm_size_stubs, like following:
>
> diff --git a/bfd/elf32-arm.c b/bfd/elf32-arm.c
> index 6a9e60f..7a216cf 100644
> --- a/bfd/elf32-arm.c
> +++ b/bfd/elf32-arm.c
> @@ -4964,7 +4964,7 @@ elf32_arm_size_stubs (bfd *output_bfd,
>          12-byte stubs.  If we exceed that, then we will fail to link.
>          The user will have to relink with an explicit group size
>          option.  */
> -      stub_group_size = 4170000;
> +      stub_group_size = 4170000 / 2;
>      }
>
>    group_sections (htab, stub_group_size, stubs_always_after_branch);
>
>     And the crash is vanished immediately.So I think the problem comes
>     from this stub_group_size.
>     Any idea on this problem?

Do you know what is the largest value that works for you? For example
could you try 4140000?


-- 
Will Newton
Toolchain Working Group, Linaro

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

Hi Will,
> 
> Do you know what is the largest value that works for you? For example
> could you try 4140000?
That's pretty misleading. The place needed to modify is the structure:
  HOWTO (R_ARM_THM_CALL,	/* type */
	 1,			/* rightshift */
	 2,			/* size (0 = byte, 1 = short, 2 = long) */
	 24,			/* bitsize */
	 TRUE,			/* pc_relative */
	 0,			/* bitpos */
	 complain_overflow_signed,/* complain_on_overflow */
	 bfd_elf_generic_reloc,	/* special_function */
	 "R_ARM_THM_CALL",	/* name */
	 FALSE,			/* partial_inplace */
	 0x07ff2fff,		/* src_mask */
	 0x07ff2fff,		/* dst_mask */
	 TRUE),			/* pcrel_offset */

bitsize field.
I have change it to 25 to make thing works.And This is the idea of
gold,check my previous mail please.And according to the manual here:
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0204ic/Cihfddaf.html
that 16-bit thumb bl/blx instructions can jump 4MB(aka 23 bitsize
field), that means the 25 makes sense(23 + 2 ,according to the code). 

--
Regards
lin zuojian

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

Hi Alan,
    that was your commit,changing bitsize field from 25 to 24.Would you
    like to look it though, again?

commit f6ebfac0855915796ac643b48c13e4020906af5c
Author: Alan Modra <amodra@gmail.com>
Date:   Tue Jan 19 03:49:43 2010 +0000

        * elf32-arm.c (elf32_arm_howto_table_1): Correct bitsize of
        R_ARM_THM_CALL entry.
        (elf32_arm_final_link_relocate): Correct calculation of
        reloc_signed_max when doing a R_ARM_THM_CALL, R_ARM_THM_XPC22,
        or R_ARM_THM_JUMP24 relocation.

--
Regards
lin zuojian

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

Hi,
    I read the code again.And I think the structure is correct.I was
    wrong about the last message. Let's focus on the stub grouping size.
--
Regards
linzu zuojian

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

Hi,
    I was completely misunderstood about this behavior.Here's what the
    source say:
      /* Default values.  */
      /* Thumb branch range is +-4MB has to be used as the default
	 maximum size (a given section can contain both ARM and Thumb
	 code, so the worst case has to be taken into account).

	 This value is 24K less than that, which allows for 2025
	 12-byte stubs.  If we exceed that, then we will fail to link.
	 The user will have to relink with an explicit group size
	 option.  */

     So this is cause by too many stubs in the same group,and I have to
     "relink with an explicit group size option".

     But will the linker do the math for me? Retry with the suitable
     parameter when fail for example.

On Fri, Mar 28, 2014 at 08:59:52PM +0800, lin zuojian wrote:
> Hi,
>     I read the code again.And I think the structure is correct.I was
>     wrong about the last message. Let's focus on the stub grouping size.
> --
> Regards
> linzu zuojian

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[Will Newton]

On 28 March 2014 13:09, lin zuojian <manjian2006@gmail.com> wrote:
> Hi,
>     I was completely misunderstood about this behavior.Here's what the
>     source say:
>       /* Default values.  */
>       /* Thumb branch range is +-4MB has to be used as the default
>          maximum size (a given section can contain both ARM and Thumb
>          code, so the worst case has to be taken into account).
>
>          This value is 24K less than that, which allows for 2025
>          12-byte stubs.  If we exceed that, then we will fail to link.
>          The user will have to relink with an explicit group size
>          option.  */
>
>      So this is cause by too many stubs in the same group,and I have to
>      "relink with an explicit group size option".
>
>      But will the linker do the math for me? Retry with the suitable
>      parameter when fail for example.

Unfortunately no, the linker doesn't have the ability to do that at
the moment. From looking at the values used it seems like
stub_group_size could sensibly be decreased a bit, but it would be
interesting to know what the highest value that works for you is.

-- 
Will Newton
Toolchain Working Group, Linaro

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

> 
> Unfortunately no, the linker doesn't have the ability to do that at
> the moment. From looking at the values used it seems like
> stub_group_size could sensibly be decreased a bit, but it would be
> interesting to know what the highest value that works for you is.
> 
> -- 
> Will Newton
> Toolchain Working Group, Linaro
Will tell you that once I figure it out.
--
Regards
lin zuojian

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

> 
> Unfortunately no, the linker doesn't have the ability to do that at
> the moment. From looking at the values used it seems like
> stub_group_size could sensibly be decreased a bit, but it would be
> interesting to know what the highest value that works for you is.
> 
> -- 
> Will Newton
> Toolchain Working Group, Linaro
Hi Will,
    I think at least ld should echo more compernesive output.Or the
    users without such experiences will google it till they die.
--
Regards
lin zuojian 

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

Hi,
    In case google would guide you here,I leave the answer here.It's to
    add an extra option to you linker,--stub-group-size=<the size
    smaller than 4170000>.I can't tell you the exact size,but you can
    narrow it down till you link.DO NOT USE A SMALLER VALUE than
    neccessary,or the binary will have a performance penlty.

sources:
4170000:
elf32-arm.c:elf32_arm_size_stubs :
  if (stub_group_size == 1)
    {
      /* Default values.  */
      /* Thumb branch range is +-4MB has to be used as the default
	 maximum size (a given section can contain both ARM and Thumb
	 code, so the worst case has to be taken into account).

	 This value is 24K less than that, which allows for 2025
	 12-byte stubs.  If we exceed that, then we will fail to link.
	 The user will have to relink with an explicit group size
	 option.  */
      stub_group_size = 4170000;
    }

stub-group-size:
armelf.em:
PARSE_AND_LIST_LONGOPTS='
...
  { "stub-group-size", required_argument, NULL, OPTION_STUBGROUP_SIZE },
and
PARSE_AND_LIST_ARGS_CASES='
...
    case OPTION_STUBGROUP_SIZE:
      {
	const char *end;

        group_size = bfd_scan_vma (optarg, &end, 0);
        if (*end)
	  einfo (_("%P%F: invalid number `%s'\''\n"), optarg);
      }
      break;
You may also check the rest of long options that armelf ld would accept
in PARSE_AND_LIST_LONGOPTS, it's interesting. 
--
Regards
lin zuojian

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[Alan Modra]

On Fri, Mar 28, 2014 at 08:40:26PM +0800, lin zuojian wrote:
> Hi Alan,
>     that was your commit,changing bitsize field from 25 to 24.Would you
>     like to look it though, again?
> 
> commit f6ebfac0855915796ac643b48c13e4020906af5c
> Author: Alan Modra <amodra@gmail.com>
> Date:   Tue Jan 19 03:49:43 2010 +0000

I may have committed the patch, but the real author is in the
changelog entry.  See
https://sourceware.org/ml/binutils/2010-01/msg00400.html

-- 
Alan Modra
Australia Development Lab, IBM

Re: Tracing Android NDK's R_ARM_THM_CALL Truncate Problem

[lin zuojian]

Hi Will,
    The working stub-group-size for our project is 4032304, the result
    of ((1 << 22) - 12*13500).Will ld do a optimized relayout for
    sections?I have used -ffunction-sections & -fdata-sections.
--
Regards
lin zuojian