Re: memcpy to an unaligned address

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/4/05, Carl Whitwell <carl.whitwell@gmail.com> wrote:
> Hi, 
>     thought I'd drop you a mail, would put it on gcc mailing list but
> haven't got time to work out how to send it there at this moment. 

The gcc mailing list is gcc@sources.redhat.com.
   
> All testing here is done on x86 processors using gcc under cygwin. 

Are you using an x86 host and an arm target?

> Testing gcc 3.3.4 showed no problems with memcpy alignment.  Using your
> example, the structure s was aligned but the member variable b was
> unaligned. The assembler code produced for the direct copy s->a = p and the
> memcpy replacement were identical. 

Did it produce an open code memcpy, and was it correct?

> Testing gcc 4.0.1 I found the structure s was unaligned such that the member
> variable b was aligned, which was odd. 
> I discovered that the structure appears to be aligned based upon the natural
> alignment of the last element within that structure. 
> Of course this means that the memcpy is now acting on an aligned member
> rather than an unaligned member, and works perfectly well. 

Interesting. Can you post an assembler snippet of this?

> This means that to cause an unaligned element within the structure I had to
> add another element to the structure and retest. 
> On doing this though I found gcc appeared to correctly replace memcpy with
> an unaligned copy. 
>   
> I completely agree that gcc should be handling all the alignment issues
> here, but I'm not sure what it thinks it's doing moving the structure about
> in 4.0.1 
> It may be worth tracking the addresses of the members in all the tests to
> make sure the tests are comparable across gcc versions. 
>   
> Regards, 
> Carl Whitwell 

Cheers,
Shaun

Re: memcpy to an unaligned address

[Christian Joensson]

On 8/4/05, Shaun Jackman <sjackman@gmail.com> wrote:

> The gcc mailing list is gcc@sources.redhat.com.

I'd say it's gcc@gcc.gnu.org though...

-- 
Cheers,

/ChJ

Re: memcpy to an unaligned address

[Carl Whitwell]

On 8/4/05, Shaun Jackman <sjackman@gmail.com> wrote:
> Are you using an x86 host and an arm target?
> 

Actually no, my major concern at the time was the large quantity of
legacy code with packed structures that we have on an embedded linux
x86 system. I was just testing that we didn't have an issue there with
the structure access.

> Did it produce an open code memcpy, and was it correct?
> 

I couldn't see any problems with the generated code - with no
optimisation memcpy was called as a normal function, under
optimisation the memcpy was replaced with the same code as the s->b =
n line produced.

> Interesting. Can you post an assembler snippet of this?
> 
I created an executable and made it report the alignment -
I used your test case and declared a variable of the packed structure then did
    printf("Address of a = %p\n", &test.a);
    printf("Address of b = %p\n", &test.b);
with interesting differences between gcc 3 and 4.

On further consideration i'd also add
    printf("Address of test = %p\n", &test);
just to see if it showed up anything interesting.

If I had more time I'd investigate where gcc 4 is putting this
variable in memory, I have a small concern that it may be leaving
padding bytes on the stack to align the structure, I'm not sure it's a
big issue though (it definitely isn't to me anyway since we're using
2.95.3 and the unmentionable 2.96 ;o) ).

Not sure any of this helps with the memcpy alignment issue though.

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/5/05, Carl Whitwell <carl.whitwell@gmail.com> wrote:
> On 8/4/05, Shaun Jackman <sjackman@gmail.com> wrote:
> > Are you using an x86 host and an arm target?
> 
> Actually no, my major concern at the time was the large quantity of
> legacy code with packed structures that we have on an embedded linux
> x86 system. I was just testing that we didn't have an issue there with
> the structure access.

The x86 includes hardware to fetch a word from an unaligned address by
fetching from two aligned address and shifting and combining to
produce the correct result. So, unaligned accesses on the x86 might
effect a slight performance hit, but they will still act correctly.
The ARM on the other hand does not include such hardware, so a load
instruction fetching from an unaligned address will behave
incorrectly. My primary concern is really just code correctness.

Cheers,
Shaun

memcpy to an unaligned address

[Shaun Jackman]

In a typical Ethernet/IP ARP header the source IP address is
unaligned. Instead of using...
	out->srcIPAddr = in->dstIPAddr;
... I used...
	memcpy(&out->srcIPAddr, &in->dstIPAddr, sizeof(uint32_t));
... to account for the unaligned destination. This worked until gcc 4,
which now generates a simple load/store.
	ldr     r3, [r6, #24]
	adds    r2, r4, #0
	adds    r2, #14
	str     r3, [r2, #0]
A nice optimisation, but in this case it's incorrect. $r4 is aligned,
and the result of adding #14 to $r4 is an unaligned pointer.

Should gcc know better, or do I need to give it a little more
information to help it out?

Please cc me in your reply. Cheers,
Shaun

RE: memcpy to an unaligned address

[Dave Korn]

----Original Message----
>From: Shaun Jackman
>Sent: 02 August 2005 18:33

> In a typical Ethernet/IP ARP header the source IP address is
> unaligned. Instead of using...
> 	out->srcIPAddr = in->dstIPAddr;
> ... I used...
> 	memcpy(&out->srcIPAddr, &in->dstIPAddr, sizeof(uint32_t));
> ... to account for the unaligned destination. This worked until gcc 4,
> which now generates a simple load/store.
> 	ldr     r3, [r6, #24]
> 	adds    r2, r4, #0
> 	adds    r2, #14
> 	str     r3, [r2, #0]
> A nice optimisation, but in this case it's incorrect. $r4 is aligned,
> and the result of adding #14 to $r4 is an unaligned pointer.
> 
> Should gcc know better, or do I need to give it a little more
> information to help it out?

  In order for anyone to answer your questions about the alignment of
various types in a struct, don't you think you should perhaps have told us a
little about what those types actually are and how the struct is laid out?
[*]


    cheers,
      DaveK

[*] - See debugging, psychic ;)
-- 
Can't think of a witty .sigline today....

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/2/05, Dave Korn <dave.korn@artimi.com> wrote:
>   In order for anyone to answer your questions about the alignment of
> various types in a struct, don't you think you should perhaps have told us a
> little about what those types actually are and how the struct is laid out?

Of course, my apologies. I was clearly overly terse. I declare the
structure packed as follows:

typedef struct {
	uint16_t a;
	uint32_t b;
} __attribute__((packed)) st;

void foo(st *s, int n)
{
	memcpy(&s->b, &n, sizeof n);
}

This code generates the unaligend store:
$ arm-elf-objdump -d packed.o
...
   0:   e24dd004        sub     sp, sp, #4      ; 0x4
   4:   e5801002        str     r1, [r0, #2]
   8:   e28dd004        add     sp, sp, #4      ; 0x4
   c:   e12fff1e        bx      lr
$ arm-elf-gcc --version | head -1
arm-elf-gcc (GCC) 4.0.1

Cheers,
Shaun

Re: memcpy to an unaligned address

[Paul Koning]

One of the things that continues to baffle me (and my colleagues) is
the bizarre way in which attributes such as "packed" work when applied
to structs.

It would be natural to assume, as Shaun did, that marking a struct
"packed" (or, for that matter, "packed,aligned(2)") would apply that
attribute to the fields of the struct.

But it doesn't work that way.  To get the right results, you have to
stick attributes all over the structure fields, one by one.  This is
highly counterintuive.

Worse yet, in this example the attribute is applied to the structure
elements to some extent but not consistently -- it causes the fields
to be packed -- hence unaligned -- but it does not do unaligned
accesses to the fields.

This sure looks like a bug.

     paul

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/2/05, Paul Koning <pkoning@equallogic.com> wrote:
> One of the things that continues to baffle me (and my colleagues) is
> the bizarre way in which attributes such as "packed" work when applied
> to structs.
> 
> It would be natural to assume, as Shaun did, that marking a struct
> "packed" (or, for that matter, "packed,aligned(2)") would apply that
> attribute to the fields of the struct.

This is exactly the behaviour suggested by the info docs:

$ info gcc 'C Ext' 'Type Attr'
...
     Specifying this attribute for `struct' and `union' types is
     equivalent to specifying the `packed' attribute on each of the
     structure or union members.

Cheers,
Shaun

RE: memcpy to an unaligned address

[Dave Korn]

----Original Message----
>From: Shaun Jackman
>Sent: 02 August 2005 20:26

> On 8/2/05, Paul Koning <pkoning@equallogic.com> wrote:
>> One of the things that continues to baffle me (and my colleagues) is
>> the bizarre way in which attributes such as "packed" work when applied
>> to structs. 
>> 
>> It would be natural to assume, as Shaun did, that marking a struct
>> "packed" (or, for that matter, "packed,aligned(2)") would apply that
>> attribute to the fields of the struct.
> 
> This is exactly the behaviour suggested by the info docs:
> 
> $ info gcc 'C Ext' 'Type Attr'
> ...
>      Specifying this attribute for `struct' and `union' types is
>      equivalent to specifying the `packed' attribute on each of the
>      structure or union members.
> 


  There are two separate issues here:

1)  Is the base of the struct aligned to the natural alignment, or can the
struct be based at any address

2)  Is there padding between the struct members to maintain their natural
alignments (on the assumption that the struct's base address is aligned.)

  I think this is where some of the ambiguity in the docs comes from.  But
I'm about to leave the office now, so I can't go into depth with this thread
right now....


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

RE: memcpy to an unaligned address

[Paul Koning]

>>>>> "Dave" == Dave Korn <dave.korn@artimi.com> writes:

 Dave> ----Original Message----
 >> From: Shaun Jackman Sent: 02 August 2005 20:26

 >> On 8/2/05, Paul Koning <pkoning@equallogic.com> wrote:
 >>> One of the things that continues to baffle me (and my colleagues)
 >>> is the bizarre way in which attributes such as "packed" work when
 >>> applied to structs.
 >>> 
 >>> It would be natural to assume, as Shaun did, that marking a
 >>> struct "packed" (or, for that matter, "packed,aligned(2)") would
 >>> apply that attribute to the fields of the struct.
 >>  This is exactly the behaviour suggested by the info docs:
 >> 
 >> $ info gcc 'C Ext' 'Type Attr' ...  Specifying this attribute for
 >> `struct' and `union' types is equivalent to specifying the
 >> `packed' attribute on each of the structure or union members.
 >> 


 Dave> There are two separate issues here:

 Dave> 1) Is the base of the struct aligned to the natural alignment,
 Dave> or can the struct be based at any address

 Dave> 2) Is there padding between the struct members to maintain
 Dave> their natural alignments (on the assumption that the struct's
 Dave> base address is aligned.)

Sure.  But in Shaun's case it looks like (2) has been applied, except
that the compiler doesn't adjust the generated code correctly.  I
would argue that "packed" applied to a whole struct should produce
BOTH effects 1 and 2.

There's a third case for which there appears to be no notation:

3) A pointer to a T that doesn't have the normal alignment of 
   the type T.

For example, as far as I can tell, GCC offers no way to say "pointer
to unaligned int" -- short of creating a one-member struct.

   paul

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/2/05, Dave Korn <dave.korn@artimi.com> wrote:
>   There are two separate issues here:
> 
> 1)  Is the base of the struct aligned to the natural alignment, or can the
> struct be based at any address

The base of the struct is aligned to the natural alignment, four bytes
in this case.
 
> 2)  Is there padding between the struct members to maintain their natural
> alignments (on the assumption that the struct's base address is aligned.)

There is no padding. The structure is defined as
__attribute__((packed)) to explicitly remove the padding. The result
is that gcc knows the unaligned four byte member is at an offset of
two bytes from the base of the struct, but uses a four byte load at
the unaligned address of base+2. I don't expect...
	p->unaligned = n;
... to work, but I definitely expect
	memcpy(&p->unaligned, &n, sizeof p->unaligned);
to work. The second case is being optimised to the first case though
and generating and unaligned store.

Cheers,
Shaun

Re: memcpy to an unaligned address

[Paul Koning]

>>>>> "Shaun" == Shaun Jackman <sjackman@gmail.com> writes:

 >> 2) Is there padding between the struct members to maintain their
 >> natural alignments (on the assumption that the struct's base
 >> address is aligned.)

 Shaun> There is no padding. The structure is defined as
 Shaun> __attribute__((packed)) to explicitly remove the padding. The
 Shaun> result is that gcc knows the unaligned four byte member is at
 Shaun> an offset of two bytes from the base of the struct, but uses a
 Shaun> four byte load at the unaligned address of base+2. I don't
 Shaun> expect...
 Shaun>		 p-> unaligned = n;
 Shaun> ... to work, ...

I would.  If you tell gcc that a thing is unaligned, it is responsible
for doing unaligned references to it.  That very definitely includes
direct references to the content in expressions.  And in general that
works.  Clearly there is a GCC bug here; GCC put the field at an
unaligned offset, but did not do unaligned references to it.

	  paul

Re: memcpy to an unaligned address

[Mike Stump]

On Aug 2, 2005, at 1:15 PM, Shaun Jackman wrote:
> There is no padding. The structure is defined as
> __attribute__((packed)) to explicitly remove the padding. The result
> is that gcc knows the unaligned four byte member is at an offset of
> two bytes from the base of the struct, but uses a four byte load at
> the unaligned address of base+2. I don't expect...
>     p->unaligned = n;
> ... to work,

Actually, that works just fine, with:

typedef struct {
   unsigned short int a;
   unsigned int b;
} __attribute__((packed)) st;

void foo(st *s, int n)
{
   s->b = n;
}

I get:

_foo:
         @ args = 0, pretend = 0, frame = 0
         @ frame_needed = 0, uses_anonymous_args = 0
         @ link register save eliminated.
         mov     r3, r1, lsr #24
         mov     r2, r1, lsr #8
         mov     ip, r1, lsr #16
         @ lr needed for prologue
         strb    r3, [r0, #5]
         strb    r2, [r0, #3]
         strb    ip, [r0, #4]
         strb    r1, [r0, #2]
         mov     pc, lr

> but I definitely expect
>     memcpy(&p->unaligned, &n, sizeof p->unaligned);
> to work.

Ah, I was having trouble getting it to fail for me...  Now I can:

#include <memory.h>

typedef struct {
   unsigned short int a;
   unsigned int b;
} __attribute__((packed)) st;

void foo(st *s, int n)
{
   memcpy(&s->b, &n, sizeof n);
}

_foo:
         @ args = 0, pretend = 0, frame = 4
         @ frame_needed = 0, uses_anonymous_args = 0
         @ link register save eliminated.
         sub     sp, sp, #4
         @ lr needed for prologue
         str     r1, [r0, #2]
         add     sp, sp, #4
         bx      lr

Yes, this is a compiler bug in the expansion of memcpy, please file a  
bug report.  The solution is for the compiler to notice the memory  
alignment of the destination and `do-the-right-thing' when it isn't  
aligned.

Re: memcpy to an unaligned address

[Andrew Pinski]

> 
> On Aug 2, 2005, at 1:15 PM, Shaun Jackman wrote:
> > There is no padding. The structure is defined as
> > __attribute__((packed)) to explicitly remove the padding. The result
> > is that gcc knows the unaligned four byte member is at an offset of
> > two bytes from the base of the struct, but uses a four byte load at
> > the unaligned address of base+2. I don't expect...
> >     p->unaligned = n;
> > ... to work,
> 
> Actually, that works just fine, with:
> 
> typedef struct {
>    unsigned short int a;
>    unsigned int b;
> } __attribute__((packed)) st;
> 
> void foo(st *s, int n)
> {
>    s->b = n;
> }
> 
> Ah, I was having trouble getting it to fail for me...  Now I can:
> 
> #include <memory.h>
> 
> typedef struct {
>    unsigned short int a;
>    unsigned int b;
> } __attribute__((packed)) st;
> 
> void foo(st *s, int n)
> {
>    memcpy(&s->b, &n, sizeof n);
> }
> 
> Yes, this is a compiler bug in the expansion of memcpy, please file a  
> bug report.  The solution is for the compiler to notice the memory  
> alignment of the destination and `do-the-right-thing' when it isn't  
> aligned.

No it is not, once you take the address (which should be rejected), it
is of type "unsigned int *" and not unaligned variable, passing it to
memcpy assumes the type alignment is the natural alignment.

-- Pinski

Re: memcpy to an unaligned address

[Ian Lance Taylor]

Andrew Pinski <pinskia@physics.uc.edu> writes:

> > Yes, this is a compiler bug in the expansion of memcpy, please file a  
> > bug report.  The solution is for the compiler to notice the memory  
> > alignment of the destination and `do-the-right-thing' when it isn't  
> > aligned.
> 
> No it is not, once you take the address (which should be rejected), it
> is of type "unsigned int *" and not unaligned variable, passing it to
> memcpy assumes the type alignment is the natural alignment.

That argument doesn't make sense to me.  memcpy takes a void*
argument, which has no presumed alignment.  The builtin should work
the same way.  That is, there is an implicit cast to void* in the
argument to memcpy.  The compiler can certainly take advantage of any
knowledge it has about the alignment, but it can't assume anything
about the alignment that it doesn't already know.

Ian

Re: memcpy to an unaligned address

[Mike Stump]

On Aug 2, 2005, at 1:45 PM, Ian Lance Taylor wrote:
> That argument doesn't make sense to me.  memcpy takes a void*
> argument, which has no presumed alignment.

The memcpy builtin uses the static type of the actual argument  
(before conversion to void*), to gain hints about the alignments of  
the data coming in.  This is so that we can producing nice fast code  
for 1-16 bytes objects.  This is actually good.  The real problem is  
formation of the address of the member doesn't produce a pointer to  
unaligned type, but rather a pointer to aligned type, this is the  
part that is wrong.  We'd have to add pointers to unaligned data to  
our type system to fix it.  That should be done, but is a hard/big  
job, and no one has stepped forward to do it.

Re: memcpy to an unaligned address

[Joe Buck]

On Tue, Aug 02, 2005 at 02:29:44PM -0700, Mike Stump wrote:
> On Aug 2, 2005, at 1:45 PM, Ian Lance Taylor wrote:
> >That argument doesn't make sense to me.  memcpy takes a void*
> >argument, which has no presumed alignment.
> 
> The memcpy builtin uses the static type of the actual argument  
> (before conversion to void*), to gain hints about the alignments of  
> the data coming in.  This is so that we can producing nice fast code  
> for 1-16 bytes objects.  This is actually good.  The real problem is  
> formation of the address of the member doesn't produce a pointer to  
> unaligned type, but rather a pointer to aligned type, this is the  
> part that is wrong.  We'd have to add pointers to unaligned data to  
> our type system to fix it.  That should be done, but is a hard/big  
> job, and no one has stepped forward to do it.

So my suggestion to just make pointers to unaligned objects void* would
work in this case, then.

Re: memcpy to an unaligned address

[Richard Henderson]

On Tue, Aug 02, 2005 at 01:45:01PM -0700, Ian Lance Taylor wrote:
> Andrew Pinski <pinskia@physics.uc.edu> writes:
> 
> > > Yes, this is a compiler bug in the expansion of memcpy, please file a  
> > > bug report.  The solution is for the compiler to notice the memory  
> > > alignment of the destination and `do-the-right-thing' when it isn't  
> > > aligned.
> > 
> > No it is not, once you take the address (which should be rejected), it
> > is of type "unsigned int *" and not unaligned variable, passing it to
> > memcpy assumes the type alignment is the natural alignment.
> 
> That argument doesn't make sense to me.

It is nevertheless correct.  Examine all of the parts of the expression.

In particular, "&s->b".  What type does it have?  In an ideal world, it
would be "pointer to unaligned integer".  But we have no such type in
our type system, so it is "pointer to integer".  This expression is ONLY
THEN passed to memcpy.  At which point we query the argument for its
alignment, and get the non-intuitive result.

If you instead pass "s" to memcpy, you should get the correct unaligned
copy.  If that isn't happening, that's a bug.


r~

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/3/05, Richard Henderson <rth@redhat.com> wrote:
> It is nevertheless correct.  Examine all of the parts of the expression.
> 
> In particular, "&s->b".  What type does it have?  In an ideal world, it
> would be "pointer to unaligned integer".  But we have no such type in
> our type system, so it is "pointer to integer".  This expression is ONLY
> THEN passed to memcpy.  At which point we query the argument for its
> alignment, and get the non-intuitive result.
> 
> If you instead pass "s" to memcpy, you should get the correct unaligned
> copy.  If that isn't happening, that's a bug.

I'm not sure I understood the last line. s is a structure, and its
address is aligned. How would you pass it to memcpy, and why would it
generate an unaligned copy?

Cheers,
Shaun

RE: memcpy to an unaligned address

[Dave Korn]

----Original Message----
>From: Shaun Jackman
>Sent: 03 August 2005 19:15

> On 8/3/05, Richard Henderson <rth@redhat.com> wrote:
>> It is nevertheless correct.  Examine all of the parts of the expression.
>> 
>> In particular, "&s->b".  What type does it have?  In an ideal world, it
>> would be "pointer to unaligned integer".  But we have no such type in
>> our type system, so it is "pointer to integer".  This expression is ONLY
>> THEN passed to memcpy.  At which point we query the argument for its
>> alignment, and get the non-intuitive result.
>> 
>> If you instead pass "s" to memcpy, you should get the correct unaligned
>> copy.  If that isn't happening, that's a bug.
> 
> I'm not sure I understood the last line. s is a structure,

  Not if "&s->b" makes any sense it isn't!


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: memcpy to an unaligned address

[Richard Henderson]

On Wed, Aug 03, 2005 at 12:15:05PM -0600, Shaun Jackman wrote:
> I'm not sure I understood the last line. s is a structure, and its
> address is aligned. How would you pass it to memcpy, and why would it
> generate an unaligned copy?

In the example I was replying to, S is a pointer to a structure,
and it wasn't aligned.


r~

Re: memcpy to an unaligned address

[Ian Lance Taylor]

Richard Henderson <rth@redhat.com> writes:

> On Tue, Aug 02, 2005 at 01:45:01PM -0700, Ian Lance Taylor wrote:
> > Andrew Pinski <pinskia@physics.uc.edu> writes:
> > 
> > > > Yes, this is a compiler bug in the expansion of memcpy, please file a  
> > > > bug report.  The solution is for the compiler to notice the memory  
> > > > alignment of the destination and `do-the-right-thing' when it isn't  
> > > > aligned.
> > > 
> > > No it is not, once you take the address (which should be rejected), it
> > > is of type "unsigned int *" and not unaligned variable, passing it to
> > > memcpy assumes the type alignment is the natural alignment.
> > 
> > That argument doesn't make sense to me.
> 
> It is nevertheless correct.  Examine all of the parts of the expression.
> 
> In particular, "&s->b".  What type does it have?  In an ideal world, it
> would be "pointer to unaligned integer".  But we have no such type in
> our type system, so it is "pointer to integer".  This expression is ONLY
> THEN passed to memcpy.  At which point we query the argument for its
> alignment, and get the non-intuitive result.

That's a good explanation for what gcc is doing, and I do now
understand better than I did before.  But I thought Andrew was arguing
that gcc's behaviour is correct, and is not a compiler bug.  I still
think that gcc's behaviour here is not correct.  It is clear that if
gcc did not open code memcpy, the right thing would happen.  If gcc
open codes memcpy in such a way that it makes an incorrect assumption
about alignment, for whatever reason, I think that is a bug in the
compiler.

And so, since I think this is a bug which we want to fix, and since we
obviously want gcc to open code memcpy, I think that gcc has to
somehow notice the memory alignments involved.

Ian

Re: memcpy to an unaligned address

[Paul Koning]

>>>>> "Richard" == Richard Henderson <rth@redhat.com> writes:

 >> > No it is not, once you take the address (which should be
 >> rejected), it > is of type "unsigned int *" and not unaligned
 >> variable, passing it to > memcpy assumes the type alignment is the
 >> natural alignment.
 >> 
 >> That argument doesn't make sense to me.

 Richard> It is nevertheless correct.  Examine all of the parts of the
 Richard> expression.

 Richard> In particular, "&s->b".  What type does it have?  In an
 Richard> ideal world, it would be "pointer to unaligned integer".
 Richard> But we have no such type in our type system, so it is
 Richard> "pointer to integer".  This expression is ONLY THEN passed
 Richard> to memcpy.  At which point we query the argument for its
 Richard> alignment, and get the non-intuitive result.

The underlying problem is that the type system in GCC isn't right.

The C type system has data of various kinds, pointers to them,
structures made up of the above, etc.  GCC extends the type system by
introducing selectable alignment.  But it doesn't do it consistently.
We have int, we have *int, we have packed int, but we don't have 
*packed int.

So the outcome is "correct" only if you treat GCC's incomplete type
system as correct, which I don't agree with.  This does mean,
unfortunately, that the fix is to correct that incompleteness, which
was said to be a nontrivial task.

    paul

Re: memcpy to an unaligned address

[Paul Koning]

>>>>> "Andrew" == Andrew Pinski <pinskia@physics.uc.edu> writes:

 >> Yes, this is a compiler bug in the expansion of memcpy, please
 >> file a bug report.  The solution is for the compiler to notice the
 >> memory alignment of the destination and `do-the-right-thing' when
 >> it isn't aligned.

 Andrew> No it is not, once you take the address (which should be
 Andrew> rejected), it is of type "unsigned int *" and not unaligned
 Andrew> variable, passing it to memcpy assumes the type alignment is
 Andrew> the natural alignment.

That seems like a misfeature.

It sounds like the workaround is to avoid memcpy, and just use
variable assignment.  Alternatively, cast the pointers to char*, which
should force memcpy to do the right thing.  Ugh.

       paul

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/2/05, Paul Koning <pkoning@equallogic.com> wrote:
> It sounds like the workaround is to avoid memcpy, and just use
> variable assignment.  Alternatively, cast the pointers to char*, which
> should force memcpy to do the right thing.  Ugh.

I swear originally, back in the gcc 2.95 days, I used memcpy because
the memcpy function checked for unaligned pointers, whereas storing to
and loading from unaligned variables generated a simple store/load
instruction which wouldn't work. It seems the tables have turned and
the exact opposite is true now with gcc 4, where memcpy doesn't work,
but unaligned variables do. I believe gcc 3 behaved the same as gcc 2
-- memcpy worked, unaligned variables didn't work. Can someone confirm
this summary is correct?

It seems to me there's an argument for a _memcpy_unaligned(3)
function, as ugly as that is.

Cheers,
Shaun

Re: memcpy to an unaligned address

[Paul Koning]

>>>>> "Shaun" == Shaun Jackman <sjackman@gmail.com> writes:

 Shaun> On 8/2/05, Paul Koning <pkoning@equallogic.com> wrote:
 >> It sounds like the workaround is to avoid memcpy, and just use
 >> variable assignment.  Alternatively, cast the pointers to char*,
 >> which should force memcpy to do the right thing.  Ugh.

 Shaun> I swear originally, back in the gcc 2.95 days, I used memcpy
 Shaun> because the memcpy function checked for unaligned pointers,
 Shaun> whereas storing to and loading from unaligned variables
 Shaun> generated a simple store/load instruction which wouldn't
 Shaun> work. It seems the tables have turned and the exact opposite
 Shaun> is true now with gcc 4, where memcpy doesn't work, but
 Shaun> unaligned variables do. I believe gcc 3 behaved the same as
 Shaun> gcc 2 -- memcpy worked, unaligned variables didn't work. Can
 Shaun> someone confirm this summary is correct?

I'm pretty sure I have used unaligned load/store with gcc 2 as well as
3 and it works correctly.  Certainly it should work correctly.  If you
tell GCC to pack stuff, and it does (assigning odd offsets in the
process) then GCC is responsible for generating code to cope with
that.  All this assuming you're not playing games with pointer
arithmetic to generate pointers that are less aligned than what GCC is
entitled to assume.  If you saw something else, that would have been a
GCC bug.

    paul

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/2/05, Paul Koning <pkoning@equallogic.com> wrote:
> It sounds like the workaround is to avoid memcpy, and just use
> variable assignment.  Alternatively, cast the pointers to char*, which
> should force memcpy to do the right thing.  Ugh.

Casting to void* does not work either. gcc keeps the alignment
information -- but not the *unalignment* information, if that
distinction makes any sense -- of a particular variable around as long
as it can, through casts and even through assignment. The unalignment
information, on the other hand, is lost immediately after the &
operator. None of these examples produce an unaligned load:

	memcpy(&s->b, &n, sizeof n);

	memcpy((void*)&s->b, &n, sizeof n);

	void *p = &s->b;
	memcpy(p, &n, sizeof n);

But as pointed out by others, this does produce an unaligned load:

	s->b = n;

Cheers,
Shaun

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/2/05, Shaun Jackman <sjackman@gmail.com> wrote:
> operator. None of these examples produce an unaligned load:

I should clarify the wording I'm using here. By "an unaligned load" I
mean code to safely load from an unaligned pointer.

Cheers,
Shaun

Re: memcpy to an unaligned address

[Mike Stump]

On Aug 2, 2005, at 1:37 PM, Andrew Pinski wrote:
> No it is not,

:-)  Ah, yes, the old, we don't have pointers to unaligned types  
problem...  anyway, we can at least agree that this is a gapping hole  
people can drive trucks though in the type system, but I'm still  
claiming it isn't a feature on theoretic grounds.

:-(

Shaun, want to do up an entry in the manual describing this?  We have  
known about this for years and years, but, we don't do a good job  
communicating it to users.  Essentially, & doesn't work as one would  
expect on unaligned data, as it produces a pointer to an aligned  
object instead of a pointer to unaligned object.  Essentially, we  
don't have a type system that contains pointer to unaligned types.   
The compiler then goes on to make codegen choices based upon the fact  
that the data are known to be aligned, and bad things happen.

Re: memcpy to an unaligned address

[Joe Buck]

On Tue, Aug 02, 2005 at 02:04:16PM -0700, Mike Stump wrote:
> Shaun, want to do up an entry in the manual describing this?  We have  
> known about this for years and years, but, we don't do a good job  
> communicating it to users.  Essentially, & doesn't work as one would  
> expect on unaligned data, as it produces a pointer to an aligned  
> object instead of a pointer to unaligned object.

I suppose we could make & on an unaligned project return a void*.  That
isn't really right, but it would at least prevent the cases that we know
don't work from compiling.

Re: memcpy to an unaligned address

[Shaun Jackman]

On 8/2/05, Joe Buck <Joe.Buck@synopsys.com> wrote:
> I suppose we could make & on an unaligned project return a void*.  That
> isn't really right, but it would at least prevent the cases that we know
> don't work from compiling.

That sounds like a dangerous idea only because I'd expect...
	int *p = &packed_struct.unaligned_member;
... to fail if unaligned_member is not an int, but if the & operator
returns a void*, it would suddenly become very permissive.

Cheers,
Shaun

Re: memcpy to an unaligned address

[Joe Buck]

On Tue, Aug 02, 2005 at 04:07:00PM -0600, Shaun Jackman wrote:
> On 8/2/05, Joe Buck <Joe.Buck@synopsys.com> wrote:
> > I suppose we could make & on an unaligned project return a void*.  That
> > isn't really right, but it would at least prevent the cases that we know
> > don't work from compiling.
> 
> That sounds like a dangerous idea only because I'd expect...
> 	int *p = &packed_struct.unaligned_member;
> ... to fail if unaligned_member is not an int, but if the & operator
> returns a void*, it would suddenly become very permissive.

Ah.  I was thinking as a C++ programmer, where void* cannot be assigned to
int* without an explicit cast.  The decision to allow this in C was the
worst mistake the standards committee made.

The problem is that the type returned by malloc is not just any void*,
but a special pointer that is guaranteed to have alignment sufficient
to store any type.  This is very different from the type of the arguments
to memcpy, which is assumed to have no alignment that can be counted on.

Re: memcpy to an unaligned address

[Falk Hueffner]

Shaun Jackman <sjackman@gmail.com> writes:

> In a typical Ethernet/IP ARP header the source IP address is
> unaligned. Instead of using...
> 	out->srcIPAddr = in->dstIPAddr;
> ... I used...
> 	memcpy(&out->srcIPAddr, &in->dstIPAddr, sizeof(uint32_t));
> ... to account for the unaligned destination. This worked until gcc 4,
> which now generates a simple load/store.
> 	ldr     r3, [r6, #24]
> 	adds    r2, r4, #0
> 	adds    r2, #14
> 	str     r3, [r2, #0]
> A nice optimisation, but in this case it's incorrect. $r4 is aligned,
> and the result of adding #14 to $r4 is an unaligned pointer.

It isn't incorrect; gcc can assume that pointers are always correctly
aligned for their type. Anything else would result in horrible code.
If your program forms a pointer that is not properly aligned, it is
already invalid, and later breakage is only a symptom of that.

-- 
	Falk

Re: memcpy to an unaligned address

[Mike Stump]

On Aug 2, 2005, at 10:32 AM, Shaun Jackman wrote:
> In a typical Ethernet/IP ARP header the source IP address is
> unaligned. Instead of using...
>     out->srcIPAddr = in->dstIPAddr;
> ... I used...
>     memcpy(&out->srcIPAddr, &in->dstIPAddr, sizeof(uint32_t));
> ... to account for the unaligned destination. This worked until gcc 4,
> which now generates a simple load/store.
>     ldr     r3, [r6, #24]
>     adds    r2, r4, #0
>     adds    r2, #14
>     str     r3, [r2, #0]
> A nice optimisation, but in this case it's incorrect. $r4 is aligned,
> and the result of adding #14 to $r4 is an unaligned pointer.
>
> Should gcc know better, or do I need to give it a little more
> information to help it out?

gcc-help is the correct list to ask this question on.  Anyway, I  
suspect people would be aided in helping you by seeing the source  
code and knowing what version of gcc you're using...  I suspect you  
don't mark the structure as packed and as using 1 or 2 byte  
alignment.  If you do that, then the compiler should generate the  
correct code, for example:

mrs $ cat t1.c
struct {
   char a[14];
   int i __attribute__((aligned(1), packed));
} s, d;

main() {
   d.i = s.i;
}


$ arm-gcc -O4 t1.c -S

gives:

_main:
         @ args = 0, pretend = 0, frame = 0
         @ frame_needed = 0, uses_anonymous_args = 0
         @ link register save eliminated.
         str     sl, [sp, #-4]!
         ldr     sl, .L3
         ldr     r2, .L3+4
.L2:
         add     sl, pc, sl
         ldr     ip, [sl, r2]
         ldr     r0, .L3+8
         ldrh    r1, [ip, #16]
         ldr     r2, [sl, r0]
         ldrh    r3, [ip, #14]
         @ lr needed for prologue
         strh    r1, [r2, #16]   @ movhi
         strh    r3, [r2, #14]   @ movhi
         ldmfd   sp!, {sl}
         mov     pc, lr

for me.  Notice the adding of 14, notice the two 16 bit moves instead  
of one 4 byte move.  If you lie to the compiler, it will make your  
life rough.  Telling it that it is aligned, when the data isn't  
aligned, is a lie.