memcpy / Language Lawyer / optimization question

memcpy / Language Lawyer / optimization question

[Steve Ellcey]

I have a question about memcpy and when it is legal to turn it into a
simple assignment.  The first memcpy in the program below will cause an
abort on IA64 when compiled with -O2 because it gets turned into integer
assignment and p is not properly aligned.  I think this is OK.  The last
memcpy, where I put in a cast to (char *), will not assume any alignment
and this compiles and runs correctly on IA64 with -O2.  Again, I think
this is OK.

My question is about the second memcpy where I cast to (void *).  This
still results in an abort on IA64 because it is still assuming integer
alignment (and thus changing the code to do an integer assignment).  Is
it legal to do this transformation with the (void *) cast?

I ran into this in the libgfortran where I am trying to replace an
assignment of a long into a (possibly) unaligned buffer with a call to
memcpy.

Steve Ellcey
sje@cup.hp.com


------------------------------

#include <string.h>

static char buffer[80];
static int *foo();

main ()
{
        int i;
        int *p;

        i = 0;
        p = foo();

/* Causes core dump with -O2 */
        memcpy(p, &i, sizeof (int));
/* Causes core dump with -O2 */
        memcpy((void *) p, (void *) &i, sizeof (int));
/* Works with -O2 */
        memcpy((char *) p, (char *) &i, sizeof (int));
}

int *foo()
{
        return (int *) &buffer[1];
}

Re: memcpy / Language Lawyer / optimization question

[Joe Buck]

On Thu, Dec 09, 2004 at 11:32:55AM -0800, Steve Ellcey wrote:
> I have a question about memcpy and when it is legal to turn it into a
> simple assignment.  The first memcpy in the program below will cause an
> abort on IA64 when compiled with -O2 because it gets turned into integer
> assignment and p is not properly aligned.  I think this is OK.

No, it's not OK; you are lying to the compiler and paying for the offense.
By declaring the return value of foo to be a pointer to int, you are
promising that the return value will be aligned.  It isn't, therefore you
get a crash.

> static char buffer[80];
> ...
> int *foo()
> {
>         return (int *) &buffer[1];
> }

Don't ever do that.  &buffer[i] is not aligned to a 4-byte boundary, so
casting it to a pointer to int is a very bad mistake.

Re: memcpy / Language Lawyer / optimization question

[Paul Jarc]

Joe Buck <Joe.Buck@synopsys.COM> wrote:
> On Thu, Dec 09, 2004 at 11:32:55AM -0800, Steve Ellcey wrote:
>> I have a question about memcpy and when it is legal to turn it into a
>> simple assignment.  The first memcpy in the program below will cause an
>> abort on IA64 when compiled with -O2 because it gets turned into integer
>> assignment and p is not properly aligned.  I think this is OK.
>
> No, it's not OK; you are lying to the compiler and paying for the offense.

I think he's saying the crash is ok in this case, not the code.

> By declaring the return value of foo to be a pointer to int, you are
> promising that the return value will be aligned.  It isn't, therefore you
> get a crash.

Is it expected that a (char*) cast avoids the crash, while a (void*)
cast does not?


paul

Re: memcpy / Language Lawyer / optimization question

[Gabriel Dos Reis]

Steve Ellcey <sje@cup.hp.com> writes:

| I have a question about memcpy and when it is legal to turn it into a
| simple assignment.  The first memcpy in the program below will cause an
| abort on IA64 when compiled with -O2 because it gets turned into integer
| assignment and p is not properly aligned.  I think this is OK.  The last
| memcpy, where I put in a cast to (char *), will not assume any alignment
| and this compiles and runs correctly on IA64 with -O2.  Again, I think
| this is OK.
| 
| My question is about the second memcpy where I cast to (void *).  This
| still results in an abort on IA64 because it is still assuming integer
| alignment (and thus changing the code to do an integer assignment).  Is
| it legal to do this transformation with the (void *) cast?

There is an explicit permission of copying an object into a possibly
non-aligned array of char or unsigned, with say memcpy(), and copy it
back with no problem.  

I believe the rule

       [#26] A pointer to void shall have the  same  representation
       and  alignment  requirements  as  a  pointer  to a character
       type.39)  Similarly, pointers to  qualified  or  unqualified
       versions   of   compatible   types   shall   have  the  same
       representation and alignment requirements.  All pointers  to
       structure  types  shall  have  the  same  representation and
       alignment requirements as each other.  All pointers to union
       types  shall  have  the  same  representation  and alignment
       requirements as each other.  Pointers to  other  types  need
       not  have the same representation or alignment requirements.

implies that any alignment derivation we make on char* shall also hold
for void*.  In particular the use of void* should inhibit the
transformation of memcpy into an object assignment.

-- Gaby

Re: memcpy / Language Lawyer / optimization question

[Joe Buck]

On Thu, Dec 09, 2004 at 11:32:55AM -0800, Steve Ellcey wrote:
> [ unaligned int and memcpy ]
...
> I ran into this in the libgfortran where I am trying to replace an
> assignment of a long into a (possibly) unaligned buffer with a call to
> memcpy.

I misunderstood your original question.  The issue of lying to the
compiler is still an important matter: if the user calls memcpy, a
standard function, the compiler is allowed to optimize it based on the
rules of C.  It is very dangerous to find a case that gcc currently
does not optimize, and rely on this to do aligned word copies, because
future work on gcc is likely to break this.

One approach would be to force a call to the real memcpy.  Ideally, there
would be a hook that would specify that an unaligned transfer must be
done, even in a case that appears aligned based on the type system.

Re: memcpy / Language Lawyer / optimization question

[Joseph S. Myers]

On Thu, 9 Dec 2004, Steve Ellcey wrote:

> My question is about the second memcpy where I cast to (void *).  This
> still results in an abort on IA64 because it is still assuming integer
> alignment (and thus changing the code to do an integer assignment).  Is
> it legal to do this transformation with the (void *) cast?

The casts in the calls to memcpy are irrelevant: the undefined behavior 
occurs before them, at the point where you cast an unaligned pointer to 
int *.  Casting a pointer to another pointer type for which it doesn't 
have the alignment yields undefined behavior.  The compiler can assume 
that the result of a sequence of pointer casts has the alignment of 
whichever type in the sequence of pointer types has the strictest 
alignment requirements.

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
    jsm@polyomino.org.uk (personal mail)
    joseph@codesourcery.com (CodeSourcery mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)

Re: memcpy / Language Lawyer / optimization question

[Steve Ellcey]

> From: "Joseph S. Myers" <joseph@codesourcery.com>
>
> The casts in the calls to memcpy are irrelevant: the undefined behavior 
> occurs before them, at the point where you cast an unaligned pointer to 
> int *.

Thanks to everyone for the info on memcpy.  I think I have the right fix
for the Fortran library.  There is a routine that returns a 'char *' and
that return value was being cast to 'int *' and put into an 'int *'
variable before being passed into memcpy.

If I remove the cast and make the variable a 'char *' then I don't need
any casts in the memcpy call and everything works correctly.

It does looks like 'void *' should have been behaving just like 'char *'
so I will look into that issue some more and either come up with a patch
or submit a PR for the problem.

Steve Ellcey
sje@cup.hp.com

Re: memcpy / Language Lawyer / optimization question

[Paul Schlie]

> From: Joe Buck <Joe.Buck@synopsys.COM>
> On Thu, Dec 09, 2004 at 05:54:08PM -0500, Paul Schlie wrote:
>> As a more general but related question: as C does not define/specify
>> everything, and even occasionally specifies something as being explicitly
>> "unspecified", giving license to the complier to express whatever incidental
>> behavior it may have; has the GCC team adopted the strategy that it will
>> strive to adopt and document the behavior which seems most generally useful
>> and/or least-fragile in such circumstances ... ?
> 
> No, because such a design decision would lead to a compiler that produces
> extremely slow code.  Some programmer might accidentally try to use
> unaligned integers on a processor that does not support unaligned word
> accesses, so your dictum would force the compiler to do all memory
> accesses one byte at a time, just so that this hypothetical programmer can
> continue to write sloppy code.
> 
> If, as a programmer, you follow the language standard, the gcc team will
> immediately accept that any failure to handle your code is a gcc bug.
> Same goes for use of a documented extension in a documented way.  Do
> otherwise, and you are on your own.

> From: Geoffrey Keating <geoffk@geoffk.org>
> No, that strategy has not been adopted.  This avoids arguments about
> what is "most generally useful" or "least-fragile".  What people
> usually mean by such statements is "whatever compiler X does" or
> "whatever the previous version of GCC does" or "this particular
> behaviour that my program expects", and generally you can find
> evidence in that form for many possible behaviours.

Thanks, understood.

-paul-

Re: memcpy / Language Lawyer / optimization question

[Joe Buck]

On Thu, Dec 09, 2004 at 05:54:08PM -0500, Paul Schlie wrote:
> As a more general but related question: as C does not define/specify
> everything, and even occasionally specifies something as being explicitly
> "unspecified", giving license to the complier to express whatever incidental
> behavior it may have; has the GCC team adopted the strategy that it will
> strive to adopt and document the behavior which seems most generally useful
> and/or least-fragile in such circumstances ... ?

No, because such a design decision would lead to a compiler that produces
extremely slow code.  Some programmer might accidentally try to use
unaligned integers on a processor that does not support unaligned word
accesses, so your dictum would force the compiler to do all memory
accesses one byte at a time, just so that this hypothetical programmer can
continue to write sloppy code.

If, as a programmer, you follow the language standard, the gcc team will
immediately accept that any failure to handle your code is a gcc bug.
Same goes for use of a documented extension in a documented way.  Do
otherwise, and you are on your own.

Re: memcpy / Language Lawyer / optimization question

[Geoffrey Keating]

Paul Schlie <schlie@comcast.net> writes:

> As a more general but related question: as C does not define/specify
> everything, and even occasionally specifies something as being explicitly
> "unspecified", giving license to the complier to express whatever incidental
> behavior it may have; has the GCC team adopted the strategy that it will
> strive to adopt and document the behavior which seems most generally useful
> and/or least-fragile in such circumstances (which it has license to do), as
> opposed to justifying any such behavior as being "legal" without regard to
> the consideration of adopting a more useful behavior, and/or emit warnings
> if the adopted behavior may have undesirable consequences when reasonable
> to do so; or is this also "unspecified"? :)

No, that strategy has not been adopted.  This avoids arguments about
what is "most generally useful" or "least-fragile".  What people
usually mean by such statements is "whatever compiler X does" or
"whatever the previous version of GCC does" or "this particular
behaviour that my program expects", and generally you can find
evidence in that form for many possible behaviours.

Re: memcpy / Language Lawyer / optimization question

[Paul Schlie]

> Joseph S. Myers wrote:
>> On Thu, 9 Dec 2004, Steve Ellcey wrote:
>> My question is about the second memcpy where I cast to (void *).  This
>> still results in an abort on IA64 because it is still assuming integer
>> alignment (and thus changing the code to do an integer assignment).  Is
>> it legal to do this transformation with the (void *) cast?
>
> The casts in the calls to memcpy are irrelevant: the undefined behavior
> occurs before them, at the point where you cast an unaligned pointer to
> int *.  Casting a pointer to another pointer type for which it doesn't
> have the alignment yields undefined behavior.  The compiler can assume
> that the result of a sequence of pointer casts has the alignment of
> whichever type in the sequence of pointer types has the strictest
> alignment requirements.

As a more general but related question: as C does not define/specify
everything, and even occasionally specifies something as being explicitly
"unspecified", giving license to the complier to express whatever incidental
behavior it may have; has the GCC team adopted the strategy that it will
strive to adopt and document the behavior which seems most generally useful
and/or least-fragile in such circumstances (which it has license to do), as
opposed to justifying any such behavior as being "legal" without regard to
the consideration of adopting a more useful behavior, and/or emit warnings
if the adopted behavior may have undesirable consequences when reasonable
to do so; or is this also "unspecified"? :)