Clarification anyone? -- was Re: Linux and aliasing?

Clarification anyone? -- was Re: Linux and aliasing?

[george]

Can someone post or otherwise direct me to a clear statement of what the new
aliasing rules are?

I've been following this thread from the beginning, but I don't have the
ANSI/ISO standard for C, and I'd like to know more about what kind of code is
conformant and what isn't.

This whole thing seems a little scary--I'm using egcs 1.1.2 on a large
project and I'd be afraid to upgrade without a clear understanding of this
particular issue and/or a clear warning from the compiler when I violate the
aliasing rules, whether I violate them on purpose or by accident.

I might argue that the important issue here is not one of deliberate
non-conformance, but one of accidental non-conformance.  Debugging large
projects is hard.  It's even harder when the compiler does something one
doesn't expect--and no warning is provided.
---
George T. Talbot
<george@moberg.com>

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Tim Hollebeek]

george@moberg.com writes ...
> 
> Can someone post or otherwise direct me to a clear statement of what the new
> aliasing rules are?
> 
> I've been following this thread from the beginning, but I don't have the
> ANSI/ISO standard for C, and I'd like to know more about what kind of code is
> conformant and what isn't.

Let me try to give a simple definition, since the discussion so far
has been dominated by technical details.  It has nothing to do with
pointer casts, though they figure prominently in lots of code that
violates the ANSI assumptions.  The "main idea" is that when you
declare something to be of a particular type, that's what it's type
actually is.  When you access that object, you must access it through
pointers with the correct type (pointer to whatever) or a pointer type
designed to represent raw memory (pointer to char).

This is actually suprisingly useful information for a compiler.
Consider something like:

void count_and_set(blah *foo, double *z, double v) {
    while (*z) {
        foo->bar->count++;
        *z++ = p;
    }
}

(&foo->bar) is obviously a loop invariant and can be hoisted out of
the loop, but not in "traditional" C!  z might point into foo
somewhere, so the write through z might change the value of foo->bar.
ANSI C says such code is nonsense, and if you really meant to play
fast and loose with the type system, you should use char *'s,
memcpy's, etc.

Hope this helps anyone who is trying to follow along.

-Tim

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Jamie Lokier]

Tim Hollebeek wrote:
> This is actually suprisingly useful information for a compiler.

Indeed, this is why we want to use the information to do really clever
optimisations, while also doing the right thing on traditional code.

We have to draw the line between traditional code that's going to keep
working and code that's going to break.  That's what the thread's about.

-- Jamie

Re: Clarification anyone? -- was Re: Linux and aliasing?

[mark]

>>>>> "george" == george  <george@moberg.com> writes:

    george> I've been following this thread from the beginning, but I
    george> don't have the ANSI/ISO standard for C, and I'd like to
    george> know more about what kind of code is conformant and what
    george> isn't.

It's pretty simple: you may only access something of type X through an
lvalue of type X (i.e., a pointer of type X*, or a variable of type X,
or some such.)  The exceptions are that you may access *anything*
though a pointer of type `char*', and that you may use a `const X*' to
access a non-const X, or an `X*' to access a `const X' (similarly for
volatile).  Also, you can use `signed int*' to access an `unsigned
int'; similarly for other integer types.

    george> This whole thing seems a little scary--I'm using egcs
    george> 1.1.2 on a large project and I'd be afraid to upgrade
    george> without a clear understanding of this particular issue
    george> and/or a clear warning from the compiler when I violate
    george> the aliasing rules, whether I violate them on purpose or
    george> by accident.

Just turn on -fno-strict-aliasing; that's the same behavior you know
and love.

--
Mark Mitchell                   mark@codesourcery.com
CodeSourcery, LLC               http://www.codesourcery.com

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Nick Ing-Simmons]

Tim Hollebeek <tim@franck.Princeton.EDU> writes:
>
>Let me try to give a simple definition, since the discussion so far
>has been dominated by technical details.  It has nothing to do with
>pointer casts, though they figure prominently in lots of code that
>violates the ANSI assumptions.  The "main idea" is that when you
>declare something to be of a particular type, that's what it's type
>actually is.  When you access that object, you must access it through
>pointers with the correct type (pointer to whatever) or a pointer type
>designed to represent raw memory (pointer to char).
>
>This is actually suprisingly useful information for a compiler.
>Consider something like:
>
>void count_and_set(blah *foo, double *z, double v) {
>    while (*z) {
>        foo->bar->count++;
>        *z++ = p;
>    }
>}
>
>(&foo->bar) is obviously a loop invariant and can be hoisted out of
>the loop, but not in "traditional" C!  z might point into foo
>somewhere, so the write through z might change the value of foo->bar.
>ANSI C says such code is nonsense, and if you really meant to play
>fast and loose with the type system, you should use char *'s,
>memcpy's, etc.

Does not sound too bad so far - so can someone give an example of 
the code that would be broken by that?

So ... just to check my understanding.

I assume unions are exempt from this?

e.g.

typedef union
 {float f;
  struct
   {
#if BIG_ENDIAN
    unsigned sgn:1;
    unsigned exp:8;
    unsigned mant:23;
#else
    unsigned mant:23;
    unsigned exp:8;
    unsigned sgn:1;
#endif
   } s;
 } ieee_sp;




What is proposed is that any mem-assign can be considered to "clobber"
any value of the same type, and an assign via a char * to clobber anything ?

And thus problem is that if I do this:

int a;
*((short *) &a) = 123;

Then 'a' is not considered clobbered?

Even in the same function?

-- 
Nick Ing-Simmons <nik@tiuk.ti.com>
Via, but not speaking for: Texas Instruments Ltd.

Re: Clarification anyone? -- was Re: Linux and aliasing?

[mark]

>>>>> "Nick" == Nick Ing-Simmons <nik@tiuk.ti.com> writes:

    Nick> I assume unions are exempt from this?

Somewhat.  Please see the manual section I just posted for more
information. 

    Nick> What is proposed is that any mem-assign can be considered to
    Nick> "clobber" any value of the same type, and an assign via a
    Nick> char * to clobber anything ?

Right.

    Nick> And thus problem is that if I do this:

    Nick> int a; *((short *) &a) = 123;

    Nick> Then 'a' is not considered clobbered?

Right.

    Nick> Even in the same function?

Right.

--
Mark Mitchell                   mark@codesourcery.com
CodeSourcery, LLC               http://www.codesourcery.com

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Ross Smith]

mark@codesourcery.com wrote:
> 
> The exceptions are that you may access *anything*
> though a pointer of type `char*',

I thought it was unsigned char*? I don't think signed types are required
to have a one-to-one correspondence between valid values and all
possible bit patterns (so converting an arbitrary pointer to char* and
dereferencing might trigger a trap representation). (But I don't have a
copy of the C standard on hand, and I couldn't find anything relevant in
the C++ standard.)

--
Ross Smith <ross.s@ihug.co.nz> The Internet Group, Auckland, New Zealand
========================================================================
  I know you're desperate, I really sympathise
  I see the morbid horror flicker in your eyes
  But rest assured I'm gonna help to ease your pain
  I'm gonna put a thousand tiny implants in your brain     [Motorhead]

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Martin v. Loewis]

> I've been following this thread from the beginning, but I don't have the
> ANSI/ISO standard for C, and I'd like to know more about what kind of code is
> conformant and what isn't.

Tim already answered that, but then also explained what optimizations
the compiler can do. Here's just the rule, in different wording:

<rule>
Any access to an object of some type through a pointer of a different,
incompatible type is invalid (unless the different type is char*).
</rule>
(Look Ma, I didn't say 'alias':)

<example>
So if you have 

  struct foo *my_foo = get_some_foo();
  struct bar *my_bar = (struct bar*)my_foo;

then writing

  my_bar->some_date = 42;

is incorrect (actually, undefined). However, writing

  struct foo *my_foo2 = (struct foo*)my_bar;
  my_foo2->some_data_inside_foo = -42;

is fine: the object behind my_foo2 is really of type 'struct foo'.
</example>

Hope this clarifies it,
Martin

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Martin v. Loewis]

>     Nick> Even in the same function?
> 
> Right.

In some cases, flow analysis will detect that the value is clobbered.
Consider

unsigned long foo()
{
   unsigned long a=15;
    *((short *) &a) = 123;
   return a;
}

Compiling this with gcc-2.95 19990606 -O2 -fomit-frame-pointer gives
me

foo:
	movl $123,%eax
	ret

so the compiler clearly detects that a has been changed. The case that
doesn't work as before is

short *bar(unsigned long *a)
{
  return (short*)a;
}

unsigned long foo()
{
   unsigned long a;
   short *addr =  *bar(&a);
   a = 15;
   *addr = 123;
   return a;
}

Regards,
Martin

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Martin v. Loewis]

> I thought it was unsigned char*? I don't think signed types are required
> to have a one-to-one correspondence between valid values and all
> possible bit patterns (so converting an arbitrary pointer to char* and
> dereferencing might trigger a trap representation).

In C99, section 6.5/7 says

>> An object shall have its stored value accessed only by an lvalue
>> expression that has one of the following types:
>> - a type compatible with the effective type of the object,
>> ...
>> - a character type.

In that context, "character type" means char, signed char, and
unsigned char.

Regards,
Martin

Re: Clarification anyone? -- was Re: Linux and aliasing?

[James Antill]

"Martin v. Loewis" <martin@mira.isdn.cs.tu-berlin.de> writes:

> > I've been following this thread from the beginning, but I don't have the
> > ANSI/ISO standard for C, and I'd like to know more about what kind of code is
> > conformant and what isn't.
> 
> Tim already answered that, but then also explained what optimizations
> the compiler can do. Here's just the rule, in different wording:
> 
> <rule>
> Any access to an object of some type through a pointer of a different,
> incompatible type is invalid (unless the different type is char*).
> </rule>
> (Look Ma, I didn't say 'alias':)
> 
> <example>
> So if you have 
> 
>   struct foo *my_foo = get_some_foo();
>   struct bar *my_bar = (struct bar*)my_foo;
> 
> then writing
> 
>   my_bar->some_date = 42;
> 
> is incorrect (actually, undefined). However, writing

 Presumably unless..

 struct bar
 {
   struct foo; /* initial member */ 
   /* more stuff */
 }

 is the case ... and maybve even struct foo being an opaque type
defined in another translation unit which is compatible with struct
bar ? 

-- 
James Antill -- james@and.org
If you go to the Third World and find 100 people who have never tasted ketchup
before, you find out two things: one is that people don't actually like tomato
ketchup, the other is that they dislike all ketchups equally. -- Rob Young.

Re: Clarification anyone? -- was Re: Linux and aliasing?

[James Antill]

"Martin v. Loewis" <martin@mira.isdn.cs.tu-berlin.de> writes:

> > I've been following this thread from the beginning, but I don't have the
> > ANSI/ISO standard for C, and I'd like to know more about what kind of code is
> > conformant and what isn't.
> 
> Tim already answered that, but then also explained what optimizations
> the compiler can do. Here's just the rule, in different wording:
> 
> <rule>
> Any access to an object of some type through a pointer of a different,
> incompatible type is invalid (unless the different type is char*).
> </rule>
> (Look Ma, I didn't say 'alias':)
> 
> <example>
> So if you have 
> 
>   struct foo *my_foo = get_some_foo();
>   struct bar *my_bar = (struct bar*)my_foo;
> 
> then writing
> 
>   my_bar->some_date = 42;
> 
> is incorrect (actually, undefined). However, writing

 Presumably unless..

 struct bar
 {
   struct foo; /* initial member */ 
   /* more stuff */
 }

 is the case ... and maybve even struct foo being an opaque type
defined in another translation unit which is compatible with struct
bar ? 

-- 
James Antill -- james@and.org
If you go to the Third World and find 100 people who have never tasted ketchup
before, you find out two things: one is that people don't actually like tomato
ketchup, the other is that they dislike all ketchups equally. -- Rob Young.

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Jamie Lokier]

Tim Hollebeek wrote:
> This is actually suprisingly useful information for a compiler.

Indeed, this is why we want to use the information to do really clever
optimisations, while also doing the right thing on traditional code.

We have to draw the line between traditional code that's going to keep
working and code that's going to break.  That's what the thread's about.

-- Jamie

Re: Clarification anyone? -- was Re: Linux and aliasing?

[mark]

>>>>> "george" == george  <george@moberg.com> writes:

    george> I've been following this thread from the beginning, but I
    george> don't have the ANSI/ISO standard for C, and I'd like to
    george> know more about what kind of code is conformant and what
    george> isn't.

It's pretty simple: you may only access something of type X through an
lvalue of type X (i.e., a pointer of type X*, or a variable of type X,
or some such.)  The exceptions are that you may access *anything*
though a pointer of type `char*', and that you may use a `const X*' to
access a non-const X, or an `X*' to access a `const X' (similarly for
volatile).  Also, you can use `signed int*' to access an `unsigned
int'; similarly for other integer types.

    george> This whole thing seems a little scary--I'm using egcs
    george> 1.1.2 on a large project and I'd be afraid to upgrade
    george> without a clear understanding of this particular issue
    george> and/or a clear warning from the compiler when I violate
    george> the aliasing rules, whether I violate them on purpose or
    george> by accident.

Just turn on -fno-strict-aliasing; that's the same behavior you know
and love.

--
Mark Mitchell                   mark@codesourcery.com
CodeSourcery, LLC               http://www.codesourcery.com

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Tim Hollebeek]

george@moberg.com writes ...
> 
> Can someone post or otherwise direct me to a clear statement of what the new
> aliasing rules are?
> 
> I've been following this thread from the beginning, but I don't have the
> ANSI/ISO standard for C, and I'd like to know more about what kind of code is
> conformant and what isn't.

Let me try to give a simple definition, since the discussion so far
has been dominated by technical details.  It has nothing to do with
pointer casts, though they figure prominently in lots of code that
violates the ANSI assumptions.  The "main idea" is that when you
declare something to be of a particular type, that's what it's type
actually is.  When you access that object, you must access it through
pointers with the correct type (pointer to whatever) or a pointer type
designed to represent raw memory (pointer to char).

This is actually suprisingly useful information for a compiler.
Consider something like:

void count_and_set(blah *foo, double *z, double v) {
    while (*z) {
        foo->bar->count++;
        *z++ = p;
    }
}

(&foo->bar) is obviously a loop invariant and can be hoisted out of
the loop, but not in "traditional" C!  z might point into foo
somewhere, so the write through z might change the value of foo->bar.
ANSI C says such code is nonsense, and if you really meant to play
fast and loose with the type system, you should use char *'s,
memcpy's, etc.

Hope this helps anyone who is trying to follow along.

-Tim

Clarification anyone? -- was Re: Linux and aliasing?

[george]

Can someone post or otherwise direct me to a clear statement of what the new
aliasing rules are?

I've been following this thread from the beginning, but I don't have the
ANSI/ISO standard for C, and I'd like to know more about what kind of code is
conformant and what isn't.

This whole thing seems a little scary--I'm using egcs 1.1.2 on a large
project and I'd be afraid to upgrade without a clear understanding of this
particular issue and/or a clear warning from the compiler when I violate the
aliasing rules, whether I violate them on purpose or by accident.

I might argue that the important issue here is not one of deliberate
non-conformance, but one of accidental non-conformance.  Debugging large
projects is hard.  It's even harder when the compiler does something one
doesn't expect--and no warning is provided.
---
George T. Talbot
<george@moberg.com>

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Martin v. Loewis]

> I thought it was unsigned char*? I don't think signed types are required
> to have a one-to-one correspondence between valid values and all
> possible bit patterns (so converting an arbitrary pointer to char* and
> dereferencing might trigger a trap representation).

In C99, section 6.5/7 says

>> An object shall have its stored value accessed only by an lvalue
>> expression that has one of the following types:
>> - a type compatible with the effective type of the object,
>> ...
>> - a character type.

In that context, "character type" means char, signed char, and
unsigned char.

Regards,
Martin

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Ross Smith]

mark@codesourcery.com wrote:
> 
> The exceptions are that you may access *anything*
> though a pointer of type `char*',

I thought it was unsigned char*? I don't think signed types are required
to have a one-to-one correspondence between valid values and all
possible bit patterns (so converting an arbitrary pointer to char* and
dereferencing might trigger a trap representation). (But I don't have a
copy of the C standard on hand, and I couldn't find anything relevant in
the C++ standard.)

--
Ross Smith <ross.s@ihug.co.nz> The Internet Group, Auckland, New Zealand
========================================================================
  I know you're desperate, I really sympathise
  I see the morbid horror flicker in your eyes
  But rest assured I'm gonna help to ease your pain
  I'm gonna put a thousand tiny implants in your brain     [Motorhead]

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Martin v. Loewis]

> I've been following this thread from the beginning, but I don't have the
> ANSI/ISO standard for C, and I'd like to know more about what kind of code is
> conformant and what isn't.

Tim already answered that, but then also explained what optimizations
the compiler can do. Here's just the rule, in different wording:

<rule>
Any access to an object of some type through a pointer of a different,
incompatible type is invalid (unless the different type is char*).
</rule>
(Look Ma, I didn't say 'alias':)

<example>
So if you have 

  struct foo *my_foo = get_some_foo();
  struct bar *my_bar = (struct bar*)my_foo;

then writing

  my_bar->some_date = 42;

is incorrect (actually, undefined). However, writing

  struct foo *my_foo2 = (struct foo*)my_bar;
  my_foo2->some_data_inside_foo = -42;

is fine: the object behind my_foo2 is really of type 'struct foo'.
</example>

Hope this clarifies it,
Martin

Re: Clarification anyone? -- was Re: Linux and aliasing?

[mark]

>>>>> "Nick" == Nick Ing-Simmons <nik@tiuk.ti.com> writes:

    Nick> I assume unions are exempt from this?

Somewhat.  Please see the manual section I just posted for more
information. 

    Nick> What is proposed is that any mem-assign can be considered to
    Nick> "clobber" any value of the same type, and an assign via a
    Nick> char * to clobber anything ?

Right.

    Nick> And thus problem is that if I do this:

    Nick> int a; *((short *) &a) = 123;

    Nick> Then 'a' is not considered clobbered?

Right.

    Nick> Even in the same function?

Right.

--
Mark Mitchell                   mark@codesourcery.com
CodeSourcery, LLC               http://www.codesourcery.com

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Martin v. Loewis]

>     Nick> Even in the same function?
> 
> Right.

In some cases, flow analysis will detect that the value is clobbered.
Consider

unsigned long foo()
{
   unsigned long a=15;
    *((short *) &a) = 123;
   return a;
}

Compiling this with gcc-2.95 19990606 -O2 -fomit-frame-pointer gives
me

foo:
	movl $123,%eax
	ret

so the compiler clearly detects that a has been changed. The case that
doesn't work as before is

short *bar(unsigned long *a)
{
  return (short*)a;
}

unsigned long foo()
{
   unsigned long a;
   short *addr =  *bar(&a);
   a = 15;
   *addr = 123;
   return a;
}

Regards,
Martin

Re: Clarification anyone? -- was Re: Linux and aliasing?

[Nick Ing-Simmons]

Tim Hollebeek <tim@franck.Princeton.EDU> writes:
>
>Let me try to give a simple definition, since the discussion so far
>has been dominated by technical details.  It has nothing to do with
>pointer casts, though they figure prominently in lots of code that
>violates the ANSI assumptions.  The "main idea" is that when you
>declare something to be of a particular type, that's what it's type
>actually is.  When you access that object, you must access it through
>pointers with the correct type (pointer to whatever) or a pointer type
>designed to represent raw memory (pointer to char).
>
>This is actually suprisingly useful information for a compiler.
>Consider something like:
>
>void count_and_set(blah *foo, double *z, double v) {
>    while (*z) {
>        foo->bar->count++;
>        *z++ = p;
>    }
>}
>
>(&foo->bar) is obviously a loop invariant and can be hoisted out of
>the loop, but not in "traditional" C!  z might point into foo
>somewhere, so the write through z might change the value of foo->bar.
>ANSI C says such code is nonsense, and if you really meant to play
>fast and loose with the type system, you should use char *'s,
>memcpy's, etc.

Does not sound too bad so far - so can someone give an example of 
the code that would be broken by that?

So ... just to check my understanding.

I assume unions are exempt from this?

e.g.

typedef union
 {float f;
  struct
   {
#if BIG_ENDIAN
    unsigned sgn:1;
    unsigned exp:8;
    unsigned mant:23;
#else
    unsigned mant:23;
    unsigned exp:8;
    unsigned sgn:1;
#endif
   } s;
 } ieee_sp;




What is proposed is that any mem-assign can be considered to "clobber"
any value of the same type, and an assign via a char * to clobber anything ?

And thus problem is that if I do this:

int a;
*((short *) &a) = 123;

Then 'a' is not considered clobbered?

Even in the same function?

-- 
Nick Ing-Simmons <nik@tiuk.ti.com>
Via, but not speaking for: Texas Instruments Ltd.