invalid offsetof from non-POD type

invalid offsetof from non-POD type

[John Quigley]

The code below triggers an "invalid offsetof from non-POD type" warning:

In this case, and in every case I've seen, the warning is a false alarm and 
the compiler produces the correct result.  Is there a case where the warning 
fires and the compiler produces an incorrect result?  (i.e.: returns the 
wrong offset)?  If there is no such case, is there anything wrong with 
providing a command line flag in gcc that allows the user to disable this 
warning?

Thanks,
- jq

#include <cstddef>
#include <iostream>
using namespace std;

class Foo
{
   public:
      int x;
      char fillerdata[256];
      int y;

      Foo()
      {
         x = 0;
         y = 0;
      }
};

int main()
{
   int yoffset = offsetof(Foo, y); // triggers warning
   cout << "yoffset: " << yoffset << endl;

   Foo foo;
   
   // get a pointer to y using the computed offset
   int* yptr = (int*)(((unsigned char*)&foo) + yoffset);
   // test it be assigning 
   *yptr = 15;
   cout << "foo values: " << foo.x << ", " << foo.y << endl;

   return 0;
}

Re: invalid offsetof from non-POD type

[Gabriel Dos Reis]

John Quigley <johnw@lowestplane.org> writes:

| The code below triggers an "invalid offsetof from non-POD type" warning:

The compiler is right.

| In this case, and in every case I've seen, the warning is a false alarm and 

No, that is not false alarm.  Your class Foo is *not* a POD, and as
such your program, technically, invokes undefined behaviour.

[...]

| class Foo
| {
|    public:
|       int x;
|       char fillerdata[256];
|       int y;
| 
|       Foo()
        ^^^^^ 

Your class has a user-defined constructor, therefore it certainly
cannot not be a POD.  Hence the warning. 


(if it were only me, I would issue a hard error and halt translation.)

-- Gaby

Re: invalid offsetof from non-POD type

[Nathan Sidwell]

John Quigley wrote:
> The code below triggers an "invalid offsetof from non-POD type" warning:
> 
> In this case, and in every case I've seen, the warning is a false alarm and 
> the compiler produces the correct result.  Is there a case where the warning 
> fires and the compiler produces an incorrect result?  (i.e.: returns the 
> wrong offset)?  If there is no such case, is there anything wrong with 
> providing a command line flag in gcc that allows the user to disable this 
> warning?
offsetof may only be applied to POD types. [18.1]/5

nathan

-- 
Nathan Sidwell    ::   http://www.codesourcery.com   ::     CodeSourcery LLC
          The voices in my head said this was stupid too
nathan@codesourcery.com : http://www.cs.bris.ac.uk/~nathan/ : nathan@acm.org

Re: invalid offsetof from non-POD type

[John Quigley]

While it is not standards compliant code, gcc still provides the correct 
result.  My question is: will gcc continue to provide the correct result in 
all cases for the foreseeable future? If so, why not add a flag to disable 
this warning and treat this as a "vendor extension" in gcc?  

It would make porting to gcc easier.  In my case I'm porting a large 
application that uses this construct throughout.  It will be quite difficult 
to make this change and will potentially introduce a lot of bugs.  So I would 
prefer to avoid it if possible.

On Monday 21 April 2003 6:39 am, Gabriel Dos Reis wrote:
> John Quigley <johnw@lowestplane.org> writes:
> | The code below triggers an "invalid offsetof from non-POD type" warning:
>
> The compiler is right.
>
> | In this case, and in every case I've seen, the warning is a false alarm
> | and
>
> No, that is not false alarm.  Your class Foo is *not* a POD, and as
> such your program, technically, invokes undefined behaviour.
>
> [...]
>
> | class Foo
> | {
> |    public:
> |       int x;
> |       char fillerdata[256];
> |       int y;
> |
> |       Foo()
>
>         ^^^^^
>
> Your class has a user-defined constructor, therefore it certainly
> cannot not be a POD.  Hence the warning.
>
>
> (if it were only me, I would issue a hard error and halt translation.)
>
> -- Gaby

Re: invalid offsetof from non-POD type

[Gabriel Dos Reis]

John Quigley <johnw@lowestplane.org> writes:

| While it is not standards compliant code, gcc still provides the correct 
| result. 

The key issue is what do you define to be the "correct" result when you
apply offsetof() to a non-POD?  

-- Gaby

Re: invalid offsetof from non-POD type

[Zack Weinberg]

Gabriel Dos Reis <gdr@integrable-solutions.net> writes:

> John Quigley <johnw@lowestplane.org> writes:
>
> | While it is not standards compliant code, gcc still provides the correct 
> | result. 
>
> The key issue is what do you define to be the "correct" result when you
> apply offsetof() to a non-POD?  

I have never really understood why the C++ standard imposes this
restriction.  There would seem to be a well-defined answer to the
question posed by offsetof(non-POD, data-member), since the data
member does exist in memory at a well-defined offset from the
beginning of the object.  If that weren't true the compiler wouldn't
be able to generate accesses to it.

zw

Re: invalid offsetof from non-POD type

[Gabriel Dos Reis]

Zack Weinberg <zack@codesourcery.com> writes:

| Gabriel Dos Reis <gdr@integrable-solutions.net> writes:
| 
| > John Quigley <johnw@lowestplane.org> writes:
| >
| > | While it is not standards compliant code, gcc still provides the correct 
| > | result. 
| >
| > The key issue is what do you define to be the "correct" result when you
| > apply offsetof() to a non-POD?  
| 
| I have never really understood why the C++ standard imposes this
| restriction.  There would seem to be a well-defined answer to the
| question posed by offsetof(non-POD, data-member), since the data
| member does exist in memory at a well-defined offset from the
| beginning of the object.  If that weren't true the compiler wouldn't
| be able to generate accesses to it.

That statement is confused.  
The issue isn't that the compiler couldn't return some random number. 
And no, the data do not always exist in memory, for example, a
subobject of "empty" class type" does not always occupy memory. Let
alone, subobject of morally virtual base classes.  Which begs my
question.

-- Gaby

Re: invalid offsetof from non-POD type

[John Quigley]

On Monday 21 April 2003 7:34 pm, Gabriel Dos Reis wrote:
> John Quigley <johnw@lowestplane.org> writes:
> | While it is not standards compliant code, gcc still provides the correct
> | result.
>
> The key issue is what do you define to be the "correct" result when you
> apply offsetof() to a non-POD?
>
> -- Gaby

This isn't a precise definition, but I would expect that I can use offsetof to 
compute the offset of a field of a non-POD type.  Then I can take a pointer 
to an instance of that non-POD, advance it by adding the offset, and the new 
pointer will be addressing the desired field in the instance.

Perhaps whoever wrote the code in gcc has a real definition, since they seemed 
to think that they were producing the right answer:

  /* Complain about other invalid uses of offsetof, even though they will
     give the right answer.  Note that we complain whether or not they
     actually used the offsetof macro, since there's no way to know at this
     point.  So we just give a warning, instead of a pedwarn.  */
  if (protect
      && CLASSTYPE_NON_POD_P (old_basetype)
      && TREE_CODE (old_datum) == INDIRECT_REF
      && integer_zerop (TREE_OPERAND (old_datum, 0)))
    warning ("\
invalid offsetof from non-POD type `%#T'; use pointer to member instead",
	     basetype);

Re: invalid offsetof from non-POD type

[Gabriel Dos Reis]

John Quigley <johnw@lowestplane.org> writes:

| On Monday 21 April 2003 7:34 pm, Gabriel Dos Reis wrote:
| > John Quigley <johnw@lowestplane.org> writes:
| > | While it is not standards compliant code, gcc still provides the correct
| > | result.
| >
| > The key issue is what do you define to be the "correct" result when you
| > apply offsetof() to a non-POD?
| >
| > -- Gaby
| 
| This isn't a precise definition, but I would expect that I can use offsetof to 
| compute the offset of a field of a non-POD type.  Then I can take a pointer 
| to an instance of that non-POD, advance it by adding the offset, and the new 
| pointer will be addressing the desired field in the instance.

That expectation is still not well formed:  the location of a
particular subobject depends on the complexity of the class hierarchy
and may require runtime computation (which means, you no longer have a
compile-time constant),  I'm leaving alone alignment issues.

| Perhaps whoever wrote the code in gcc has a real definition, since they seemed 
| to think that they were producing the right answer:

As I pointed out in the followup to Zack's message, what does it mean
to produce the "right" answer for a member whose type has a virtual
base class?  What is the "right" answer for member from a virtual base
class? 

Being able to describe a computational process is quite different from
knowing the outcome of that process at the description time.

If your point is that there are unaccuarate comments in the GCC source
code, then I agree with you.


Just FYI, the offsetof macro issue had had been  hashed in the past by
the C++ committee; from the archive, I can locate at least two major
debates, one in 1990, the second (the most "heated") in 1995.
Meanwhile, it has been debated to depth at committee meetings.  The
current wording is a result of consensus (from the opponents, I can
identify major library implementers). 

-- Gaby

Re: invalid offsetof from non-POD type

[Kai Henningsen]

gdr@integrable-solutions.net (Gabriel Dos Reis)  wrote on 22.04.03 in <m3u1cr6xgo.fsf@uniton.integrable-solutions.net>:

> Zack Weinberg <zack@codesourcery.com> writes:
>
> | Gabriel Dos Reis <gdr@integrable-solutions.net> writes:
> |
> | > John Quigley <johnw@lowestplane.org> writes:
> | >
> | > | While it is not standards compliant code, gcc still provides the
> | > correct | result.
> | >
> | > The key issue is what do you define to be the "correct" result when you
> | > apply offsetof() to a non-POD?
> |
> | I have never really understood why the C++ standard imposes this
> | restriction.  There would seem to be a well-defined answer to the
> | question posed by offsetof(non-POD, data-member), since the data
> | member does exist in memory at a well-defined offset from the
> | beginning of the object.  If that weren't true the compiler wouldn't
> | be able to generate accesses to it.
>
> That statement is confused.

Not really. You might quibble over details, but I completely agree with  
the basic content.

> The issue isn't that the compiler couldn't return some random number.

Nobody is asking for some random number. The number asked for is well- 
defined for the cases in which one typically asks for it, as far as I can  
see.

MfG Kai

Re: invalid offsetof from non-POD type

[Gabriel Dos Reis]

kaih@khms.westfalen.de (Kai Henningsen) writes:

| gdr@integrable-solutions.net (Gabriel Dos Reis)  wrote on 22.04.03 in <m3u1cr6xgo.fsf@uniton.integrable-solutions.net>:
| 
| > Zack Weinberg <zack@codesourcery.com> writes:
| >
| > | Gabriel Dos Reis <gdr@integrable-solutions.net> writes:
| > |
| > | > John Quigley <johnw@lowestplane.org> writes:
| > | >
| > | > | While it is not standards compliant code, gcc still provides the
| > | > correct | result.
| > | >
| > | > The key issue is what do you define to be the "correct" result when you
| > | > apply offsetof() to a non-POD?
| > |
| > | I have never really understood why the C++ standard imposes this
| > | restriction.  There would seem to be a well-defined answer to the
| > | question posed by offsetof(non-POD, data-member), since the data
| > | member does exist in memory at a well-defined offset from the
| > | beginning of the object.  If that weren't true the compiler wouldn't
| > | be able to generate accesses to it.
| >
| > That statement is confused.
| 
| Not really. You might quibble over details, but I completely agree with  
| the basic content.

I'm not quibbling over the details: Actually the real semantics are in
the detail.  How do you handle multiple and virtual inheritance?

| > The issue isn't that the compiler couldn't return some random number.
| 
| Nobody is asking for some random number.

The question wasn't phrased in that term, but in effect, that is what
it boils down to.

| The number asked for is well- 
| defined for the cases in which one typically asks for it, as far as I can  
| see.

It remains to define "one typically asks for it".  Something I've been
asking for since the beginning, yet no answer :-(

-- Gaby

Re: invalid offsetof from non-POD type

[Nathan Sidwell]

Gabriel Dos Reis wrote:

> I'm not quibbling over the details: Actually the real semantics are in
> the detail.  How do you handle multiple and virtual inheritance?
The std already contains wording (wrt to base casting, IIRC), where you can
do something via a non-virtual base path that you cannot do via a virtual
base path. offsetof is the same.
1) empty members and bases are not a problem. They have an address/offset,
but you cannot touch the storage at that location.
2) multiple non-virtual inheritance is no more a problem than in other data
member access situations.

Like Zack, I think it strange that I can write &thing.member but not
offsetof(thing, member) (modulo virtual bases). I can even write
'(char *)&thing.member - (char *)&thing', and all sane compilers will
evaluate it at compile time.

It is interesting that John Quigley has a large body of real code
that relies on the more lenient interpretation - if there's a lot of
it out there, offsetof should be extended.

nathan

-- 
Nathan Sidwell    ::   http://www.codesourcery.com   ::     CodeSourcery LLC
          The voices in my head said this was stupid too
nathan@codesourcery.com : http://www.cs.bris.ac.uk/~nathan/ : nathan@acm.org

Re: invalid offsetof from non-POD type

[Gabriel Dos Reis]

Nathan Sidwell <nathan@codesourcery.com> writes:

| Gabriel Dos Reis wrote:
| 
| > I'm not quibbling over the details: Actually the real semantics are in
| > the detail.  How do you handle multiple and virtual inheritance?
| The std already contains wording (wrt to base casting, IIRC), where you can
| do something via a non-virtual base path that you cannot do via a virtual
| base path. offsetof is the same.

That is an interesting point of view -- because it is -not- the way
offset currently works, or else you have a different wording in mind.  

Note that this new point of view has not been expressed when I
repeatedly asked for what constitutes the "right answer".  Using those
standard wordings implies that access control need to be applied. 

| 1) empty members and bases are not a problem. They have an address/offset,
| but you cannot touch the storage at that location.

And the original poster says he wanted to touch the storage of the
member.  

| 2) multiple non-virtual inheritance is no more a problem than in other data
| member access situations.

Then, one neeeds to spell out the constraints: Is member access
checked (presumably, yes since you proposed one takes standard
wordings)?  What about multiple or ambiguous members? 

| Like Zack, I think it strange that I can write &thing.member but not

The committee archive contains past debate on this issue.  See

  * c++std-lib-36
  * c++std-lib-56
  * thread starting at c++std-lib-4276 (read John Skaller's proposal)
  * thread starting at c++core-6935
  * thread starting at c++std-core-7310 (which mentions why offsetof
                doesn't play nicely with  members of reference types).


C++ semantics is not as simple as that of C.  In C++ &t is not always
the the address of t.


| offsetof(thing, member) (modulo virtual bases). I can even write
| '(char *)&thing.member - (char *)&thing', and all sane compilers will
| evaluate it at compile time.

I'm not sure "sane" is appropriate to conduct this technical discussion.
But... 

What a sane compile give you depends on what sane contructs you give
it. If you define the above to be your sane offsetof, why not just
wrap it into a macro or a template so that you're sure that your sane
offsetof always matches your definition of sane?

Point is "sane" is not an appropriate criterion for language
extension/design, because  what is "sane" depends on ones own
definition of "sane".

| It is interesting that John Quigley has a large body of real code
| that relies on the more lenient interpretation - if there's a lot of
| it out there, offsetof should be extended.

I'm not opposed in principle to extending offsetof.  What I'm
objecting to is the kind of description of semantics that have been
offered right now.

-- Gaby

Re: invalid offsetof from non-POD type

[Gabriel Dos Reis]

Nathan Sidwell <nathan@codesourcery.com> writes:

[...]

| Like Zack, I think it strange that I can write &thing.member but not
| offsetof(thing, member) (modulo virtual bases). I can even write
| '(char *)&thing.member - (char *)&thing', and all sane compilers will
| evaluate it at compile time.

I don't know whether you take GCC to be a "sane" compiler, but it
can't evaluate this at compile-time -- and if it does, then I would say
it is broken.

  #include <iostream>
  #include <cstddef>

  struct A {
     double d;
     int& ri;
     A(double v, int& i) : d(v), ri(i) { }
  };

  int main()
  {
     int i = 9;
     A a(3.4, i);

     std::cout << "(char*)&a.i - (char*)&a = " << ((char*)&a.ri - (char*)&a)
               << std::endl;
  }

That expression may be negative (and it is so on some platforms).

I think, we need a better technical term and criterion than "sane".

-- Gaby

Re: invalid offsetof from non-POD type

[Zack Weinberg]

Gabriel Dos Reis <gdr@integrable-solutions.net> writes:

> | 2) multiple non-virtual inheritance is no more a problem than in other data
> | member access situations.
>
> Then, one neeeds to spell out the constraints: Is member access
> checked (presumably, yes since you proposed one takes standard
> wordings)?  What about multiple or ambiguous members? 

Suppose the following definition:

For any object X of type T with data member m of type t, 
offsetof(T, m) is defined to be a value of type ptrdiff_t which
satisfies the equation

    (t *)((char *)&X + offsetof(T, m)) == &X.m

If (for whatever reason) &X.m would be a constraint violation, or if
&X.m is legitimate but there isn't exactly one value which satisfies
the equation, then offsetof(T, m) is a constraint violation.

I do not claim to be a C++ expert.  Does that cover all the
circumstances you are worried about?  An intended property is that
under no circumstances does offsetof(T, m) induce undefined behavior;
it is either well-defined or a constraint violation (-> diagnostic,
translation unit rejected).

zw

Re: invalid offsetof from non-POD type

[Joe Buck]

On Tue, Apr 22, 2003 at 05:08:07AM +0200, Gabriel Dos Reis wrote:
> Zack Weinberg <zack@codesourcery.com> writes:
> 
> | Gabriel Dos Reis <gdr@integrable-solutions.net> writes:
> | 
> | > John Quigley <johnw@lowestplane.org> writes:
> | >
> | > | While it is not standards compliant code, gcc still provides the correct 
> | > | result. 
> | >
> | > The key issue is what do you define to be the "correct" result when you
> | > apply offsetof() to a non-POD?  
> | 
> | I have never really understood why the C++ standard imposes this
> | restriction.  There would seem to be a well-defined answer to the
> | question posed by offsetof(non-POD, data-member), since the data
> | member does exist in memory at a well-defined offset from the
> | beginning of the object.  If that weren't true the compiler wouldn't
> | be able to generate accesses to it.
> 
> That statement is confused.  
> The issue isn't that the compiler couldn't return some random number. 
> And no, the data do not always exist in memory, for example, a
> subobject of "empty" class type" does not always occupy memory. Let
> alone, subobject of morally virtual base classes.  Which begs my
> question.

There are cases where offsetof() could not possibly return a meaningful
value; in particular, for virtual base classes, the compiler must in
general dereference one or more pointers to get to the field, so there
is no fixed offset between the class object address and the field address
(in GCC's implementation of a virtual base class, the derived class object
must follow a pointer to get to the base class fields).

For empty fields, as Gaby mentions, there is also an issue, because the
object is not in fact stored in the struct/class at all (it has no fields,
and all its presence does is provide operations for the class as a whole),
however, the user is unlikely to care about this (there isn't a reason why
the user would want to use offsetof for an empty field).

However, I believe that the specification in the ISO standard is too
strict, and sometimes leads to inconveniences for the developer.
Frequently in mixed C/C++ systems, it is convenient to have data types
that are POD in every way except for the presence of a constructor, for
easier initialization.  Clearly for such types, offsetof can be
well-defined.  This means that the restrictions on offsetof() could be
relaxed.

Unfortunately, to do this right, some thought would have to go into a
specification of the extension.  And then there's the interaction of
offsetof issues with type-based aliasing, which is a whole new can of
worms.

Re: invalid offsetof from non-POD type

[Joe Buck]

On Tue, Apr 22, 2003 at 12:03:04PM +0200, Gabriel Dos Reis wrote:
> Nathan Sidwell <nathan@codesourcery.com> writes:
> 
> [...]
> 
> | Like Zack, I think it strange that I can write &thing.member but not
> | offsetof(thing, member) (modulo virtual bases). I can even write
> | '(char *)&thing.member - (char *)&thing', and all sane compilers will
> | evaluate it at compile time.
> 
> I don't know whether you take GCC to be a "sane" compiler, but it
> can't evaluate this at compile-time -- and if it does, then I would say
> it is broken.

[ example deleted ]

Your example uses references, so Nathan needs at least one more "modulo".

Re: invalid offsetof from non-POD type

[Daniel Jacobowitz]

On Tue, Apr 22, 2003 at 11:19:07AM -0700, Joe Buck wrote:
> On Tue, Apr 22, 2003 at 05:08:07AM +0200, Gabriel Dos Reis wrote:
> > Zack Weinberg <zack@codesourcery.com> writes:
> > 
> > | Gabriel Dos Reis <gdr@integrable-solutions.net> writes:
> > | 
> > | > John Quigley <johnw@lowestplane.org> writes:
> > | >
> > | > | While it is not standards compliant code, gcc still provides the correct 
> > | > | result. 
> > | >
> > | > The key issue is what do you define to be the "correct" result when you
> > | > apply offsetof() to a non-POD?  
> > | 
> > | I have never really understood why the C++ standard imposes this
> > | restriction.  There would seem to be a well-defined answer to the
> > | question posed by offsetof(non-POD, data-member), since the data
> > | member does exist in memory at a well-defined offset from the
> > | beginning of the object.  If that weren't true the compiler wouldn't
> > | be able to generate accesses to it.
> > 
> > That statement is confused.  
> > The issue isn't that the compiler couldn't return some random number. 
> > And no, the data do not always exist in memory, for example, a
> > subobject of "empty" class type" does not always occupy memory. Let
> > alone, subobject of morally virtual base classes.  Which begs my
> > question.
> 
> There are cases where offsetof() could not possibly return a meaningful
> value; in particular, for virtual base classes, the compiler must in
> general dereference one or more pointers to get to the field, so there
> is no fixed offset between the class object address and the field address
> (in GCC's implementation of a virtual base class, the derived class object
> must follow a pointer to get to the base class fields).

Is there an obvious reason why it would not be correct to do exactly
this?  i.e. follow the virtual base pointers in order to establish the
result of offsetof?

[Is offsetof considered to be an integer constant expression?  I have
no idea and I don't have the standard available.]

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: invalid offsetof from non-POD type

[Joe Buck]

On Tue, Apr 22, 2003 at 10:19:03AM -0700, Zack Weinberg wrote:
> Suppose the following definition:
> 
> For any object X of type T with data member m of type t, 
> offsetof(T, m) is defined to be a value of type ptrdiff_t which
> satisfies the equation
> 
>     (t *)((char *)&X + offsetof(T, m)) == &X.m

Try it a different way: type T is a struct or class type that does not
have any direct or indirect virtual base classes.  type t is not a
reference, and it is not an empty class or struct.  In all such cases (at
least for the GCC implementation) type T is then laid out in one unit, and
the offset of m is uniquely defined.

Re: invalid offsetof from non-POD type

[Joe Buck]

On Tue, Apr 22, 2003 at 02:31:40PM -0400, Daniel Jacobowitz wrote:
> > There are cases where offsetof() could not possibly return a meaningful
> > value; in particular, for virtual base classes, the compiler must in
> > general dereference one or more pointers to get to the field, so there
> > is no fixed offset between the class object address and the field address
> > (in GCC's implementation of a virtual base class, the derived class object
> > must follow a pointer to get to the base class fields).
> 
> Is there an obvious reason why it would not be correct to do exactly
> this?  i.e. follow the virtual base pointers in order to establish the
> result of offsetof?

The typical use of offsetof is to create a transformation: given a
struct, I can map it to a field by just adding the offset to the pointer
to the base.  The problem is, when you have a virtual base class, this
transformation is broken.  Given a pointer to Foo, it might really point
to a class derived from Foo, and in this derived class, the distance to
a given field in the base class will be different.

For non-virtual derivation, the offset has the correct value even for
derived classes.

> [Is offsetof considered to be an integer constant expression?  I have
> no idea and I don't have the standard available.]

I believe that it is.  But even if it's only an integer expression, and
you try to compute it at runtime and use it later, it won't work right
for virtual bases, because the value sometimes

Re: invalid offsetof from non-POD type

[Gabriel Dos Reis]

Daniel Jacobowitz <drow@mvista.com> writes:

[...]

| > There are cases where offsetof() could not possibly return a meaningful
| > value; in particular, for virtual base classes, the compiler must in
| > general dereference one or more pointers to get to the field, so there
| > is no fixed offset between the class object address and the field address
| > (in GCC's implementation of a virtual base class, the derived class object
| > must follow a pointer to get to the base class fields).
| 
| Is there an obvious reason why it would not be correct to do exactly
| this?  i.e. follow the virtual base pointers in order to establish the
| result of offsetof?

We could do that if we introduce the notion of dynamic "offsetof" and
we don't always return an integral constant.
(I believe the necessary machinery is already in place in dynamic_cast<>)._

| [Is offsetof considered to be an integer constant expression?  I have
| no idea and I don't have the standard available.]

Yes, both C and C++ requires that the value of offsetof() be an
integral constant. 

In 1995, when John Skaller raised again the offsetof() issue, it made
a proposal to make it an operator, i.e. part of the core language -- I
believe that would have been a better approach the standard more of
less requires some magic to get a sensible definition of offsetof.

In C++, the needs for a powerful offsetof is mitigeted by the notion
of pointer to members.  If we could subtract two pointer to data members
(or complete that algebraic type, borrowing words from John Skaller)
then I believe we would not have to reinvent whell anew for making
offsetof play nicely witg non-PODs.

As I said, this is not a new topic.

-- Gaby
 

Re: invalid offsetof from non-POD type

[Gareth Pearce]

> In C++, the needs for a powerful offsetof is mitigeted by the notion
> of pointer to members.  If we could subtract two pointer to data members
> (or complete that algebraic type, borrowing words from John Skaller)
> then I believe we would not have to reinvent whell anew for making
> offsetof play nicely witg non-PODs.

I had a not too large piece of code which used offset style mechanics, which
I changed over to pointers to members (due to wanting toget rid of said
warning).  The issues I seem to remember getting annoyed at was, the lack of
common type.  Offsetof is just a number.  If you want to store a list of
different offsets, its easy.  With pointers to members, if you want to store
a list of pointers to members, where the targets arent all of the same type,
life gets much uglier.  I'm sure being strongly typed is useful, but a
'generic member pointer' type would seem nice.

I might be completely wrong here, was a while ago.

Gareth

Re: invalid offsetof from non-POD type

[John Quigley]

> As I pointed out in the followup to Zack's message, what does it mean
> to produce the "right" answer for a member whose type has a virtual
> base class?  What is the "right" answer for member from a virtual base
> class?

Maybe there is no right answer for these cases, but doesn't gcc already have 
code to flag them as an error? (in gcc 3.2.2, gcc/cp/typeck.c line 2178)  It 
seems as if it already distinguishes between the undefined and the 
well-defined cases.

Perhaps it would be useful if I described how the code that I'm working on 
uses offsetof.  It is the Torque game engine from www.garagegames.com.  The 
engine has an internal c-like scripting language.  The fields and methods of 
the c++ classes in the engine can be exposed to this scripting language.  For 
each field, the engine uses offsetof to compute the offset of the field in 
the class.  Later on, when the scripts read or write to the field, the script 
runtime locates the field by adding the offset to the base pointer of the 
object in question.

Thanks for considering this issue.

Re: invalid offsetof from non-POD type

[Nathan Sidwell]

John Quigley wrote:
> Perhaps it would be useful if I described how the code that I'm working on 
> uses offsetof.  It is the Torque game engine from www.garagegames.com.  The 
> engine has an internal c-like scripting language.  The fields and methods of 
> the c++ classes in the engine can be exposed to this scripting language.  For 
> each field, the engine uses offsetof to compute the offset of the field in 
> the class.  Later on, when the scripts read or write to the field, the script 
> runtime locates the field by adding the offset to the base pointer of the 
> object in question.
yeah, I understand the idiom. what you really want is a pointer to void data
member. One way to get what you want is something like
#define myoff(T,m) ((size_t)(&((T *)1)->m) - 1)
that
a) circumvents g++'s invalid offsetof check
b) allows m to be from a base, and DTRT if the base is not at offset zero
c) DTWT for virtual bases and reference members

BTW, do people do things like
	offsetof (T, m.sub_m)
? I don't think it blessed by the C/c++ std, but falls out of the common
implementation of offsetof.

nathan

-- 
Nathan Sidwell    ::   http://www.codesourcery.com   ::     CodeSourcery LLC
          The voices in my head said this was stupid too
nathan@codesourcery.com : http://www.cs.bris.ac.uk/~nathan/ : nathan@acm.org

Re: invalid offsetof from non-POD type

[Gareth Pearce]

> BTW, do people do things like
> offsetof (T, m.sub_m)
> ? I don't think it blessed by the C/c++ std, but falls out of the common
> implementation of offsetof.

The code I changed to use pointer to members used this kind of thing, and
working around it was terrible.

Gareth

Re: invalid offsetof from non-POD type

[Paul Koning]

>>>>> "Nathan" == Nathan Sidwell <nathan@codesourcery.com> writes:

 Nathan> John Quigley wrote:
 >> Perhaps it would be useful if I described how the code that I'm
 >> working on uses offsetof.  It is the Torque game engine from
 >> www.garagegames.com.  The engine has an internal c-like scripting
 >> language.  The fields and methods of the c++ classes in the engine
 >> can be exposed to this scripting language.  For each field, the
 >> engine uses offsetof to compute the offset of the field in the
 >> class.  Later on, when the scripts read or write to the field, the
 >> script runtime locates the field by adding the offset to the base
 >> pointer of the object in question.

 Nathan> yeah, I understand the idiom. what you really want is a
 Nathan> pointer to void data member. One way to get what you want is
 Nathan> something like

 Nathan> #define myoff(T,m) ((size_t)(&((T *)1)->m) - 1)

 Nathan> that
 Nathan> a) circumvents g++'s invalid offsetof check
 Nathan> b) allows m to be from a base, and DTRT if the base is not at
 Nathan> offset zero
 Nathan> c) DTWT for virtual bases and reference members

I noticed that seems to work, but I'd rather have a
-Wno-offsetof-null-check to suppress the warning.  That way our builds
(which are -Wall -Werror) will work again without having to hack the
code with ever more bizarre hacks.

 Nathan> BTW, do people do things like offsetof (T, m.sub_m)

I think so, yes.  Certainly I would expect that to work just as well
as the "regular" offsetof().

   paul

Re: invalid offsetof from non-POD type

[Matthias Benkmann]

On Thu, 24 Apr 2003 21:09:23 -0700 John Quigley <johnw@lowestplane.org>
wrote:

> Perhaps it would be useful if I described how the code that I'm working
> on uses offsetof.  It is the Torque game engine from
> www.garagegames.com.  The engine has an internal c-like scripting
> language.  The fields and methods of the c++ classes in the engine can
> be exposed to this scripting language.  For each field, the engine uses
> offsetof to compute the offset of the field in the class.  Later on,
> when the scripts read or write to the field, the script runtime locates
> the field by adding the offset to the base pointer of the object in
> question.

Can't you wrap the POD-stuff in a real POD like this:

#include <cstddef>
#include <iostream>  
using namespace std;

class Foo   
{
   public:
      struct foodata{
        int x;
        char fillerdata[256];
        int y;
      } data;
  
      Foo()
      {        
         data.x = 0;
         data.y = 0;
      }
};

int main()
{
   int yoffset = offsetof(Foo::foodata, y);
   cout << "yoffset: " << yoffset << endl;

   Foo foo;
   
   // get a pointer to y using the computed offset
   int* yptr = (int*)(((unsigned char*)&foo.data) + yoffset);
   // test it be assigning 
   *yptr = 15;
   cout << "foo values: " << foo.data.x << ", " << foo.data.y << endl;

   return 0;
}
 

MSB

-- 
Support bacteria - they're the only culture some people have.

Re: invalid offsetof from non-POD type

[Nathan Sidwell]

Matthias Benkmann wrote:

> Can't you wrap the POD-stuff in a real POD like this:
yes, but then you (well I've had to in the past), do something like

class Foo {
	public:
	struct Pod { ...};
	...
};

class BiggerFoo : public Foo {
	public
	struct Pod : Foo::Pod { ... }; // this is a lie now
	...
};

It gets annoying to have to write
	struct Pod { struct Foo::Pod base; ... };
as that exposes the hierachy to all BiggerFoo::Pod users.
	
nathan

-- 
Nathan Sidwell    ::   http://www.codesourcery.com   ::     CodeSourcery LLC
          The voices in my head said this was stupid too
nathan@codesourcery.com : http://www.cs.bris.ac.uk/~nathan/ : nathan@acm.org

Re: invalid offsetof from non-POD type

[Matthias Benkmann]

On Sat, 26 Apr 2003 11:44:59 +0100 Nathan Sidwell
<nathan@codesourcery.com> wrote:

> Matthias Benkmann wrote:
> 
> > Can't you wrap the POD-stuff in a real POD like this:
> yes, but then you (well I've had to in the past), do something like
> 
> class Foo {
> 	public:
> 	struct Pod { ...};
> 	...
> };
> 
> class BiggerFoo : public Foo {
> 	public
> 	struct Pod : Foo::Pod { ... }; // this is a lie now
> 	...
> };
> 
> It gets annoying to have to write
> 	struct Pod { struct Foo::Pod base; ... };
> as that exposes the hierachy to all BiggerFoo::Pod users.

Doesn't inheritance always yield a non-POD? So this wouldn't even work
with offsetof, would it?

In any case I wouldn't use offsetof in the 1st place for exposing certain
fields to a scripting language. I'd prefer a  solution such as this:

#include <cstddef>
#include <iostream>
using namespace std;


class ThingPtr
{
public:
  enum Type {Int, Long, Char};
  int& toInt() {if (t!=Int) throw 13; else return *((int*)p);};
  long& toLong() {if (t!=Long) throw 13; else return *((long*)p);};
  char& toChar() {if (t!=Char) throw 13; else return *((char*)p);};
  Type t;
  void* p;
};

class Foo
{
public:
    int x;
    char fillerdata[256];
    int y;

    static void getX(Foo& this_,ThingPtr& target)
      {target.t=ThingPtr::Int; target.p=&this_.x;};
    static void getY(Foo& this_,ThingPtr& target)  
      {target.t=ThingPtr::Int; target.p=&this_.y;};

    Foo()
    {
       x = 0;
       y = 0;
    }
     
    virtual ~Foo(){}; //need at least 1 virtual method for dynamic_cast<>
};

class Bar:public Foo
{
  char z;
public:  
  static void getZ(Foo& this_,ThingPtr& target)
      {target.t=ThingPtr::Char; target.p=&dynamic_cast<Bar&>(this_).z;};
};
  
typedef void (*getField)(Foo&,ThingPtr&);

int main()
{
   getField ygetter = &Foo::getY;
   getField zgetter = &Bar::getZ;

   Foo foo;
   
   // get a reference to y using the computed offset
   ThingPtr tp;
   (*ygetter)(foo,tp);
   int& yref=tp.toInt();
   // test it by assigning
   yref = 15;
   cout << "foo values: " << foo.x << ", " << foo.y << endl;
   
   //test type checking
   try{
     tp.toLong();
   }catch(...)   
   {
     cout<<"Type checking on field type works"<<endl;
   }
    
   try{
     (*zgetter)(foo,tp);
   }catch(...)
   {
     cout<<"Type checking on object type works"<<endl;
   }
    
   return 0;
}
 


This gives you much more flexibility and type safety. Some macro-magic can
be used to eliminate all the type-checking in the release build including
all the dynamic_casts (although of course you should only do this if the
little bit of extra performance really makes a *measurable* difference,
which is doubtful if an interpreted scripting language is involved).

MSB

-- 
Want to learn long division in ternary?
Contact me at ICQ 2101001122212.01/0.00002

Re: invalid offsetof from non-POD type

[Matt Austern]

On Sunday, April 27, 2003, at 03:57 AM, Matthias Benkmann wrote:

> On Sat, 26 Apr 2003 11:44:59 +0100 Nathan Sidwell
> <nathan@codesourcery.com> wrote:
>
>> Matthias Benkmann wrote:
>>
>>> Can't you wrap the POD-stuff in a real POD like this:
>> yes, but then you (well I've had to in the past), do something like
>>
>> class Foo {
>> 	public:
>> 	struct Pod { ...};
>> 	...
>> };
>>
>> class BiggerFoo : public Foo {
>> 	public
>> 	struct Pod : Foo::Pod { ... }; // this is a lie now
>> 	...
>> };
>>
>> It gets annoying to have to write
>> 	struct Pod { struct Foo::Pod base; ... };
>> as that exposes the hierachy to all BiggerFoo::Pod users.
>
> Doesn't inheritance always yield a non-POD? So this wouldn't even work
> with offsetof, would it?

I've got three claims about this discussion.

First, it's happening in the wrong place.  We don't want to be changing 
the language the g++ accepts from C++ to something else.  If C++98's 
restrictions on offsetof are too restrictive, then we should be having 
a discussion on the standards committee reflector to have those 
restrictions loosened in C++0x.  At present g++ is already pretty 
lenient: all we're doing  is issuing a warning.  We shouldn't be 
silencing that warning.  Users ought to be warned when they're doing 
something that's explicitly forbidden by the language standard.  I 
would strongly oppose any change to this in our compiler until we've 
brought up the issue with the standards committee.  We do not want 
half-specified language extensions.

Second, we should be careful about which restrictions we think are too 
loose and which are too tight.  The C++ standard gives implementers a 
lot of freedom about object layout, even if g++ doesn't use all of that 
freedom.  For example, imagine an implementation in which all base 
classes, not just virtual bases, are accessed through a pointer.  (For 
extra credit, imagine why an implementation like that might even be 
useful.)

Third, I do agree that POD is too restrictive a criterion.  It comes 
closer to what we want than other concepts we've already got in the 
language standard, which was probably why it was chosen: nobody wanted 
to come up with a new concept just for offsetof.  But that's just 
standardization technicality.  If we think it's worth the effort, I'm 
sure it's possible to define things more precisely.

			--Matt

Re: invalid offsetof from non-POD type

[Gabriel Dos Reis]

Arnold Cole <y09kpa@yahoo.com> writes:

|                                       In my mind,
| offsetof stands just on the same level as sizeof.

sizeof applies uniformly to all data type.

offsetof operates at a different level: that of datatypes with no
nearly no abstraction layer.

[...]

| #define offsetof(cls,member) (((size_t)&((cls
| *)0x100->member)-0x100)

it seems you have your own idea of what offsetof should mean for a
non-POD class.  That idea does not agree with mine.  For example,
given the above implementation of offsetof, what does it return for

   #include <iostream>
   struct A {
     int& r;
     A(int& i) : r(i) { }
   };

   int main()
   {
      std::cout << offset(A, r) << std::endl;
   }

on my machine (i686-pc-linux-gnu) I get a SIGSEV.

What about
 
    struct B  {
      enum { E = offset (A, r) };
    };

?

leave alone polymorphic classes.

[...]

| If any of the standard gurus out there would care to
| explain to me why offsetof shouldn't work for non POD
| types (and what POD types are), I'd appreciate it.

the issue has been hashed here and there in the past.

Anyway, it has been raised in the C++ standards committee.

-- Gaby

Re: invalid offsetof from non-POD type

[Arnold Cole]

This is regarding a thread from Apr 2003. At that time
someone complained against this warning issued by the
C compiler.

Apparently the standard mandates that 'offsets may
only be applied to POD types. [18.1]/5'

While this may be so, it doesn't solve my coding
problem and doesn't explain to me why my class, with a
user-defined constructor (hence not a POD - whatever
this is) may not be used in offsetof. In my mind,
offsetof stands just on the same level as sizeof.
There are legitimate uses for them both, regardless of
user defined constructor.

While gcc 3.2 issues the warning, I figured a way to
redefine the macro such that it doesn't complain
anymore. Here it is for you guys out there to use it,
until we all agree whether offsetof is good or bad to
use.

#define offsetof(cls,member) (((size_t)&((cls
*)0x100->member)-0x100)

The trick is to use 0x100 (or any other value) instead
of 0. I used 0x100 to avoid any potential alignment 
issues.

If any of the standard gurus out there would care to
explain to me why offsetof shouldn't work for non POD
types (and what POD types are), I'd appreciate it.

All the best!
Adrian



__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com

Re: invalid offsetof from non-POD type

[skaller]

On Thu, 2003-08-14 at 05:57, Matt Austern wrote:

> >> I think the warning can be disabled using `-Wno-invalid-offsetof'.
> >
> > OK, can someone check please? I only have gcc 3.2.2, and the
> > option appears not available there:
> 
> I added this option three months ago.  It will appear in 3.4.

Ok, thanks!

Re: invalid offsetof from non-POD type

[Matt Austern]

On Tuesday, August 12, 2003, at 02:22  PM, skaller wrote:

> On Tue, 2003-08-12 at 23:21, Fergus Henderson wrote:
>> The warning text appears to have been changed.
>> The new warning is
>>
>>       warning ("invalid access to non-static data member `%D' of NULL 
>> object",
>>                    member);
>>       warning  ("(perhaps the `offsetof' macro was used 
>> incorrectly)");
>>
>> I think the warning can be disabled using `-Wno-invalid-offsetof'.
>
> OK, can someone check please? I only have gcc 3.2.2, and the
> option appears not available there:

I added this option three months ago.  It will appear in 3.4.

			--Matt

Re: invalid offsetof from non-POD type

[skaller]

On Tue, 2003-08-12 at 23:21, Fergus Henderson wrote:
> The warning text appears to have been changed.
> The new warning is
> 
>       warning ("invalid access to non-static data member `%D' of NULL object",
>                    member);
>       warning  ("(perhaps the `offsetof' macro was used incorrectly)");
> 
> I think the warning can be disabled using `-Wno-invalid-offsetof'.

OK, can someone check please? I only have gcc 3.2.2, and the
option appears not available there:

> g++ -Wno-invalid-offsetof -ansi -pedantic -shared -I. tut/examples/tut116.cpp -o tut/examples/tut116.so
  .. ERROR CODE 0x100
cc1plus: unrecognized option `-Wno-invalid-offsetof'

[with my 34K modem downloading the *binary* of gcc is out of the
question -- forget about the source :[

Re: invalid offsetof from non-POD type

[Fergus Henderson]

The warning text appears to have been changed.
The new warning is

      warning ("invalid access to non-static data member `%D' of NULL object",
                   member);
      warning  ("(perhaps the `offsetof' macro was used incorrectly)");

I think the warning can be disabled using `-Wno-invalid-offsetof'.

On 12-Aug-2003, skaller <skaller@ozemail.com.au> wrote:
> My understanding is:
> 
> 1. g++ is entitled to issue a warning on non-standard use
> use of offsetof. However, g++ 3.2.2 also issues the same
> diagnostic for the calculation
> 
> 	(unsigned char*)(X*)0 - (unsigned char*)(void*)
> 		(&(((X*)0)->x))
> 
> which it is very misleading.

That is an unfortunate artifact of the implementation.
However, I don't agree that the new warning is misleading.

> If this calculation is invalid,
> the fault lies entirely in the subexpression
> 
> 	((X*)0)->x
> 
> which is probably undefined even if X is a POD.

That expression should IMHO not be undefined if X is a POD and the
expression occurs in an lvalue context (e.g. as the operand of unary &).
Even if the standard implies that `&(((X*)0)->x)' is undefined when X is a POD
(I'm not sure that it does), this is a very pedantic issue; all existing C++
implementations support that kind of thing, and GNU C++ should not warn
about unless `-pedantic' is specified, and maybe not even then.

If X is not a POD, then the design criteria that standard C++ should
support systems that use highly indirect implementations for non-POD
data types implies that that expression should be undefined.
Hence GCC should warn about it.

> 2. The recommendation to use 'pointers to members' instead
> should be removed.

That appears to have been done already.

> 3. At least in g++ 3.2.2 there seems to be no way
> to turn off this warning individually: I am forced
> to use -w to turn off all warnings, which makes me
> very uncomfortable. (Have I missed something?)

Did you try `-Wno-invalid-offsetof'?

> 4. The rule with offsetof was originally formulated as
> is to (a) ensure C compatibility and (b) otherwise
> provide implementors with the maximum freedom.
> 
> Unfortunately, the rule is grossly overzealous,
> and a change in the Standard must be considered necessary.
> 
> An ambitious change would involve fixing pointers to members
> so they might be adequate for tasks currently performed using
> integers, memory pointers and casts.

Sure... but that is an ambitious change.

> A less ambitious change
> would relax the requirements so that provided the member
> was accessible and the access path from the structure unambiguous
> and not crossing a virtual base boundary, the offsetof() macro
> should have a well defined result, being the number of bytes
> offset the member is from the start of the designated structure.

That would prohibit a class of C++ implementations that the C++
committee wanted to permit: those that implement ordinary inheritance
similar to multiple inheritence.   Such implementations avoid compile-time
dependencies on base class sizes, which can drastically reduce the amount
of recompilation required when a base class changes.

-- 
Fergus Henderson <fjh@cs.mu.oz.au>  |  "I have always known that the pursuit
The University of Melbourne         |  of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh>  |     -- the last words of T. S. Garp.

Re: invalid offsetof from non-POD type

[skaller]

This email refers to an archived thread on this topic.
My question is whether some action has been taken with
respect to this issue.

My understanding is:

1. g++ is entitled to issue a warning on non-standard use
use of offsetof. However, g++ 3.2.2 also issues the same
diagnostic for the calculation

	(unsigned char*)(X*)0 - (unsigned char*)(void*)
		(&(((X*)0)->x))

which it is very misleading. If this calculation is invalid,
the fault lies entirely in the subexpression

	((X*)0)->x

which is probably undefined even if X is a POD.

2. The recommendation to use 'pointers to members' instead
should be removed. The pointer to member semantics in C++
are incomplete and inadequate for certain tasks where
an integral offset is required, for example by a lower
level memory management routine.

3. At least in g++ 3.2.2 there seems to be no way
to turn off this warning individually: I am forced
to use -w to turn off all warnings, which makes me
very uncomfortable. (Have I missed something?)

4. The rule with offsetof was originally formulated as
is to (a) ensure C compatibility and (b) otherwise
provide implementors with the maximum freedom.

Unfortunately, the rule is grossly overzealous,
and a change in the Standard must be considered necessary.

An ambitious change would involve fixing pointers to members
so they might be adequate for tasks currently performed using
integers, memory pointers and casts. A less ambitious change
would relax the requirements so that provided the member
was accessible and the access path from the structure unambiguous
and not crossing a virtual base boundary, the offsetof() macro
should have a well defined result, being the number of bytes
offset the member is from the start of the designated structure.

In the meantime however, I think g++ would best serve the
community by continuing to issue a warning by default,
a hard error if enough strictness options are enabled,
and neither an error nor diagnostic if a specific switch
is set: it seems likely g++ already generates the correct code
in cases it warns about.

My particular product generates a large number of data structures
which are intended to be collected by an exact garbage collector,
and in order for the collector to operate it must know the location
of all pointers in the system. My tool generates tables containing
this data, but it must use the offsetof() macro to do so.

The only alternative I have is to bypass C++ completely and
layout all data structures manually -- forgoing any real opportunity
for C/C++ interoperability, which is one of the products key features.

I *cannot* live with the warning. It is out of the question for
a compiler to systematically generate a large number of these
spurious warnings, which it is guarranteed to do for every
piece of code my tool generates.

invalid offsetof from non-POD type

[Chris Croswhite]

Hello all,

Perhaps someone could give me had with understanding this error:
"foo.c:15: warning: invalid offsetof from non-POD type `struct foo_struct';
use pointer to member instead"

Here is the simplest test:

#include <stdio.h>
#include <linux/stddef.h>

class Myclass {
          int _val;
};

struct foo_struct {
          Myclass a;
          void *bar;
};

int main(void)
{
          printf("offset is %d\n", offsetof(struct foo_struct, bar));
}


The class within the struct is causing offsetof warning.  Could someone
describe why this is a warning and if this produces invalid code.  If this
is correect code, is there a way to turn off this warning?

TIA,
Chris