[Bug libstdc++/56158] New: bad enum values computed by operator~ in ios_base.h

[Bug libstdc++/56158] New: bad enum values computed by operator~ in ios_base.h

[richard-gccbugzilla at metafoo dot co.uk]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

             Bug #: 56158
           Summary: bad enum values computed by operator~ in ios_base.h
    Classification: Unclassified
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libstdc++
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: richard-gccbugzilla@metafoo.co.uk


The overloaded operator~s defined for the enumerations in ios_base.h have the
following form:

  Enum operator~(Enum e) { return Enum(~static_cast<int>(e)); }

The ~ creates values outside the range of values of the enumeration type, so
the cast back to the Enum type has an unspecified value (see
[expr.static.cast]p10), and in practice it produces an Enum value outside the
range of representable values for the Enum type, so behavior is undefined.

Fix:

--- include/bits/ios_base.h
+++ include/bits/ios_base.h
@@ -87,7 +87,7 @@

   inline _GLIBCXX_CONSTEXPR _Ios_Fmtflags
   operator~(_Ios_Fmtflags __a)
-  { return _Ios_Fmtflags(~static_cast<int>(__a)); }
+  { return _Ios_Fmtflags(static_cast<int>(__a) ^
static_cast<int>(_S_ios_fmtflags_end - 1)); }

   inline const _Ios_Fmtflags&
   operator|=(_Ios_Fmtflags& __a, _Ios_Fmtflags __b)
@@ -127,7 +127,7 @@

   inline _GLIBCXX_CONSTEXPR _Ios_Openmode
   operator~(_Ios_Openmode __a)
-  { return _Ios_Openmode(~static_cast<int>(__a)); }
+  { return _Ios_Openmode(static_cast<int>(__a) ^
static_cast<int>(_S_ios_openmode_end - 1)); }

   inline const _Ios_Openmode&
   operator|=(_Ios_Openmode& __a, _Ios_Openmode __b)
@@ -165,7 +165,7 @@

   inline _GLIBCXX_CONSTEXPR _Ios_Iostate
   operator~(_Ios_Iostate __a)
-  { return _Ios_Iostate(~static_cast<int>(__a)); }
+  { return _Ios_Iostate(static_cast<int>(__a) ^
static_cast<int>(_S_ios_iostate_end - 1)); }

   inline const _Ios_Iostate&
   operator|=(_Ios_Iostate& __a, _Ios_Iostate __b)

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2013-01-31
     Ever Confirmed|0                           |1

--- Comment #1 from Jonathan Wakely <redi at gcc dot gnu.org> 2013-01-31 10:04:39 UTC ---
Thanks, Richard

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bkoz at redhat dot com

--- Comment #2 from Paolo Carlini <paolo.carlini at oracle dot com> 2013-01-31 10:08:33 UTC ---
Crazy nobody noticed so far. So... Is this something we can commit right now?
Assuming there aren't ABI implications, I would be in favor. I'm also adding
Benjamin in CC, I think I wasn't contributing yet, when this code went in ;)

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |redi at gcc dot gnu.org

--- Comment #3 from Paolo Carlini <paolo.carlini at oracle dot com> 2013-02-06 09:50:50 UTC ---
I'm going to regression test the fix.

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

--- Comment #4 from Paolo Carlini <paolo.carlini at oracle dot com> 2013-02-06 15:29:38 UTC ---
I'm wondering: before doing anything in v3, is this a C++11 issue? Because in
17.5.2.1.3 I see a fixed underlying type but otherwise I see exactly
~static_cast<int_type>(X) like in v3?!?

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

--- Comment #5 from Jonathan Wakely <redi at gcc dot gnu.org> 2013-02-06 15:59:51 UTC ---
[dcl.enum]/7 "For an enumeration whose underlying type is fixed, the values of
the enumeration are the values of the underlying type."

Because the underlying type in 17.5.2.1.3 is fixed those operations cannot
create a value outside the range of the enumeration type. See "The underlying
type should be fixed" in
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3110.html

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

--- Comment #6 from Paolo Carlini <paolo.carlini at oracle dot com> 2013-02-06 16:40:09 UTC ---
Oh, I was missing that, thanks. Now, I don't know if we should really try to
fix this now after so many years. I'm tempted to just leave it alone until we
break the ABI, unless we are really sure that the value returned by the amended
operator can intetoperate with old code and viceversa. Do you jnow in practice
which values the current operator is returning for GCC?

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

--- Comment #7 from Paolo Carlini <paolo.carlini at oracle dot com> 2013-02-07 11:22:32 UTC ---
We should double check but I'm pretty sure that *in practice* *for GCC* things
are Ok, because the sizeof of these enums is 4 (and in practice the systems we
support have sizeof int <= 4).

If we don't have a concrete testcase, we definitely want to change this later.

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

--- Comment #8 from Jonathan Wakely <redi at gcc dot gnu.org> 2013-02-07 11:39:31 UTC ---
I think since 4.6 the default behaviour (i.e. without -fstrict-enums) is safe.

With -fstrict-enums (or in releases before 4.6) the optimisers can assume that
no invalid values are ever produced, so Enum(~static_cast<int>(e)) has
undefined behaviour as Richard says.

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

--- Comment #9 from Paolo Carlini <paolo.carlini at oracle dot com> 2013-02-07 11:50:03 UTC ---
Sure, sure. If we really want to support -fstrict-enums, I'm afraid we are
going to open a big can of worms... Still, are you sure it causes problems
*here*? I'm asking because we have the final 1L << 16.

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[trippels at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

Markus Trippelsdorf <trippels at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |doko at gcc dot gnu.org

--- Comment #10 from Markus Trippelsdorf <trippels at gcc dot gnu.org> ---
*** Bug 66624 has been marked as a duplicate of this bug. ***

[Bug libstdc++/56158] bad enum values computed by operator~ in ios_base.h

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=56158

--- Comment #11 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Now that sanitisers are complaining about this we should really fix it.