[Bug c++/52938] New: std::string::reserve request is not maintained if object is used in other object copy ctore

[Bug c++/52938] New: std::string::reserve request is not maintained if object is used in other object copy ctore

[abdul.tohmaz at emc dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

             Bug #: 52938
           Summary: std::string::reserve request is not maintained if
                    object is used in other object copy ctore
    Classification: Unclassified
           Product: gcc
           Version: 4.6.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: abdul.tohmaz@emc.com


In the line below I use 2 std::string (x and y) and I call x.reserve.  But I
found out that if I create string y from x (using copy ctor), the reserved
capacity is transferred to y and it is no longer associated with x.  


std::string x;
x.reserve(20);     /// x capacity is 20
std::cout<<x.capacity()<<' '<<(void*)(x.c_str())<<"\n";  
std::string y(x);
x="aaa";    /// x capacity should = 20 
std::cout<<x.capacity()<<' '<<(void*)(x.c_str())<<"\n";
x += "bbb";   /// x capacity should = 20 
std::cout<<x.capacity()<<' '<<(void*)(x.c_str())<<"\n";

compile and run:
20 0xf40028                                                                     
3 0xf40068                                                                      
6 0xf40098   

Notice how the pointer to string buffer is being reallocated (it shouldn't
since I have asked for x.reserve(20).

The code will behave correctly if I remove the line std::string y(x).

BTW, MS Visual c++ handles this correctly.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctore

[pinskia at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|c++                         |libstdc++

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> 2012-04-11 20:26:04 UTC ---
Why do you think this is a bug?  I don't see why x="aaa"; would not change the
reserve.  

The main reason why it changes the reserve is that the backing part is shared
between x and y.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctore

[abdul.tohmaz at emc dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #2 from Abdul Tohmaz <abdul.tohmaz at emc dot com> 2012-04-11 20:32:26 UTC ---
(In reply to comment #1)
> Why do you think this is a bug?  I don't see why x="aaa"; would not change the
> reserve.  
> 
> The main reason why it changes the reserve is that the backing part is shared
> between x and y.

I have called:
x.reserve(20); 
which I expect that 20 bytes (at least) to be reserved to this string.  So I
would expect no memory reallocation as long as my string can fit in my reserved
request.

Then I called x="aaa"; 
and then
x += "bbb";

Why do I see memory is being reallocated even though the string x content can
fit within what I requested in my reserve call?

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctore

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #3 from Jonathan Wakely <redi at gcc dot gnu.org> 2012-04-11 20:57:18 UTC ---
(In reply to comment #2)
> Why do I see memory is being reallocated even though the string x content can
> fit within what I requested in my reserve call?

Because GCC's std::string uses copy-on-write, so the original memory is shared
between x and y.  When x is modified that memory must be un-shared (i.e. make a
copy of the shared data before doing a write to x).  It's not possible to alter
y when assigning to x, so x must be altered.

I suppose it would be possible to ensure that when x copies the data it copies
the full capacity, but I  don't think we want to change std::string in that
way, so I suggest this should be closed WONTFIX.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctore

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #4 from Jonathan Wakely <redi at gcc dot gnu.org> 2012-04-11 21:08:16 UTC ---
N.B. you can get the behaviour you want by calling reserve after creating y

std::string x;
std::string y(x);
x.reserve(20);     /// x capacity is 20

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |WONTFIX

--- Comment #5 from Paolo Carlini <paolo.carlini at oracle dot com> 2012-04-11 22:33:44 UTC ---
Indeed, and definitely not the kind of issue we may even think handling within
our reference-counted implementation. Also note that <ext/vstring.h> already
behaves as wanted, thus the same will happen for our C++11 std::string.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[abdul.tohmaz at emc dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #6 from Abdul Tohmaz <abdul.tohmaz at emc dot com> 2012-04-11 23:50:21 UTC ---
Guys, I do value your points, but please consider the following:
I can't always be in control when a copy is made of my string.  Here is a quick
example.

string x;
x.reserve(1024); //no memory realloc for my string up to 1024
x="aaa";  
x+= "bb";  // so far no memory realloc.

//now I call a function by third party library whose
//signature is void doit(const string&).  
doit(x);

x+= "cc";  //I am expecting no memory realloc here.

//but in fact the last statement will reallocate memory if 
//doit put a copy of my string in a vector:  ex
//void doit (const std::string& val){ vec.push_back(val);}

Don't you think this is a problem?  I called reserve on x to avoid memory
reallocation, and yet I am faced with memory reallocation just because I call
doit; whose signature (const ref) suggests that my object should not be
changed.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[abdul.tohmaz at emc dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

Abdul Tohmaz <abdul.tohmaz at emc dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |UNCONFIRMED
         Resolution|WONTFIX                     |

--- Comment #7 from Abdul Tohmaz <abdul.tohmaz at emc dot com> 2012-04-11 23:53:04 UTC ---
Guys, I do value your points, but please consider the following:
I can't always be in control when a copy is made of my string.  Here is a quick
example.

string x;
x.reserve(1024); //no memory realloc for my string up to 1024
x="aaa";  
x+= "bb";  // so far no memory realloc.

//now I call a function by third party library whose
//signature is void doit(const string&).  
doit(x);

x+= "cc";  //I am expecting no memory realloc here.

//but in fact the last statement will reallocate memory if 
//doit put a copy of my string in a vector:  ex
//void doit (const std::string& val){ vec.push_back(val);}

Don't you think this is a problem?  I called reserve on x to avoid memory
reallocation, and yet I am faced with memory reallocation just because I call
doit; whose signature (const ref) suggests that my object should not be
changed.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #8 from Paolo Carlini <paolo.carlini at oracle dot com> 2012-04-11 23:57:34 UTC ---
What we tried to explain is that this sort of issue is well known and, more or
less, affects *any* reference counted implementation (I'm pretty sure you can
even find duplicates in Bugzilla or discussions in the mailing lists). That is
not the case when reference counting is not used and indeed it will *not* be
used (per the new C++11 Standard) in a new implementation of std::string which
we are currently showcasing as <ext/vstring.h>. Therefore, until a new "C++11"
ABI will become alive, and with it a new std::string, you can only work around
the issues, or experiment with <ext/vstring.h>, sorry.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[paolo.carlini at oracle dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

Paolo Carlini <paolo.carlini at oracle dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |WONTFIX

--- Comment #9 from Paolo Carlini <paolo.carlini at oracle dot com> 2012-04-11 23:58:21 UTC ---
.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[abdul.tohmaz at emc dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #10 from Abdul Tohmaz <abdul.tohmaz at emc dot com> 2012-04-12 00:11:34 UTC ---
Thanks for the clarification,
I will switch to ext/vstring.h...

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #11 from Jonathan Wakely <redi at gcc dot gnu.org> 2012-04-12 00:22:01 UTC ---
(In reply to comment #7)
> Don't you think this is a problem?  I called reserve on x to avoid memory
> reallocation, and yet I am faced with memory reallocation just because I call
> doit; whose signature (const ref) suggests that my object should not be
> changed.

If it didn't behave like that then you'd instead get an allocation when the
third party library copied the string.  You can't avoid the allocation
entirely, the difference is just where it happens, earlier or later.

You can always construct examples where it's apparently a problem, but I can
also show plenty of examples where that behaviour is beneficial, or even
required by the standard.

Consider:

  string s1("abc");
  s1.reserve(4);
  const string s2 = s1;
  const char* p = &s2[0];
  s1 += 'a';
  assert( p == &s2[0] );

For this to work with a reference-counted string the memory originally
allocated for s1 must still be pointed to by p, so appending to s1 must
allocate new memory.

The benefits and problems of reference-counted strings are well known and it's
not going to be changed now, std::string is stable and effectively frozen
except for serious bugs.  This isn't a bug, it's behaviour allowed by the C++
2003 standard, even if it's not the behaviour you expect or want.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[abdul.tohmaz at emc dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #12 from Abdul Tohmaz <abdul.tohmaz at emc dot com> 2012-04-12 23:07:32 UTC ---
(In reply to comment #11)

> The benefits and problems of reference-counted strings are well known and it's
> not going to be changed now, std::string is stable and effectively frozen
> except for serious bugs.  This isn't a bug, it's behaviour allowed by the C++
> 2003 standard, even if it's not the behavior you expect or want.

Now I understand the reason for choosing not to fix.  But I can't agree with
the statement saying this is not a bug.  Here is why.

The user could care less if we use copy-on-write in the string implementation. 
The standard doesn't force that (21.3.6).  

We are violating what the standard says about reserve (20.3.10/20.3.11).  Here
it is:
"Effects: after reserve(), capacity() is greater or equal to the argument of
reserve".

That is it.  if I call reserve(1024) on a string, from that point on, capacity 
should return at least 1024. 

As a user (with access to the standards), I don't see where it says that it is
ok to violate 21.3.10/20.3.11.  If you find it, please let me know.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[redi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #13 from Jonathan Wakely <redi at gcc dot gnu.org> 2012-04-13 08:31:13 UTC ---
> The user could care less if we use copy-on-write in the string implementation. 
> The standard doesn't force that (21.3.6).  

But it does allow it.

> We are violating what the standard says about reserve (20.3.10/20.3.11).  Here
> it is:
> "Effects: after reserve(), capacity() is greater or equal to the argument of
> reserve".
> 
> That is it.  if I call reserve(1024) on a string, from that point on, capacity 
> should return at least 1024. 

Immediately after you call reserve it returns at least 1024.  But not
necessarily from that point on for ever and ever.  If you call swap() to
exchange it with another string it's capacity could shrink, or in C++11 if you
move assign another string to it its capacity could change. Or, in C++03 for
reference-counted strings, it could change because the previously-shared string
is no longer shared.

This is a pointless discussion anyway, it's not going to change.

[Bug libstdc++/52938] std::string::reserve request is not maintained if object is used in other object copy ctor

[abdul.tohmaz at emc dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52938

--- Comment #14 from Abdul Tohmaz <abdul.tohmaz at emc dot com> 2012-04-13 13:37:47 UTC ---
(In reply to comment #13)

> Immediately after you call reserve it returns at least 1024.  But not
> necessarily from that point on for ever and ever.  If you call swap() to
> exchange it with another string it's capacity could shrink, or in C++11 if you
> move assign another string to it its capacity could change. Or, in C++03 for
> reference-counted strings, it could change because the previously-shared string
> is no longer shared.
> 
> This is a pointless discussion anyway, it's not going to change.

I agree with you on this discussion being pointless.
I am looking at this from the end user perspective and what the standards
dictates while you are looking at from the way gcc implements string.  Your
point about C++11 doesn't apply here (completely different ball game).  The
standard doesn't say anything about capacity for swap (21.3.5.8).  The user
expects the standard to be followed and when it says that capacity to be >= to
the reserve argument, he certainly expects that.  Unless you can show me
somewhere in the standards where it says the requested capacity via reserve
calls may be affected by copy constructor in the case an implementer chose to
use copy-on-write.  That is all I ask.