public inbox for gcc-bugs@sourceware.org help / color / mirror / Atom feed
* [Bug libstdc++/42857] New: std::istream::ignore(std::streamsize n) calls unnecessary underflow @ 2010-01-24 19:46 tommi at tntnet dot org 2010-01-27 21:10 ` [Bug libstdc++/42857] " paolo dot carlini at oracle dot com 2010-09-20 12:15 ` paolo dot carlini at oracle dot com 0 siblings, 2 replies; 11+ messages in thread From: tommi at tntnet dot org @ 2010-01-24 19:46 UTC (permalink / raw) To: gcc-bugs When ignoring the number of bytes available in a streambuf using std::istream::ignore, ignore calls underflow. This should not happen. I feel the easiest way to reproduce it is to look at strace output when ignoring bytes from ifstream: #include <iostream> #include <fstream> int main(int argc, char* argv[]) { std::ifstream in(argv[0]); in.get(); // trigger filling of input buffer by calling underflow in.ignore(in.rdbuf()->in_avail()); // this triggers another underflow } The strace output shows, that 2 calls to read with 8191 bytes occures. When ignoring one byte less and then another byte, only one read is done: #include <iostream> #include <fstream> int main(int argc, char* argv[]) { std::ifstream in(argv[0]); in.get(); // trigger filling of input buffer in.ignore(in.rdbuf()->in_avail() - 1); in.ignore(1); // this does not trigger underflow } -- Summary: std::istream::ignore(std::streamsize n) calls unnecessary underflow Product: gcc Version: 4.4.1 Status: UNCONFIRMED Severity: normal Priority: P3 Component: libstdc++ AssignedTo: unassigned at gcc dot gnu dot org ReportedBy: tommi at tntnet dot org GCC target triplet: i486-linux-gnu http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow 2010-01-24 19:46 [Bug libstdc++/42857] New: std::istream::ignore(std::streamsize n) calls unnecessary underflow tommi at tntnet dot org @ 2010-01-27 21:10 ` paolo dot carlini at oracle dot com 2010-09-20 12:15 ` paolo dot carlini at oracle dot com 1 sibling, 0 replies; 11+ messages in thread From: paolo dot carlini at oracle dot com @ 2010-01-27 21:10 UTC (permalink / raw) To: gcc-bugs ------- Comment #1 from paolo dot carlini at oracle dot com 2010-01-27 21:09 ------- Putting aside the strange inconsistency of the second example, which could be easily fixed, and probably should anyway (we have an overload corresponding to n == 1 which calls sbumpc and should probably call snextc instead for a consistent behavior), I think the issue boils down to a very old interpretation issue in these sections of the standard, where it uses wording of the form "any of the following occurs" (the same happens for get, getline, the issue isn't limited to ignore) and it's not clear at all whether, when the target n is reached, thus n characters are extracted, the second termination condition, which involves checking for end-of-file, is still relevant or not, or, in principle could even have been computed first, before checking the value of n. In short, I understand your request as a request of evaluating the conditions exactly in the order written in the Standard, and as soon as one holds, the following ones simply ignored. It's not what we have been doing, uniformly (modulo the buglet above) across this area, but indeed makes sense. Note however that a rather noticeable consequence of your interpretation is that ignoring exactly to end-of-line would not set the eofbit anymore in the stream, because we would not do getc on it. Now instead we uniformly set eofbit when it's actually the case (and setting it could even throw, a very noticeable behavior) Thus, I believe that, besides the buglet above, we should give this issue much more thought, likely it's even too late for 4.5.0, and I'd like to involve Nathan in the discussion, I'm pretty sure I learned from him, some time ago, about the tricky point of the "any of the following occurs"-type specifications. -- paolo dot carlini at oracle dot com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ncm at cantrip dot org http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow 2010-01-24 19:46 [Bug libstdc++/42857] New: std::istream::ignore(std::streamsize n) calls unnecessary underflow tommi at tntnet dot org 2010-01-27 21:10 ` [Bug libstdc++/42857] " paolo dot carlini at oracle dot com @ 2010-09-20 12:15 ` paolo dot carlini at oracle dot com 1 sibling, 0 replies; 11+ messages in thread From: paolo dot carlini at oracle dot com @ 2010-09-20 12:15 UTC (permalink / raw) To: gcc-bugs ------- Comment #2 from paolo dot carlini at oracle dot com 2010-09-20 12:15 ------- I was having a second look to this issue, and noticed something more which I missed the first time: the Standard, *only* in the case of getline(char_type*, streamsize, char_type) explicitly says "These conditions are tested in the order shown.". In my opinion that means that the get and ignore overloads using the famous "any of the following occurs" can in principle check the conditions in *any* implementation defined order and being conforming. Now, we have a case here where we have an additional underflow because in our implementation we uniformly insist on always checking whether end-of-file occurs in the sequence, thus setting eofbit (besides the special case of ignore(), as already noticed) in that case, like the above mentioned getline does, for example. I want to understand how critical this additional underflow is, performance-wise, which, as far as I can see normally can be triggered only by passing in_avail to ignore, because otherwise, frankly, I find our consistent implementation defined behavior across the various get, getline, ignore overloads pretty nice. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 ^ permalink raw reply [flat|nested] 11+ messages in thread
[parent not found: <bug-42857-4@http.gcc.gnu.org/bugzilla/>]
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow [not found] <bug-42857-4@http.gcc.gnu.org/bugzilla/> @ 2011-12-22 0:56 ` paolo.carlini at oracle dot com 2013-02-24 22:05 ` bugs at mm dot beanwood.com ` (6 subsequent siblings) 7 siblings, 0 replies; 11+ messages in thread From: paolo.carlini at oracle dot com @ 2011-12-22 0:56 UTC (permalink / raw) To: gcc-bugs http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 Paolo Carlini <paolo.carlini at oracle dot com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |claytongdavis at gmail dot | |com --- Comment #3 from Paolo Carlini <paolo.carlini at oracle dot com> 2011-12-22 00:55:04 UTC --- *** Bug 51651 has been marked as a duplicate of this bug. *** ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow [not found] <bug-42857-4@http.gcc.gnu.org/bugzilla/> 2011-12-22 0:56 ` paolo.carlini at oracle dot com @ 2013-02-24 22:05 ` bugs at mm dot beanwood.com 2014-04-18 16:19 ` cubbi at cubbi dot org ` (5 subsequent siblings) 7 siblings, 0 replies; 11+ messages in thread From: bugs at mm dot beanwood.com @ 2013-02-24 22:05 UTC (permalink / raw) To: gcc-bugs http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 Andrew Ayer <bugs at mm dot beanwood.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bugs at mm dot beanwood.com --- Comment #4 from Andrew Ayer <bugs at mm dot beanwood.com> 2013-02-24 22:05:26 UTC --- (In reply to comment #2) > I want to understand how critical this additional underflow is, > performance-wise There are also correctness implications: what if you're trying to ignore all the bytes you know are available to be read without blocking? Then the extra underflow blocks, possibly forever if the program is prevented from taking action that would result in the underflow completing. I got bitten by this because I was trying to ignore data from a socket. Note that the standard has similar language for read(char_type*, streamsize) ("either of the following conditions") but the library in effect checks for n characters being reached before checking for eof: if you read() exactly the number of characters left in the stream, eofbit is not set. So the library isn't currently consistent across all the various istream functions. I would argue that the current behavior of ignore() is actually contrary to the standard. If the standard says "until any" then ignore() needs to terminate when n characters have been extracted. But if it first checks for eof and blocks forever, then it doesn't terminate. Perhaps this is why getline explicitly mentions the order: in that case the standard really does want eof to be checked first. ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow [not found] <bug-42857-4@http.gcc.gnu.org/bugzilla/> 2011-12-22 0:56 ` paolo.carlini at oracle dot com 2013-02-24 22:05 ` bugs at mm dot beanwood.com @ 2014-04-18 16:19 ` cubbi at cubbi dot org 2014-07-23 21:47 ` bugs at mm dot beanwood.com ` (4 subsequent siblings) 7 siblings, 0 replies; 11+ messages in thread From: cubbi at cubbi dot org @ 2014-04-18 16:19 UTC (permalink / raw) To: gcc-bugs http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 Sergey Zubkov <cubbi at cubbi dot org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cubbi at cubbi dot org --- Comment #5 from Sergey Zubkov <cubbi at cubbi dot org> --- Looking at existing implementations, libstdc++ is the odd one out: LLVM libc++, IBM XL C++, Microsoft Visual Studio, and Oracle libCstd and stlport4 all let me ignore exactly the number of characters requested, without an extra read from the socket. ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow [not found] <bug-42857-4@http.gcc.gnu.org/bugzilla/> ` (2 preceding siblings ...) 2014-04-18 16:19 ` cubbi at cubbi dot org @ 2014-07-23 21:47 ` bugs at mm dot beanwood.com 2020-11-10 23:46 ` ncm at cantrip dot org ` (3 subsequent siblings) 7 siblings, 0 replies; 11+ messages in thread From: bugs at mm dot beanwood.com @ 2014-07-23 21:47 UTC (permalink / raw) To: gcc-bugs https://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 --- Comment #6 from Andrew Ayer <bugs at mm dot beanwood.com> --- Any word on if this will be fixed in GCC? To summarize, GCC's current behavior is wrong because: * Underflowing after ignoring the requested number of bytes could block forever, breaking applications. * The standard says "characters are extracted until ANY of the following occurs: ... n characters are extracted" (emphasis mine). If ignore() blocks forever after extracting n characters, it's a violation of the standard because it doesn't terminate as is required. * GCC's std::basic_istream::read() implementation does NOT underflow after reading the requested number of bytes (even though the standard uses similar language for read and ignore). This inconsistent behavior is confusing as one would expect read and ignore to behave equivalently. * Other major C++ implementations don't underflow after ignoring the requested number of bytes. (Thanks, Sergey, for checking this.) I wrote about this in greater depth at [1] and also wrote my own non-member ignore() which I'm using in place of basic_istream::ignore() [2] which others may find useful until this bug is fixed. [1] https://www.agwa.name/blog/post/gccs_implementation_of_basicistreamignore_is_broken [2] https://www.agwa.name/blog/post/gccs_implementation_of_basicistreamignore_is_broken/media/ignore.h ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow [not found] <bug-42857-4@http.gcc.gnu.org/bugzilla/> ` (3 preceding siblings ...) 2014-07-23 21:47 ` bugs at mm dot beanwood.com @ 2020-11-10 23:46 ` ncm at cantrip dot org 2020-11-11 0:01 ` redi at gcc dot gnu.org ` (2 subsequent siblings) 7 siblings, 0 replies; 11+ messages in thread From: ncm at cantrip dot org @ 2020-11-10 23:46 UTC (permalink / raw) To: gcc-bugs https://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 --- Comment #7 from ncm at cantrip dot org --- This bug appears not to manifest in g++-10. ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow [not found] <bug-42857-4@http.gcc.gnu.org/bugzilla/> ` (4 preceding siblings ...) 2020-11-10 23:46 ` ncm at cantrip dot org @ 2020-11-11 0:01 ` redi at gcc dot gnu.org 2020-11-11 0:06 ` ncm at cantrip dot org 2020-11-11 0:24 ` redi at gcc dot gnu.org 7 siblings, 0 replies; 11+ messages in thread From: redi at gcc dot gnu.org @ 2020-11-11 0:01 UTC (permalink / raw) To: gcc-bugs https://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 --- Comment #8 from Jonathan Wakely <redi at gcc dot gnu.org> --- Probably changed by one of the patches for PR 94749 or PR 96161, although I still see two reads for the first example. ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow [not found] <bug-42857-4@http.gcc.gnu.org/bugzilla/> ` (5 preceding siblings ...) 2020-11-11 0:01 ` redi at gcc dot gnu.org @ 2020-11-11 0:06 ` ncm at cantrip dot org 2020-11-11 0:24 ` redi at gcc dot gnu.org 7 siblings, 0 replies; 11+ messages in thread From: ncm at cantrip dot org @ 2020-11-11 0:06 UTC (permalink / raw) To: gcc-bugs https://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 --- Comment #9 from ncm at cantrip dot org --- (In reply to Jonathan Wakely from comment #8) > Probably changed by one of the patches for PR 94749 or PR 96161, although I > still see two reads for the first example. Thank you, I was mistaken. This bug is still present in g++-10. ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Bug libstdc++/42857] std::istream::ignore(std::streamsize n) calls unnecessary underflow [not found] <bug-42857-4@http.gcc.gnu.org/bugzilla/> ` (6 preceding siblings ...) 2020-11-11 0:06 ` ncm at cantrip dot org @ 2020-11-11 0:24 ` redi at gcc dot gnu.org 7 siblings, 0 replies; 11+ messages in thread From: redi at gcc dot gnu.org @ 2020-11-11 0:24 UTC (permalink / raw) To: gcc-bugs https://gcc.gnu.org/bugzilla/show_bug.cgi?id=42857 --- Comment #10 from Jonathan Wakely <redi at gcc dot gnu.org> --- Untested patch: --- a/libstdc++-v3/src/c++98/compatibility.cc +++ b/libstdc++-v3/src/c++98/compatibility.cc @@ -88,7 +88,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION { __sb->__safe_gbump(__size); _M_gcount += __size; - __c = __sb->sgetc(); + if (_M_gcount < __n + || __n == __gnu_cxx::__numeric_traits<streamsize>::__max) + __c = __sb->sgetc(); } else { A similar change would be needed in the other versions of this function (the wistream specialization, and the generic version). ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2020-11-11 0:24 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-01-24 19:46 [Bug libstdc++/42857] New: std::istream::ignore(std::streamsize n) calls unnecessary underflow tommi at tntnet dot org 2010-01-27 21:10 ` [Bug libstdc++/42857] " paolo dot carlini at oracle dot com 2010-09-20 12:15 ` paolo dot carlini at oracle dot com [not found] <bug-42857-4@http.gcc.gnu.org/bugzilla/> 2011-12-22 0:56 ` paolo.carlini at oracle dot com 2013-02-24 22:05 ` bugs at mm dot beanwood.com 2014-04-18 16:19 ` cubbi at cubbi dot org 2014-07-23 21:47 ` bugs at mm dot beanwood.com 2020-11-10 23:46 ` ncm at cantrip dot org 2020-11-11 0:01 ` redi at gcc dot gnu.org 2020-11-11 0:06 ` ncm at cantrip dot org 2020-11-11 0:24 ` redi at gcc dot gnu.org
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).