[Bug preprocessor/14438] New: -E of #pragma poison has negative line number

[Bug preprocessor/14438] New: -E of #pragma poison has negative line number

[lindsayd at cisco dot com]

A i686-pc-linux-gnu compiler built from 
  gcc-3.4-20040303.tar.bz2
exhibits this:

echo "#pragma GCC poison foo" > _dummy.c
gcc -E -nostdinc _dummy.c
  # 1 "_dummy.c"
  # 1 "<built-in>"
  # 1 "<command line>"
  # 1 "_dummy.c"       
  # 4294967218 "_dummy.c"

Umm, ah, line number 4294967218 ? This bug was brought to my attention by GAS,
when it said:
   line numbers must be positive; line number -15 rejected

The bug is in GCC-3.3.2 but isn't in 3.3.1.

-- 
           Summary: -E of #pragma poison has negative line number
           Product: gcc
           Version: 3.4.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: preprocessor
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: lindsayd at cisco dot com
                CC: gcc-bugs at gcc dot gnu dot org
  GCC host triplet: i686-pc-linux-gnu
GCC target triplet: native


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.4/3.5 Regression] -E of #pragma poison has negative line number

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2004-03-05 08:20 -------
Confirmed, a regression from 3.3.1

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|                            |1
      Known to fail|                            |3.4.0 3.5.0
      Known to work|                            |3.3.1
   Last reconfirmed|0000-00-00 00:00:00         |2004-03-05 08:20:00
               date|                            |
            Summary|-E of #pragma poison has    |[3.4/3.5 Regression] -E of
                   |negative line number        |#pragma poison has negative
                   |                            |line number
   Target Milestone|---                         |3.4.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[reichelt at gcc dot gnu dot org]

------- Additional Comments From reichelt at gcc dot gnu dot org  2004-03-10 16:22 -------
This just bit me, too.

Alexandre, I strongly suspect your patches

http://gcc.gnu.org/ml/gcc-cvs/2003-09/msg00418.html
http://gcc.gnu.org/ml/gcc-cvs/2003-09/msg00715.html

to be responsible for this. Could you please have a look?


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |aoliva at redhat dot com,
                   |                            |reichelt at gcc dot gnu dot
                   |                            |org
      Known to fail|3.4.0 3.5.0                 |3.3.2 3.3.3 3.4.0 3.5.0
            Summary|[3.4/3.5 Regression] -E of  |[3.3/3.4/3.5 Regression] -E
                   |#pragma poison has negative |of #pragma has negative line
                   |line number                 |number
   Target Milestone|3.4.0                       |3.3.4


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[aoliva at gcc dot gnu dot org]

------- Additional Comments From aoliva at gcc dot gnu dot org  2004-03-11 06:55 -------
You are right, it's the second cb_line_change, that I added in the second patch,
that is the culprit.  It doesn't seem to always (or ever?) have accurate line
number information in pfile->cur_token after we handle pragma poison.  I've
taken it out in my local tree and the problem went away.  Would you be willing
to give the patch a round of testing, perhaps even for the next release out of
the 3.3 branch, and post the patch for review by the cpp maintainers?  If not,
let me know, and I'll try to get to it.

Thanks for bring the problem to my attention, and sorry about the lossage.



-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
   Last reconfirmed|2004-03-05 08:20:00         |2004-03-11 06:55:05
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[reichelt at gcc dot gnu dot org]

------- Additional Comments From reichelt at gcc dot gnu dot org  2004-03-20 21:17 -------
Alexandre, I tested your suggestion on 3.3 branch, 3.4 branch and mainline.

It cured the problem of the submitter with
  #pragma GCC poison foo

It also cured the problem for standard header files that contain
  #pragma GCC system_header
For example, preprocessing the C++ header
  #include<list>
without your patch resulted in

  # 1 "PR14438A.cc"
  # 1 "<built-in>"
  # 1 "<command line>"
  # 1 "PR14438A.cc"
  # 1
"/mygcc/bin/../lib/gcc/i686-pc-linux-gnu/3.4.0/../../../../include/c++/3.4.0/list"
1 3
  # 64
"/mygcc/bin/../lib/gcc/i686-pc-linux-gnu/3.4.0/../../../../include/c++/3.4.0/list"
3

  # 65
"/mygcc/bin/../lib/gcc/i686-pc-linux-gnu/3.4.0/../../../../include/c++/3.4.0/list"
3
  # 4294967211
"/mygcc/bin/../lib/gcc/i686-pc-linux-gnu/3.4.0/../../../../include/c++/3.4.0/list"
3
    ^^^^^^^^^^

Alas, I couldn't do regression testing, since my DejaGnu seems to be out
of date. Maybe it's some different problem, but right know I don't have
the time to hunt the problem down.

Since it's your patch, anyway, could you please do that and get it reviewed?
(I'll attach the patches to the PR, one for the 3.3 and 3.4 branch and one
for mainline - which differs only in whitespace.)

Btw, I don't have the slightest idea to how to prepare a test for this kind
of problem for the testsuite. Any ideas?


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |patch


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[zack at gcc dot gnu dot org]

------- Additional Comments From zack at gcc dot gnu dot org  2004-03-21 20:16 -------
I'd like to know why pfile->cur_token is garbage at this point; my suspicion is
that this should be fixed there instead.  The existing patch would be OK for
3.3/4, though, if it's too hard to fix this properly.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[mmitchel at gcc dot gnu dot org]

------- Additional Comments From mmitchel at gcc dot gnu dot org  2004-03-21 20:18 -------
Based on Zack's comments, would you please apply this patch to 3.4 *only*.

Leave the bug open for mainline.

Alexandre, would you please follow up on that issue so that we can resolve this
fully for 3.5?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[gdr at integrable-solutions dot net]

------- Additional Comments From gdr at integrable-solutions dot net  2004-03-21 22:11 -------
Subject: Re:  [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

"mmitchel at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org> writes:

| Based on Zack's comments, would you please apply this patch to 3.4 *only*.

Apply it to gcc-3_3-branch too.

| Leave the bug open for mainline.

Agreed.

-- Gaby


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[aoliva at gcc dot gnu dot org]

------- Additional Comments From aoliva at gcc dot gnu dot org  2004-03-23 00:45 -------
Err...   Is it really a problem in 3.4?  I didn't think so.  I understood it was
only broken in 3.3.  I believe 3.4 is already fixed in that it didn't have
garbage there at that point.  I'll check whether the patch is necessary for 3.4
and, if so, check it in, and then proceed to regression-testing the patch for
3.3.  I'm pretty sure it's not an issue in mainline, but I'll verify as well.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[lindsayd at cisco dot com]

------- Additional Comments From lindsayd at cisco dot com  2004-03-23 00:55 -------
Subject: Re:  [3.3/3.4/3.5 Regression] -E of #pragma
 has negative line number

> Err...   Is it really a problem in 3.4? 

The bug was first reported in a 3.4 snapshot.

---
Don




-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[aoliva at gcc dot gnu dot org]

------- Additional Comments From aoliva at gcc dot gnu dot org  2004-03-23 01:13 -------
So it is a problem in 3.4 and mainline as well.  Sorry, I had misread the bug
report.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[cvs-commit at gcc dot gnu dot org]

------- Additional Comments From cvs-commit at gcc dot gnu dot org  2004-03-24 03:18 -------
Subject: Bug 14438

CVSROOT:	/cvs/gcc
Module name:	gcc
Branch: 	gcc-3_3-branch
Changes by:	aoliva@gcc.gnu.org	2004-03-24 03:18:44

Modified files:
	gcc            : ChangeLog cpplib.c 

Log message:
	PR preprocessor/14438
	* cpplib.c (do_pragma): Remove line_change call after pragma
	handler.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_3-branch&r1=1.16114.2.950&r2=1.16114.2.951
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cpplib.c.diff?cvsroot=gcc&only_with_tag=gcc-3_3-branch&r1=1.326.2.3&r2=1.326.2.4



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[cvs-commit at gcc dot gnu dot org]

------- Additional Comments From cvs-commit at gcc dot gnu dot org  2004-03-24 03:19 -------
Subject: Bug 14438

CVSROOT:	/cvs/gcc
Module name:	gcc
Branch: 	gcc-3_4-branch
Changes by:	aoliva@gcc.gnu.org	2004-03-24 03:19:02

Modified files:
	gcc            : ChangeLog cpplib.c 

Log message:
	PR preprocessor/14438
	* cpplib.c (do_pragma): Remove line_change call after pragma
	handler.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=2.2326.2.367&r2=2.2326.2.368
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cpplib.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.355.4.1&r2=1.355.4.2



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.3/3.4/3.5 Regression] -E of #pragma has negative line number

[cvs-commit at gcc dot gnu dot org]

------- Additional Comments From cvs-commit at gcc dot gnu dot org  2004-03-24 03:19 -------
Subject: Bug 14438

CVSROOT:	/cvs/gcc
Module name:	gcc
Changes by:	aoliva@gcc.gnu.org	2004-03-24 03:19:38

Modified files:
	gcc            : ChangeLog cpplib.c 

Log message:
	PR preprocessor/14438
	* cpplib.c (do_pragma): Remove line_change call after pragma
	handler.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&r1=2.3273&r2=2.3274
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cpplib.c.diff?cvsroot=gcc&r1=1.361&r2=1.362



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.5, needs investigation] -E of #pragma has negative line number

[aoliva at gcc dot gnu dot org]

------- Additional Comments From aoliva at gcc dot gnu dot org  2004-03-24 03:27 -------
Regression fixed, still need to investigate, per Zack's request.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to fail|3.3.2 3.3.3 3.4.0 3.5.0     |3.3.2 3.3.3
            Summary|[3.3/3.4/3.5 Regression] -E |[3.5, needs investigation] -
                   |of #pragma has negative line|E of #pragma has negative
                   |number                      |line number


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.5, needs investigation] -E of #pragma has negative line number

[aoliva at redhat dot com]

------- Additional Comments From aoliva at redhat dot com  2004-03-24 04:31 -------
Subject: Re: preprocessor/14438

The reason why the patch had introduced a regression was that
pfile->cur_token pointed to uninitialized memory, one past the last
initialized token.  _cpp_lex_direct() doesn't skip to the next line
because we're processing a directive, so it just sets result->type to
CPP_EOF, along with its correct location, after incrementing
pfile->cur_token past result, without any form of initialization to
this value.  In fact, I'm a bit concerned as to what might happen if
we were to call _cpp_lex_direct() an unlimited number of times while
processing a directive; I don't see anything that would prevent
cur_token from running past the memory area reserved for it.

I suppose one way to fix this issue would be to decrement cur_token
again when returning in the CPP_EOF case, such that we'd return the
same EOF token over and over; another would be to just refrain from
dereferencing cur_token after reading an EOF token, but how would we
tell?  And, more importantly, why would we want to?  Clearly we don't
need it in the case I thought we did when I added the code I just
removed; I don't think we need it elsewhere.

Is this analysis enough for us to close this bug report now?



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.5, needs investigation] -E of #pragma has negative line number

[neroden at gcc dot gnu dot org]

------- Additional Comments From neroden at gcc dot gnu dot org  2004-03-24 04:35 -------
>I suppose one way to fix this issue would be to decrement cur_token 
>again when returning in the CPP_EOF case, such that we'd return the 
>same EOF token over and over; 
Actually, that sounds like a fairly robust thing to do -- in _cpp_lex_direct, 
perhaps? 
 

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.5, needs investigation] -E of #pragma has negative line number

[zack at codesourcery dot com]

------- Additional Comments From zack at codesourcery dot com  2004-03-24 04:58 -------
Subject: Re: preprocessor/14438

Alexandre Oliva <aoliva@redhat.com> writes:

> The reason why the patch had introduced a regression was that
> pfile->cur_token pointed to uninitialized memory, one past the last
> initialized token.  _cpp_lex_direct() doesn't skip to the next line
> because we're processing a directive, so it just sets result->type to
> CPP_EOF, along with its correct location, after incrementing
> pfile->cur_token past result, without any form of initialization to
> this value.  In fact, I'm a bit concerned as to what might happen if
> we were to call _cpp_lex_direct() an unlimited number of times while
> processing a directive; I don't see anything that would prevent
> cur_token from running past the memory area reserved for it.

Oh, ugh.

> I suppose one way to fix this issue would be to decrement cur_token
> again when returning in the CPP_EOF case, such that we'd return the
> same EOF token over and over; another would be to just refrain from
> dereferencing cur_token after reading an EOF token, but how would we
> tell?  And, more importantly, why would we want to?  Clearly we don't
> need it in the case I thought we did when I added the code I just
> removed; I don't think we need it elsewhere.

Suggest you add defensive code to prevent running cur_token off the
end of the buffer, as you suggest, and then go ahead and close the bug
report.

zw


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] [3.5, needs investigation] -E of #pragma has negative line number

[aoliva at redhat dot com]

------- Additional Comments From aoliva at redhat dot com  2004-03-30 21:59 -------
Subject: Re: preprocessor/14438

On Mar 24, 2004, Zack Weinberg <zack@codesourcery.com> wrote:

> Alexandre Oliva <aoliva@redhat.com> writes:
>> In fact, I'm a bit concerned as to what might happen if
>> we were to call _cpp_lex_direct() an unlimited number of times while
>> processing a directive; I don't see anything that would prevent
>> cur_token from running past the memory area reserved for it.

> Oh, ugh.

> Suggest you add defensive code to prevent running cur_token off the
> end of the buffer, as you suggest, and then go ahead and close the bug
> report.

It turns out that the most common caller of _cpp_lex_direct(),
cpp_lex_token(), already takes care of that, and other callers seem to
take care of buffer management already, except perhaps for the code
that reads the initial file and directory name, but those should be
the first few tokens, therefore they should fall within the initially
allocated buffer and never fail.  I'm a bit wary of introducing
overhead in _cpp_lex_direct(), since it's quite performance critical,
and it appears that we're safe, so I'm leaning towards leaving it as
is.  If you agree, please go ahead and mark the bug as closed.
Otherwise, I suggest that you, being more familiar with the cpp code,
take it over and refactor it a big to make it safer without
introducing unnecessary overhead in the common paths.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] Potential need for buffer overflow checks in _cpp_lex_direct

[zack at gcc dot gnu dot org]

------- Additional Comments From zack at gcc dot gnu dot org  2004-03-31 00:07 -------
I will need to study the code a bit to figure out whether checking in
_cpp_lex_direct is appropriate or not.  Can't say when I will get to this,
but given what you say it is not critical.  Reassigning to myself, retargeting
for 3.5.0, updating title and priority.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|aoliva at gcc dot gnu dot   |zack at gcc dot gnu dot org
                   |org                         |
           Keywords|patch                       |
      Known to fail|3.3.2 3.3.3                 |
      Known to work|3.3.1                       |
           Priority|P2                          |P3
            Summary|[3.5, needs investigation] -|Potential need for buffer
                   |E of #pragma has negative   |overflow checks in
                   |line number                 |_cpp_lex_direct
   Target Milestone|3.3.4                       |3.5.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] Potential need for buffer overflow checks in _cpp_lex_direct

[zack at gcc dot gnu dot org]

------- Additional Comments From zack at gcc dot gnu dot org  2004-03-31 00:08 -------
(From update of attachment 5950)
obsoleting patches which were checked in and do not address the current ambit
of this bug report.


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
Attachment #5950 is|0                           |1
           obsolete|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] Potential need for buffer overflow checks in _cpp_lex_direct

[zack at gcc dot gnu dot org]

------- Additional Comments From zack at gcc dot gnu dot org  2004-03-31 00:09 -------
(From update of attachment 5951)
obsoleting patches which were checked in and do not address the current ambit
of this bug report.


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
Attachment #5951 is|0                           |1
           obsolete|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] Potential need for buffer overflow checks in _cpp_lex_direct

[mmitchel at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.0.0                       |4.0.1


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] Potential need for buffer overflow checks in _cpp_lex_direct

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2005-07-05 02:14 -------
Unassigning from Zack since he is now gone from GCC development.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|zack at codesourcery dot com|unassigned at gcc dot gnu
                   |                            |dot org
             Status|ASSIGNED                    |NEW


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] Potential need for buffer overflow checks in _cpp_lex_direct

[mmitchel at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.0.1                       |4.0.2


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438

[Bug preprocessor/14438] Potential need for buffer overflow checks in _cpp_lex_direct

[pinskia at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.0.2                       |---


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14438