[Bug middle-end/18785] New: isdigit builtin function fails with EBCDIC character sets

[Bug middle-end/18785] New: isdigit builtin function fails with EBCDIC character sets

[darcypj at us dot ibm dot com]

Configured with: /cteam/darcypj/200410bld/hbtree/configure
--target=s390x-ibm-tpf --prefix=/opt/tpf-cross-tools2
--with-gcc-version-trigger=/cteam/darcypj/200410bld/hbtree/gcc/version.c
--program-prefix=s390x-ibm-tpf-
--with-headers=/cteam/darcypj/200410bld/tpfheaders/include
--build=s390x-ibm-linux --without-newlib --enable-languages=c,c++
--with-libs=/opt/tpf-cross-tools2/lib --with-lib-path=/opt/tpf-cross-tools2/lib
--host=s390x-ibm-linux
Thread model: tpf
gcc version 4.0.0 20041107 (experimental)

Compilation command line:
s390x-ibm-tpf-gcc -c digit.c -fexec-charset=IBM1047 -fverbose-asm -gdwarf-2
-Wa,-aldhs=listing.lst --save-temps -Wall

No compiler warnings or errors emitted.

The isdigit builtin function is using ASCII '0' for the comparison to the
character passed to isdigit.  This causes a failure in this case, because the
IBM1047 codepage (and most EBCDIC based codepages) has a wholly different
representation for numeric characters than plain old ASCII.

Note: You don't need to be building for this specific target to get these
results, just use any version from the trunk after isdigit was inserted as a
builtin function, and make sure you use an EBCDIC character set for the
-fexec-charset.  I have .i files from s390x-ibm-tpf target and from a native
i686-gnu-linux build.

I will attach a sample .i file shortly from each architecture.

-- 
           Summary: isdigit builtin function fails with EBCDIC character
                    sets
           Product: gcc
           Version: 4.0.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: middle-end
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: darcypj at us dot ibm dot com
                CC: gcc-bugs at gcc dot gnu dot org
 GCC build triplet: s390x-ibm-linux
  GCC host triplet: s390x-ibm-linux
GCC target triplet: s390x-ibm-tpf


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] isdigit builtin function fails with EBCDIC character sets

[darcypj at us dot ibm dot com]

------- Additional Comments From darcypj at us dot ibm dot com  2004-12-02 14:25 -------
Created an attachment (id=7657)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=7657&action=view)
s390x-ibm-tpf ".i" file as example.  COmpile command given in problem
statement.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[pinskia at gcc dot gnu dot org]

------- Additional Comments From pinskia at gcc dot gnu dot org  2004-12-02 14:28 -------
The problem is that TARGET_DIGIT0 is not dependent on the exec-charset.

This was caused by:
2004-04-14  Kaveh R. Ghazi  <ghazi@caip.rutgers.edu>
        
        * builtins.c (fold_builtin_isdigit): New.
        (fold_builtin): Handle BUILT_IN_ISDIGIT.
        * defaults.h: Add TARGET_DIGIT0 and sort.
        * doc/tm.texi: Add TARGET_BS and TARGET_DIGIT0.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ghazi at caip dot rutgers
                   |                            |dot edu
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|                            |1
           Keywords|                            |wrong-code
   Last reconfirmed|0000-00-00 00:00:00         |2004-12-02 14:28:58
               date|                            |
            Summary|isdigit builtin function    |[4.0 Regression] isdigit
                   |fails with EBCDIC character |builtin function fails with
                   |sets                        |EBCDIC character sets
   Target Milestone|---                         |4.0.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[kghazi at verizon dot net]

------- Additional Comments From kghazi at verizon dot net  2004-12-02 15:08 -------
Subject: Re:  [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

The .i attachments you supplied use C header macro versions of isdigit, not
the builtin function.  E.g.:

retval = ((*__ctype_b_loc ())[(int) (('3'))] & (unsigned short int)
_ISdigit);

So I don't see how this interacts with the builtin.  Were these files
generated in error?

It's possible in some cases to disable these ctype macros by supplying -D
flags, but you don't indicate that you did this in your tests.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[darcypj at us dot ibm dot com]

------- Additional Comments From darcypj at us dot ibm dot com  2004-12-02 15:27 -------
It appears it did use these macros in the ix86 example, I can obsolete that
attachment.

However, the s390x-ibm-tpf example does not use those macros.

I can attach a listing, since I doubt that you have this compiler built in
place, and it will be apparent that the builtin is behaving improperly,
comparing against ascii '0' even though the compiler is translating all
character literals into IBM1047.

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[kghazi at verizon dot net]

------- Additional Comments From kghazi at verizon dot net  2004-12-02 15:56 -------
Subject: Re:  [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

> However, the s390x-ibm-tpf example does not use those macros.

No really it doesn't.  I just check both again.

> I can attach a listing, since I doubt that you have this compiler built in
> place, and it will be apparent that the builtin is behaving improperly,
> comparing against ascii '0' even though the compiler is translating all
> character literals into IBM1047.

No need, I believe you that there's a problem (I just wanted to make sure we
had the right testcases.)
The feedback I got for the original patch was inconsistent though.
http://gcc.gnu.org/ml/gcc-patches/2004-04/msg00593.html
http://gcc.gnu.org/ml/gcc-patches/2004-04/msg00851.html

I'm not sure how to proceed.  What I'd like to do is disable the builtin if
input_charset != narrow_charset (using the libcpp names.)  I don't know if
we have access to the charset bits or iconv stuff in the middle-end.  Most
of that is handled by libcpp or the C family frontends.



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[roger at eyesopen dot com]

------- Additional Comments From roger at eyesopen dot com  2004-12-04 03:44 -------
Hi P.J.,

Thanks for removing the i686 version of the attachment.  But it looks like the
390x-ibm-tpf version of digit.i (attachment #1), also reveals that the isdigit
call is being expanded as a macro in native <ctype.h>.

Can you confirm that there's actually a problem with the GCC builtin, for
example, by showing that adding the command line option -fno-builtin-isdigit
resolves the
failure?  If specifying -fno-builtin-isdigit doesn't correct the failure, then
obviously its not GCC's builtin implementation at fault.

You should be able to factor out problems with the system headers by using a
test case that doesn't include <ctype.h> at all.  For example, just:

int foo(char ch)
{
  return isdigit(ch);
}


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[pinskia at gcc dot gnu dot org]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P2                          |P3


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[kghazi at verizon dot net]

------- Additional Comments From kghazi at verizon dot net  2004-12-06 22:06 -------
Subject: Re:  [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

Out of curiousity, do you have any problems with builtin printf's conversion
of printf("hello world\n") -> puts("hello world") ?  That transformation
doesn't even use TARGET_NEWLINE, instead checking directly for '\n'.
--Kaveh



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[darcypj at us dot ibm dot com]

------- Additional Comments From darcypj at us dot ibm dot com  2004-12-07 15:21 -------
Wow - curiously, I don't have a problem with it, but that is because when I
added the -fexec-charset option, I no longer got the branch to puts().

Examples follow:

===================================
If I compile with the following options, I see the puts:

-c print.c -fverbose-asm -Wa,-aldhs=print.lst -gdwarf-2

   4:print.c       ****   printf("Hello world\n");
  47                            .loc 1 4 0
  48 001c C0200000              larl    %r2,.LC0        #,
  48      0000
  49 0022 C0E50000              brasl   %r14,puts       #,
  49      0000


=======================
And with these options (just the -fexec-charset addded) I see a call to printf().

-c print.c -fverbose-asm -Wa,-aldhs=print.lst -gdwarf-2 -fexec-charset=IBM1047

   4:print.c       ****   printf("Hello world\n");
  47                            .loc 1 4 0
  48 001c C0200000              larl    %r2,.LC0        #,
  48      0000
  49 0022 C0E50000              brasl   %r14,printf     #,
  49      0000


Note, I get this warning either way since I am not including any headers, in
order to try and force the builtin printf()..

print.c: In function 'main':
print.c:4: warning: incompatible implicit declaration of built-in function 'printf'

So it appears to me that the compiler builtin knows not to activate when the
-fexec-charset is invoked?

-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[kghazi at verizon dot net]

------- Additional Comments From kghazi at verizon dot net  2004-12-07 20:08 -------
Subject: Re:  [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

> So it appears to me that the compiler builtin knows not to activate when
the
> -fexec-charset is invoked?

No, there's no special -fexec-charset handling in the builtin printf either.
What's happening here is that the trailing newline in "hello world\n" is not
matching the '\n' in the test in builtins.c:expand_builtin_printf.  This
simply causes it to avoid doing the transformation into puts.  Marginally
suboptimal, but harmless.

However, it will probably erroneously match the values for ASCII newline
\012, as in "hello world\012", if that mapping appears by chance as the last
character in your printf string.  (I don't know what \012 maps to in
IBM1047.)  That would cause code to mistakenly change the call into puts
with a string of length one less, adding the newline from puts.  This no
longer matches the intended behavior of the original program if '\n' in the
source charset != '\n' in the exec charset.

So I think whatever solution applies to isdigit should also be applied to
printf  (and perhaps elsewhere, I haven't done a full audit.)  I'm not
entirely sure how to fix it nor do I have the time to fully investigate at
the moment, so some help would be appreciated.
Thanks,
--Kaveh



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[roger at eyesopen dot com]

------- Additional Comments From roger at eyesopen dot com  2004-12-22 16:40 -------
Possible patch at http://gcc.gnu.org/ml/gcc-patches/2004-12/msg01711.html


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |roger at eyesopen dot com
                   |dot org                     |
             Status|NEW                         |ASSIGNED
           Keywords|                            |patch


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[zack at codesourcery dot com]

------- Additional Comments From zack at codesourcery dot com  2005-02-20 17:07 -------
Patch checked in.  Related bugs remain, which should receive separate PRs.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785

[Bug middle-end/18785] [4.0 Regression] isdigit builtin function fails with EBCDIC character sets

[cvs-commit at gcc dot gnu dot org]

------- Additional Comments From cvs-commit at gcc dot gnu dot org  2005-02-20 17:01 -------
Subject: Bug 18785

CVSROOT:	/cvs/gcc
Module name:	gcc
Changes by:	zack@gcc.gnu.org	2005-02-20 17:01:32

Modified files:
	gcc            : ChangeLog builtins.c c-common.c c-common.h 
	                 c-objc-common.h c-pretty-print.c defaults.h 
	                 langhooks-def.h langhooks.c langhooks.h 
	                 system.h 
	gcc/config/arm : arm.c 
	gcc/config/mips: mips.c 
	gcc/cp         : ChangeLog cp-objcp-common.h 
	gcc/doc        : tm.texi 
	gcc/testsuite  : ChangeLog 
	libcpp         : ChangeLog charset.c 
	libcpp/include : cpplib.h 
Added files:
	gcc/testsuite/gcc.dg/charset: builtin1.c 

Log message:
	PR 18785
	libcpp:
	* charset.c (LAST_POSSIBLY_BASIC_SOURCE_CHAR): New helper macro.
	(cpp_host_to_exec_charset): New function.
	* include/cpplib.h: Declare cpp_host_to_exec_charset.
	
	gcc:
	* langhooks.h (struct lang_hooks): Add to_target_charset.
	* langhooks.c (lhd_to_target_charset): New function.
	* langhooks-def.h: Declare lhd_to_target_charset.
	(LANG_HOOKS_TO_TARGET_CHARSET): New macro.
	(LANG_HOOKS_INITIALIZER): Update.
	* c-common.c (c_common_to_target_charset): New function.
	* c-common.h: Declare it.
	* c-objc-common.h (LANG_HOOKS_TO_TARGET_CHARSET): Set to
	c_common_to_target_charset.
	
	* defaults.c (TARGET_BELL, TARGET_BS, TARGET_CR, TARGET_DIGIT0)
	(TARGET_ESC, TARGET_FF, TARGET_NEWLINE, TARGET_TAB, TARGET_VT):
	Delete definitions.
	* system.h: Poison them.
	* doc/tm.texi: Don't discuss them.
	* builtins.c (fold_builtin_isdigit): Use lang_hooks.to_target_charset.
	* c-pretty-print.c (pp_c_integer_constant): Don't use pp_c_char.
	(pp_c_char): Do not attempt to generate letter escapes for
	newline, tab, etc.
	* config/arm/arm.c (output_ascii_pseudo_op): Likewise.
	* config/mips/mips.c (mips_output_ascii): Likewise.
	gcc/cp:
	* cp-objcp-common.h (LANG_HOOKS_TO_TARGET_CHARSET): Set to
	c_common_to_target_charset.  Delete bogus comment.
	gcc/testsuite:
	* gcc.dg/charset/builtin1.c: New test.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&r1=2.7540&r2=2.7541
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/builtins.c.diff?cvsroot=gcc&r1=1.424&r2=1.425
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/c-common.c.diff?cvsroot=gcc&r1=1.604&r2=1.605
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/c-common.h.diff?cvsroot=gcc&r1=1.276&r2=1.277
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/c-objc-common.h.diff?cvsroot=gcc&r1=2.2&r2=2.3
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/c-pretty-print.c.diff?cvsroot=gcc&r1=1.57&r2=1.58
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/defaults.h.diff?cvsroot=gcc&r1=1.167&r2=1.168
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/langhooks-def.h.diff?cvsroot=gcc&r1=1.96&r2=1.97
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/langhooks.c.diff?cvsroot=gcc&r1=1.79&r2=1.80
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/langhooks.h.diff?cvsroot=gcc&r1=1.102&r2=1.103
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/system.h.diff?cvsroot=gcc&r1=1.242&r2=1.243
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/config/arm/arm.c.diff?cvsroot=gcc&r1=1.429&r2=1.430
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/config/mips/mips.c.diff?cvsroot=gcc&r1=1.485&r2=1.486
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/ChangeLog.diff?cvsroot=gcc&r1=1.4638&r2=1.4639
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/cp/cp-objcp-common.h.diff?cvsroot=gcc&r1=1.5&r2=1.6
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/doc/tm.texi.diff?cvsroot=gcc&r1=1.412&r2=1.413
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/ChangeLog.diff?cvsroot=gcc&r1=1.5058&r2=1.5059
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/gcc.dg/charset/builtin1.c.diff?cvsroot=gcc&r1=NONE&r2=1.1
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libcpp/ChangeLog.diff?cvsroot=gcc&r1=1.55&r2=1.56
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libcpp/charset.c.diff?cvsroot=gcc&r1=1.4&r2=1.5
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/libcpp/include/cpplib.h.diff?cvsroot=gcc&r1=1.8&r2=1.9



-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18785