[PATCH v2 06/10] Avoid undefined behavior in parse_number

[Tom Tromey <tom@tromey.com>]

-fsanitize=undefined pointed out that c-exp.y relied on undefined
behavior here:

      if (c != 'l' && c != 'u')
	n *= base;

...when a large hex constant "just fit" into a LONGEST, causing the
high bit to be set.

This fixes the problem by having the function work in an unsigned
type.

gdb/ChangeLog
2018-10-01  Tom Tromey  <tom@tromey.com>

	* c-exp.y (parse_number): Work in unsigned.  Remove casts.
---
 gdb/ChangeLog |  4 ++++
 gdb/c-exp.y   | 10 ++++------
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/gdb/c-exp.y b/gdb/c-exp.y
index 0326ee090e..09e31d2283 100644
--- a/gdb/c-exp.y
+++ b/gdb/c-exp.y
@@ -1760,10 +1760,8 @@ static int
 parse_number (struct parser_state *par_state,
 	      const char *buf, int len, int parsed_float, YYSTYPE *putithere)
 {
-  /* FIXME: Shouldn't these be unsigned?  We don't deal with negative values
-     here, and we do kind of silly things like cast to unsigned.  */
-  LONGEST n = 0;
-  LONGEST prevn = 0;
+  ULONGEST n = 0;
+  ULONGEST prevn = 0;
   ULONGEST un;
 
   int i = 0;
@@ -1922,7 +1920,7 @@ parse_number (struct parser_state *par_state,
 	 on 0x123456789 when LONGEST is 32 bits.  */
       if (c != 'l' && c != 'u' && n != 0)
 	{	
-	  if ((unsigned_p && (ULONGEST) prevn >= (ULONGEST) n))
+	  if (unsigned_p && prevn >= n)
 	    error (_("Numeric constant too large."));
 	}
       prevn = n;
@@ -1940,7 +1938,7 @@ parse_number (struct parser_state *par_state,
      the case where it is we just always shift the value more than
      once, with fewer bits each time.  */
 
-  un = (ULONGEST)n >> 2;
+  un = n >> 2;
   if (long_p == 0
       && (un >> (gdbarch_int_bit (parse_gdbarch (par_state)) - 2)) == 0)
     {
-- 
2.17.1