Re: [PATCH] Clear upper bits during sign extension

[Doug Evans <dje@google.com>]

On Sun, Dec 28, 2014 at 7:37 PM, Yao Qi <yao@codesourcery.com> wrote:
> Joel Brobecker <brobecker@adacore.com> writes:
>
>> I think you can simplify the above with just:
>>
>>         value &= ((LONGEST) 1 << bit) - 1;
>>
>> ? I don't know if the cast to LONGEST is really necessary, but we use
>> it for signbit, so I did the same for the mask...
>
> Yeah, that is better, or even we can left shift signbit by one.
>
> --
> Yao (齐尧)
>
> gdb:
>
> 2014-12-29  Yao Qi  <yao@codesourcery.com>
>
>         * utils.c (gdb_sign_extend): Clear bits from BIT in VALUE.
>
> diff --git a/gdb/utils.c b/gdb/utils.c
> index 47adb67..61873f7 100644
> --- a/gdb/utils.c
> +++ b/gdb/utils.c
> @@ -3032,6 +3032,9 @@ gdb_sign_extend (LONGEST value, int bit)
>      {
>        LONGEST signbit = ((LONGEST) 1) << (bit - 1);
>
> +      /* Clear upper bits from bit BIT.  */
> +      value &= (signbit << 1) - 1;
> +
>        value = (value ^ signbit) - signbit;
>      }

It's not immediately clear to this reader that undefined behaviour is
avoided here.
E.g., what if sizeof (LONGEST) == 8 && bit == 64.

There's an assert at the beginning of the function that bit could be 64.