Dwarf_Op

Dwarf_Op

[Sasha Da Rocha Pinheiro]

Hi all,

I have a Dwarf_Op object whose member "number" has size of 8 bytes. 
Its value although is 0x00000000FFFFFFE8. 
Shouldn't it be 0xFFFFFFFFFFFFFFE8 instead?
Since it means an offset, for the current operation, shouldn't it be of a signed integer type instead of unsigned?

Regards
Sasha

Re: Dwarf_Op

[Mark Wielaard]

Hi Sasha,

On Thu, 2019-04-25 at 23:59 +0000, Sasha Da Rocha Pinheiro wrote:
> I have a Dwarf_Op object whose member "number" has size of 8 bytes. 
> Its value although is 0x00000000FFFFFFE8. 
> Shouldn't it be 0xFFFFFFFFFFFFFFE8 instead?
> Since it means an offset, for the current operation, shouldn't it be
> of a signed integer type instead of unsigned?

Could you provide more information?
Which function did you use to get the Dwarf_Op?
What was its associated attribute or frame?
What is its atom_value?
How are you using the Dwarf_Op that requires interpreting its number
fields?

Thanks,

Mark

Re: Dwarf_Op

[Sasha Da Rocha Pinheiro]

Hi Mark,

The output from dwarfdump:

<    5><0x00400918:0x00400974><main><cie offset 0x000000b4::cie index     0><fde offset 0x000000b0 length: 0x00000024>
       <eh aug data len 0x0>
        0x00400918: <off cfa=00(r31) > 
        0x0040091c: <off cfa=32(r31) > <off r29=-32(cfa) > <off r30=-24(cfa) > 
        0x00400920: <off cfa=32(r29) > <off r29=-32(cfa) > <off r30=-24(cfa) > 
        0x00400970: <off cfa=00(r31) > 

I'm getting the rules for r30 at PC 0x0040091c.
The value you see -24 from cfa is passed delivered by libdw as:

(gdb) p/x locations[i]
$19 = {atom = 0x23, number = 0xffffffffffffffe8, number2 = 0x0, offset = 0x0}

The atom here is DW_OP_plus_uconst.
Previously I have said it was 0xffffffe8, because the printf with specifier %x printed that way. But gdb seems to print its full 8 bytes.
I used dwarf_frame_register.

Even though, the member number is of unsigned int type with signed encoded values. Am I correct?

Sasha




From: Mark Wielaard <mark@klomp.org>
Sent: Friday, April 26, 2019 3:20 AM
To: Sasha Da Rocha Pinheiro; elfutils-devel@sourceware.org
Subject: Re: Dwarf_Op
 
Hi Sasha,

On Thu, 2019-04-25 at 23:59 +0000, Sasha Da Rocha Pinheiro wrote:
> I have a Dwarf_Op object whose member "number" has size of 8 bytes.
> Its value although is 0x00000000FFFFFFE8.
> Shouldn't it be 0xFFFFFFFFFFFFFFE8 instead?
> Since it means an offset, for the current operation, shouldn't it be
> of a signed integer type instead of unsigned?

Could you provide more information?
Which function did you use to get the Dwarf_Op?
What was its associated attribute or frame?
What is its atom_value?
How are you using the Dwarf_Op that requires interpreting its number
fields?

Thanks,

Mark

Re: Dwarf_Op

[Mark Wielaard]

On Fri, Apr 26, 2019 at 04:48:49PM +0000, Sasha Da Rocha Pinheiro wrote:

> The output from dwarfdump:
> 
> <    5><0x00400918:0x00400974><main><cie offset 0x000000b4::cie index     0><fde offset 0x000000b0 length: 0x00000024>
>        <eh aug data len 0x0>
>         0x00400918: <off cfa=00(r31) > 
>         0x0040091c: <off cfa=32(r31) > <off r29=-32(cfa) > <off r30=-24(cfa) > 
>         0x00400920: <off cfa=32(r29) > <off r29=-32(cfa) > <off r30=-24(cfa) > 
>         0x00400970: <off cfa=00(r31) > 
> 
> I'm getting the rules for r30 at PC 0x0040091c.
> The value you see -24 from cfa is passed delivered by libdw as:
> 
> (gdb) p/x locations[i]
> $19 = {atom = 0x23, number = 0xffffffffffffffe8, number2 = 0x0, offset = 0x0}
> 
> The atom here is DW_OP_plus_uconst.
> Previously I have said it was 0xffffffe8, because the printf with specifier %x printed that way. But gdb seems to print its full 8 bytes.
> I used dwarf_frame_register.
> 
> Even though, the member number is of unsigned int type with signed encoded values. Am I correct?

Yes, a Dwarf_Op struct holds two number elements which are (unsigned 64bit)
Dwarf_Words:

typedef struct
{
  uint8_t atom;                 /* Operation */
  Dwarf_Word number;            /* Operand */
  Dwarf_Word number2;           /* Possible second operand */
  Dwarf_Word offset;            /* Offset in location expression */
} Dwarf_Op;

I assume you are using dwarf_frame_register () to turn the CFI for a
particular register into a location expression (array of Dwarf_Ops).
Which in this case would indeed produce DW_OP_call_frame_cfa,
DW_OP_plus_uconst (with the constant -24 in number).

Unfortunately there is no better/precise way to turn the CFI register
rule into a Dwarf Expression Location.

But note that if you use the Dwarf_Ops to calculate an Dwarf_Addr,
then the 64bit unsigned arithmetic will work out (because the plus of
the big unsigned constant will wrap around and turn into a small minus
constant).

Cheers,

Mark

Re: Dwarf_Op

[Sasha Da Rocha Pinheiro]

Hi Mark,

yes, I did use dwarf_frame_register(), I believe I mentioned that :).
In the case I brought up you're saying it was an elfutils' libdw decision to provide negative number as DW_OP_plus_uconst (unsigned constant)? So there would be no harm to cast it to a signed int??

Do you know if the rules to find the registers will always have only another one register associated to it? Because libdwarf function dwarf_get_fde_info_for_reg3() returns a register_num, which is described as:
"Argument register_num should point to a location which will hold the register number associated with the register rule."

elfutils only gives us a location description in an array of Dwarf_Op. Does it provide methods to "execute" it so we can get the result of the expressions? Or it delegates it to the consumer?

Regards
Sasha




From: Mark Wielaard <mark@klomp.org>
Sent: Saturday, April 27, 2019 12:57 PM
To: Sasha Da Rocha Pinheiro
Cc: elfutils-devel@sourceware.org
Subject: Re: Dwarf_Op
 
On Fri, Apr 26, 2019 at 04:48:49PM +0000, Sasha Da Rocha Pinheiro wrote:

> The output from dwarfdump:
>
> <    5><0x00400918:0x00400974><main><cie offset 0x000000b4::cie index     0><fde offset 0x000000b0 length: 0x00000024>
>        <eh aug data len 0x0>
>         0x00400918: <off cfa=00(r31) >
>         0x0040091c: <off cfa=32(r31) > <off r29=-32(cfa) > <off r30=-24(cfa) >
>         0x00400920: <off cfa=32(r29) > <off r29=-32(cfa) > <off r30=-24(cfa) >
>         0x00400970: <off cfa=00(r31) >
>
> I'm getting the rules for r30 at PC 0x0040091c.
> The value you see -24 from cfa is passed delivered by libdw as:
>
> (gdb) p/x locations[i]
> $19 = {atom = 0x23, number = 0xffffffffffffffe8, number2 = 0x0, offset = 0x0}
>
> The atom here is DW_OP_plus_uconst.
> Previously I have said it was 0xffffffe8, because the printf with specifier %x printed that way. But gdb seems to print its full 8 bytes.
> I used dwarf_frame_register.
>
> Even though, the member number is of unsigned int type with signed encoded values. Am I correct?

Yes, a Dwarf_Op struct holds two number elements which are (unsigned 64bit)
Dwarf_Words:

typedef struct
{
  uint8_t atom;                 /* Operation */
  Dwarf_Word number;            /* Operand */
  Dwarf_Word number2;           /* Possible second operand */
  Dwarf_Word offset;            /* Offset in location expression */
} Dwarf_Op;

I assume you are using dwarf_frame_register () to turn the CFI for a
particular register into a location expression (array of Dwarf_Ops).
Which in this case would indeed produce DW_OP_call_frame_cfa,
DW_OP_plus_uconst (with the constant -24 in number).

Unfortunately there is no better/precise way to turn the CFI register
rule into a Dwarf Expression Location.

But note that if you use the Dwarf_Ops to calculate an Dwarf_Addr,
then the 64bit unsigned arithmetic will work out (because the plus of
the big unsigned constant will wrap around and turn into a small minus
constant).

Cheers,

Mark

Re: Dwarf_Op

[Mark Wielaard]

On Wed, May 01, 2019 at 05:08:15PM +0000, Sasha Da Rocha Pinheiro wrote:
> yes, I did use dwarf_frame_register(), I believe I mentioned that :).

Sorry, I missed that.

> In the case I brought up you're saying it was an elfutils' libdw
> decision to provide negative number as DW_OP_plus_uconst (unsigned
> constant)? So there would be no harm to cast it to a signed int??

Yes, since I don't believe there is a better representation.
There is no DW_OP_plus_sconst for example.

> Do you know if the rules to find the registers will always have only
> another one register associated to it?  Because libdwarf function
> dwarf_get_fde_info_for_reg3() returns a register_num, which is
> described as: "Argument register_num should point to a location
> which will hold the register number associated with the register
> rule."

I am not familiar with libdwarf, so don't know how to answer this question

> elfutils only gives us a location description in an array of
> Dwarf_Op. Does it provide methods to "execute" it so we can get the
> result of the expressions? Or it delegates it to the consumer?

It doesn't have a generic location description "executor". It really
should have. But to make it generic it needs a lot of context.

libdwfl does contain a partial operator interpreter for unwinding. See
expr_eval in libdwfl/frame_unwind.c. To turn that into something more
generic we would need some way to provide it with things like memory
accessors and register values. libdwfl provides some of that through
dwfl_attach_state.

Cheers,

Mark

Re: Dwarf_Op

[Sasha Da Rocha Pinheiro]

So why not a DW_OP_constu or DW_OP_consts and then a DW_OP_plus?

About the Dwarf_Op array we get from dwarf_frame_register(), (which are a sequence of Dwarf expressions): are they suppose to be an abstraction to all types of dwarf location descriptions? Including Single (single and composite) and Location Lists?

Is there more documentation about dwfl? What does dwfl stand for? (dwarf frame library?!)
We already have access to the process memory space and registers, so would expr_eval be suffice?

In case of finding the registers values at some PC, would it always gives us a single expression (like an address or an offset from a register)?

In the comment of dwarf_frame_register() in the file libdw.h, there is:
"Note the last operation is DW_OP_stack_value if there is no mutable location but only a computable value."
What is a mutable location, I didn't find this concept in the Dwarf Debugging Information Format v4.

Also there there is:

1057    For common simple expressions *OPS is OPS_MEM.  For arbitrary DWARF
1058    expressions in the CFI, *OPS is an internal pointer that can be used as
1059    long as the Dwarf_CFI used to create FRAME remains alive.  */

Why simple expression can only have 3 operations? (since Dwarf_Op ops_mem[3])
And what's an arbitrary Dwarf expression? What's a non-arbitrary?


Regards,
Sasha



From: Mark Wielaard <mark@klomp.org>
Sent: Wednesday, May 1, 2019 6:02 PM
To: Sasha Da Rocha Pinheiro
Cc: elfutils-devel@sourceware.org
Subject: Re: Dwarf_Op
 
On Wed, May 01, 2019 at 05:08:15PM +0000, Sasha Da Rocha Pinheiro wrote:
> yes, I did use dwarf_frame_register(), I believe I mentioned that :).

Sorry, I missed that.

> In the case I brought up you're saying it was an elfutils' libdw
> decision to provide negative number as DW_OP_plus_uconst (unsigned
> constant)? So there would be no harm to cast it to a signed int??

Yes, since I don't believe there is a better representation.
There is no DW_OP_plus_sconst for example.

> Do you know if the rules to find the registers will always have only
> another one register associated to it?  Because libdwarf function
> dwarf_get_fde_info_for_reg3() returns a register_num, which is
> described as: "Argument register_num should point to a location
> which will hold the register number associated with the register
> rule."

I am not familiar with libdwarf, so don't know how to answer this question

> elfutils only gives us a location description in an array of
> Dwarf_Op. Does it provide methods to "execute" it so we can get the
> result of the expressions? Or it delegates it to the consumer?

It doesn't have a generic location description "executor". It really
should have. But to make it generic it needs a lot of context.

libdwfl does contain a partial operator interpreter for unwinding. See
expr_eval in libdwfl/frame_unwind.c. To turn that into something more
generic we would need some way to provide it with things like memory
accessors and register values. libdwfl provides some of that through
dwfl_attach_state.

Cheers,

Mark

Re: Dwarf_Op

[Mark Wielaard]

On Thu, May 02, 2019 at 07:29:33PM +0000, Sasha Da Rocha Pinheiro wrote:
> So why not a DW_OP_constu or DW_OP_consts and then a DW_OP_plus?

Probably because that is multiple operators and less efficiently
encoded.  But yes, it would be slightly more "correct" semantically.
In this case however it is for computing (unsigned) addresses (CFA
plus offset) in which case unsigned arithmetic is fine.

> About the Dwarf_Op array we get from dwarf_frame_register(), (which
> are a sequence of Dwarf expressions): are they suppose to be an
> abstraction to all types of dwarf location descriptions? Including
> Single (single and composite) and Location Lists?

In theory yes, it is not a special kind of Dwarf Location
Expression. It should be usable generially, although clearly it
resolves in the context of the CFI (current pc and register values,
memory, etc.). It turns the CFI for getting a register into a Dwarf
Location Expression that you can use to resolve the value of that
register.

> Is there more documentation about dwfl? What does dwfl stand for?
> (dwarf frame library?!)

All documentation is in libdwfl.h for now.  I am not sure dwfl
officially stands for anything.  But if ebl is the Elf Backend
Library, then dwfl could be the libDW Frontend Library.

> We already have access to the process memory space and registers, so
> would expr_eval be suffice?

Yes, if you are just looking for CFI/Unwinding then that should be
enough. But expr_eval isn't a generic DWARF Expression evaluator so
you cannot simply use it as is for generic DWARF Location Expressions
coming from other parts of DWARF.

> In case of finding the registers values at some PC, would it always
> gives us a single expression (like an address or an offset from a
> register)?

Yes.

> In the comment of dwarf_frame_register() in the file libdw.h, there
> is: "Note the last operation is DW_OP_stack_value if there is no
> mutable location but only a computable value."  What is a mutable
> location, I didn't find this concept in the Dwarf Debugging
> Information Format v4.

I believe "mutable locations" is what DWARF calls Location
Descriptions, which point to a memory location or register where the
value can be found (and so can be changed in that location). And what
is called a "computable value" is what DWARF calls Implicit Location
Descriptions, which provides a value itself (so there is no real
location that can be changed).

> Also there there is:
> 
> 1057    For common simple expressions *OPS is OPS_MEM.  For arbitrary DWARF
> 1058    expressions in the CFI, *OPS is an internal pointer that can be used as
> 1059    long as the Dwarf_CFI used to create FRAME remains alive.  */
> 
> Why simple expression can only have 3 operations? (since Dwarf_Op ops_mem[3])
> And what's an arbitrary Dwarf expression? What's a non-arbitrary?

It is "common simple expressions" vs "arbitrary expressions".  The
interface is made to make memory management easier (for the
implementation of the function). But might be slightly confusing if
you don't know that (you really shouldn't know about the internals).

The idea is that most often the CFI expression can be turned into 1, 2
or 3 DWARF operators. The caller is asked to provide memory to store
those. If the expression is bigger, then a pointer to internal storage
is returned. In both cases you would just use *OPS for the *NOPS
number of operations (in the first case *OPS points to your *MEM_OPS
provided array, in the second case it points to some internal storage.

Cheers,

Mark

Re: Dwarf_Op

[Sasha Da Rocha Pinheiro]

About the *ops returned from dwarf_frame_register():
Is it correct to say that if we don't get a DW_OP_stack_value as the last operation, the value on top of the stack will always be a memory address? Meaning you must dereference it? Even though there is no DW_OP_deref provided as last operation? If not, what else could it be?

Another issue:
I am trying to decode the following *ops:    DW_OP_call_frame_cfa    DW_OP_stack_value
This was returned after I requested register 31 (SP in aarch64) at address 0x40091c.
As you can see below, I pasted the CIE and FDE relative to this address using eu-readelf and dwarfdump.
Is it that, in this case, DW_OP_call_frame_cfa means "get the CFA value at 0x00400918"? Why isn't SP rule undefined or same_value, instead of those two operations?

=============
 dwarfdump output 
=============
<    5><0x00400918:0x00400974><main><cie offset 0x000000b4::cie index     0><fde offset 0x000000b0 length: 0x00000024>
       <eh aug data len 0x0>
        0x00400918: <off cfa=00(r31) > 
        0x0040091c: <off cfa=32(r31) > <off r29=-32(cfa) > <off r30=-24(cfa) > 
        0x00400920: <off cfa=32(r29) > <off r29=-32(cfa) > <off r30=-24(cfa) > 
        0x00400970: <off cfa=00(r31) > 
cie:
<    0> version                         1
        cie section offset              0 0x00000000
        augmentation                    zR
        code_alignment_factor           4
        data_alignment_factor           -8
        return_address_register         30
        eh aug data len 0x1 bytes 0x1b 
        bytes of initial instructions   7
        cie length                      20
        initial instructions
         0 DW_CFA_def_cfa r31 0
         3 DW_CFA_nop
         4 DW_CFA_nop
         5 DW_CFA_nop
         6 DW_CFA_nop


============
 eu-readelf output 
============

[     0] CIE length=20
   CIE_id:                   0
   version:                  1
   augmentation:             "zR"
   code_alignment_factor:    4
   data_alignment_factor:    -8
   return_address_register:  30
   Augmentation data:        0x1b (FDE address encoding: sdata4 pcrel)

   Program:
     def_cfa r31 (reg31) at offset 0
     nop
     nop
     nop
     nop
...
[    b0] FDE length=36 cie=[     0]
   CIE_pointer:              180
   initial_location:         0x0000000000400918 <main> (offset: 0x918)
   address_range:            0x5c (end offset: 0x974)

   Program:
     advance_loc 1 to 0x91c
     def_cfa_offset 32
     offset r29 (reg29) at cfa-32
     offset r30 (reg30) at cfa-24
     advance_loc 1 to 0x920
     def_cfa_register r29 (reg29)
     advance_loc 20 to 0x970
     restore r30 (reg30)
     restore r29 (reg29)
     def_cfa r31 (reg31) at offset 0
     nop
     nop
     nop
     nop
     nop
     nop
     nop





From: Mark Wielaard <mark@klomp.org>
Sent: Friday, May 3, 2019 4:09 PM
To: Sasha Da Rocha Pinheiro
Cc: elfutils-devel@sourceware.org
Subject: Re: Dwarf_Op
 
On Thu, May 02, 2019 at 07:29:33PM +0000, Sasha Da Rocha Pinheiro wrote:
> So why not a DW_OP_constu or DW_OP_consts and then a DW_OP_plus?

Probably because that is multiple operators and less efficiently
encoded.  But yes, it would be slightly more "correct" semantically.
In this case however it is for computing (unsigned) addresses (CFA
plus offset) in which case unsigned arithmetic is fine.

> About the Dwarf_Op array we get from dwarf_frame_register(), (which
> are a sequence of Dwarf expressions): are they suppose to be an
> abstraction to all types of dwarf location descriptions? Including
> Single (single and composite) and Location Lists?

In theory yes, it is not a special kind of Dwarf Location
Expression. It should be usable generially, although clearly it
resolves in the context of the CFI (current pc and register values,
memory, etc.). It turns the CFI for getting a register into a Dwarf
Location Expression that you can use to resolve the value of that
register.

> Is there more documentation about dwfl? What does dwfl stand for?
> (dwarf frame library?!)

All documentation is in libdwfl.h for now.  I am not sure dwfl
officially stands for anything.  But if ebl is the Elf Backend
Library, then dwfl could be the libDW Frontend Library.

> We already have access to the process memory space and registers, so
> would expr_eval be suffice?

Yes, if you are just looking for CFI/Unwinding then that should be
enough. But expr_eval isn't a generic DWARF Expression evaluator so
you cannot simply use it as is for generic DWARF Location Expressions
coming from other parts of DWARF.

> In case of finding the registers values at some PC, would it always
> gives us a single expression (like an address or an offset from a
> register)?

Yes.

> In the comment of dwarf_frame_register() in the file libdw.h, there
> is: "Note the last operation is DW_OP_stack_value if there is no
> mutable location but only a computable value."  What is a mutable
> location, I didn't find this concept in the Dwarf Debugging
> Information Format v4.

I believe "mutable locations" is what DWARF calls Location
Descriptions, which point to a memory location or register where the
value can be found (and so can be changed in that location). And what
is called a "computable value" is what DWARF calls Implicit Location
Descriptions, which provides a value itself (so there is no real
location that can be changed).

> Also there there is:
>
> 1057    For common simple expressions *OPS is OPS_MEM.  For arbitrary DWARF
> 1058    expressions in the CFI, *OPS is an internal pointer that can be used as
> 1059    long as the Dwarf_CFI used to create FRAME remains alive.  */
>
> Why simple expression can only have 3 operations? (since Dwarf_Op ops_mem[3])
> And what's an arbitrary Dwarf expression? What's a non-arbitrary?

It is "common simple expressions" vs "arbitrary expressions".  The
interface is made to make memory management easier (for the
implementation of the function). But might be slightly confusing if
you don't know that (you really shouldn't know about the internals).

The idea is that most often the CFI expression can be turned into 1, 2
or 3 DWARF operators. The caller is asked to provide memory to store
those. If the expression is bigger, then a pointer to internal storage
is returned. In both cases you would just use *OPS for the *NOPS
number of operations (in the first case *OPS points to your *MEM_OPS
provided array, in the second case it points to some internal storage.

Cheers,

Mark

Re: Dwarf_Op

[Mark Wielaard]

On Wed, May 08, 2019 at 06:03:55PM +0000, Sasha Da Rocha Pinheiro wrote:
> About the *ops returned from dwarf_frame_register():
> Is it correct to say that if we don't get a DW_OP_stack_value as the
> last operation, the value on top of the stack will always be a
> memory address? Meaning you must dereference it? Even though there
> is no DW_OP_deref provided as last operation? If not, what else
> could it be?

There could be no operations at all, an empty location description,
then there simply is no location/value. It could be a single register
location description operation (DW_OP_reg[0..31] or DW_OP_regx) that
says the value is in that particular register (note this is different
from the DW_OP_bregx operations which pushes the contents of the
register on the expression stack). Or it is one or more of the other
operations which leave the location (an address) on the expression
stack. If the last operation is DW_OP_stack_value then the value on
the top of the expression stack is the value itself (and not a
location where the value can be changed).

Technically it can also be an DW_OP_implicit_value (again a value, not
a location), and DW_OP_implicit_pointer (a "fake" pointer/location,
but really a description of the value of that location), or a
composite location description (DW_OP_[bit_]piece, multiple location
descriptions that have to be combined to get the value). But these
don't come from dwarf_frame_register at the moment.

> Another issue: I am trying to decode the following *ops:
> DW_OP_call_frame_cfa DW_OP_stack_value This was returned after I
> requested register 31 (SP in aarch64) at address 0x40091c.  As you
> can see below, I pasted the CIE and FDE relative to this address
> using eu-readelf and dwarfdump.  Is it that, in this case,
> DW_OP_call_frame_cfa means "get the CFA value at 0x00400918"? Why
> isn't SP rule undefined or same_value, instead of those two
> operations?

Yes, that looks circular, but it is meant to be "efficient". Often
registers can be found at an offset of the current CFA (or in this
case, it is the CFA value). You get the CFA for a frame with
dwarf_frame_cfa (), which works similar to dwarf_frame_register (),
the CFA itself doesn't have/is a register number however.

Hope that helps,

Mark