[binutils-gdb] libctf: get the encoding of non-ints/fps in the dynamic space right

[binutils-gdb] libctf: get the encoding of non-ints/fps in the dynamic space right

[Jose E.Marchesi]

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=9c1a2295e84170d2de06ef3c828f0c9f5933867e

commit 9c1a2295e84170d2de06ef3c828f0c9f5933867e
Author: Nick Alcock <nick.alcock@oracle.com>
Date:   Fri Aug 9 22:53:50 2019 +0100

    libctf: get the encoding of non-ints/fps in the dynamic space right
    
    If you call ctf_type_encoding() on a slice, you are meant to get the
    encoding of the slice with the format of the underlying type.  If
    you call it on a non-int, non-fp, non-slice, you're meant to get the
    error ECTF_INTNOTFP.
    
    None of this was implemented for types in the dynamic space (which, now,
    is *all* types in writable containers).  Instead, we were always
    returning the encoding as if it were a float, which for all other types
    consulted the wrong part of a discriminated union and returned garbage.
    (Curiously, existing users were more disturbed by the lack of an error
    in the non-int/fp/slice case than they were about getting garbage back.)
    
    libctf/
    	* ctf-types.c (ctf_type_encoding): Fix the dynamic case to
    	work right for non-int/fps.

Diff:
---
 libctf/ChangeLog   |  5 +++++
 libctf/ctf-types.c | 22 +++++++++++++++++++++-
 2 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/libctf/ChangeLog b/libctf/ChangeLog
index 8f41ece..74dfbef 100644
--- a/libctf/ChangeLog
+++ b/libctf/ChangeLog
@@ -1,3 +1,8 @@
+2019-08-09  Nick Alcock  <nick.alcock@oracle.com>
+
+	* ctf-types.c (ctf_type_encoding): Fix the dynamic case to
+	work right for non-int/fps.
+
 2019-08-08  Nick Alcock  <nick.alcock@oracle.com>
 
 	* ctf-types.c (ctf_type_name): Don't strlen a potentially-
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index ec221d7..27cbfb9 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -739,7 +739,27 @@ ctf_type_encoding (ctf_file_t *fp, ctf_id_t type, ctf_encoding_t *ep)
 
   if ((dtd = ctf_dynamic_type (ofp, type)) != NULL)
     {
-      *ep = dtd->dtd_u.dtu_enc;
+      switch (LCTF_INFO_KIND (fp, tp->ctt_info))
+	{
+	case CTF_K_INTEGER:
+	case CTF_K_FLOAT:
+	  *ep = dtd->dtd_u.dtu_enc;
+	  break;
+	case CTF_K_SLICE:
+	  {
+	    const ctf_slice_t *slice;
+	    ctf_encoding_t underlying_en;
+	    slice = &dtd->dtd_u.dtu_slice;
+
+	    data = ctf_type_encoding (fp, slice->cts_type, &underlying_en);
+	    ep->cte_format = underlying_en.cte_format;
+	    ep->cte_offset = slice->cts_offset;
+	    ep->cte_bits = slice->cts_bits;
+	    break;
+	  }
+	default:
+	  return (ctf_set_errno (ofp, ECTF_NOTINTFP));
+	}
       return 0;
     }