Re: [PATCH 20/20] symtab: Add support for MODVERSIONS (CRC checksums)

[Matthias Maennich <maennich@google.com>]

Hi!

On Wed, Mar 17, 2021 at 11:29:13PM +0000, Giuliano Procida wrote:
>Hi Dodji.
>
>Not sure if Matthias would say exactly the same things.
>
>On Wed, 17 Mar 2021 at 17:13, Dodji Seketeli <dodji@seketeli.org> wrote:
>
>> Matthias Maennich <maennich@google.com> a écrit:
>>
>> > The Linux Kernel has a mechanism (MODVERSIONS) to checksum symbols based
>> > on their type. In a way similar to what libabigail does, but different.
>> > The CRC values for symbols can be extracted from the symtab either by
>> > following the __kcrctab_<symbol> entry or by using the __crc_<symbol>
>> > value directly.
>> >
>> > This patch adds support for extracting those CRC values and storing them
>> > as a property of elf_symbol. Subsequently, 'crc' gets emitted as an
>> > attribute of 'elf-symbol' in the XML representation.
>>
>> This is pretty cool btw.
>>
>> Personally, I would have tied the CRC to the decl instead of the ELF
>> symbol.  I know in the kernel land, symbols and decls are usually
>> thought of interchangeably but conceptually, they are not the same
>> thing.
>>
>> The linux kernel CRCs are computed from the types of the declarations
>> that are associated to the ELF symbol.  I am talking about the genksyms
>> machinery.  Se we are really talking about the declarations and types
>> here.  They are storing in ELF because there is no concept of decls and
>> types in ELF.  But we do have that in libabigail.  So I really think it
>> makes more sense to have this tied to decls instead of ELF symbols.
>>
>>
>I can think of a few arguments in the opposite direction:
>
>The link between ELF symbols and declarations isn't completely reliable.
>Examples here include things like strlen and friends which are declared in
>.h
>but defined in .S files. Arguably this is a bug somewhere needing a fix.
>
>I opened a libabigail bug in the last few days regarding a missing parameter
>diff. Having CRCs meant we didn't miss this and checking at the symbol
>level protects us from issues in the much more complex type difference code.
>
>CRCs are part of the module link-loader interface which is very much about
>symbol-level compatibility rather than type-level compatibility. We risk
>diverging from the kernel's notion of module compatibility if we interpret
>CRCs as anything else.
>
>A CRC can change even if the decl type does not. Internal dependencies on

That was my first thought. In fact, I can construct a case where the
same underlying decl is referenced by two elf symbols with different CRC
values: with symbol aliases.

Consider this patch to the existing modversions test case.

   | diff --git a/tests/data/test-symtab/kernel/one_of_each.c b/tests/data/test-symtab/kernel/one_of_each.c
   | index 9d461fbd0053..488dbf38d3d9 100644
   | --- a/tests/data/test-symtab/kernel/one_of_each.c
   | +++ b/tests/data/test-symtab/kernel/one_of_each.c
   | @@ -6,6 +6,9 @@ EXPORT_SYMBOL(exported_function);
   |  void exported_function_gpl(void) {}
   |  EXPORT_SYMBOL_GPL(exported_function_gpl);
   |
   | +void aliased_function(void) __attribute__ ((alias("exported_function")));
   | +EXPORT_SYMBOL_GPL(aliased_function);
   | +

It creates a symbol table layout like this

  | $ readelf -sW one_of_each.ko | egrep "__(ksy|cr).*function"
  |   0000000000000000     0 NOTYPE  LOCAL  DEFAULT    3 __ksymtab_exported_function
  |   000000000000000c     0 NOTYPE  LOCAL  DEFAULT    5 __ksymtab_exported_function_gpl
  |   0000000000000000     0 NOTYPE  LOCAL  DEFAULT    5 __ksymtab_aliased_function
  |   00000000e52d5bcf     0 NOTYPE  GLOBAL DEFAULT  ABS __crc_exported_function
  |   00000000f9cc3f69     0 NOTYPE  GLOBAL DEFAULT  ABS __crc_aliased_function
  |   00000000fda43846     0 NOTYPE  GLOBAL DEFAULT  ABS __crc_exported_function_gpl

In particular, the aliased function obviously maps to the same function
location, but received a different CRC value (probably due to having a
different name).

This creates (simplified)

   <elf-symbol name='aliased_function' type='func-type' ...  crc='0xf9cc3f69'/>
   <elf-symbol name='exported_function' type='func-type' ... alias='aliased_function' crc='0xe52d5bcf'/>
   <elf-symbol name='exported_function_gpl' type='func-type' ... crc='0xfda43846'/>

and 'aliased_function' is not directly connected to any own decl. It
delegates that to 'exported_function'. So, connecting it to a decl
would make us lose information. Without debug information we could not
even surely tell which one is the alias and which one is the original,
and the CRC would be taken rather randomly from one of them.

But we can think this further through. I usually associate the decl and
type information with DWARF and the ELF symbol information with what we
can deduct from the symbol table. For stripped binaries that do not have
debug information, we can only work with symbol information. Now, since
the CRC is part of the symbol table, it is actually not stripped off and
still retained. Hence, libabigail is able at abidw or abidiff time to
extract the CRC from stripped kernel binaries and is able to do somewhat
reasonable ABI analysis based on CRC values and other properties encoded
in the symbol table. That is actually a big win and we would likely give
up on this if we would push this to the decls. (We still could create
the decls just for this property, but this does sound like a hack and
would still not be accurate for aliases.)

>such a decl (admittedly not possible in pure C, I think) should not be
>reported
>as having changed indirectly, just because the CRC has.
>
>It's less and simpler code.

Besides everything I said above, this is the main reason I would argue
to keep this attached to the ELF symbol. Based on the new symtab reader
implementation, this is a fairly simple piece of code. ~100 lines of
code.

>
>If CRCs are ever replaced by something better, it will likely be DWARF or
>perhaps even BTF-based. At this point the argument that it's type info
>and not symbol info will be stronger. BTF would be less sensitive to changes
>than DWARF in odd cases (multidimensional arrays are flattened, for
>example).
>
>If tomorrow we build or get a hash for decls and types we'll have to
>> re-do this again, at the decl level.
>>
>> As a matter of fact, the feature already exists for type units in DWARF
>> for instance.  We just don't support that yet.  But we might have to
>> support it in the future.
>>
>> So the more I think about this, the more I think we should add an
>> "artifact_hash" property to type_or_decl_base instead of putting this
>> into the ELF symbol.
>>
>> What do you think?

I think it belongs to the elf symbol and it makes things much simpler :-)

Cheers,
Matthias

>>
>>
>It's an interesting one.
>
>
>> [...]
>>
>> Cheers,
>>
>> --
>>                 Dodji
>>
>
>Regards,
>Giuliano.