From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from emagii.se (www.emagii.com [185.133.207.17]) by sourceware.org (Postfix) with ESMTPS id 3F8C53858D33 for ; Wed, 22 Feb 2023 18:24:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3F8C53858D33 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=emagii.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=emagii.com Received: from [10.175.196.145] (84-55-68-216.customers.ownit.se [84.55.68.216]) by emagii.se (Postfix) with ESMTPSA id 85D9A120224; Wed, 22 Feb 2023 19:23:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=emagii.com; s=default; t=1677090242; bh=chMHy+MScCkF6YpFK7pAoTQYp8e2CLlFbypf690Vs6o=; h=Subject:To:From; b=JtmQwR4YJy/4sWoHZexJP4KhXAzrhiwH8Yxc7z0exH06RLiUC3+BuOx9h6zk653p5 CvlYXCKnXnFW6kfC5ccCiSeRdZ0r3vw5lZtD338YLlLNfVTOK3kdgVPlOtlrGEWtzd ivVqKK4vI4d3MpxYlvxEJSMNORF7fy/RmxqfwlVM= Authentication-Results: emagii.beebytevps.io; spf=pass (sender IP is 84.55.68.216) smtp.mailfrom=binutils@emagii.com smtp.helo=[10.175.196.145] Received-SPF: pass (emagii.beebytevps.io: connection is authenticated) Message-ID: Date: Wed, 22 Feb 2023 19:23:56 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: [PATCH v5 0/10 Add support for CRC64 generation in linker Content-Language: en-US To: binutils@sourceware.org Cc: nickc@redhat.com References: <20230222161609.239928-1-binutils@emagii.com> From: Ulf Samuelsson In-Reply-To: <20230222161609.239928-1-binutils@emagii.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-PPP-Message-ID: <167709024107.2614789.14039471697662950081@localhost.localdomain> X-PPP-Vhost: emagii.com X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,SPF_HELO_FAIL,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Discovered two bugs (which I fixed locally) so any review should be for general concepts. I will modify the tests so that the CRCs will be computed on known test data. As an example: The CRC64-ECMA run over the 9 characters in the string "123456789" should generate 0x6C40DF5F0B497347. So the revised test for CRC64-ECMA will calculate over this string and store the result, which means that most of the test data can be ignored. Here I store the CRC at 0x1120, and the expected value in 0x1110.      5     1100 434f4445 deadbeef 00000000 00000000 CODE............      6     1110 4773490b 5fdf406c 00000000 00000000 GsI._.@l........      7     1120 4773490b 5fdf406c 00000000 00000000 GsI._.@l........      8     1130 31323334 35363738 3900ffff ffffffff 123456789....... They match, so this seems to work. Best Regards Ulf Samuelsson On 2023-02-22 17:15, Ulf Samuelsson via Binutils wrote: > Fifth patchset. > > +* The linker script has a new command to insert a timestamp > + TIMESTAMP > + inserts the current time (seconds since Epoch) as a 64-bit value > + > +* The linker script syntax has new commands for debugging a linker script > + DEBUG ON turns on debugging > + DEBUG OFF turns off debugging > + > +* The linker script syntax has new commands for handling CRC-32/64 calculations > + on the '.text' section > + It uses code from https://www.libcrc.org/ > + > + DIGEST "CRC32" (start, end) > + DIGEST "CRC64-ECMA" (start, end) > + DIGEST "CRC64-ISO" (start, end) > + DIGEST "CRC64-WE" (start, end) > + DIGEST POLY (size, polynome)(start, end) > + DIGEST POLYI(size, polynome)(start, end) (inversion during CRC calculation) > + DIGEST TABLE > + DIGEST SECTION "
> > ran indent on the new files. > > The *.d files in the testsuite was edited to > reduce size and to remove all '(' characters. > The new linker can link using the testsuite/ld-script/*.t > linker files, but they do not pass the 'make check-ld' > test. > > Fourth patchset. > Get rid of binaries in testsuite. > > Here is the third patchset for introducing CRC64 generation. > This has focused on supporting maintainability of the feature. > In order to do so, CRC32 generation has been introduced to > detect issues when extending the linker with another algoritm > for checking integrity of the image. > This will simplify adding other algorithms like SHA algoritms > in the future. > > In addition, the TIMESTAMP command is used to store the current time > in seconds since Epoch is stored in the image. > > The main routine for calculating CRCs has been broken down > in subroutines which allow other algorithms. > > An algorithm typically needs > *init__tab( poly ) > calc__inv( const unsigned char *input_str, size_t num_bytes ); > calc_ ( const unsigned char *input_str, size_t num_bytes ); > void lang_add__syndrome(bool invert, bfd_vma poly) > void lang_add__table(void) > > The common functions are: > * bool get_text_section_contents(void) > This reads the '.text' section and its contents > Pointers are stored in the global variables > asection *text_section > char *text_contents > * bool set_crc_checksum(bfd_vma crc, bfd_vma addr, bfd_vma size) > Stores the CRC at the index addr. The size of the CRC in bytes is specified. > Storing something larger that 64-bit is not supported here. > * bool symbol_lookup(char *name, bfd_vma *val) > Gets the value of a symbol into 'val'. Returns false if not found. > > To add CRC32 support means adding the following commands > * CRC32 CRC32 '(' crc_start ',' crc_end ')' > * CRC32 POLY '[' mustbe_exp ']' '(' crc_start ',' crc_end ')' > * CRC32 POLYI '[' mustbe_exp ']' '(' crc_start ',' crc_end ')' > > ================ > Here is the second patchset for introducing CRC64 generation in the linker > This is mainly looking at indentation and whitespace errors. > The patches applies cleanly without whitespace error > except for the last patch which contains the testsuite. > When the contents of .text is printed out, sometimes > the last byte of a line in the CRC table is a space. > This is reported as a whitespace error. > > Supports the following new linker commands: > * CRC64 ECMA '(' crc_start ',' crc_end ')' > * CRC64 ISO '(' crc_start ',' crc_end ')' > * CRC64 POLY '[' mustbe_exp ']' '(' crc_start ',' crc_end ')' > * CRC64 POLYI '[' mustbe_exp ']' '(' crc_start ',' crc_end ')' > * CRC64 TABLE > > ECMA == 0x42F0E1EBA9EA3693 > ISO == 0xD800000000000000 > POLY == Allows your own polynome > POLYI == Allows your own polynome, with inversion during calc > > The CRC is calculated from crc_start to crc_end (not included) > > The "CRC64 command > * Allows specifying the polynom (ECMA(default), ISO or your own) > * Allows for inversion in the CRC calculation (CRC64-WE) > * Allows specifying the area that should be checked. > * reserves room for the CRC (8 bytes) > * Declares a symbol ___CRC64___ for the address of the CRC. > * Declares a symbol ___CRC64_START___ for the first address of the checked area > * Declares a symbol ___CRC64_END___ for the first address after the checked area > > The symbols names are declared in "checksum.h". > If the names are unsuitable, it is easy to change. > > The CRC TABLE command > This is used to speed up the CRC calculation. > * Creates a 2kB table which speeds up the CRC calculation > * Can insert the 2kB table into the .text section at current location. > * Declares a symbol ___CRC64_TABLE___ if the table is inserted. > > Comments on the code: > > The process of calculation involves: > * Check if CRC is requested > * Validation of CRC prerequisites > * get .text section > * get .text section contents > * calculate CRC over the area > * insert the CRC in the correct location > ==================================== > > This version also supports the > * DEBUG ON > * DEBUG OFF > turning on/off yydebug > I find it useful for debugging the build, but it can easly be removed. > > This patch contains a lot of debug output, that is enabled by > #define DEBUG_CRC 1 > This should be removed in the final version. > > The ld.texi stuff needs some more work. Not very experienced with that. > > Added an entry in NEWS > > Added 4 test cases for the different CRC64 polynome commands. > All testcases generate a CRC table. > > The code is using the libcrc released under an MIT license found in > * https://www.libcrc.org/ > * https://github.com/lammertb/libcrc/tree/v2.0 > Author: Lammert Bies > A license for libcrc is added to the patch. > > [PATCH v5 01/10] TIMESTAMP: ldlang: process > [PATCH v5 02/10] DIGEST: NEWS > [PATCH v5 03/10] DIGEST: ld.texi > [PATCH v5 04/10] LIBCRC: license > [PATCH v5 05/10] DIGEST: ldlex.l > [PATCH v5 06/10] DIGEST: ldgram.y > [PATCH v5 07/10] DIGEST: Makefile.am: add new files > [PATCH v5 08/10] DIGEST: CRC-32/64 algorithms > [PATCH v5 09/10] DIGEST: ldmain.c: add CRC calculation > [PATCH v5 10/10] DIGEST: testsuite >