From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 11376 invoked by alias); 27 Jun 2019 12:28:31 -0000 Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-owner@gcc.gnu.org Received: (qmail 11360 invoked by uid 89); 27 Jun 2019 12:28:31 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-16.9 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,SPF_PASS autolearn=ham version=3.3.1 spammy=lto_output, section_type, sk:lto_out, ltrans X-HELO: mx1.suse.de Received: from mx2.suse.de (HELO mx1.suse.de) (195.135.220.15) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 27 Jun 2019 12:28:29 +0000 Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 43DB4AEAE; Thu, 27 Jun 2019 12:28:27 +0000 (UTC) Subject: [PATCH] Add .gnu.lto_.lto section. To: Richard Biener Cc: Jan Hubicka , Thomas Koenig , Andrew Pinski , Jeff Law , GCC Development , GCC Patches References: <122f53d2-5ae0-b801-81ab-36ce69f9efda@netcologne.de> <9417a329-c71c-4682-872c-60ff3524c47e@suse.cz> <20190620120225.s7ouywid2mntmhlm@kam.mff.cuni.cz> <20190621125718.qv2m5iygqj2b43fe@kam.mff.cuni.cz> <498ec44b-60ad-6f70-3c7a-b521f9fb6c56@suse.cz> <48330aa3-d678-89a1-aa86-09e948059733@suse.cz> <06d3a9f1-2d2a-8138-99d1-c6ad4fc4c020@su se.cz> From: =?UTF-8?Q?Martin_Li=c5=a1ka?= Message-ID: <35e5f4a6-36a8-f529-81c5-20eecb6a189e@suse.cz> Date: Thu, 27 Jun 2019 12:28:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------A4EF3D9FA98F3B32386D0835" X-IsSubscribed: yes X-SW-Source: 2019-06/txt/msg00332.txt.bz2 This is a multi-part message in MIME format. --------------A4EF3D9FA98F3B32386D0835 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-length: 494 Hi. So this is a patch candidate for the .gnu.lto_.lto ELF section. As mentioned, the section contains information about bytecode version, compression algorithm used and slim LTO object flag. One minor issues is that I need to append random suffix to the section: [ 6] .gnu.lto_.lto.36e74acfb145b04b PROGBITS 0000000000000000 000086 000008 00 E 0 0 1 That's because of in WPA we load more object files and the hash is used for mapping. Is the idea of the patch right? Martin --------------A4EF3D9FA98F3B32386D0835 Content-Type: text/x-patch; name="0001-Add-.gnu.lto_.lto-section.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-Add-.gnu.lto_.lto-section.patch" Content-length: 7559 >From ec550463df26855982b7b70393766cc8e7c6cc8e Mon Sep 17 00:00:00 2001 From: Martin Liska Date: Fri, 21 Jun 2019 12:14:04 +0200 Subject: [PATCH] Add .gnu.lto_.lto section. --- gcc/lto-section-in.c | 9 +++------ gcc/lto-section-out.c | 2 -- gcc/lto-streamer-out.c | 40 +++++++++++++++++++++++++--------------- gcc/lto-streamer.h | 25 +++++++++++++++++++++---- gcc/lto/lto-common.c | 15 +++++++++++++++ 5 files changed, 64 insertions(+), 27 deletions(-) diff --git a/gcc/lto-section-in.c b/gcc/lto-section-in.c index 4cfc0cad4be..4e7d1181f23 100644 --- a/gcc/lto-section-in.c +++ b/gcc/lto-section-in.c @@ -52,10 +52,10 @@ const char *lto_section_name[LTO_N_SECTION_TYPES] = "icf", "offload_table", "mode_table", - "hsa" + "hsa", + "lto" }; - /* Hooks so that the ipa passes can call into the lto front end to get sections. */ @@ -146,7 +146,7 @@ lto_get_section_data (struct lto_file_decl_data *file_data, /* WPA->ltrans streams are not compressed with exception of function bodies and variable initializers that has been verbatim copied from earlier compilations. */ - if (!flag_ltrans || decompress) + if ((!flag_ltrans || decompress) && section_type != LTO_section_lto) { /* Create a mapping header containing the underlying data and length, and prepend this to the uncompression buffer. The uncompressed data @@ -167,9 +167,6 @@ lto_get_section_data (struct lto_file_decl_data *file_data, data = buffer.data + header_length; } - lto_check_version (((const lto_header *)data)->major_version, - ((const lto_header *)data)->minor_version, - file_data->file_name); return data; } diff --git a/gcc/lto-section-out.c b/gcc/lto-section-out.c index c91e58f0465..7ae102164ef 100644 --- a/gcc/lto-section-out.c +++ b/gcc/lto-section-out.c @@ -285,8 +285,6 @@ lto_destroy_simple_output_block (struct lto_simple_output_block *ob) /* Write the header which says how to decode the pieces of the t. */ memset (&header, 0, sizeof (struct lto_simple_header)); - header.major_version = LTO_major_version; - header.minor_version = LTO_minor_version; header.main_size = ob->main_stream->total_size; lto_write_data (&header, sizeof header); diff --git a/gcc/lto-streamer-out.c b/gcc/lto-streamer-out.c index dc68429303c..7dee770aa11 100644 --- a/gcc/lto-streamer-out.c +++ b/gcc/lto-streamer-out.c @@ -1974,10 +1974,6 @@ produce_asm (struct output_block *ob, tree fn) /* The entire header is stream computed here. */ memset (&header, 0, sizeof (struct lto_function_header)); - /* Write the header. */ - header.major_version = LTO_major_version; - header.minor_version = LTO_minor_version; - if (section_type == LTO_section_function_body) header.cfg_size = ob->cfg_stream->total_size; header.main_size = ob->main_stream->total_size; @@ -2270,10 +2266,6 @@ lto_output_toplevel_asms (void) /* The entire header stream is computed here. */ memset (&header, 0, sizeof (header)); - /* Write the header. */ - header.major_version = LTO_major_version; - header.minor_version = LTO_minor_version; - header.main_size = ob->main_stream->total_size; header.string_size = ob->string_stream->total_size; lto_write_data (&header, sizeof header); @@ -2390,6 +2382,29 @@ prune_offload_funcs (void) DECL_PRESERVE_P (fn_decl) = 1; } +/* Produce LTO section that contains global information + about LTO bytecode. */ + +static void +produce_lto_section () +{ + /* Stream LTO meta section. */ + output_block *ob = create_output_block (LTO_section_lto); + + char * section_name = lto_get_section_name (LTO_section_lto, NULL, NULL); + lto_begin_section (section_name, false); + free (section_name); + + lto_compression compression = ZLIB; + + bool slim_object = flag_generate_lto && !flag_fat_lto_objects; + lto_section s + = { LTO_major_version, LTO_minor_version, slim_object, compression, 0 }; + lto_write_data (&s, sizeof s); + lto_end_section (); + destroy_output_block (ob); +} + /* Main entry point from the pass manager. */ void @@ -2412,6 +2427,8 @@ lto_output (void) /* Initialize the streamer. */ lto_streamer_init (); + produce_lto_section (); + n_nodes = lto_symtab_encoder_size (encoder); /* Process only the functions with bodies. */ for (i = 0; i < n_nodes; i++) @@ -2827,10 +2844,6 @@ lto_write_mode_table (void) struct lto_simple_header_with_strings header; memset (&header, 0, sizeof (header)); - /* Write the header. */ - header.major_version = LTO_major_version; - header.minor_version = LTO_minor_version; - header.main_size = ob->main_stream->total_size; header.string_size = ob->string_stream->total_size; lto_write_data (&header, sizeof header); @@ -2901,9 +2914,6 @@ produce_asm_for_decls (void) lto_output_decl_state_streams (ob, fn_out_state); } - header.major_version = LTO_major_version; - header.minor_version = LTO_minor_version; - /* Currently not used. This field would allow us to preallocate the globals vector, so that it need not be resized as it is extended. */ header.num_nodes = -1; diff --git a/gcc/lto-streamer.h b/gcc/lto-streamer.h index 3b1e07b4bfc..2f481aebeb2 100644 --- a/gcc/lto-streamer.h +++ b/gcc/lto-streamer.h @@ -234,6 +234,7 @@ enum lto_section_type LTO_section_offload_table, LTO_section_mode_table, LTO_section_ipa_hsa, + LTO_section_lto, LTO_N_SECTION_TYPES /* Must be last. */ }; @@ -378,18 +379,31 @@ public: unsigned int len; }; +/* Compression algorithm used for compression of LTO bytecode. */ -/* The is the first part of the record for a function or constructor - in the .o file. */ -struct lto_header +enum lto_compression +{ + ZLIB, + ZSTD +}; + +/* Structure that represents LTO ELF section with information + about the format. */ + +struct lto_section { int16_t major_version; int16_t minor_version; + unsigned char slim_object: 1; + lto_compression compression: 4; + int32_t reserved0: 27; }; +STATIC_ASSERT (sizeof (lto_section) == 8); + /* The is the first part of the record in an LTO file for many of the IPA passes. */ -struct lto_simple_header : lto_header +struct lto_simple_header { /* Size of main gimple body of function. */ int32_t main_size; @@ -589,6 +603,9 @@ struct GTY(()) lto_file_decl_data /* Mode translation table. */ const unsigned char *mode_table; + + /* Read LTO section. */ + lto_section lto_section_header; }; typedef struct lto_file_decl_data *lto_file_decl_data_ptr; diff --git a/gcc/lto/lto-common.c b/gcc/lto/lto-common.c index 0d38ee6b57b..1cfbda0f554 100644 --- a/gcc/lto/lto-common.c +++ b/gcc/lto/lto-common.c @@ -2060,6 +2060,21 @@ lto_file_finalize (struct lto_file_decl_data *file_data, lto_file *file) #else file_data->mode_table = lto_mode_identity_table; #endif + + /* Read and verify LTO section. */ + data = lto_get_section_data (file_data, LTO_section_lto, NULL, &len, false); + if (data == NULL) + { + fatal_error (input_location, "bytecode stream in file %qs generated " + "with GCC compiler older than 10.0", file_data->file_name); + return; + } + + file_data->lto_section_header = *(const lto_section *)data; + lto_check_version (file_data->lto_section_header.major_version, + file_data->lto_section_header.minor_version, + file_data->file_name); + data = lto_get_section_data (file_data, LTO_section_decls, NULL, &len); if (data == NULL) { -- 2.21.0 --------------A4EF3D9FA98F3B32386D0835--