From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by sourceware.org (Postfix) with ESMTPS id 1B918385801D for ; Mon, 8 Nov 2021 11:46:20 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 1B918385801D Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-511-2UEcqK9xO2SsYRz2ejUWHA-1; Mon, 08 Nov 2021 06:46:15 -0500 X-MC-Unique: 2UEcqK9xO2SsYRz2ejUWHA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 69C5D80DDEC; Mon, 8 Nov 2021 11:46:14 +0000 (UTC) Received: from tucnak.zalov.cz (unknown [10.39.192.54]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 026325F4E9; Mon, 8 Nov 2021 11:46:13 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.16.1/8.16.1) with ESMTPS id 1A8Bk5ss013034 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Mon, 8 Nov 2021 12:46:10 +0100 Received: (from jakub@localhost) by tucnak.zalov.cz (8.16.1/8.16.1/Submit) id 1A8Bk4xe013033; Mon, 8 Nov 2021 12:46:04 +0100 Date: Mon, 8 Nov 2021 12:46:04 +0100 From: Jakub Jelinek To: Iain Sandoe Cc: Richard Biener , GCC Patches Subject: Re: [PATCH 0/4] config: Allow a host to opt out of PCH. Message-ID: <20211108114604.GI2710@tucnak> Reply-To: Jakub Jelinek References: <20211104200218.24159-1-iain@sandoe.co.uk> <20211105095411.GG304296@tucnak> <20211105152515.GK304296@tucnak> MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-5.7 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Nov 2021 11:46:22 -0000 On Fri, Nov 05, 2021 at 04:37:09PM +0000, Iain Sandoe wrote: > It is hard to judge the relative effort in the two immediately visible solutions: > > 1. relocatable PCH > 2. taking the tree streamer from the modules implementation, moving its home > to c-family and adding hooks so that each FE can stream its own special trees. > > ISTM, that part of the reason people dislike PCH is because the implementation is > mixed up with the GC solution - the rendering is non-transparent etc. > > So, in some ways, (2) above would be a better investment - the process of PCH is: > generate: > “get to the end of parsing a TU” .. stream the AST > consume: > .. see a header .. stream the PCH AST in if there is one available for the header. > > There is no reason for this to be mixed into the GC solution - the read in (currently) > happens to an empty TU and there should be nothing in the AST that carries any > reference to the compiler’s executable. I'm afraid (2) is much more work though even just for C++, because handling just modules streaming and arbitrary headers is quite different. Anyway, I've rebuilt my cc1plus as PIE (and am invoking it under gdb wrapper which has ASLR disabled when building x86_64-pc-linux-gnu/bits/stdc++.h.gch/O2g.gch). With the hack patch I've posted earlier, the results are much shorter than before, in particular those scalar at messages only for ovl_op_info array and then object 0x7fffe9f6b3c0 at 0x7fffe9f6b3c8 points to executable 0x555556a2a180 object 0x7fffe9f6b3c0 at 0x7fffe9f6b3d0 points to executable 0x55555772f9b9 object 0x7fffe9f6b3a0 at 0x7fffe9f6b3a8 points to executable 0x555556a2a180 object 0x7fffe9f6b3a0 at 0x7fffe9f6b3b0 points to executable 0x55555767da08 object 0x7fffe9f6b400 at 0x7fffe9f6b408 points to executable 0x555556a2a180 object 0x7fffe9f6b400 at 0x7fffe9f6b410 points to executable 0x55555772f9d2 object 0x7fffe9f6b480 at 0x7fffe9f6b488 points to executable 0x555556a306d0 object 0x7fffe9f6b3e0 at 0x7fffe9f6b3e8 points to executable 0x555556a2a180 object 0x7fffe9f6b3e0 at 0x7fffe9f6b3f0 points to executable 0x55555772f9c0 object 0x7fffe9f6b420 at 0x7fffe9f6b428 points to executable 0x555556a2b880 object 0x7fffe9f6b440 at 0x7fffe9f6b448 points to executable 0x555556a2b7a0 object 0x7fffe9f6b460 at 0x7fffe9f6b468 points to executable 0x555556a30710 object 0x7fffe9f79168 at 0x7fffe9f791d8 points to executable 0x5555576832b9 object 0x7ffff7fca000 at 0x7ffff7fca048 points to executable 0x55555670b7d0 object 0x7ffff7fca000 at 0x7ffff7fca050 points to executable 0x5555561eb040 If I look at the unique addresses in the last column after subtracing my PIE base of 0x555555554000, they are: 0000000c97040 _Z20ggc_round_alloc_sizem 00000011b77d0 _ZL20realloc_for_line_mapPvm 00000014d6180 _Z21output_section_asm_opPKv 00000014d77a0 _ZL10emit_localP9tree_nodePKcmm 00000014d7880 _ZL15emit_tls_commonP9tree_nodePKcmm 00000014dc6d0 _ZL8emit_bssP9tree_nodePKcmm 00000014dc710 _ZL11emit_commonP9tree_nodePKcmm 0000002129a08 "\t.text" 000000212f2b9 "GNU C++17" 00000021db9b9 "\t.data" 00000021db9c0 "\t.section" 00000021db9d2 "\t.bss" For ovl_op_info array, I've mentioned that the array has: struct GTY(()) ovl_op_info_t { tree identifier; const char *name; const char *mangled_name; // And a bunch of scalar members. }; and while .name and .mangled_name are initially initialized to NULL or string literals, init_operators then (at least in my understanding not based on any command line switches and therefore probably always the same way) reorders some of the elements plus creates those identifier trees. I said I didn't know what exactly PCH does with const char * or char * members. Looking in more detail, gt_ggc_m_S clearly supports: 1) NULL 2) non-GC addresses (so most likely const literals): /* Look up the page on which the object is alloced. If it was not GC allocated, gracefully bail out. */ entry = safe_lookup_page_table_entry (p); if (!entry) return; 3) GC addresses not pointing to start of objects - here it assumes it points to STRING_CST payload and marks the STRING_CST 4) GC addresses which are starts of objects And then as can be seen in gt_pch_note_object during PCH, it only has an exception for NULL and (void *) 1, otherwise for gt_pch_p_S it remembers the pointer and uses strlen (pointer) + 1 to determine the size. While I haven't verified it yet, my understanding is that if PCH save is done and some GC object or e.g. that ovl_op_info[?][?].{,mangled_}name points to a .rodata string literal that when the PCH is saved, we actually make the .gch file not point it to the string literal in .rodata, but allocate in GC that string literal and so when PCH is loaded, they will point to some GC allocated memory containing a copy of that string literal. So, in theory ovl_op_info would be fine, my printout happens for scalars when saving the scalar data, but after that we do write_pch_globals and we have: { &ovl_op_info[0][0].identifier, 1 * (2) * (OVL_OP_MAX), sizeof (ovl_op_info[0][0]), >_ggc_mx_tree_node, >_pch_nx_tree_node }, { &ovl_op_info[0][0].name, 1 * (2) * (OVL_OP_MAX), sizeof (ovl_op_info[0][0]), (gt_pointer_walker) >_ggc_m_S, (gt_pointer_walker) >_pch_n_S }, { &ovl_op_info[0][0].mangled_name, 1 * (2) * (OVL_OP_MAX), sizeof (ovl_op_info[0][0]), (gt_pointer_walker) >_ggc_m_S, (gt_pointer_walker) >_pch_n_S }, registered which after restoring those overwrites those 3 members with something different (for the latter two with those pointers to GC allocated copies of the string literals). Sure, we could move the ovl_op_info[?][?].identifier out of the structure to another array, which would be GTY marked and this one wouldn't, or name and mangled_name could be changed from const char * to char [10] and char [12] arrays. Now, looking at the fortunately just very few other pointers into the PIE. We clearly have: typedef bool (*noswitch_section_callback) (tree decl, const char *name, unsigned HOST_WIDE_INT size, unsigned HOST_WIDE_INT rounded); struct GTY(()) noswitch_section { struct section_common common; noswitch_section_callback GTY ((skip)) callback; }; which covers 00000014d77a0 _ZL10emit_localP9tree_nodePKcmm 00000014d7880 _ZL15emit_tls_commonP9tree_nodePKcmm 00000014dc6d0 _ZL8emit_bssP9tree_nodePKcmm 00000014dc710 _ZL11emit_commonP9tree_nodePKcmm typedef void *(*line_map_realloc) (void *, size_t); typedef size_t (*line_map_round_alloc_size_func) (size_t); class GTY(()) line_maps { ... line_map_realloc reallocator; line_map_round_alloc_size_func round_alloc_size; ... }; which covers 0000000c97040 _Z20ggc_round_alloc_sizem 00000011b77d0 _ZL20realloc_for_line_mapPvm typedef void (*unnamed_section_callback) (const void *); struct GTY(()) unnamed_section { struct section_common common; unnamed_section_callback GTY ((skip)) callback; const void *GTY ((skip)) data; section *next; }; which covers 00000014d6180 _Z21output_section_asm_opPKv For the strings, I wonder about struct GTY(()) tree_translation_unit_decl { struct tree_decl_common common; const char * GTY((skip(""))) language; }; (whether we don't want to drop that GTY((skip(""))) stuff. And the remaining one is again above, the data pointers passed to get_unnnamed_section. So, if we want to make PCH work for PIEs, I'd say we can: 1) add a new GTY option, say callback, which would act like skip for non-PCH and for PCH would make us skip it but remember for address bias translation 2) drop the skip for tree_translation_unit_decl::language 3) change get_unnamed_section to have const char * as last argument instead of const void *, change unnamed_section::data also to const char * and update everything related to that 4) maybe add a host hook whether it is ok to support binaries changing addresses (the only thing I'm worried is if some host that uses function descriptors allocates them dynamically instead of having them somewhere in the executable) 5) maybe add a gengtype warning if it sees in GTY tracked structure a function pointer without that new callback option Jakub