From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-f170.google.com (mail-lj1-f170.google.com [209.85.208.170]) by sourceware.org (Postfix) with ESMTPS id C3A893857014 for ; Thu, 22 Jul 2021 14:33:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org C3A893857014 Received: by mail-lj1-f170.google.com with SMTP id b29so7565902ljf.11 for ; Thu, 22 Jul 2021 07:33:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qYXjD3dBnghSZMl9IMl+gi2cLt2N2Gv+idIO+eG4Qrs=; b=dBScEus+pDKZ7cWY20SSq9TlwhG2BPw2Z6Y/9pSwXInTyuBzWVROO6RVVEedo3Fkh5 RGokTd+5G1uadmjddbqHhvwMIpUoLpxTtgU1zeTKc1UZEgdRyyLXo1XXsEiYRvmtBjac a6v09/2qR8/XtNZP7lnemGux6IzYZjeZRTiT/9C20ugvzxbL+aW3CeoRUdv3kh/JELal jhejO454BpVmd3nPkvW50PbwAeYyl+lK2kkDHSXwXmHqrxiqgvkdfIZ1UE+e6H7r2BSc Bmn1/O1shTegF5EELjR+oUGqy2rjd2Z93v5+DzLfBVV1AomVVS4YfMwiXR/RgWhlm8tr ER9Q== X-Gm-Message-State: AOAM532Bv6CrCRJqYAhr3HR6wzdYK+TSMaYqqLrkSvsIQmpEWkrhzLia 7moJuepMF6yVB8u5f+cO3wS2byZVD57PVg== X-Google-Smtp-Source: ABdhPJwZ6ahn1Ucz+YHgTLeUEuiSjeSwPye6i8pXCco0Uj7ktYYf0rSLS9auWMROz+gLPXNitB3bGA== X-Received: by 2002:a05:651c:983:: with SMTP id b3mr199118ljq.287.1626964383297; Thu, 22 Jul 2021 07:33:03 -0700 (PDT) Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com. [209.85.208.177]) by smtp.gmail.com with ESMTPSA id x3sm3118566lji.12.2021.07.22.07.33.03 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 22 Jul 2021 07:33:03 -0700 (PDT) Received: by mail-lj1-f177.google.com with SMTP id b16so6354302ljq.9 for ; Thu, 22 Jul 2021 07:33:03 -0700 (PDT) X-Received: by 2002:a2e:a16d:: with SMTP id u13mr226117ljl.222.1626964382645; Thu, 22 Jul 2021 07:33:02 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Erick Ochoa Date: Thu, 22 Jul 2021 16:32:51 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: tree decl stored during LGEN does not map to a symtab_node during WPA To: Richard Biener Cc: Jan Hubicka , GCC Development Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-0.8 required=5.0 tests=BAYES_00, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, KAM_ASCII_DIVIDERS, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2021 14:33:06 -0000 > > But the addresses are at LGEN time? The following is what runs at WPA time unsigned long pid = streamer_read_uhwi (&ib); unsigned long id = streamer_read_uhwi (&ib); lto_symtab_encoder_t encoder = file_data->symtab_node_encoder; cgraph_node *cnode = dyn_cast(lto_symtab_encoder_deref(encoder, id)); logger ("%s %ld %ld %p\n", cnode->name (), pid, id, cnode); > Note the nodes are actually > streamed to different instances by input_symtab, then decls are merged > (lto_symtab_merge_decls), then I think the IPA > pass summaries are read in (to different unmerged instances!), _then_ > the symtab merging process starts (lto_symtab_merge_symbols). > I think the last step eventually calls the cgraph/varpool removal hook > IPA passes registered. Ah, so what you are saying is that during the read_summary stage they will still be different, but during execute or write_optimization_summary (), will they be finally merged? I think maybe the terminology of LGEN/WPA/LTRANS should be expanded to be lgen_gen, lgen_write, lwpa_read, lwpa_exec/lwpa_write, ltrans_read, ltrans_exec? So, just to be a bit more concrete, when initializing the ipa_opt_pass_d instance one has to write functions which will be called by a parent process. Normally I see the following comments with them: generate_summary write_summary read_summary write_optimization_summary read_optimization_summary and finally there's the execute function that gets called. I am doing the following: generate_summary, /* generating pid */ write_summary /* generating id and writing pid and id */ read_summary /* reading and printing the info I told about */ write_optimization_summary /* nothing yet */ read_optimization_summary /* nothing yet */ execute /* nothing yet */ And I think these correspond to the following "LGEN/WPA/LTRANS" stages 1. lgen (multiple processes) generate_summary 2. lgen (multiple process) write_summary 3. wpa (single process) read_summary 4. wpa (single process) execute 5. wpa? (single process?) write_optimization_summary 6 ltrans (multiple processes) read_optimization_summary And you are telling me that cgraph_node and varpool_nodes will have the same address only after the beginning of the execute stage but not before that? Is the above correct? I did try printing cnode->name() during execute and it segfaulted, so perhaps those function bodies where merged to something else? Note, that some names were successfully printed out. I'm wondering, can I use the function lto_symtab_encoder_deref during execute? I think this is unlikely... because in the past I've tried to use lto_symtab_encoder_encode during generate_summary and it caused segfaults. I'll still give it a try. Perhaps this is still a bit of progress? But now I'm wondering, if I can't use lto_symtab_encoder_deref and the nodes were indeed merged, do some of the varpool_node* I saved during read_summary are pointing to random memory? How am I able to tell which ones survived? > > It might be that you need a replace hook to do what you want, I think > that for example IPA CP encodes references to global vars aka &global > as IPA_REF and those are transparently re-written. > > As said, I think it can be made work but the details, since this is the > first IPA pass needing this, can be incomplete infra-structure-wise. > > Basically you have summaries like > > 'global = _3' > > where the should eventually be implicit and the constraints > grouped into constraints generated from the respective function body > and constraints generated by call stmts (not sure here), and constraints > for global variable init. But for the above constraint the point is to > make the 'global' references from different LGEN units the same by > some means (but not streaming and comparing the actual assembler name). > I'll need some more time to read through how ipa-cp encodes references to global variables. Thanks for the suggestion! I don't really follow the paragraph that details what you think my summaries look like. I'm thinking that for global = _3 global is a variable? and _3 states that it is an SSA variable in function 1? I think that can be a possible notation. I prefer to just use integers. What do you mean by implicit? But the idea is to essentially "compile" down all variables/functions/locals/ssa/etc into integers. And have the program represented as integers and relation of integers. For example: int* a extern void foo (int* c); int main () { int b; a = &b; foo (a) // defined in a different function } Should have the following at LGEN time (more specifically write_summary) variable -> long integer encoding -------------------------------------------- abstract null -> $null_id cgraph main -> 0 cgraph foo -> 1 varpool a -> 2 tree b -> 0 x 0 // corresponds to main position 0 real arg c -> 1 x 0 // corresponds to foo position 0 Technically we can also map the other way around, we just need to know in which "table" the information is stored. (I.e., the symbol_table, the local_decl table or the ssa_table...) Then, we give them a unique id id for lgen <-> variable <-> long integer encoding -------------------------------------------------------------- $null_id <-> abstract null -> $null_id 0 <-> cgraph main -> 0 1 <-> cgraph foo -> 1 2 <-> varpool a -> 2 3 <-> tree b -> 0 x 0 4 <-> real arg c -> 1 x 0 Then we can generate the constraints 2 = &3 // a = &b 4 = 2 // parm c = a call foo The problem is that because this is happening in parallel the other partition might generate the following constraints: void foo(int *c) { c = NULL; } abstract null -> $null_id cgraph foo -> 0 formal arg c -> 0 x 0 Give the following global id: $null_id <-> abstract null -> $null_id 0 <-> cgraph foo -> 0 1 <-> formal arg c -> 0 x 0 And have the following constraint: 1 = $null_id and so if we were to merge the constraints from both partitions naively, we would get that 0 and 1 refer to different parts of the program. I am trying to get the primary ID's to match at WPA time to be something like: FROM PARTITON pid 1 0 <-> cgraph main -> 0 1 <-> cgraph foo -> 1 2 <-> varpool a -> 2 3 <-> tree b -> 0 x 0 4 <-> real arg c -> 1 x 0 2 = &3 // a = &b 4 = 2 // parm c = a call 1 FROM PARTITION pid 2 $null_id <-> abstract null -> $null_id 0 <-> cgraph foo -> 0 1 <-> formal arg c -> 0 x 0 1 = $null_id MERGED with a map back to their old PID wpa id, pid x lgen id, var, 0 <-> 1 x 0 <-> cgraph main -> 0 1 <-> 1 x 1 <-> cgraph foo -> 1 1 <-> 2 x 0 <-> cgraph foo -> 0 2 <-> 1 x 2 <-> varpool a -> 2 3 <-> 1 x 3 <-> tree b -> 0 x 0 4 <-> 1 x 4 <-> real arg c -> 1 x 0 5 <-> 2 x 1 <-> formal arg c -> 1 x 0 2 = &3 // a = &b 4 = 2 // real arg c = a call 1 // call foo 5 = $null_id // formal arg c = NULL Finally, with this information we can run points-to analysis using integers standing in for memory locations and can output a pointer pointee relationship also as integers. I don't want to go through the whole derivation (and I already omitted details and probably have made some silly mistakes here) but in the end, for example we should at least have: Pointer, pointee --------------------- 2, 3 // a may-points-to b 4, 3 // real arg c may-points-to b 2, $null_id // a may-points-to NULL 5, $null_id // formal arg c may-points-to NULL 5, 3 // formal arg c may-points-to b And we can use these numbers to map back to the gimple source. This might be inefficient and there's room for removing some redundancy, but that's kinda what I'm thinking about. > > One node is dropped and all references are adjusted. And somehow > IPA passes are notified about this _after_they have read their > summaries. > > Richard.