From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id C5A8A3858D37 for ; Wed, 1 Feb 2023 10:48:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C5A8A3858D37 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675248489; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=3EJlfLQiIM6NZosbNV4+XUyPCld/0vLpf2fCySQSnDc=; b=c2DcLPekeXTbaDMAHjahZo5n07tMoMB71W7lD7n6sFuYeSE+TQeUeFQHKlzpk2xybx6hoC ksNn1gVnfpqOJALCs5Nd8VH4GxJUvzfdurCvNkCsRxXf9Q5H3/n+vxTE3YFfoHJAyxMxpK o0C5XkmM+6M10sGobojX1uXQFen/T5A= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-604-iKG4M2uXNmmYNjanozxsMQ-1; Wed, 01 Feb 2023 05:48:08 -0500 X-MC-Unique: iKG4M2uXNmmYNjanozxsMQ-1 Received: by mail-qk1-f200.google.com with SMTP id u11-20020a05620a430b00b007052a66d201so11031492qko.23 for ; Wed, 01 Feb 2023 02:48:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:user-agent:message-id:date:organization:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3EJlfLQiIM6NZosbNV4+XUyPCld/0vLpf2fCySQSnDc=; b=6SMVsOXzETbHTphnzkvBTAxcp2oUI6S0YF6PWMbC1NXjhGSHwfUFDii2BVnXAYbW1Q KkDEOTVMosgwHfc1UYeUrkoFalI6MHACqqOWLspn+KoHJpVvRam3PsnLMyFQDlYa1JZY KkpLKgPmsi+IlXhvMO+/aAvEkvL5XtK1M9XdMJnGDeM1w+nJ/u/NvNFnqGrOLMPHzsXD FKGw3uX5mgxg466qBSbWA9NJpK2A7z8OqmmdsZBr8cUh9QgQp3KPZYTtFtvDwiV2K9Qh VlSflgU/hcEkBgblfRSrfbwjHW1zzFb4ajnTPUWWbLqEB+HsUKLYSYxzFSKQfYHPhFFV IR7Q== X-Gm-Message-State: AO0yUKU0Q+QYg32f4LW7t5x0Qn/GjypGmAXujhSxMQUDoh4lkL22sMvs x1NaGkq7IX84GOXOh+0o+xeg1PjOwQaKIzaDIo2llV2RHOjH9zhVReOMEuHFzUCe4JUndet1Xtk l/XADgN69KrKwFsbPIAW/ X-Received: by 2002:a05:622a:4cc:b0:3b8:385f:d72e with SMTP id q12-20020a05622a04cc00b003b8385fd72emr3298072qtx.48.1675248487708; Wed, 01 Feb 2023 02:48:07 -0800 (PST) X-Google-Smtp-Source: AK7set81ne+Tz/LIoPiMmV+3h9oIxSc4SrKhw0Kh1QJG9HXS27RhNnpEK5STzuW3FlKcCU7SInyImw== X-Received: by 2002:a05:622a:4cc:b0:3b8:385f:d72e with SMTP id q12-20020a05622a04cc00b003b8385fd72emr3298059qtx.48.1675248487448; Wed, 01 Feb 2023 02:48:07 -0800 (PST) Received: from localhost ([88.120.130.27]) by smtp.gmail.com with ESMTPSA id o4-20020a05620a0d4400b006ea7f9d8644sm11834429qkl.96.2023.02.01.02.48.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 02:48:07 -0800 (PST) Received: by localhost (Postfix, from userid 1000) id C4C75C1B74E1; Wed, 1 Feb 2023 11:48:04 +0100 (CET) From: Dodji Seketeli To: gprocida@google.com Cc: libabigail@sourceware.org Subject: [PATCH, RFC] {dwarf,elf_based}-reader,writer: Avoid duplicating corpora in corpus_group Organization: Red Hat / France X-Operating-System: AlmaLinux 9.1 X-URL: http://www.redhat.com Date: Wed, 01 Feb 2023 11:48:04 +0100 Message-ID: <878rhh8xwb.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hello Giuliano, You recently made me aware (on #libabigail IRC channel on oftc) of an issue regarding duplicated corpora in the output of "abidw --linux-tree". Thank you for the heads-up and sorry for the disagreement. Below is a patch I am proposing to address that problem. Could you please tell me if it addresses the issue at hand? If yes, I'd be happy to apply it. Otherwise, I'll get back to the black board. Thank you in advance. It's been brought to my attention on IRC that running abidw --linux-tree would result in a corpus group that duplicates every single corpus in the resulting abixml. Oops. This is because both dwarf::reader::read_corpus() and elf_based_reader::read_and_add_corpus_to_group() add the corpus to the corpus_group, and yet, the later function calls the former. So the corpus is added to the corpus_group twice. This patch ensures that elf_based_reader::read_and_add_corpus_to_group() is the only one to add the corpus to the group. It also ensures that this happens before the corpus is constructed from the debug info because that is useful for sharing types among the various corpora. Otherwise, those types are potentially duplicated in the IR of each corpus. The patch also ensures the abixml writer enforces the fact that each corpus is emitted only once. * src/abg-dwarf-reader.cc (reader::read_debug_info_into_corpus): Do not add the corpus to the group here ... * src/abg-elf-based-reader.cc (elf_based_reader::read_and_add_corpus_to_group): ... because it's already added here. But then, let's add it here /before/ reading type & symbols information into the corpus. * src/abg-writer.cc (write_context::m_emitted_corpora_set): Add new data member. (write_context::{corpus_is_emitted, record_corpus_as_emitted}): Define new member functions. (write_corpus): Invoke the new write_context::record_corpus_as_emitted here. (write_corpus_group): Ensure that each corpus is emitted only once. Signed-off-by: Dodji Seketeli --- src/abg-dwarf-reader.cc | 3 --- src/abg-elf-based-reader.cc | 4 +--- src/abg-writer.cc | 45 +++++++++++++++++++++++++++++++++++-- 3 files changed, 44 insertions(+), 8 deletions(-) diff --git a/src/abg-dwarf-reader.cc b/src/abg-dwarf-reader.cc index ce6b52d4..c67edc47 100644 --- a/src/abg-dwarf-reader.cc +++ b/src/abg-dwarf-reader.cc @@ -2121,9 +2121,6 @@ public: corpus()->set_soname(dt_soname()); corpus()->set_needed(dt_needed()); corpus()->set_architecture_name(elf_architecture()); - if (corpus_group_sptr group = corpus_group()) - group->add_corpus(corpus()); - // Set symbols information to the corpus. corpus()->set_symtab(symtab()); diff --git a/src/abg-elf-based-reader.cc b/src/abg-elf-based-reader.cc index cd7b59b6..d1d9a2df 100644 --- a/src/abg-elf-based-reader.cc +++ b/src/abg-elf-based-reader.cc @@ -92,10 +92,8 @@ ir::corpus_sptr elf_based_reader::read_and_add_corpus_to_group(ir::corpus_group& group, fe_iface::status& status) { + group.add_corpus(corpus()); ir::corpus_sptr corp = read_corpus(status); - - if (status & fe_iface::STATUS_OK) - group.add_corpus(corp); return corp; } diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 1fb067b8..9fe3dec7 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -230,7 +230,8 @@ class write_context class_tmpl_shared_ptr_map m_class_tmpl_id_map; string_elf_symbol_sptr_map_type m_fun_symbol_map; string_elf_symbol_sptr_map_type m_var_symbol_map; - unordered_set m_emitted_decls_set; + unordered_set m_emitted_decls_set; + unordered_set m_emitted_corpora_set; write_context(); @@ -818,6 +819,42 @@ public: m_emitted_decls_set.insert(irepr); } + /// Test if a corpus has already been emitted. + /// + /// A corpus is emitted if it's been recorded as having been emitted + /// by the function record_corpus_as_emitted(). + /// + /// @param corp the corpus to consider. + /// + /// @return true iff the corpus @p corp has been emitted. + bool + corpus_is_emitted(const corpus_sptr& corp) + { + if (!corp) + return false; + + if (m_emitted_corpora_set.find(corp->get_path()) + == m_emitted_corpora_set.end()) + return false; + + return true; + } + + /// Record the corpus has having been emitted. + /// + /// @param corp the corpus to consider. + void + record_corpus_as_emitted(const corpus_sptr& corp) + { + if (!corp) + return; + + const string& path = corp->get_path(); + ABG_ASSERT(!path.empty()); + + m_emitted_corpora_set.insert(path); + } + /// Get the set of types that have been emitted. /// /// @return the set of types that have been emitted. @@ -4588,6 +4625,7 @@ write_corpus(write_context& ctxt, out << "\n"; ctxt.clear_referenced_types(); + ctxt.record_corpus_as_emitted(corpus); return true; } @@ -4639,7 +4677,10 @@ std::ostream& out = ctxt.get_ostream(); group->get_corpora().begin(); c != group->get_corpora().end(); ++c) - write_corpus(ctxt, *c, get_indent_to_level(ctxt, indent, 1), true); + { + ABG_ASSERT(!ctxt.corpus_is_emitted(*c)); + write_corpus(ctxt, *c, get_indent_to_level(ctxt, indent, 1), true); + } do_indent_to_level(ctxt, indent, 0); out << "\n"; -- 2.31.1 Cheers, -- Dodji