From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <> Received: from fx304.security-mail.net (mxout.security-mail.net [85.31.212.48]) by sourceware.org (Postfix) with ESMTPS id 051DD383801A for ; Tue, 17 Aug 2021 23:34:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 051DD383801A Received: from localhost (localhost [127.0.0.1]) by fx304.security-mail.net (Postfix) with SMTP id B871962721 for ; Wed, 18 Aug 2021 01:34:12 +0200 (CEST) Received: from fx304 (localhost [127.0.0.1]) by fx304.security-mail.net (Postfix) with ESMTP id B4A16626FC for ; Wed, 18 Aug 2021 01:34:12 +0200 (CEST) X-Virus-Scanned: E-securemail Secumail-id: <1110a.611c4774.5f1fc.0> Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx304.security-mail.net (Postfix) with ESMTPS id 60F78626ED for ; Wed, 18 Aug 2021 01:34:12 +0200 (CEST) Received: by zimbra2.kalray.eu (Postfix) id 40DCE27E03AF; Wed, 18 Aug 2021 01:34:12 +0200 (CEST) Date: Wed, 18 Aug 2021 01:34:12 +0200 (CEST) From: MAILER-DAEMON@zimbra2.kalray.eu (Mail Delivery System) Subject: Undelivered Mail Returned to Sender To: gcc@gcc.gnu.org Auto-Submitted: auto-replied MIME-Version: 1.0 Content-Type: multipart/report; report-type=delivery-status; boundary="513B827E041D.1629243252/zimbra2.kalray.eu" Content-Transfer-Encoding: 8bit Message-Id: <20210817233412.40DCE27E03AF@zimbra2.kalray.eu> X-Virus-Scanned: by Secumail X-Spam-Status: No, score=-5.6 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Aug 2021 23:34:24 -0000 This is a MIME-encapsulated message. --513B827E041D.1629243252/zimbra2.kalray.eu Content-Description: Notification Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit This is the mail system at host zimbra2.kalray.eu. I'm sorry to have to inform you that your message could not be delivered to one or more recipients. It's attached below. For further assistance, please send mail to postmaster. If you do so, please include this problem report. You can delete your own text from the attached returned message. The mail system : host zimbra2.kalray.eu[192.168.40.202] said: 452 4.2.2 Over quota (in reply to end of DATA command) --513B827E041D.1629243252/zimbra2.kalray.eu Content-Description: Delivery report Content-Type: message/delivery-status Content-Transfer-Encoding: 8bit Reporting-MTA: dns; zimbra2.kalray.eu X-Postfix-Queue-ID: 513B827E041D X-Postfix-Sender: rfc822; gcc@gcc.gnu.org Arrival-Date: Fri, 13 Aug 2021 01:16:03 +0200 (CEST) Final-Recipient: rfc822; bddinechin@kalray.eu Original-Recipient: rfc822;benoit.dinechin@kalray.eu Action: failed Status: 4.2.2 Remote-MTA: dns; zimbra2.kalray.eu Diagnostic-Code: smtp; 452 4.2.2 Over quota --513B827E041D.1629243252/zimbra2.kalray.eu Content-Description: Undelivered Message Content-Type: message/rfc822 Content-Transfer-Encoding: 8bit Return-Path: Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 513B827E041D for ; Fri, 13 Aug 2021 01:16:03 +0200 (CEST) Authentication-Results: zimbra2.kalray.eu (amavisd-new); dkim=fail (1024-bit key) reason="fail (body has been altered)" header.d=gcc.gnu.org Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CTlypabSpkrF for ; Fri, 13 Aug 2021 01:16:03 +0200 (CEST) Received: from fx303.security-mail.net (mxout.security-mail.net [85.31.212.46]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 0CBE127E03AE for ; Fri, 13 Aug 2021 01:16:03 +0200 (CEST) Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) by fx303.security-mail.net (Postfix) with ESMTPS id 3569C1C3276 for ; Fri, 13 Aug 2021 01:15:59 +0200 (CEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 31722382C417 for ; Thu, 12 Aug 2021 23:15:57 +0000 (GMT) Received: from mail-ot1-x334.google.com (mail-ot1-x334.google.com [IPv6:2607:f8b0:4864:20::334]) by sourceware.org (Postfix) with ESMTPS id 52DDC384F003 for ; Thu, 12 Aug 2021 23:15:46 +0000 (GMT) Received: by mail-ot1-x334.google.com with SMTP id c2-20020a0568303482b029048bcf4c6bd9so9901112otu.8 for ; Thu, 12 Aug 2021 16:15:46 -0700 (PDT) Received: from [192.168.0.41] (97-118-104-61.hlrn.qwest.net. [97.118.104.61]) by smtp.gmail.com with ESMTPSA id 31sm224391oti.63.2021.08.12.16.15.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 12 Aug 2021 16:15:45 -0700 (PDT) X-Quarantine-ID: X-Virus-Scanned: E-securemail, by Secumail X-Spam-Status: No, score=-2.576 tagged_above=-1000 required=7.5 tests=[AB_ENVFROM_LONG_40=0.5, AB_FAKE_WORD_1=0.2, AB_IN_REPLY_TO_EXISTS=-1, AB_LONG_SUBJ_30=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-1, DKIM_VALID_AU=-0.1, FSL_RCVD_EX_4=0.01, FSL_RCVD_UT_4=0.01, HEAD_NEWS=-0.5, MISSING_MID=0.14, MM_ENVFROM_BOUNCE=1, NICE_REPLY_A=1, RCVD_IN_DNSWL_MED=-1.3, RDNS_DYNAMIC=0.363, S_FROM_GREY_MINUS_2=-2] autolearn=disabled Authentication-Results: fx303.security-mail.net (amavisd-new); dkim=pass (1024-bit key) header.d=gcc.gnu.org Secumail-id: <9960.6115abaf.e5515.0> DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 31722382C417 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1628810157; bh=STvNmdM4DjA34ztUz4Wuv77mJtedtcje56508RRmIaQ=; h=Subject:To:References:Date:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=OpdDJKV8Ar/nJqV08jYebvylL3fRiNH6gEVS65yGO21yUhtHZp2tndL2WCyLVV6Lu 5Cp98oCByWr2ZfF04fHv20FZ08seBpyIlYMGpM0NTthnUrEkoc3bHO+gN3OA/z2oc+ 4tzIGi2qNY4LV6V+3n2+OHPJifU+ZTWMoKbXro64= X-Original-To: gcc@gcc.gnu.org Delivered-To: gcc@gcc.gnu.org DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 52DDC384F003 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=STvNmdM4DjA34ztUz4Wuv77mJtedtcje56508RRmIaQ=; b=UyQYGabBgaB3yCvn8GJOz6Ae6rBjYCaPxwKSn4jX8oPjNKdNEhMttNziiRGyrQ6K18 TBg9o26Zewie48hhjiBL3twl/rvDzqMgFe+CUGST9VctmJjDo0oz/w7rvnbJ0rmL3+9T pyTELV3XFrr/tojc0/zy//D2b3nYDho7Hdd9D9p7+xdk7SoO3FrrvqMNBg5CJvsV1epX Ezu4SyX9Wd9ZgvwHUbXFV2JEN2Oi82VMt9JpkBIvgKQnW13ASBhQaH+09OYcOosUCrpr 1N/W/G7VYRQhgO1Mn8ot0oMeHLDsK88185EYIdvWmE9uzg/O9dtmXSpm/+JDlkwdY2X/ pjBA== X-Gm-Message-State: AOAM531yDrfo3fOn0XhKNCfSpY2nnRV7epK0ba8+Wy5in2aIxjqQO4pf 9v2VO4v4n3QFacO7ksWC6WW3kYV4aMA= X-Google-Smtp-Source: ABdhPJyyPvFCKMbwpC2TIsKiESeGw/oie+Lj5RHlB/n7vbROP3ribJbaE7Y7Hg2XuLuNY6jIKFNlng== X-Received: by 2002:a9d:7d90:: with SMTP id j16mr3350547otn.186.1628810145490; Thu, 12 Aug 2021 16:15:45 -0700 (PDT) Subject: Re: 'hash_map>' To: Thomas Schwinge , gcc@gcc.gnu.org References: <87r1f6qzmx.fsf@euler.schwinge.homeip.net> Message-ID: Date: Thu, 12 Aug 2021 17:15:44 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: <87r1f6qzmx.fsf@euler.schwinge.homeip.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-us Content-Transfer-Encoding: 8bit X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Martin Sebor via Gcc Reply-To: Martin Sebor Errors-To: gcc-bounces+benoit.dinechin=kalray.eu@gcc.gnu.org Sender: Gcc X-ALTERMIMEV2_in: done On 8/6/21 10:57 AM, Thomas Schwinge wrote: > Hi! > > So I'm trying to do some C++... ;-) > > Given: > > /* A map from SSA names or var decls to record fields. */ > typedef hash_map field_map_t; > > /* For each propagation record type, this is a map from SSA names or var decls > to propagate, to the field in the record type that should be used for > transmission and reception. */ > typedef hash_map record_field_map_t; > > Thus, that's a 'hash_map>'. (I may do that, > right?) Looking through GCC implementation files, very most of all uses > of 'hash_map' boil down to pointer key ('tree', for example) and > pointer/integer value. Right. Because most GCC containers rely exclusively on GCC's own uses for testing, if your use case is novel in some way, chances are it might not work as intended in all circumstances. I've wrestled with hash_map a number of times. A use case that's close to yours (i.e., a non-trivial value type) is in cp/parser.c: see class_to_loc_map_t. (I don't remember if I tested it for leaks though. It's used to implement -Wmismatched-tags so compiling a few tests under Valgrind should show if it does leak.) > > Then: > > record_field_map_t field_map ([...]); // see below > for ([...]) > { > tree record_type = [...]; > [...] > bool existed; > field_map_t &fields > = field_map.get_or_insert (record_type, &existed); > gcc_checking_assert (!existed); > [...] > for ([...]) > fields.put ([...], [...]); > [...] > } > [stuff that looks up elements from 'field_map'] > field_map.empty (); > > This generally works. > > If I instantiate 'record_field_map_t field_map (40);', Valgrind is happy. > If however I instantiate 'record_field_map_t field_map (13);' (where '13' > would be the default for 'hash_map'), Valgrind complains: > > 2,080 bytes in 10 blocks are definitely lost in loss record 828 of 876 > at 0x483DD99: calloc (vg_replace_malloc.c:762) > by 0x175F010: xcalloc (xmalloc.c:162) > by 0xAF4A2C: hash_table, tree_node*> >::hash_entry, false, xcallocator>::hash_table(unsigned long, bool, bool, bool, mem_alloc_origin) (hash-table.h:275) > by 0x15E0120: hash_map, tree_node*> >::hash_map(unsigned long, bool, bool, bool) (hash-map.h:143) > by 0x15DEE87: hash_map, tree_node*> >, simple_hashmap_traits, hash_map, tree_node*> > > >::get_or_insert(tree_node* const&, bool*) (hash-map.h:205) > by 0x15DD52C: execute_omp_oacc_neuter_broadcast() (omp-oacc-neuter-broadcast.cc:1371) > [...] > > (That's with '#pragma GCC optimize "O0"' at the top of the 'gcc/*.cc' > file.) > > My suspicion was that it is due to the 'field_map' getting resized as it > incrementally grows (and '40' being big enough for that to never happen), > and somehow the non-POD (?) value objects not being properly handled > during that. Working my way a bit through 'gcc/hash-map.*' and > 'gcc/hash-table.*' (but not claiming that I understand all that, off > hand), it seems as if my theory is right: I'm able to plug this memory > leak as follows: > > --- gcc/hash-table.h > +++ gcc/hash-table.h > @@ -820,6 +820,8 @@ hash_table::expand () > { > value_type *q = find_empty_slot_for_expand (Descriptor::hash (x)); > new ((void*) q) value_type (std::move (x)); > + //BAD Descriptor::remove (x); // (doesn't make sense and) a ton of "Invalid read [...] inside a block of size [...] free'd" > + x.~value_type (); //GOOD This seems to work! -- but does it make sense? > } > > p++; > > However, that doesn't exactly look like a correct fix, does it? I'd > expect such a manual destructor call in combination with placement new > (that is being used here, obviously) -- but this is after 'std::move'? > However, this also survives a smoke-test-like run of parts of the GCC > testsuite, bootstrap and complete run now ongoing. If explicitly calling the dtor on the moved object is the right thing to do I'd expect to see such invocations elsewhere in hash_table but I don't. It does seem like removed elements ought to be destroyed, but it also seems like the destruction should go through some traits class (e.g., call Descriptor::remove and mark_deleted or do some similar dance), and be called from other member functions that move elements. Martin > > > Grüße > Thomas > ----------------- > Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955 > To declare a filtering error, please use the following link : https://www.security-mail.net/reporter.php?mid=9960.6115abaf.e5515.0&r=benoit.dinechin%40kalray.eu&s=gcc-bounces%2Bbenoit.dinechin%3Dkalray.eu%40gcc.gnu.org&o=Re%3A+%27hash_map%3Ctree%2C+hash_map%3Ctree%2C+tree%3E%3E%27&verdict=C&c=5ce25cdf0f1a5a418e600a7ee6f6fd6fdcca4695 --513B827E041D.1629243252/zimbra2.kalray.eu--