From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <fweimer@redhat.com>
Received: from us-smtp-delivery-124.mimecast.com
 (us-smtp-delivery-124.mimecast.com [216.205.24.124])
 by sourceware.org (Postfix) with ESMTP id B029A3857C60
 for <gcc@gcc.gnu.org>; Fri, 30 Apr 2021 12:50:42 +0000 (GMT)
DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org B029A3857C60
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
 [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
 us-mta-449--GM3kcPDNQehGZ1bVwe_Hg-1; Fri, 30 Apr 2021 08:50:38 -0400
X-MC-Unique: -GM3kcPDNQehGZ1bVwe_Hg-1
Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com
 [10.5.11.16])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7A62280ED8E;
 Fri, 30 Apr 2021 12:50:37 +0000 (UTC)
Received: from oldenburg.str.redhat.com (ovpn-115-124.ams2.redhat.com
 [10.36.115.124])
 by smtp.corp.redhat.com (Postfix) with ESMTPS id 422795C1A3;
 Fri, 30 Apr 2021 12:50:36 +0000 (UTC)
From: Florian Weimer <fweimer@redhat.com>
To: Richard Biener <richard.guenther@gmail.com>
Cc: Feng Xue OS via Gcc <gcc@gcc.gnu.org>,  JiangNing OS
 <jiangning@os.amperecomputing.com>
Subject: Re: [RFC] A memory gathering optimization for loop
References: <MWHPR01MB23044C033788140F0CE726EAF7A80@MWHPR01MB2304.prod.exchangelabs.com>
 <87mttgb7xy.fsf@oldenburg.str.redhat.com>
 <CAFiYyc2j+FcVwpUZ6MWJ3SqfWZVb+1cWTRRxwUpEdQtXLb9myQ@mail.gmail.com>
Date: Fri, 30 Apr 2021 14:50:49 +0200
In-Reply-To: <CAFiYyc2j+FcVwpUZ6MWJ3SqfWZVb+1cWTRRxwUpEdQtXLb9myQ@mail.gmail.com>
 (Richard Biener's message of "Fri, 30 Apr 2021 14:26:17 +0200")
Message-ID: <87bl9w542e.fsf@oldenburg.str.redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain
X-Spam-Status: No, score=-6.5 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH,
 DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW,
 RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS,
 TXREP autolearn=ham autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
 server2.sourceware.org
X-BeenThere: gcc@gcc.gnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Gcc mailing list <gcc.gcc.gnu.org>
List-Unsubscribe: <https://gcc.gnu.org/mailman/options/gcc>,
 <mailto:gcc-request@gcc.gnu.org?subject=unsubscribe>
List-Archive: <https://gcc.gnu.org/pipermail/gcc/>
List-Post: <mailto:gcc@gcc.gnu.org>
List-Help: <mailto:gcc-request@gcc.gnu.org?subject=help>
List-Subscribe: <https://gcc.gnu.org/mailman/listinfo/gcc>,
 <mailto:gcc-request@gcc.gnu.org?subject=subscribe>
X-List-Received-Date: Fri, 30 Apr 2021 12:50:44 -0000

* Richard Biener:

>> Can you change this optimization so that it can use a fixed-size buffer?
>> This would avoid all issues around calling calloc.  If iter_count can be
>> very large, allocating that much extra memory might not be beneficial
>> anyway.
>
> It would need to be TLS storage though or protected against concurrent
> access (maybe an atomic is enough if we drop to the original loop when
> concurrent access is detected).

If it can be constrained to a small window, then it can be allocated on
the stack.

But I suspect the point of the optimization is to do this data
reorganization when huge amounts of temporary memory are needed.  I
don't see a way to make this safe in general.

glibc's qsort implementation does something quite similar manually,
including the extra memory allocations, and we occasionally receive
complaints about it.

Thanks,
Florian