From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtpout2.vodafonemail.de (smtpout2.vodafonemail.de [145.253.239.133]) by sourceware.org (Postfix) with ESMTPS id 3BAAE3870858 for ; Mon, 21 Dec 2020 20:52:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 3BAAE3870858 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=nexgo.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=Stromeko@nexgo.de Received: from smtp.vodafone.de (unknown [10.2.0.33]) by smtpout2.vodafonemail.de (Postfix) with ESMTP id 60AB61230B3 for ; Mon, 21 Dec 2020 21:52:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nexgo.de; s=vfde-smtpout-mb-15sep; t=1608583947; bh=GMdSBhUI2Fx6f0KAg4oRWFyZHQH6i+FUg3nWjYH7xPw=; h=From:To:Subject:Date; b=M+NyWH4hAgczBYKPle/dV98INRKAOS82bgoLrpkANYoxgE9gap+yN+AeB/V5oAbZ0 EIRzIZ7wQuKxuT46rPk1vI3tl4ZgNr+Vdu+5bmn1CvQxD5gUmfbPyqam1j3y48Cpxi BsE2LsjYfCgUv+tP0ZTkcuhhL0SXCgn+4g7mNXOk= Received: from Gertrud (p54a0ca05.dip0.t-ipconnect.de [84.160.202.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (2048 bits)) (No client certificate requested) by smtp.vodafone.de (Postfix) with ESMTPSA id 0531014257F for ; Mon, 21 Dec 2020 20:52:27 +0000 (UTC) From: Achim Gratz To: cygwin-apps@cygwin.com Subject: Extreme slowdown due to malloc? Date: Mon, 21 Dec 2020 21:52:26 +0100 Message-ID: <87mty66fw5.fsf@Rainer.invalid> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-purgate-type: clean X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate: clean X-purgate-size: 1782 X-purgate-ID: 155817::1608583947-000007FA-CA36D124/0/0 X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-apps@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin package maintainer discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Dec 2020 20:52:30 -0000 I've been experimenting a bit with ZStandard dictionaries. The dictionary builder is probably not the most optimized piece of software and if you feed it large amounts of data it needs quite a lot of cycles. So I thought I run some of this on Cygwin since that machine is faster and has more threads than my Linux box. Unfortunately that plan shattered due to extreme slowness of the first (single-threaded) part of the dictionary builder that sets up the partial suffix array. |------+---------------+---------------| | | E3-1225v3 | E3-1276v3 | | | 4C/4T | 4C/8T | | | 3.2/3.6GHz | 3.6/4.0GHz | |------+---------------+---------------| | 100 | 00:14 / 55s | 00:23 / 126s | | 200 | 00:39 / 145s | 01:10 / 241s | | 400 | 01:12 / 266s | 01:25 / 322s | | 800 | 02:06 / 466s | 11:12 / 1245s | | 1600 | 03:57 / 872s | > 2hr | | 3200 | 08:03 / 1756s | n/a | | 6400 | 16:17 / 3581s | n/a | |------+---------------+---------------| The obvious difference is that I/O takes a lot longer on Cygwin (roughly a minute for reading all the data) and that I have an insane amount of page faults on Windows (as reported by time) vs. none on Linux. While doing that I also noticed that top shows the program taking 100% CPU in the multithreaded portion of the program, while it should show close to 800% at that time. I'm not sure if that information just isn't available on Windows or if procps-ng needs to look someplace else for that to be shown as expected. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs