From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from omta002.cacentral1.a.cloudfilter.net (omta002.cacentral1.a.cloudfilter.net [3.97.99.33]) by sourceware.org (Postfix) with ESMTPS id 0C1363858CD1 for ; Sat, 2 Sep 2023 19:59:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0C1363858CD1 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=Shaw.ca Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=shaw.ca Received: from shw-obgw-4002a.ext.cloudfilter.net ([10.228.9.250]) by cmsmtp with ESMTP id cSMQqHF7r6NwhcWlqqGqpJ; Sat, 02 Sep 2023 19:59:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=shaw.ca; s=s20180605; t=1693684742; bh=8mc1JgNXvlOebx71tUe4n1KYmlot6i+PhjDOw0KnyLg=; h=Date:Reply-To:Subject:To:References:From:Cc:In-Reply-To; b=mKfzC/fQb5jm6jpWtY1cZIuPWUnmv6hSqcfEcvJ5xskMPkaZoeNHTOOxzSV5Wbo0J Znd9/ZI6LI/c2Bx/viqtVWpUHZiDGrl2YtGxbPG7GEJT9v2PtGXPxbUrwpC3BeoIjn ri6oEE/20blszCUs7UICByL7vlOAWI0EPeZaTmQiQmu6ZhvoF0sc12HidSyYm7VSw+ vXr+8yHnM+khH420MN9HaQ44nu+9sGi+xa0Yv1Bimdlqd/ztuP2QmZKZGGzFhJ1jFk qeyNarECW01UZtDcJ1ILGoMp1TurZhUX3G/UCQKslUz8JfCHCEAaOXuf3kfXXnO849 i4gqJpljAKmlw== Received: from [10.0.0.5] ([184.64.102.149]) by cmsmtp with ESMTP id cWlpq8ZxiyAOecWlqqYfZ5; Sat, 02 Sep 2023 19:59:02 +0000 X-Authority-Analysis: v=2.4 cv=e5oV9Il/ c=1 sm=1 tr=0 ts=64f39406 a=DxHlV3/gbUaP7LOF0QAmaA==:117 a=DxHlV3/gbUaP7LOF0QAmaA==:17 a=IkcTkHD0fZMA:10 a=6whCB0ajAAAA:8 a=YbKMr47jpe0mhzbFp54A:9 a=QEXdDO2ut3YA:10 a=yGKlR3sVXOPn9h0UBopn:22 Message-ID: <07386659-68b3-a35d-1402-22684f8e5755@Shaw.ca> Date: Sat, 2 Sep 2023 13:59:01 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.15.0 Reply-To: cygwin@cygwin.com Subject: Re: posix thread scaling issue Content-Language: en-CA To: cygwin@cygwin.com References: <550e8950-8f7a-4765-b23e-57d0e710fde0@jeffunit.com> <2cfbcf8d-911f-a64b-8916-12b005c9f6f6@Shaw.ca> From: Brian Inglis Organization: Inglis Cc: jeff In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-CMAE-Envelope: MS4xfBpkrTKBTW4q0iFAOBHgGzMpj3ulU7G3AvD+N3KuZQQG1rOinpHaje29O0yKk9M/mmH6haO1kTPCAaLmZZSKpLk5LAONBt0iCJe2hCVcEkPhpSoMzYf7 8khRfAPHY6VGoZOAkAohNTJtEar0zcNqw6IiXhWZbWAwumDZYy8P0dzEjIze+6zaufGjljfKMXIWbFLGPKiA2ezZHKnhZeFsCH8= X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 2023-09-02 12:27, jeff via Cygwin wrote: > On 9/2/2023 10:56, Brian Inglis wrote: >> On 2023-09-02 08:57, jeff via Cygwin wrote: >>> I have a program that is embarrassing parallel. >>> On my older computer which has an epyc 7302 (16 cores,  32 threads) it scales >>> very well using cygwin, and fully utilized all threads. >>> On my new computer which has an epyc 7B13 (64 cores, 128 threads) it does not >>> scale very well. >>> According to the windows task manager, it only uses 74% of the cpu resources. >>> The time it takes the program to run on windows is 166 seconds. >>> Using the same hardware on a recent version of linux, I can get 100% cpu >>> utilization and the program takes 100 seconds to run. >>> I suspect there may be something in cygwin that doesn't scale well with lots >>> of posix threads. Both Windows and Cygwin support multiple processor groups, as some developers, maintainers, and users need support on such systems, and the process and thread support has been added to Cygwin. >>> I know this is a bit of an unusual situation, but you can buy a 128 core / >>> 256 thread system now. >>> Enclosed is the output of cygcheck. >>> I updated my version of cygwin to be current as of today, Sep 2 2023. >> What Windows edition and version are you running? >> For details run: >> >> $ reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" \ >> | sed '/^\s\+\.*\s/!d;/^.\{80,\}/d' >> >> Some retail editions limit you to 64 threads and that seems to be your case: >> >>     NUMBER_OF_PROCESSORS = '64' >> >> To make full use of your processors, you may have to upgrade your Windows to a >> commercial licence (and installation) of Windows 10/11 Pro for Workstations, >> enabling server features on non-server "Worskations" ~ HEDTs (High-End >> DeskTops); see: >> >> https://www.anandtech.com/show/15483/amd-threadripper-3990x-review/3 >> >> or just run Linux! >> >> Watch out for terms misused like processor == socket on some sites! >> >> Also, you have to consider these are server systems, mainly designed for VM >> not HPC (High Performance Computing) parallelism. >> >> Your older system has higher base and boost/turbo clocks 3.0-3.3GHz: your >> newer system has lower clocks 2.25-2.65/3/3.5GHz which seems to depend on OEM >> target. >> >> You may also need to upgrade your memory, as each core could run ~10GB/s >> instructions, and these workstations are often provisioned with 128-256GB >> (2-4GB/core), so that may also need a Windows edition upgrade. > I am running windows 10 professional. Using the task manager, 64 cores and 128 > threads shows up for my processor. As the linked AnandTech article shows and explains with Task Manager/ Performance tab, Win 10 Pro may think you have dual sockets, that limits the maximum thread parallelism you can achieve: "Now the thing is, Workstation and Enterprise are built with multiple processor groups in mind, whereas Pro is not." > Here is the output of your regex: >     SystemRoot    REG_SZ    C:\Windows >     BaseBuildRevisionNumber    REG_DWORD    0x1 >     BuildBranch    REG_SZ    vb_release >     BuildGUID    REG_SZ    ffffffff-ffff-ffff-ffff-ffffffffffff >     BuildLab    REG_SZ    19041.vb_release.191206-1406 >     BuildLabEx    REG_SZ    19041.1.amd64fre.vb_release.191206-1406 >     CompositionEditionID    REG_SZ    Enterprise >     CurrentBuild    REG_SZ    19045 >     CurrentBuildNumber    REG_SZ    19045 >     CurrentMajorVersionNumber    REG_DWORD    0xa >     CurrentMinorVersionNumber    REG_DWORD    0x0 >     CurrentType    REG_SZ    Multiprocessor Free >     CurrentVersion    REG_SZ    6.3 >     EditionID    REG_SZ    Professional >     EditionSubManufacturer    REG_SZ >     EditionSubstring    REG_SZ >     EditionSubVersion    REG_SZ >     InstallationType    REG_SZ    Client >     InstallDate    REG_DWORD    0x61e2300a >     ProductName    REG_SZ    Windows 10 Pro >     ReleaseId    REG_SZ    2009 >     SoftwareType    REG_SZ    System >     UBR    REG_DWORD    0xcfc >     PathName    REG_SZ    C:\Windows >     ProductId    REG_SZ    00330-80000-00000-AA073 >     DisplayVersion    REG_SZ    22H2 >     RegisteredOwner    REG_SZ    jdeifik >     RegisteredOrganization    REG_SZ >     InstallTime    REG_QWORD    0x1d809b6d4ce7b09 > > In practice, but the new and old processors typically run at about 3ghz when > under load. > When idling, both processors use about the same amount of power. > > I have 128gb of ram, in 4 slots. Using that configuration, I can get 100% load > and significant faster performance on linux. > Therefore I conclude the issue is either with windows or cygwin, and is not a > hardware issue. > > When I run cinebench, I can get to 100% cpu utulization (at around 3ghz) on > windows. Chances are the benchmark is designed to handle that: "When the program is running inside the group, unless it is processor group aware, then it can only access other threads in the same group. This means that if a multi-threaded program can use 128 threads, if it isn’t built with processor groups in mind, then it might only spawn with access to 64." I also do not know how you would program for that in Cygwin to map onto the equivalent Windows function required. Perhaps one of the developers involved could comment here? > As for what the processors are 'designed' for, I really don't care. > I want a reliable, fast computer with ECC memory, and I can get that with an > EPYC processor. > If a workload needs more than 128gb of memory, you pretty much need to use > server processors. > I can put in up to 2tb of memory in my system, if I have the need for that. As I suggested above, and as the AT tests suggest, with your configuration, you may get better results disabling multithreading on your current system, or running Pro for Workstations, which you may be able to test using a generic key. Pro for Workstations is used and recommended by video shops, with much lower costs and power consumption running AMD than Intel, as a designer's workstation alternative getting better performance and reponsiveness than using servers for the same task. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry