From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-il1-x12f.google.com (mail-il1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) by sourceware.org (Postfix) with ESMTPS id 22B403858CDA for ; Sat, 2 Sep 2023 20:04:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 22B403858CDA Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=jeffunit.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=avicarmi.com Received: by mail-il1-x12f.google.com with SMTP id e9e14a558f8ab-34ca22e3b3dso994395ab.3 for ; Sat, 02 Sep 2023 13:04:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jeffunit-com.20230601.gappssmtp.com; s=20230601; t=1693685052; x=1694289852; darn=cygwin.com; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:to:subject:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=AjZN+g9kL5AwgCQBhaGqQ85XYjqqW7hpzFz5AYHuOyk=; b=Ifhoti16N5uPtLVbhvFiM6if7aDLoHLLsz5mhZY5XdP+XjLZZcPbuQFLHMsJOu+gm8 yBfnSSCHtaIi02h0G6v5RM63g17DdxhW0ocqK0zSObj03ASUKla8kd40Pl7qqXrRniEw Dln/AADW3whLNNniJsbLFNxJnyfkqDoC0Fwnjzb9knBdbCm1hBorivzL6PiPS/d6jxP3 86cSfvmm9fQEuEK603LhtQWyql0BpB+gU97pDyABA6FK+aGYJHmSwRTV+i8XixwDE8Ma ieFV7D6/gAhGL8uYNGYi0mFw/ry3rNCHJSuX/QSd22oIHoAcWz0yzKEpq6rBweiQRZEq Kd3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693685052; x=1694289852; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:to:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AjZN+g9kL5AwgCQBhaGqQ85XYjqqW7hpzFz5AYHuOyk=; b=e+pd6Lc7OUAjD/fi7y5DWoUM1mYmz03WWdmhulchjTinn40NO3bWSSqK5bORxBR8hF 87T0Y/Hj2Vhn8SPQ9QRlTrW/QQssTZw+Bj69gP7Mtj3mebNNGIhjdvE92YpP9HfyDzna zXEV5HJJGAa0nv4fQH4GJR5KcElLEmZCDnpV84uIBzk084BvgAOyoJ0O5QKEa24rhLYh 2Vy/kA6WUsn81UONge+fw/ifIcEknqj5OpSZgzjAGqHd+zEkRw62H+56vl3CPS6s91Jq TgxNNvsmU6eGWmy8uD/mhOEIR8ikIXaLO2T8Z64A4k2qp65O0YNOqYuvNlU0w30BxHFy d0EQ== X-Gm-Message-State: AOJu0YxYOkaDUtEA9IY60EeTCMivfNJNE3LGH88yiNQ2zKpTdfkHU1QM uiu7GsnKG2Ub3R1jA8rjpN3OdGVc/fHdUXO52Yg= X-Google-Smtp-Source: AGHT+IGbBum8hfVjFUpFxpbUVK2NAI63FopKwAW/OmXnmF559pV+jErL9F8A7L4BeKb3+kRbXFu5Zw== X-Received: by 2002:a05:6e02:11b0:b0:346:d51:9922 with SMTP id 16-20020a056e0211b000b003460d519922mr5820597ilj.13.1693685052147; Sat, 02 Sep 2023 13:04:12 -0700 (PDT) Received: from ?IPV6:2603:8001:833f:6440:2008:c9bc:a358:68c? (2603-8001-833f-6440-2008-c9bc-a358-068c.res6.spectrum.com. [2603:8001:833f:6440:2008:c9bc:a358:68c]) by smtp.gmail.com with ESMTPSA id v1-20020a17090abb8100b00273744e6eccsm607584pjr.12.2023.09.02.13.04.11 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 02 Sep 2023 13:04:11 -0700 (PDT) Message-ID: <076456ca-2c5d-4bb3-a64b-97ed4bdd26ce@jeffunit.com> Date: Sat, 2 Sep 2023 13:04:10 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: posix thread scaling issue To: cygwin@cygwin.com References: <550e8950-8f7a-4765-b23e-57d0e710fde0@jeffunit.com> <2cfbcf8d-911f-a64b-8916-12b005c9f6f6@Shaw.ca> <07386659-68b3-a35d-1402-22684f8e5755@Shaw.ca> Content-Language: en-US From: jeff Autocrypt: addr=jeff@jeffunit.com; keydata= xsFNBFFKfi0BEACrMpYE2LikN1PK/gcaqARn7RYzGX6MNgwYwzkNJH9kO3EUp+zWlsiPkSJs iojDBWuel1KySD1YPU1a3D4j5IsDlsuL1qAfU2pWszdR4m5wQtYNSMBTcWbAzeTuadcsuDsp 7NqjA+J2sbOJ5p1VUW+lsdqVaR3xYhAemyyFouqsCyvsbghkmp+/MHLUaFFGETyrDBYhzyrF N2ZDUpsVsnSiMEWToC2TR6kdXnKOnTcNteyUR6QbpXopPj7oTrIqodetrqwH0Tx/zr4CvURG g5XR7D5yGEbUEt4WIBBFNRD1yNdUYb1iqcpwZ2CD5AfG7Dh0/I3i5ECSmMDmFYyLh41IElvy MELxK8DpMxCw42vcyPcAvWDr9F5AoMQTxj8SNa42WAHicp/vqa0d+OF7IYMBz0BrN2Ry3NzD mmMMdSw+tY7eqjFUC9KEIbUlryIshZEMbAHfAmMUXVrxTYDf+48ZhBNbAuPcca5Sd/qhfSF2 8+KL7UY79/wZ0VCuuQbV4cjCj1wu97sVVwcXwV4hgdfa17TF476pc7ZjGLpK4sXKRlANSnd5 AIhMx9MV8tf1lOt8GIWE07X2iufoMvXG8GkRQ7qX+VIdAmox0FackOnBWzJYB1l+eHTWQpA5 gAm6eJ7jmiw8+RS+0lzMYD3iKgaWWMRmTa8TYySIZ6xNvjxd9wARAQABzR9KZWZmIERlaWZp ayA8amVmZkBqZWZmdW5pdC5jb20+wsF3BBMBAgAhBQJRSn4tAhsDBgsJCAcDAgYVCAIJCgsD FgIBAh4BAheAAAoJECzT/SfdIo+7segP/iqmAM6JdnQz+S6gp4xOCDpUcaq+FzlRaLBfb7H2 KCjFAaPt5T1LdAHC8OM3TS05qfLKX2XIk0jKhC43LSiKNkbOeMKNUG2TuNNjqSY2ANe+pJan POsxbcsASFYD/wqs+K6yxbr2MUgrTTA38lmlrmi9QoyLGH/SI/Y/M0e2PWjhCL/Zf6RwZQOt kBEvMeNKxi8kfPmB1gNHVfVSfpj6RqD7P/NPAjENTb/ZzSeoB2p0CLry1HyvtNHax78xi2jq TOyS6YVAgDtTkj3hIS7eEOjHavitKb92gdTt0YdF0kn6RcsOA/JBIDodYb0UQ9tbIn9+tlkC kwo70WHvchaORj/TJpnXG0GywkRbQbb3kV8Py1IXLtHadlYJ1sKuwPXIplg1s0Jk4S4yattC vRFzt15jkZoovLtITkAKUp/9jDeWD4JXJ+QUHqVK9Zesl3eo1S53qL8VLG60O3rJt1uWkn4f hD5IqOxzzC0xxMJUSyPYC5GAxvL1bZDldzHC5i45341Z17t24MbqQ+35LjOpDvowIXSZxuoT d/CHK+HbQPDXBn+gSHUfTO9YkJk+32AZOaQG1gh/UK4iaj2pXTxmS4UkwuGpCezfhppGnATl tTegn2IlZkdcRogtEg291mmZWfEHZUvPTxshUnYpxIRa5CpCJOJrGnPuOyn55URD/J2dzsFN BFFKfi0BEAC1Qa8OZwNFFJpP9JkTg7bNtnRwOVUH15tQ/O5i/uH4T7k3gT9OUHLlxm1+69Jp LiNQuZJ3bOH0+zNEum7oXoMq90KhWG3abvZ5cGhjKql1pqukfID19kfMYoaLnTux/S7xTj00 Glcsq3oEZkVpB6Ro8IUSjht5ZuIRQeMdtgZ5xX2J6HGluuqIjzkzchwP0fpqgN/fjBv9AJQu oVvozl43Jqlcp6Jlr9rmLSG8jT92JESYaZyR63mdyUmNGsG4i13s9RxdMkd3CMUZFODG9lFk GJ83GUgn3iyoykOxDGf+yOSDnlyog3qY7fjs+zDt0Ty6eM3QzcAZK5z3vLScfNdbC9PmaXfB 95lDRorqw2kpPr/QcYctDWkBwoj95JK2m5xekMiblT2Whhw3vy5NP6tOefWIzBXKEHfW6b9N 3ONcMg0vizpj2xyJU0/n45QnBdObYaOI2Q+spfzdQK1CudC5m0UMeA5wRv5kdcrQuCbUjGG1 2xyYpLVH1Z2ThQ7x9wxXAnRaAAACmUm2K72sy25NdVV+rmEym7+Xk3HSWtQb87iH3EAiBA1y uqP8bhrjLGhWEhtbbB/SE0F6rLF7evIJcHrpt7t4duBfychvD3D0dcEh8fs9KWRDh8Y/PTVh HWxontkFufnG2JTbW7xc8r5I7riVON8DdVQYCW1ySGm0+QARAQABwsFfBBgBAgAJBQJRSn4t AhsMAAoJECzT/SfdIo+7EcYP/2++R3u/JtD0xRGnpaPxrHhWiBUoBc7m93E7pLVWND1KBxQd whbvQFZiKel4+t4huT3Yziu/kVFZDVkhExUTD4xKKL3MuyYe0tK3kEHGbc9xLA+btzWrjwJa vB+c/TBeZzto/+So+bga8ldpYzmhy65u2f7cKUgNfB65ber2WRjXErfwcWsnM/DpGR6eGCnm nkpFh5E51k3fovd0ArNAzQBPzcCh/28F0nEiiRAYUhIb9YvM2jVjCiTpib9VQldqWHJlL5tf 5lD2pN2WdAhFi2d5GwjcWytLTTIfeLEEf9Gp2iRFAEpXVbfiTia18O5f9R954byYke1+FgOo b2yZ5IFeBtKsQnSEF0K0ravSqYXsiVZgDCSXLWMP3J8vdhJtAvajXs4Mpa34ckRBOg2Cwd07 Eav6e1A1qk4l1wnoXijQOkXVW78aQ4CCJP4cSXPJnkbr13sCsOwzj0A1NO71Qtzf5sst3d5h QDnSPe9uLLlhuN0CsJtS77qpz7+tklLTamOl87bJXlIG6hFkYl+ihXlKz9z2nNgA/kf7nREY lWKxc6dL37LdzkRF0TH1Ga/eNlYxbkPsBwA3zSudZOjNIntxyLgOgimxeBvSuseohZ6KxYUM DbaOL/wbvdgYz5IAgBn84By+TXS5wUjNrr9m7kaZIIY2L413mUGjZgBrFhVS In-Reply-To: <07386659-68b3-a35d-1402-22684f8e5755@Shaw.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-0.2 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 9/2/2023 12:59, Brian Inglis wrote: > On 2023-09-02 12:27, jeff via Cygwin wrote: >> On 9/2/2023 10:56, Brian Inglis wrote: >>> On 2023-09-02 08:57, jeff via Cygwin wrote: >>>> I have a program that is embarrassing parallel. >>>> On my older computer which has an epyc 7302 (16 cores,  32 threads) >>>> it scales very well using cygwin, and fully utilized all threads. >>>> On my new computer which has an epyc 7B13 (64 cores, 128 threads) >>>> it does not scale very well. >>>> According to the windows task manager, it only uses 74% of the cpu >>>> resources. >>>> The time it takes the program to run on windows is 166 seconds. >>>> Using the same hardware on a recent version of linux, I can get >>>> 100% cpu utilization and the program takes 100 seconds to run. >>>> I suspect there may be something in cygwin that doesn't scale well >>>> with lots of posix threads. > > Both Windows and Cygwin support multiple processor groups, as some > developers, maintainers, and users need support on such systems, and > the process and thread support has been added to Cygwin. > >>>> I know this is a bit of an unusual situation, but you can buy a 128 >>>> core / 256 thread system now. >>>> Enclosed is the output of cygcheck. >>>> I updated my version of cygwin to be current as of today, Sep 2 2023. > >>> What Windows edition and version are you running? >>> For details run: >>> >>> $ reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows >>> NT\CurrentVersion" \ >>> | sed '/^\s\+\.*\s/!d;/^.\{80,\}/d' >>> >>> Some retail editions limit you to 64 threads and that seems to be >>> your case: >>> >>>     NUMBER_OF_PROCESSORS = '64' >>> >>> To make full use of your processors, you may have to upgrade your >>> Windows to a commercial licence (and installation) of Windows 10/11 >>> Pro for Workstations, enabling server features on non-server >>> "Worskations" ~ HEDTs (High-End DeskTops); see: >>> >>> https://www.anandtech.com/show/15483/amd-threadripper-3990x-review/3 >>> >>> or just run Linux! >>> >>> Watch out for terms misused like processor == socket on some sites! >>> >>> Also, you have to consider these are server systems, mainly designed >>> for VM not HPC (High Performance Computing) parallelism. >>> >>> Your older system has higher base and boost/turbo clocks 3.0-3.3GHz: >>> your newer system has lower clocks 2.25-2.65/3/3.5GHz which seems to >>> depend on OEM target. >>> >>> You may also need to upgrade your memory, as each core could run >>> ~10GB/s instructions, and these workstations are often provisioned >>> with 128-256GB (2-4GB/core), so that may also need a Windows edition >>> upgrade. > >> I am running windows 10 professional. Using the task manager, 64 >> cores and 128 threads shows up for my processor. > > As the linked AnandTech article shows and explains with Task Manager/ > Performance tab, Win 10 Pro may think you have dual sockets, that > limits the maximum thread parallelism you can achieve: According to the task manager, it says 'Sockets: 1'. jeff