From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 22111 invoked by alias); 16 Apr 2019 08:19:40 -0000 Mailing-List: contact cygwin-developers-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-developers-owner@cygwin.com Mail-Followup-To: cygwin-developers@cygwin.com Received: (qmail 22100 invoked by uid 89); 16 Apr 2019 08:19:39 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-1.7 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.1 spammy=membership, variations, researching, numa X-HELO: m0.truegem.net Received: from m0.truegem.net (HELO m0.truegem.net) (69.55.228.47) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 16 Apr 2019 08:19:37 +0000 Received: from localhost (mark@localhost) by m0.truegem.net (8.12.11/8.12.11) with ESMTP id x3G8JZhr093879 for ; Tue, 16 Apr 2019 01:19:36 -0700 (PDT) (envelope-from mark@maxrnd.com) Date: Tue, 16 Apr 2019 08:19:00 -0000 From: Mark Geisert To: cygwin-developers@cygwin.com Subject: Re: Implement sched_[gs]etaffinity() In-Reply-To: <20190412074533.GT4248@calimero.vinschen.de> Message-ID: References: <20190411082516.GN4248@calimero.vinschen.de> <20190411083745.GO4248@calimero.vinschen.de> <20190412074533.GT4248@calimero.vinschen.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-IsSubscribed: yes X-SW-Source: 2019-04/txt/msg00045.txt.bz2 On Fri, 12 Apr 2019, Corinna Vinschen wrote: > On Apr 11 13:52, Mark Geisert wrote: >> On Thu, 11 Apr 2019, Corinna Vinschen wrote: >>> On Apr 11 10:25, Corinna Vinschen wrote: [...] >>>> Your patch is nicely done, but what about machines with more than 64 >>>> CPUs? Your patch only uses the standard API for up to 64 CPUs, so a >>>> process can never use more than 64 CPUs or use CPUs from different CPU >>>> groups. There was also the case of this weird machine Achim Gratz once >>>> worked on, which had less than 64 CPUs but *still* used multiple CPU >>>> groups under Windows, for some reason. >>>> >>>> Any chance you could update your patch to support this functionality? >>>> For some info, see MSDN: >>>> >>>> https://docs.microsoft.com/en-us/windows/desktop/ProcThread/processor-groups >>>> >>>> Also, there's already some code in fhandler_proc.cc, function >>>> format_proc_cpuinfo to handle CPU groups. You can use the >>>> wincap.has_processor_groups() method to check if the system >>>> supports CPU groups. >>> >>> Btw., Glibc's cpu_set_t supports up to 1024 CPUs. See >>> https://sourceware.org/git/?p=glibc.git;a=blob;f=posix/bits/cpu-set.h >>> This may be ok for the foreseable future, I guess. >> >> Hi Corinna, >> I will look into CPU group support; thanks for the pointers. I also need to >> fix the assumption I made about which flavor of pid would be handed to the >> functions.. they will be Cygwin pids but need conversion to Windows pids >> internally. > > Yeah, right, I missed to notice that. I'll add a few notes inline > over @ cygwin-patches. I've updated my code locally to account for your notes on cygwin-patches; thanks! I've also spent some time researching Windows affinities vs Linux affinities and have come to some conclusions. I'm airing these for review before I start coding in earnest. I appreciate all comments from anybody interested. (1) On Linux, one deals with processor affinities using a huge mask that allows to choose from all processors on the system. On Windows, one deals with processor affinities for only the current processor group, max 64 processors in a group. This implies conversion between the two "views" when getting or setting processor affinities on Cygwin. (2) On Linux, sched_get/setaffinity() take a pid_t argument, but it can be either a process id or a thread id. If one selects a process id, the action affects just the main thread of that process. On Windows, selecting the process id affects all threads of that process. (3) For completeness, Linux's pthread_get/setaffinity_np() should probably be supplied by the proposed code too. (4) I was looking at Cygwin's fhandler_proc.cc, function format_proc_cpuinfo(). There's a call to __get_cpus_per_group() which is implemented in miscfuncs.cc. I haven't seen in the MSDN docs whether each processor group is guaranteed to have the same number of processors. I might even expect variations on a NUMA system. Anybody know if one can depend on the group membership of the first processor group to apply to all groups? (5) On Windows, a process starts out in a particular processor group. One can then change thread affinities in such a way that some threads run in a different processor group than other threads of the same process. The process becomes a "multi-group" process. This has implications for the conversions discussed in (1). (6) On Linux, processor affinity is inherited across fork() and execve(). I'll need to ensure Cygwin's implementation of those calls handle affinity the same way. So this is looking like a more substantial project :-). Thanks for reading, ..mark