[Bug libgomp/64719] New: omp_get_num_procs should not account for cpu affinity

[Bug libgomp/64719] New: omp_get_num_procs should not account for cpu affinity

[geir at cray dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64719

            Bug ID: 64719
           Summary: omp_get_num_procs should not account for cpu affinity
           Product: gcc
           Version: 4.9.2
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libgomp
          Assignee: unassigned at gcc dot gnu.org
          Reporter: geir at cray dot com
                CC: jakub at gcc dot gnu.org

Created attachment 34525
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=34525&action=edit
omp.c

The description for the routine omp_get_num_procs is stated as:

  Returns the number of processors online on that device. 

The GCC implementation behavior of omp_get_num_procs does not follow this
description, but rather does the following:

  Returns the number of processors that have affinity to the calling
process/thread.

The Cray and PGI versions of omp_get_num_procs will return the actual number of
processors available on the device without regard to affinity. [I suppose there
could be some interpretation what is meant by the term "online", but then this
should be further explained in the description of omp_get_num_procs.]


Our job launcher will set affinity to optimize performance of parallel programs
(MPI, SHMEM, Gloab Arrays, UPC, Coarray Fortran, ...).  In the following
example two processes that have three OpenMP threads are being invoked:

  GCC compiled program:

$ export OMP_NUM_THREADS=3
$ aprun -n 2 -d 3 ./omp.gcc
Sysconf get nprocs reports  24 processors
Sysconf get nprocs reports  24 processors
Hello from pid 14544, thread 0 (core affinity = 0) (Available OpenMP processors
= 1)
Hello from pid 14544, thread 2 (core affinity = 2) (Available OpenMP processors
= 1)
Hello from pid 14544, thread 1 (core affinity = 1) (Available OpenMP processors
= 1)
Hello from pid 14545, thread 0 (core affinity = 3) (Available OpenMP processors
= 1)
Hello from pid 14545, thread 2 (core affinity = 5) (Available OpenMP processors
= 1)
Hello from pid 14545, thread 1 (core affinity = 4) (Available OpenMP processors
= 1)
$


  Cray compiler compiled program:

$ export OMP_NUM_THREADS=3
$ aprun -n 2 -d 3 ./omp.cray
Sysconf get nprocs reports  24 processors
Sysconf get nprocs reports  24 processors
Hello from pid 14620, thread 2 (core affinity = 2) (Available OpenMP processors
= 24)
Hello from pid 14620, thread 0 (core affinity = 0) (Available OpenMP processors
= 24)
Hello from pid 14620, thread 1 (core affinity = 1) (Available OpenMP processors
= 24)
Hello from pid 14621, thread 0 (core affinity = 3) (Available OpenMP processors
= 24)
Hello from pid 14621, thread 2 (core affinity = 5) (Available OpenMP processors
= 24)
Hello from pid 14621, thread 1 (core affinity = 4) (Available OpenMP processors
= 24)
$

   Here is the GCC compiled version with CPU affinity turned off:

$ aprun -cc none -n 2 -d 3 ./omp.gcc
Sysconf get nprocs reports  24 processors
Sysconf get nprocs reports  24 processors
Hello from pid 15044, thread 1 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15044, thread 2 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15044, thread 0 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15045, thread 2 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15045, thread 0 (core affinity = 0-23) (Available OpenMP
processors = 24)
Hello from pid 15045, thread 1 (core affinity = 0-23) (Available OpenMP
processors = 24)
$

----------------------

I realize that this bug will very likely be closed as INVALID or WONTFIX, but
the documentation needs to be fixed to make clear the behavior of
omp_get_num_procs.

[Bug libgomp/64719] omp_get_num_procs should not account for cpu affinity

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64719

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |INVALID

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
This is very much intentional.  If you confine the process to certain subset of
cores/HW threads, then normally all threads that it creates are confined that
way.  So as far as OpenMP is concerned, the other cores/HW threads are not
online for this process.