What is the point of IPC_PRIVATE of shmget?

What is the point of IPC_PRIVATE of shmget?

[Peng Yu]

Hi,

I don't get the point of IPC_PRIVATE of shmget(). Since it is just
used by the current process, why not just use malloc?

Can anybody give a real example in which IPC_PRIVATE must be used, but
 malloc or other variant of *alloc functions are not appropriate to
use? Thanks.

-- 
Regards,
Peng

Re: What is the point of IPC_PRIVATE of shmget?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

On Mon, Apr 12, 2021 at 10:54:20AM -0500, Peng Yu via Libc-help wrote:
> Hi,
> 
> I don't get the point of IPC_PRIVATE of shmget(). Since it is just
> used by the current process, why not just use malloc?
> 
> Can anybody give a real example in which IPC_PRIVATE must be used, but
>  malloc or other variant of *alloc functions are not appropriate to
> use? Thanks.

Typically you get a big chunk of memory via some cousin of shmget (it is,
as far as I know, rather mmap or sbrk) and you deal out small pieces of
it via malloc ("big" and "small" here are somewhat flexible terms).

With an allocator like malloc you don't want to incur the cost of a
system call.

See the mallopt(3) man page to get a rough idea on how to tune the
interplay of malloc and the underlying "big chunk" [1] provider.

Cheers

[1] Such a big chunk is often called "arena" in allocator parlance.

-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: What is the point of IPC_PRIVATE of shmget?

[Florian Weimer]

* Peng Yu via Libc-help:

> I don't get the point of IPC_PRIVATE of shmget(). Since it is just
> used by the current process, why not just use malloc?
>
> Can anybody give a real example in which IPC_PRIVATE must be used, but
>  malloc or other variant of *alloc functions are not appropriate to
> use? Thanks.

shmget(2) says this:

| BUGS
|        The name choice IPC_PRIVATE was perhaps unfortunate, IPC_NEW
|        would more clearly show its function.

Does this answer your question?

Thanks,
Florian

Re: What is the point of IPC_PRIVATE of shmget?

[Peng Yu]

On Mon, Apr 12, 2021 at 11:07 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Peng Yu via Libc-help:
>
> > I don't get the point of IPC_PRIVATE of shmget(). Since it is just
> > used by the current process, why not just use malloc?
> >
> > Can anybody give a real example in which IPC_PRIVATE must be used, but
> >  malloc or other variant of *alloc functions are not appropriate to
> > use? Thanks.
>
> shmget(2) says this:
>
> | BUGS
> |        The name choice IPC_PRIVATE was perhaps unfortunate, IPC_NEW
> |        would more clearly show its function.
>
> Does this answer your question?

No. That is just a naming issue. It is still privately used by the
current process instead of other processes.

-- 
Regards,
Peng

Re: What is the point of IPC_PRIVATE of shmget?

[Florian Weimer]

* Peng Yu:

> On Mon, Apr 12, 2021 at 11:07 AM Florian Weimer <fweimer@redhat.com> wrote:
>>
>> * Peng Yu via Libc-help:
>>
>> > I don't get the point of IPC_PRIVATE of shmget(). Since it is just
>> > used by the current process, why not just use malloc?
>> >
>> > Can anybody give a real example in which IPC_PRIVATE must be used, but
>> >  malloc or other variant of *alloc functions are not appropriate to
>> > use? Thanks.
>>
>> shmget(2) says this:
>>
>> | BUGS
>> |        The name choice IPC_PRIVATE was perhaps unfortunate, IPC_NEW
>> |        would more clearly show its function.
>>
>> Does this answer your question?
>
> No. That is just a naming issue. It is still privately used by the
> current process instead of other processes.

Oh, it can be shared if the process forks.  With malloc, you'd lose the
sharing.

Thanks,
Florian

Re: What is the point of IPC_PRIVATE of shmget?

[Florian Weimer]

* Florian Weimer:

> * Peng Yu:
>
>> On Mon, Apr 12, 2021 at 11:07 AM Florian Weimer <fweimer@redhat.com> wrote:
>>>
>>> * Peng Yu via Libc-help:
>>>
>>> > I don't get the point of IPC_PRIVATE of shmget(). Since it is just
>>> > used by the current process, why not just use malloc?
>>> >
>>> > Can anybody give a real example in which IPC_PRIVATE must be used, but
>>> >  malloc or other variant of *alloc functions are not appropriate to
>>> > use? Thanks.
>>>
>>> shmget(2) says this:
>>>
>>> | BUGS
>>> |        The name choice IPC_PRIVATE was perhaps unfortunate, IPC_NEW
>>> |        would more clearly show its function.
>>>
>>> Does this answer your question?
>>
>> No. That is just a naming issue. It is still privately used by the
>> current process instead of other processes.
>
> Oh, it can be shared if the process forks.  With malloc, you'd lose the
> sharing.

There is also this part:

| RETURN VALUE
|        On success, a valid shared memory identifier is returned.  On
|        error, -1 is returned, and errno is set to indicate the error.

That is, IPC_PRIVATE provides the identifier for future invocations.

Florian

Re: What is the point of IPC_PRIVATE of shmget?

[Peng Yu]

> Oh, it can be shared if the process forks.  With malloc, you'd lose the
> sharing.

OK. This makes much more sense now.

-- 
Regards,
Peng

Re: What is the point of IPC_PRIVATE of shmget?

[Manfred]

On 4/12/2021 6:14 PM, Florian Weimer via Libc-help wrote:
> * Peng Yu:
> 
>> On Mon, Apr 12, 2021 at 11:07 AM Florian Weimer <fweimer@redhat.com> wrote:
>>>
>>> * Peng Yu via Libc-help:
>>>
>>>> I don't get the point of IPC_PRIVATE of shmget(). Since it is just
>>>> used by the current process, why not just use malloc?
>>>>
>>>> Can anybody give a real example in which IPC_PRIVATE must be used, but
>>>>   malloc or other variant of *alloc functions are not appropriate to
>>>> use? Thanks.
>>>
>>> shmget(2) says this:
>>>
>>> | BUGS
>>> |        The name choice IPC_PRIVATE was perhaps unfortunate, IPC_NEW
>>> |        would more clearly show its function.
>>>
>>> Does this answer your question?
>>
>> No. That is just a naming issue. It is still privately used by the
>> current process instead of other processes.
> 
> Oh, it can be shared if the process forks.  With malloc, you'd lose the
> sharing.

Out of curiosity,
Does it need to fork? Can the identifier be shared in some other way?

Thanks,
Manfred

> 
> Thanks,
> Florian
> 

Re: What is the point of IPC_PRIVATE of shmget?

[J Lumby]

For the (common) case of parent process which forks children and wants 
all to be able to access common memory,  another alternative is to use 
mmap.    postgresql used to use shmget but since V9.3 has switched most 
shared memory areas to mmap.   Here is why:

from src/backend/port/sysv_shmem.c

  * This works around the problem that many
  * systems have very low limits on the amount of System V shared memory
  * that can be allocated.  Even a limit of a few megabytes will be enough
  * to run many copies of PostgreSQL without needing to adjust system 
settings.

Cheers,   John

On 4/12/21 1:23 PM, Peng Yu via Libc-help wrote:
>> Oh, it can be shared if the process forks.  With malloc, you'd lose the
>> sharing.
> OK. This makes much more sense now.
>

Re: What is the point of IPC_PRIVATE of shmget?

[Peng Yu]

On Mon, Apr 12, 2021 at 1:59 PM Manfred via Libc-help
<libc-help@sourceware.org> wrote:
>
>
>
> On 4/12/2021 6:14 PM, Florian Weimer via Libc-help wrote:
> > * Peng Yu:
> >
> >> On Mon, Apr 12, 2021 at 11:07 AM Florian Weimer <fweimer@redhat.com> wrote:
> >>>
> >>> * Peng Yu via Libc-help:
> >>>
> >>>> I don't get the point of IPC_PRIVATE of shmget(). Since it is just
> >>>> used by the current process, why not just use malloc?
> >>>>
> >>>> Can anybody give a real example in which IPC_PRIVATE must be used, but
> >>>>   malloc or other variant of *alloc functions are not appropriate to
> >>>> use? Thanks.
> >>>
> >>> shmget(2) says this:
> >>>
> >>> | BUGS
> >>> |        The name choice IPC_PRIVATE was perhaps unfortunate, IPC_NEW
> >>> |        would more clearly show its function.
> >>>
> >>> Does this answer your question?
> >>
> >> No. That is just a naming issue. It is still privately used by the
> >> current process instead of other processes.
> >
> > Oh, it can be shared if the process forks.  With malloc, you'd lose the
> > sharing.
>
> Out of curiosity,
> Does it need to fork? Can the identifier be shared in some other way?

I had the same question. I made a test case for it. It seems that it
doesn't matter whether fork is used or not. My test shows that as long
as another process obtains the id somehow, it can access it. Can this
conclusion be safely made?

==> main.c <==
// vim: set noexpandtab tabstop=2:

#include <stdlib.h>
#include <stdio.h>
#include <sys/shm.h>

int main(int argc, char *argv[]) {
    int shmid = atoi(argv[1]);
    char *str = shmat(shmid, NULL, 0);
    printf("%s\n",str);
    return 0;
}

==> main_create.c <==
// vim: set noexpandtab tabstop=2:

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/shm.h>

int main(int argc, char *argv[]) {
    key_t key = atoi(argv[1]);
    int shmid;
    if((shmid = shmget(key, 1024, 0666|IPC_CREAT)) == -1) {
        perror("shmget()");
        return 1;
    }
    printf("%d\n", shmid);
    char *str = shmat(shmid, NULL, 0);
    strcpy(str, argv[2]);
    return 0;
}

$ shmid=$(./main_create 0 xyz); ./main.exe "$shmid"
xyz

-- 
Regards,
Peng