fopen from within a pthread

fopen from within a pthread

[Martin Beynon]

Hi,

I've noticed that if I fopen a file from a program in main nothing unusual
happens. However, if do the same from a pthread my virtual memory size
(VMDATA) increases fairly dramatically (~66MB -- 14716 vs 80252).

On closer inspection, with ltrace, I see that between the fopen and the
SYS_open call there is a SYS_mmap followed immediately by SYS_munmap when
called from within a pthread, but not if called from main(..).

Is this erroneous behaviour, or can this be explained in some way?


Sample Code:

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <stddef.h>

void* thread_fn(void *args)
{
    FILE* f = fopen("afile", "w");//vmsize = 80252KB

    while (1) {
        sleep(1);
    }

    return NULL;
}

int main(int argc, char**argv)
{
    pthread_t thread;
//    FILE* f = fopen("afile", "w");//vmsize = 14716KB


    pthread_create(&thread, NULL, thread_fn, NULL);
    while (1) {
        sleep(1);
    }


    return NULL;
}

Compiled with:
gcc -o main main.c -lpthread

With ltrace -fS ./main (fopen in main):
[pid 11405] fopen("afile", "w" <unfinished ...>
[pid 11405] SYS_brk(0)  = 0xdd0000
[pid 11405] SYS_brk(0xdf1000)  = 0xdf1000
[pid 11405] SYS_open("afile", 577, 0666) = 3
[pid 11405] <... fopen resumed> )  = 0xdd0010

cat /proc/../status:
VmPeak:       14716 kB
VmSize:       14716 kB
VmLck:           0 kB
VmPin:           0 kB
VmHWM:         872 kB
VmRSS:         872 kB
VmData:        8400 kB
VmStk:         132 kB
VmExe:           4 kB
VmLib:        2040 kB
VmPTE:          36 kB
VmPMD:          12 kB
VmSwap:           0 kB

With ltrace -fS ./main (fopen within pthread):
[pid 11435] fopen("afile", "w" <unfinished ...>
[pid 11435] SYS_mmap(0, 0x8000000, 0, 0x4022) = 0x7f579d9e3000
[pid 11435] SYS_munmap(0x7f579d9e3000, 39964672) = 0
[pid 11435] SYS_munmap(0x7f57a4000000, 27144192) = 0
[pid 11435] SYS_mprotect(0x7f57a0000000, 135168, 3)  = 0
[pid 11435] SYS_open("afile", 577, 0666)  = 3
[pid 11435] <... fopen resumed> ) = 0x7f57a00008c0

cat /proc/../status:
VmPeak:      145788 kB
VmSize:       80252 kB
VmLck:           0 kB
VmPin:           0 kB
VmHWM:         872 kB
VmRSS:         872 kB
VmData:       73936 kB
VmStk:         132 kB
VmExe:           4 kB
VmLib:        2040 kB
VmPTE:          40 kB
VmPMD:          12 kB
VmSwap:           0 kB


I'm running on Xubuntu x64 16.04.04 / glibc 2.23 (ldd (Ubuntu GLIBC
2.23-0ubuntu10) 2.23) / gcc version: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9)
5.4.0 20160609 and for good measure; Linux 4.4.0-127-generic #153-Ubuntu
SMP Sat May 19 10:58:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux



Thanks,

Martin

Re: fopen from within a pthread

[Paul Pluzhnikov via libc-help]

On Sat, Jun 2, 2018 at 3:31 PM Martin Beynon <mebeyn@gmail.com> wrote:
>
> Is this erroneous behaviour, or can this be explained in some way?


There is nothing erroneous about it.

What you are observing is the creation of thread-specific arena, from
which malloc will return memory to the new thread.

GLIBC uses thread-specific arenas to avoid having to lock malloc
internal structures when multiple threads allocate and free memory.
Additional info can be found here:
https://sourceware.org/glibc/wiki/MallocInternals



-- 
Paul Pluzhnikov

Re: fopen from within a pthread

[Carlos O'Donell]

On 06/02/2018 06:48 PM, Paul Pluzhnikov via libc-help wrote:
> On Sat, Jun 2, 2018 at 3:31 PM Martin Beynon <mebeyn@gmail.com> wrote:
>>
>> Is this erroneous behaviour, or can this be explained in some way?
> 
> 
> There is nothing erroneous about it.
> 
> What you are observing is the creation of thread-specific arena, from
> which malloc will return memory to the new thread.
> 
> GLIBC uses thread-specific arenas to avoid having to lock malloc
> internal structures when multiple threads allocate and free memory.
> Additional info can be found here:
> https://sourceware.org/glibc/wiki/MallocInternals

Also note that VM size is not the same as RSS.

The new thread arena, usually ~64MiB in size on 64-bit, is only taking
up VM space, but not consuming any real memory (RSS).

Therefore this is just an accounting procedure. Real memory will only
be used as you dirty the pages.

Cheers,
Carlos.

Re: fopen from within a pthread

[Martin Beynon]

Thanks Carlos (and Paul),

I am aware of this...what I wasn't aware of was thread specific arenas.

However that said and understood...it does generate a couple of questions
in my mind:

1) Why doesn't the main program [thread] allocate a 64MB arena? In that
little test program I posted the heap is only 132KB; which leaves me to
wonder why can't the same approach be used for child threads? If I reduce
the MALLOC_ARENA_MAX to 1, the [virtual] memory consumption is the same for
both tests (fopen in thread or in main)...

2) Why can't the initial / default [thread-specific] arena size be changed?

Cheers,

Martin

On 4 June 2018 at 15:38, Carlos O'Donell <carlos@redhat.com> wrote:

> On 06/02/2018 06:48 PM, Paul Pluzhnikov via libc-help wrote:
> > On Sat, Jun 2, 2018 at 3:31 PM Martin Beynon <mebeyn@gmail.com> wrote:
> >>
> >> Is this erroneous behaviour, or can this be explained in some way?
> >
> >
> > There is nothing erroneous about it.
> >
> > What you are observing is the creation of thread-specific arena, from
> > which malloc will return memory to the new thread.
> >
> > GLIBC uses thread-specific arenas to avoid having to lock malloc
> > internal structures when multiple threads allocate and free memory.
> > Additional info can be found here:
> > https://sourceware.org/glibc/wiki/MallocInternals
>
> Also note that VM size is not the same as RSS.
>
> The new thread arena, usually ~64MiB in size on 64-bit, is only taking
> up VM space, but not consuming any real memory (RSS).
>
> Therefore this is just an accounting procedure. Real memory will only
> be used as you dirty the pages.
>
> Cheers,
> Carlos.
>
>