* GLIBC malloc behavior question
@ 2023-02-07 15:06 Nikolay Shustov
2023-02-07 16:16 ` Paulo César Pereira de Andrade
0 siblings, 1 reply; 8+ messages in thread
From: Nikolay Shustov @ 2023-02-07 15:06 UTC (permalink / raw)
To: libc-alpha
[-- Attachment #1.1: Type: text/plain, Size: 2885 bytes --]
Hi,
I have a question about the malloc() behavior which I observe.
The synopsis is that the during the stress load, the application
aggressively allocates virtual memory without any upper limit.
Just to note, after application is loaded just with the peak of activity
and goes idle, its virtual memory doesn't scale back (I do not expect
much of that though - should I?).
The application is heavily multithreaded; at its peak of its activitiy
it creates new threads and destroys them at a pace of approx. 100/second.
After the long and tedious investigation I dare to say that there are no
memory leaks involved.
(Well, there were memory leaks and I first went after those; found and
fixed - but the result did not change much.)
The application is cross-platform and runs on Windows and some other
platforms too.
There is an OS abstraction layer that provides the unified thread and
memory allocation API for business logic, but the business logic that
triggers memory allocations is platform-independent.
There are no progressive memory allocations in OS abstraction layer
which could be blamed for the memory growth.
The thing is, on Windows, for the same activity there is no such
application memory growth at all.
It allocates memory moderately and scales back after peak of activity.
This makes me think it is not the business logic to be blamed (to the
extent of that it does not leak memory).
I used valigrind to profile for memory leaks and heap usage.
Please see massif outputs attached (some callstacks had to be trimmed out).
I am also attaching the memory map for the application (run without
valgrind); snapshot is taken after all the threads but main were
destroyed and application is idle.
The pace of the virtual memory growth is not quite linear.
From my observation, it allocates a big hunk in the beginning of the
peak loading, then in some time starts to grow in steps of ~80Mb / 10
seconds, then after some times starts to steadily grow it at pace of
~2Mb/second.
Some stats from the host:
OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
ldd -version
ldd (GNU libc) 2.17
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There
is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
uname -a
Linux <skipped> 3.10.0-1160.53.1.el7.x86_64 #1 SMP Thu Dec 16
10:19:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
At a peak load, the number of application threads is ~180.
If application is left running, I did not observe it would hit any max
virtual memory threshold and eventually ends up with hitting ulimit.
My questions are:
- Is this memory growth an expected behavior?
- What can be done to prevent it from happening?
Thanks in advance,
- Nikolay
[-- Attachment #2: massif.out --]
[-- Type: text/plain, Size: 40162 bytes --]
--------------------------------------------------------------------------------
Command: mt-multiconnect-server 65000
Massif arguments: --pages-as-heap=yes
ms_print arguments: massif.out.9052
--------------------------------------------------------------------------------
GB
2.868^ #
| :::::::@:@:#
| ::::::::::::::: :::: @:@:#
| ::::::::::::@:::::: : :: :: :: :::: @:@:#
| @@::::::::::::::: :: :: ::: @: :::: : :: :: :: :::: @:@:#
| @:::@:::@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| @@@@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| @ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
| :@ @@: :@: :@:@ :: : :::: ::: : :: :: ::: @: :::: : :: :: :: :::: @:@:#
0 +----------------------------------------------------------------------->Gi
0 50.60
Number of snapshots: 54
Detailed snapshots: [2, 3, 4, 7, 10, 12, 31, 48, 50, 53 (peak)]
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
0 0 69,632 69,632 0 0
1 1,245,519,421 1,873,780,736 1,873,780,736 0 0
2 2,119,197,635 2,242,240,512 2,242,240,512 0 0
100.00% (2,242,240,512B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->95.90% (2,150,264,832B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->50.88% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->50.88% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->50.88% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->20.95% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->20.95% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| |
| ->41.90% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->41.90% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->41.90% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->08.98% (201,326,592B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->08.98% (201,326,592B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| |
| ->02.93% (65,683,456B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
<..callstacks trimmed..>
| | ->01.70% (38,150,144B) in 2 places, all below massif's threshold (1.00%)
| | |
| | ->01.23% (27,533,312B) 0x6293867: XXthread::start() (XXthread.cpp:190)
| | ->01.23% (27,533,312B) in 4 places, all below massif's threshold (1.00%)
| |
| ->00.19% (4,206,592B) in 1+ places, all below ms_print's threshold (01.00%)
|
->03.91% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->03.81% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->03.81% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->03.52% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->03.52% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->03.52% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->03.42% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->03.42% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.42% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.42% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->03.42% (76,787,712B) 0x1: ???
| | | | ->03.42% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->03.42% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.10% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.29% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.10% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.19% (4,313,088B) in 1+ places, all below ms_print's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
3 3,449,905,980 2,296,811,520 2,296,811,520 0 0
100.00% (2,296,811,520B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->96.00% (2,204,835,840B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->49.67% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->49.67% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->49.67% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->20.45% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->20.45% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| |
| ->40.91% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->40.91% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->40.91% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| |
| ->05.10% (117,157,888B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
<..callstacks trimmed..>
| | | ->02.16% (49,532,928B) in 4 places, all below massif's threshold (1.00%)
| | |
| | ->01.56% (35,778,560B) 0x6298088: XXthread_pool::set_min_threads(int) (XXthread_pool.cpp:347)
| | | ->01.56% (35,778,560B) in 3 places, all below massif's threshold (1.00%)
| | |
| | ->01.39% (31,846,400B) 0x629719C: XXthread_pool::submit(XXrunnable*) (XXthread_pool.cpp:187)
| | ->01.39% (31,846,400B) in 2 places, all below massif's threshold (1.00%)
| |
| ->00.32% (7,303,168B) in 1+ places, all below ms_print's threshold (01.00%)
|
->03.82% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->03.72% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->03.72% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->03.44% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->03.44% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->03.44% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->03.34% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->03.34% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.34% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.34% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->03.34% (76,787,712B) 0x1: ???
| | | | ->03.34% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->03.34% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.10% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.28% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.10% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.19% (4,313,088B) in 1+ places, all below ms_print's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
4 4,323,508,142 2,328,596,480 2,328,596,480 0 0
100.00% (2,328,596,480B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->96.05% (2,236,620,800B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->48.99% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->48.99% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->48.99% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->20.17% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->20.17% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| |
| ->40.35% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->40.35% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->40.35% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->08.65% (201,326,592B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->08.65% (201,326,592B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| |
| ->06.32% (147,173,376B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
<..callstacks trimmed..>
| | | ->02.69% (62,705,664B) in 4 places, all below massif's threshold (1.00%)
| | |
| | ->01.90% (44,199,936B) 0x6298088: XXthread_pool::set_min_threads(int) (XXthread_pool.cpp:347)
| | | ->01.90% (44,199,936B) in 3 places, all below massif's threshold (1.00%)
| | |
| | ->01.73% (40,267,776B) 0x629719C: XXthread_pool::submit(XXrunnable*) (XXthread_pool.cpp:187)
| | ->01.73% (40,267,776B) in 2 places, all below massif's threshold (1.00%)
| |
| ->00.39% (9,072,640B) in 1+ places, all below ms_print's threshold (01.00%)
|
->03.76% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->03.67% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->03.67% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->03.39% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->03.39% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->03.39% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->03.30% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->03.30% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.30% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.30% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->03.30% (76,787,712B) 0x1: ???
| | | | ->03.30% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->03.30% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.09% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.28% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.10% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.19% (4,313,088B) in 1+ places, all below ms_print's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
5 5,176,557,792 2,353,487,872 2,353,487,872 0 0
6 6,074,463,273 2,386,395,136 2,386,395,136 0 0
7 7,158,241,065 2,407,047,168 2,407,047,168 0 0
100.00% (2,407,047,168B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->96.17% (2,314,928,128B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->47.40% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->47.40% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->47.40% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->19.52% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->19.52% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| |
| ->39.03% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->39.03% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->39.03% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->08.36% (201,326,592B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->08.36% (201,326,592B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| |
| ->09.22% (221,941,760B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
<..callstacks trimmed..>
| | | | ->01.31% (31,580,160B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->01.31% (31,580,160B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| | ->01.19% (28,688,384B) 0x409D4D: main (mt-multiconnect-server.cpp:99)
| |
| ->00.52% (12,611,584B) in 1+ places, all below ms_print's threshold (01.00%)
|
->03.64% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->03.55% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->03.55% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->03.28% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->03.28% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->03.28% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->03.19% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->03.19% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.19% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.19% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->03.19% (76,787,712B) 0x1: ???
| | | | ->03.19% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->03.19% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.09% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.27% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.09% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.19% (4,456,448B) in 1+ places, all below ms_print's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
8 7,836,933,298 2,416,521,216 2,416,521,216 0 0
9 9,139,037,941 2,431,938,560 2,431,938,560 0 0
10 9,823,680,957 2,444,943,360 2,444,943,360 0 0
100.00% (2,444,943,360B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->96.23% (2,352,824,320B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->46.66% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->46.66% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->46.66% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->19.21% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->19.21% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| |
| ->38.43% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->38.43% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->38.43% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->08.23% (201,326,592B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->08.23% (201,326,592B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| |
| ->10.63% (259,837,952B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
<..callstacks trimmed..>
| | | | ->01.81% (44,212,224B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->01.81% (44,212,224B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| | | ->00.17% (4,198,400B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
<..callstacks trimmed..>
| | ->01.17% (28,688,384B) 0x409D4D: main (mt-multiconnect-server.cpp:99)
| |
| ->00.52% (12,611,584B) in 1+ places, all below ms_print's threshold (01.00%)
|
->03.59% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->03.49% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->03.49% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->03.23% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->03.23% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->03.23% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->03.14% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->03.14% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.14% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.14% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->03.14% (76,787,712B) 0x1: ???
| | | | ->03.14% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->03.14% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.09% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.26% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.09% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.18% (4,456,448B) in 1+ places, all below ms_print's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
11 10,904,278,966 2,457,571,328 2,457,571,328 0 0
12 12,008,403,416 2,474,508,288 2,474,508,288 0 0
100.00% (2,474,508,288B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->96.27% (2,382,221,312B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->46.10% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->46.10% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->46.10% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->18.98% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->18.98% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| ->37.97% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->37.97% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->37.97% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->08.14% (201,326,592B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->08.14% (201,326,592B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| ->11.69% (289,312,768B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
| | ->11.69% (289,312,768B) 0x57623F7: abc_thread_create_stack (abc-threadops.c:1553)
<..callstacks trimmed..>
| | | | ->02.04% (50,528,256B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->02.04% (50,528,256B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| | | | ->01.16% (28,688,384B) 0x409D4D: main (mt-multiconnect-server.cpp:99)
| | | |
<..callstacks trimmed..>
| |
| ->00.51% (12,533,760B) in 1+ places, all below ms_print's threshold (01.00%)
|
->03.54% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->03.45% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->03.45% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->03.19% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->03.19% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->03.19% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->03.10% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->03.10% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.10% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->03.10% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->03.10% (76,787,712B) 0x1: ???
| | | | ->03.10% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->03.10% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.09% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.26% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.09% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.19% (4,624,384B) in 1+ places, all below ms_print's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
13 12,891,986,496 2,486,087,680 2,486,087,680 0 0
14 13,996,030,351 2,501,877,760 2,501,877,760 0 0
15 15,103,870,730 2,517,377,024 2,517,377,024 0 0
16 16,664,833,194 2,518,614,016 2,518,614,016 0 0
17 17,562,811,101 2,525,892,608 2,525,892,608 0 0
18 18,447,077,562 2,526,187,520 2,526,187,520 0 0
19 19,554,021,032 2,545,065,984 2,545,065,984 0 0
20 20,438,291,032 2,557,845,504 2,557,845,504 0 0
21 21,316,909,078 2,574,368,768 2,574,368,768 0 0
22 22,216,329,042 2,604,269,568 2,604,269,568 0 0
23 23,747,577,863 2,626,297,856 2,626,297,856 0 0
24 25,075,044,417 2,645,319,680 2,645,319,680 0 0
25 26,177,389,068 2,657,587,200 2,657,587,200 0 0
26 27,284,044,806 2,673,377,280 2,673,377,280 0 0
27 28,612,914,970 2,704,588,800 2,704,588,800 0 0
28 29,497,940,392 2,704,883,712 2,704,883,712 0 0
29 30,618,069,290 2,710,335,488 2,710,335,488 0 0
30 31,503,881,389 2,715,746,304 2,715,746,304 0 0
31 32,606,647,955 2,743,095,296 2,743,095,296 0 0
100.00% (2,743,095,296B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->96.64% (2,650,865,664B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->41.59% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->41.59% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->41.59% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->17.13% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->17.13% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| ->34.25% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->34.25% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->34.25% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->07.34% (201,326,592B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->07.34% (201,326,592B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| ->20.45% (560,902,144B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
| | ->20.45% (560,902,144B) 0x57623F7: abc_thread_create_stack (abc-threadops.c:1553)
<..callstacks trimmed..>
| | | | ->04.30% (117,899,264B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->04.30% (117,899,264B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
|
->03.20% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->03.11% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->03.11% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->02.88% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->02.88% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->02.88% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->02.80% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->02.80% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->02.80% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->02.80% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->02.80% (76,787,712B) 0x1: ???
| | | | ->02.80% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->02.80% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.08% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.23% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.08% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.17% (4,567,040B) in 1+ places, all below ms_print's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
32 33,703,180,442 2,766,155,776 2,766,155,776 0 0
33 34,810,962,814 2,782,097,408 2,782,097,408 0 0
34 35,686,442,245 2,807,853,056 2,807,853,056 0 0
35 36,570,903,917 2,824,990,720 2,824,990,720 0 0
36 37,680,911,263 2,839,437,312 2,839,437,312 0 0
37 39,232,942,671 2,861,662,208 2,861,662,208 0 0
38 40,563,098,118 2,880,569,344 2,880,569,344 0 0
39 41,450,021,529 2,893,201,408 2,893,201,408 0 0
40 42,793,892,602 2,909,937,664 2,909,937,664 0 0
41 43,680,316,082 2,910,154,752 2,910,154,752 0 0
42 44,567,764,699 2,916,544,512 2,916,544,512 0 0
43 45,899,708,011 2,935,492,608 2,935,492,608 0 0
44 46,795,509,789 2,946,727,936 2,946,727,936 0 0
45 47,896,679,627 2,963,914,752 2,963,914,752 0 0
46 48,783,591,707 2,976,546,816 2,976,546,816 0 0
47 49,672,015,371 2,989,178,880 2,989,178,880 0 0
48 51,011,050,577 3,011,362,816 3,011,362,816 0 0
100.00% (3,011,362,816B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->96.93% (2,919,055,360B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->37.88% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->37.88% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->37.88% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->15.60% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->15.60% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| ->31.20% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->31.20% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->31.20% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->06.69% (201,326,592B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->06.69% (201,326,592B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
| | | |
<..callstacks trimmed..>
| ->27.51% (828,280,832B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
| | ->27.51% (828,280,832B) 0x57623F7: abc_thread_create_stack (abc-threadops.c:1553)
<..callstacks trimmed..>
| | | | ->06.33% (190,533,632B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->06.33% (190,533,632B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| ->00.35% (10,399,744B) in 1+ places, all below ms_print's threshold (01.00%)
|
->02.91% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->02.84% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->02.84% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->02.62% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->02.62% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->02.62% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->02.55% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->02.55% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->02.55% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->02.55% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->02.55% (76,787,712B) 0x1: ???
| | | | ->02.55% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->02.55% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.07% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.21% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.07% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.15% (4,644,864B) in 1+ places, all below ms_print's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
49 51,887,443,774 3,028,869,120 3,028,869,120 0 0
50 52,330,973,600 3,047,944,192 3,047,944,192 0 0
100.00% (3,047,944,192B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->96.97% (2,955,636,736B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->37.43% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->37.43% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->37.43% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->15.41% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->15.41% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| ->30.82% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->30.82% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->30.82% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->06.61% (201,326,592B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->06.61% (201,326,592B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| ->28.31% (863,019,008B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
| | ->28.31% (863,019,008B) 0x57623F7: abc_thread_create_stack (abc-threadops.c:1553)
<..callstacks trimmed..>
| | | | ->06.46% (196,849,664B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->06.46% (196,849,664B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
|
->02.88% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->02.80% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->02.80% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->02.59% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->02.59% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->02.59% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->02.52% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->02.52% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->02.52% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->02.52% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->02.52% (76,787,712B) 0x1: ???
| | | | ->02.52% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->02.52% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.07% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.21% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.07% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.15% (4,644,864B) in 1+ places, all below ms_print's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
51 53,015,150,958 3,060,576,256 3,060,576,256 0 0
52 53,878,740,129 3,070,050,304 3,070,050,304 0 0
53 54,327,243,557 3,079,892,992 3,079,892,992 0 0
100.00% (3,079,892,992B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc.
->97.00% (2,987,585,536B) 0x928BFCA: mmap (in /usr/lib64/libc-2.17.so)
| ->37.04% (1,140,850,688B) 0x92122D0: new_heap (in /usr/lib64/libc-2.17.so)
| | ->37.04% (1,140,850,688B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->37.04% (1,140,850,688B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->15.25% (469,762,048B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->15.25% (469,762,048B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| |
| ->30.51% (939,524,096B) 0x921225F: new_heap (in /usr/lib64/libc-2.17.so)
| | ->30.51% (939,524,096B) 0x92128D3: arena_get2.isra.3 (in /usr/lib64/libc-2.17.so)
| | ->30.51% (939,524,096B) 0x921877D: malloc (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| | | | ->06.54% (201,326,592B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->06.54% (201,326,592B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
| ->29.05% (894,599,168B) 0x8F7F7FC: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
<..callstacks trimmed..>
| | | | ->06.70% (206,323,712B) 0x8F7EEA4: start_thread (in /usr/lib64/libpthread-2.17.so)
| | | | ->06.70% (206,323,712B) 0x9291B0C: clone (in /usr/lib64/libc-2.17.so)
<..callstacks trimmed..>
|
->02.85% (87,662,592B) 0x40197DA: mmap (in /usr/lib64/ld-2.17.so)
| ->02.77% (85,417,984B) 0x40064B6: _dl_map_object_from_fd (in /usr/lib64/ld-2.17.so)
| | ->02.77% (85,417,984B) 0x4008669: _dl_map_object (in /usr/lib64/ld-2.17.so)
| | ->02.56% (78,991,360B) 0x400CC61: openaux (in /usr/lib64/ld-2.17.so)
| | | ->02.56% (78,991,360B) 0x400F7D3: _dl_catch_error (in /usr/lib64/ld-2.17.so)
| | | ->02.56% (78,991,360B) 0x400D46C: _dl_map_object_deps (in /usr/lib64/ld-2.17.so)
| | | ->02.49% (76,787,712B) 0x4003692: dl_main (in /usr/lib64/ld-2.17.so)
| | | | ->02.49% (76,787,712B) 0x401800D: _dl_sysdep_start (in /usr/lib64/ld-2.17.so)
| | | | ->02.49% (76,787,712B) 0x4001BD0: _dl_start (in /usr/lib64/ld-2.17.so)
| | | | ->02.49% (76,787,712B) 0x4001147: ??? (in /usr/lib64/ld-2.17.so)
| | | | ->02.49% (76,787,712B) 0x1: ???
| | | | ->02.49% (76,787,712B) 0x1FFEFFFFF6: ???
| | | | ->02.49% (76,787,712B) 0x1FFF00000D: ???
| | | |
| | | ->00.07% (2,203,648B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.21% (6,426,624B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.07% (2,244,608B) in 1+ places, all below ms_print's threshold (01.00%)
|
->00.15% (4,644,864B) in 1+ places, all below ms_print's threshold (01.00%)
[-- Attachment #3: massif.out.9052 --]
[-- Type: text/plain, Size: 7192 bytes --]
desc: --pages-as-heap=yes
cmd: mt-multiconnect-server 65000
time_unit: i
#-----------
snapshot=0
#-----------
time=0
mem_heap_B=69632
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=1
#-----------
time=1245519421
mem_heap_B=1873780736
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=2
#-----------
time=2119197635
mem_heap_B=2242240512
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=detailed
<..callstacks trimmed..>
#-----------
snapshot=3
#-----------
time=3449905980
mem_heap_B=2296811520
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=detailed
<..callstacks trimmed..>
#-----------
snapshot=4
#-----------
time=4323508142
mem_heap_B=2328596480
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=detailed
<..callstacks trimmed..>
#-----------
snapshot=5
#-----------
time=5176557792
mem_heap_B=2353487872
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=6
#-----------
time=6074463273
mem_heap_B=2386395136
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=7
#-----------
time=7158241065
mem_heap_B=2407047168
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=detailed
<..callstacks trimmed..>
#-----------
snapshot=8
#-----------
time=7836933298
mem_heap_B=2416521216
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=9
#-----------
time=9139037941
mem_heap_B=2431938560
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=10
#-----------
time=9823680957
mem_heap_B=2444943360
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=detailed
<..callstacks trimmed..>
#-----------
snapshot=11
#-----------
time=10904278966
mem_heap_B=2457571328
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=12
#-----------
time=12008403416
mem_heap_B=2474508288
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=detailed
<..callstacks trimmed..>
#-----------
snapshot=13
#-----------
time=12891986496
mem_heap_B=2486087680
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=14
#-----------
time=13996030351
mem_heap_B=2501877760
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=15
#-----------
time=15103870730
mem_heap_B=2517377024
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=16
#-----------
time=16664833194
mem_heap_B=2518614016
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=17
#-----------
time=17562811101
mem_heap_B=2525892608
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=18
#-----------
time=18447077562
mem_heap_B=2526187520
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=19
#-----------
time=19554021032
mem_heap_B=2545065984
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=20
#-----------
time=20438291032
mem_heap_B=2557845504
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=21
#-----------
time=21316909078
mem_heap_B=2574368768
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=22
#-----------
time=22216329042
mem_heap_B=2604269568
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=23
#-----------
time=23747577863
mem_heap_B=2626297856
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=24
#-----------
time=25075044417
mem_heap_B=2645319680
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=25
#-----------
time=26177389068
mem_heap_B=2657587200
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=26
#-----------
time=27284044806
mem_heap_B=2673377280
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=27
#-----------
time=28612914970
mem_heap_B=2704588800
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=28
#-----------
time=29497940392
mem_heap_B=2704883712
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=29
#-----------
time=30618069290
mem_heap_B=2710335488
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=30
#-----------
time=31503881389
mem_heap_B=2715746304
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=31
#-----------
time=32606647955
mem_heap_B=2743095296
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=detailed
<..callstacks trimmed..>
#-----------
snapshot=32
#-----------
time=33703180442
mem_heap_B=2766155776
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=33
#-----------
time=34810962814
mem_heap_B=2782097408
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=34
#-----------
time=35686442245
mem_heap_B=2807853056
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=35
#-----------
time=36570903917
mem_heap_B=2824990720
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=36
#-----------
time=37680911263
mem_heap_B=2839437312
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=37
#-----------
time=39232942671
mem_heap_B=2861662208
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=38
#-----------
time=40563098118
mem_heap_B=2880569344
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=39
#-----------
time=41450021529
mem_heap_B=2893201408
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=40
#-----------
time=42793892602
mem_heap_B=2909937664
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=41
#-----------
time=43680316082
mem_heap_B=2910154752
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=42
#-----------
time=44567764699
mem_heap_B=2916544512
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=43
#-----------
time=45899708011
mem_heap_B=2935492608
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=44
#-----------
time=46795509789
mem_heap_B=2946727936
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=45
#-----------
time=47896679627
mem_heap_B=2963914752
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=46
#-----------
time=48783591707
mem_heap_B=2976546816
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=47
#-----------
time=49672015371
mem_heap_B=2989178880
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=48
#-----------
time=51011050577
mem_heap_B=3011362816
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=detailed
<..callstacks trimmed..>
#-----------
snapshot=49
#-----------
time=51887443774
mem_heap_B=3028869120
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=50
#-----------
time=52330973600
mem_heap_B=3047944192
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=detailed
<..callstacks trimmed..>
#-----------
snapshot=51
#-----------
time=53015150958
mem_heap_B=3060576256
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=52
#-----------
time=53878740129
mem_heap_B=3070050304
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=empty
#-----------
snapshot=53
#-----------
time=54327243557
mem_heap_B=3079892992
mem_heap_extra_B=0
mem_stacks_B=0
heap_tree=peak
<..callstacks trimmed..>
[-- Attachment #4: mt-multiconnect-server.txt --]
[-- Type: text/plain, Size: 92006 bytes --]
Stack trace for process id 14854
Tue Feb 7 09:55:20 EST 2023
mt-multiconnect-server 65000
14854 mt-multiconnect-server 65000
#0 0x00007fd422ac19dd in accept () from /lib64/libpthread.so.0
<...stacktrace trimmed...>
#7 0x0000000000409d4e in main (argc=2, argv=0x7ffe3ebb9508) at mt-multiconnect-server.cpp:99
/proc/14854/maps:
00400000-00411000 r-xp 00000000 fd:02 303396032 <..trimmed..>/mt-multiconnect-server
00611000-00613000 r--p 00011000 fd:02 303396032 <..trimmed..>/mt-multiconnect-server
00613000-00614000 rw-p 00013000 fd:02 303396032 <..trimmed..>/mt-multiconnect-server
00963000-00e9f000 rw-p 00000000 00:00 0 [heap]
7fd363ca0000-7fd363ca1000 ---p 00000000 00:00 0
7fd363ca1000-7fd363da1000 rw-p 00000000 00:00 0
7fd363da1000-7fd363da2000 ---p 00000000 00:00 0
7fd363da2000-7fd363ea2000 rw-p 00000000 00:00 0
7fd363fa3000-7fd363fa4000 ---p 00000000 00:00 0
7fd363fa4000-7fd3640a4000 rw-p 00000000 00:00 0
7fd3642a6000-7fd3642a7000 ---p 00000000 00:00 0
7fd3642a7000-7fd3643a7000 rw-p 00000000 00:00 0
7fd3643a7000-7fd3643a8000 ---p 00000000 00:00 0
7fd3643a8000-7fd3644a8000 rw-p 00000000 00:00 0
7fd3645a9000-7fd3645aa000 ---p 00000000 00:00 0
7fd3645aa000-7fd3646aa000 rw-p 00000000 00:00 0
7fd3648ac000-7fd3648ad000 ---p 00000000 00:00 0
7fd3648ad000-7fd3649ad000 rw-p 00000000 00:00 0
7fd3649ad000-7fd3649ae000 ---p 00000000 00:00 0
7fd3649ae000-7fd364aae000 rw-p 00000000 00:00 0
7fd364baf000-7fd364bb0000 ---p 00000000 00:00 0
7fd364bb0000-7fd364cb0000 rw-p 00000000 00:00 0
7fd364eb2000-7fd364eb3000 ---p 00000000 00:00 0
7fd364eb3000-7fd364fb3000 rw-p 00000000 00:00 0
7fd364fb3000-7fd364fb4000 ---p 00000000 00:00 0
7fd364fb4000-7fd3650b4000 rw-p 00000000 00:00 0
7fd3651b5000-7fd3651b6000 ---p 00000000 00:00 0
7fd3651b6000-7fd3652b6000 rw-p 00000000 00:00 0
7fd3654b8000-7fd3654b9000 ---p 00000000 00:00 0
7fd3654b9000-7fd3655b9000 rw-p 00000000 00:00 0
7fd3655b9000-7fd3655ba000 ---p 00000000 00:00 0
7fd3655ba000-7fd3656ba000 rw-p 00000000 00:00 0
7fd3657bb000-7fd3657bc000 ---p 00000000 00:00 0
7fd3657bc000-7fd3658bc000 rw-p 00000000 00:00 0
7fd365abe000-7fd365abf000 ---p 00000000 00:00 0
7fd365abf000-7fd365bbf000 rw-p 00000000 00:00 0
7fd365bbf000-7fd365bc0000 ---p 00000000 00:00 0
7fd365bc0000-7fd365cc0000 rw-p 00000000 00:00 0
7fd365dc1000-7fd365dc2000 ---p 00000000 00:00 0
7fd365dc2000-7fd365ec2000 rw-p 00000000 00:00 0
7fd3660c4000-7fd3660c5000 ---p 00000000 00:00 0
7fd3660c5000-7fd3661c5000 rw-p 00000000 00:00 0
7fd3661c5000-7fd3661c6000 ---p 00000000 00:00 0
7fd3661c6000-7fd3662c6000 rw-p 00000000 00:00 0
7fd3663c7000-7fd3663c8000 ---p 00000000 00:00 0
7fd3663c8000-7fd3664c8000 rw-p 00000000 00:00 0
7fd3666ca000-7fd3666cb000 ---p 00000000 00:00 0
7fd3666cb000-7fd3667cb000 rw-p 00000000 00:00 0
7fd3669cd000-7fd3669ce000 ---p 00000000 00:00 0
7fd3669ce000-7fd366ace000 rw-p 00000000 00:00 0
7fd366cd0000-7fd366cd1000 ---p 00000000 00:00 0
7fd366cd1000-7fd366dd1000 rw-p 00000000 00:00 0
7fd366fd3000-7fd366fd4000 ---p 00000000 00:00 0
7fd366fd4000-7fd3670d4000 rw-p 00000000 00:00 0
7fd3672d6000-7fd3672d7000 ---p 00000000 00:00 0
7fd3672d7000-7fd3673d7000 rw-p 00000000 00:00 0
7fd3675d9000-7fd3675da000 ---p 00000000 00:00 0
7fd3675da000-7fd3676da000 rw-p 00000000 00:00 0
7fd3678dc000-7fd3678dd000 ---p 00000000 00:00 0
7fd3678dd000-7fd3679dd000 rw-p 00000000 00:00 0
7fd367bdf000-7fd367be0000 ---p 00000000 00:00 0
7fd367be0000-7fd367ce0000 rw-p 00000000 00:00 0
7fd367ee2000-7fd367ee3000 ---p 00000000 00:00 0
7fd367ee3000-7fd367fe3000 rw-p 00000000 00:00 0
7fd3681e5000-7fd3681e6000 ---p 00000000 00:00 0
7fd3681e6000-7fd3682e6000 rw-p 00000000 00:00 0
7fd3682e6000-7fd3682e7000 ---p 00000000 00:00 0
7fd3682e7000-7fd3683e7000 rw-p 00000000 00:00 0
7fd3684e8000-7fd3684e9000 ---p 00000000 00:00 0
7fd3684e9000-7fd3685e9000 rw-p 00000000 00:00 0
7fd3687eb000-7fd3687ec000 ---p 00000000 00:00 0
7fd3687ec000-7fd3688ec000 rw-p 00000000 00:00 0
7fd368aee000-7fd368aef000 ---p 00000000 00:00 0
7fd368aef000-7fd368bef000 rw-p 00000000 00:00 0
7fd368df1000-7fd368df2000 ---p 00000000 00:00 0
7fd368df2000-7fd368ef2000 rw-p 00000000 00:00 0
7fd3690f4000-7fd3690f5000 ---p 00000000 00:00 0
7fd3690f5000-7fd3691f5000 rw-p 00000000 00:00 0
7fd3693f7000-7fd3693f8000 ---p 00000000 00:00 0
7fd3693f8000-7fd3694f8000 rw-p 00000000 00:00 0
7fd3696fa000-7fd3696fb000 ---p 00000000 00:00 0
7fd3696fb000-7fd3697fb000 rw-p 00000000 00:00 0
7fd3699fd000-7fd3699fe000 ---p 00000000 00:00 0
7fd3699fe000-7fd369afe000 rw-p 00000000 00:00 0
7fd369d00000-7fd369d01000 ---p 00000000 00:00 0
7fd369d01000-7fd369e01000 rw-p 00000000 00:00 0
7fd36a003000-7fd36a004000 ---p 00000000 00:00 0
7fd36a004000-7fd36a104000 rw-p 00000000 00:00 0
7fd36a306000-7fd36a307000 ---p 00000000 00:00 0
7fd36a307000-7fd36a407000 rw-p 00000000 00:00 0
7fd36a609000-7fd36a60a000 ---p 00000000 00:00 0
7fd36a60a000-7fd36a70a000 rw-p 00000000 00:00 0
7fd36a90c000-7fd36a90d000 ---p 00000000 00:00 0
7fd36a90d000-7fd36aa0d000 rw-p 00000000 00:00 0
7fd36ac0f000-7fd36ac10000 ---p 00000000 00:00 0
7fd36ac10000-7fd36ad10000 rw-p 00000000 00:00 0
7fd36af12000-7fd36af13000 ---p 00000000 00:00 0
7fd36af13000-7fd36b013000 rw-p 00000000 00:00 0
7fd36b013000-7fd36b014000 ---p 00000000 00:00 0
7fd36b014000-7fd36b114000 rw-p 00000000 00:00 0
7fd36b215000-7fd36b216000 ---p 00000000 00:00 0
7fd36b216000-7fd36b316000 rw-p 00000000 00:00 0
7fd36b518000-7fd36b519000 ---p 00000000 00:00 0
7fd36b519000-7fd36b619000 rw-p 00000000 00:00 0
7fd36b81b000-7fd36b81c000 ---p 00000000 00:00 0
7fd36b81c000-7fd36b91c000 rw-p 00000000 00:00 0
7fd36bb1e000-7fd36bb1f000 ---p 00000000 00:00 0
7fd36bb1f000-7fd36bc1f000 rw-p 00000000 00:00 0
7fd36be21000-7fd36be22000 ---p 00000000 00:00 0
7fd36be22000-7fd36bf22000 rw-p 00000000 00:00 0
7fd36c124000-7fd36c125000 ---p 00000000 00:00 0
7fd36c125000-7fd36c225000 rw-p 00000000 00:00 0
7fd36c427000-7fd36c428000 ---p 00000000 00:00 0
7fd36c428000-7fd36c528000 rw-p 00000000 00:00 0
7fd36c72a000-7fd36c72b000 ---p 00000000 00:00 0
7fd36c72b000-7fd36c82b000 rw-p 00000000 00:00 0
7fd36ca2d000-7fd36ca2e000 ---p 00000000 00:00 0
7fd36ca2e000-7fd36cb2e000 rw-p 00000000 00:00 0
7fd36cd30000-7fd36cd31000 ---p 00000000 00:00 0
7fd36cd31000-7fd36ce31000 rw-p 00000000 00:00 0
7fd36d033000-7fd36d034000 ---p 00000000 00:00 0
7fd36d034000-7fd36d134000 rw-p 00000000 00:00 0
7fd36d336000-7fd36d337000 ---p 00000000 00:00 0
7fd36d337000-7fd36d437000 rw-p 00000000 00:00 0
7fd36d639000-7fd36d63a000 ---p 00000000 00:00 0
7fd36d63a000-7fd36d73a000 rw-p 00000000 00:00 0
7fd36d93c000-7fd36d93d000 ---p 00000000 00:00 0
7fd36d93d000-7fd36da3d000 rw-p 00000000 00:00 0
7fd36dc3f000-7fd36dc40000 ---p 00000000 00:00 0
7fd36dc40000-7fd36dd40000 rw-p 00000000 00:00 0
7fd36dd40000-7fd36dd41000 ---p 00000000 00:00 0
7fd36dd41000-7fd36de41000 rw-p 00000000 00:00 0
7fd36df42000-7fd36df43000 ---p 00000000 00:00 0
7fd36df43000-7fd36e043000 rw-p 00000000 00:00 0
7fd36e245000-7fd36e246000 ---p 00000000 00:00 0
7fd36e246000-7fd36e346000 rw-p 00000000 00:00 0
7fd36e548000-7fd36e549000 ---p 00000000 00:00 0
7fd36e549000-7fd36e649000 rw-p 00000000 00:00 0
7fd36e84b000-7fd36e84c000 ---p 00000000 00:00 0
7fd36e84c000-7fd36e94c000 rw-p 00000000 00:00 0
7fd36eb4e000-7fd36eb4f000 ---p 00000000 00:00 0
7fd36eb4f000-7fd36ec4f000 rw-p 00000000 00:00 0
7fd36ee51000-7fd36ee52000 ---p 00000000 00:00 0
7fd36ee52000-7fd36ef52000 rw-p 00000000 00:00 0
7fd36f154000-7fd36f155000 ---p 00000000 00:00 0
7fd36f155000-7fd36f255000 rw-p 00000000 00:00 0
7fd36f457000-7fd36f458000 ---p 00000000 00:00 0
7fd36f458000-7fd36f558000 rw-p 00000000 00:00 0
7fd36f75a000-7fd36f75b000 ---p 00000000 00:00 0
7fd36f75b000-7fd36f85b000 rw-p 00000000 00:00 0
7fd36f95c000-7fd36f95d000 ---p 00000000 00:00 0
7fd36f95d000-7fd36fa5d000 rw-p 00000000 00:00 0
7fd36fa5d000-7fd36fa5e000 ---p 00000000 00:00 0
7fd36fa5e000-7fd36fb5e000 rw-p 00000000 00:00 0
7fd36fb5e000-7fd36fb5f000 ---p 00000000 00:00 0
7fd36fb5f000-7fd36fc5f000 rw-p 00000000 00:00 0
7fd36fc5f000-7fd36fc60000 ---p 00000000 00:00 0
7fd36fc60000-7fd36fd60000 rw-p 00000000 00:00 0
7fd36fd60000-7fd36fd61000 ---p 00000000 00:00 0
7fd36fd61000-7fd36fe61000 rw-p 00000000 00:00 0
7fd36fe61000-7fd36fe62000 ---p 00000000 00:00 0
7fd36fe62000-7fd36ff62000 rw-p 00000000 00:00 0
7fd370063000-7fd370064000 ---p 00000000 00:00 0
7fd370064000-7fd370164000 rw-p 00000000 00:00 0
7fd370164000-7fd370165000 ---p 00000000 00:00 0
7fd370165000-7fd370265000 rw-p 00000000 00:00 0
7fd370265000-7fd370266000 ---p 00000000 00:00 0
7fd370266000-7fd370366000 rw-p 00000000 00:00 0
7fd370366000-7fd370367000 ---p 00000000 00:00 0
7fd370367000-7fd370467000 rw-p 00000000 00:00 0
7fd370467000-7fd370468000 ---p 00000000 00:00 0
7fd370468000-7fd370568000 rw-p 00000000 00:00 0
7fd370669000-7fd37066a000 ---p 00000000 00:00 0
7fd37066a000-7fd37076a000 rw-p 00000000 00:00 0
7fd37076a000-7fd37076b000 ---p 00000000 00:00 0
7fd37076b000-7fd37086b000 rw-p 00000000 00:00 0
7fd37096c000-7fd37096d000 ---p 00000000 00:00 0
7fd37096d000-7fd370a6d000 rw-p 00000000 00:00 0
7fd370a6d000-7fd370a6e000 ---p 00000000 00:00 0
7fd370a6e000-7fd370b6e000 rw-p 00000000 00:00 0
7fd370c6f000-7fd370c70000 ---p 00000000 00:00 0
7fd370c70000-7fd370d70000 rw-p 00000000 00:00 0
7fd370e71000-7fd370e72000 ---p 00000000 00:00 0
7fd370e72000-7fd370f72000 rw-p 00000000 00:00 0
7fd370f72000-7fd370f73000 ---p 00000000 00:00 0
7fd370f73000-7fd371073000 rw-p 00000000 00:00 0
7fd371073000-7fd371074000 ---p 00000000 00:00 0
7fd371074000-7fd371174000 rw-p 00000000 00:00 0
7fd371174000-7fd371175000 ---p 00000000 00:00 0
7fd371175000-7fd371275000 rw-p 00000000 00:00 0
7fd371275000-7fd371276000 ---p 00000000 00:00 0
7fd371276000-7fd371376000 rw-p 00000000 00:00 0
7fd371376000-7fd371377000 ---p 00000000 00:00 0
7fd371377000-7fd371477000 rw-p 00000000 00:00 0
7fd371578000-7fd371579000 ---p 00000000 00:00 0
7fd371579000-7fd371679000 rw-p 00000000 00:00 0
7fd371679000-7fd37167a000 ---p 00000000 00:00 0
7fd37167a000-7fd37177a000 rw-p 00000000 00:00 0
7fd37177a000-7fd37177b000 ---p 00000000 00:00 0
7fd37177b000-7fd37187b000 rw-p 00000000 00:00 0
7fd37187b000-7fd37187c000 ---p 00000000 00:00 0
7fd37187c000-7fd37197c000 rw-p 00000000 00:00 0
7fd37197c000-7fd37197d000 ---p 00000000 00:00 0
7fd37197d000-7fd371a7d000 rw-p 00000000 00:00 0
7fd371b7e000-7fd371b7f000 ---p 00000000 00:00 0
7fd371b7f000-7fd371c7f000 rw-p 00000000 00:00 0
7fd371c7f000-7fd371c80000 ---p 00000000 00:00 0
7fd371c80000-7fd371d80000 rw-p 00000000 00:00 0
7fd371e81000-7fd371e82000 ---p 00000000 00:00 0
7fd371e82000-7fd371f82000 rw-p 00000000 00:00 0
7fd371f82000-7fd371f83000 ---p 00000000 00:00 0
7fd371f83000-7fd372083000 rw-p 00000000 00:00 0
7fd372184000-7fd372185000 ---p 00000000 00:00 0
7fd372185000-7fd372285000 rw-p 00000000 00:00 0
7fd372285000-7fd372286000 ---p 00000000 00:00 0
7fd372286000-7fd372386000 rw-p 00000000 00:00 0
7fd372487000-7fd372488000 ---p 00000000 00:00 0
7fd372488000-7fd372588000 rw-p 00000000 00:00 0
7fd372588000-7fd372589000 ---p 00000000 00:00 0
7fd372589000-7fd372689000 rw-p 00000000 00:00 0
7fd37278a000-7fd37278b000 ---p 00000000 00:00 0
7fd37278b000-7fd37288b000 rw-p 00000000 00:00 0
7fd37288b000-7fd37288c000 ---p 00000000 00:00 0
7fd37288c000-7fd37298c000 rw-p 00000000 00:00 0
7fd372a8d000-7fd372a8e000 ---p 00000000 00:00 0
7fd372a8e000-7fd372b8e000 rw-p 00000000 00:00 0
7fd372b8e000-7fd372b8f000 ---p 00000000 00:00 0
7fd372b8f000-7fd372c8f000 rw-p 00000000 00:00 0
7fd372d90000-7fd372d91000 ---p 00000000 00:00 0
7fd372d91000-7fd372e91000 rw-p 00000000 00:00 0
7fd372e91000-7fd372e92000 ---p 00000000 00:00 0
7fd372e92000-7fd372f92000 rw-p 00000000 00:00 0
7fd373093000-7fd373094000 ---p 00000000 00:00 0
7fd373094000-7fd373194000 rw-p 00000000 00:00 0
7fd373194000-7fd373195000 ---p 00000000 00:00 0
7fd373195000-7fd373295000 rw-p 00000000 00:00 0
7fd373396000-7fd373397000 ---p 00000000 00:00 0
7fd373397000-7fd373497000 rw-p 00000000 00:00 0
7fd373497000-7fd373498000 ---p 00000000 00:00 0
7fd373498000-7fd373598000 rw-p 00000000 00:00 0
7fd373699000-7fd37369a000 ---p 00000000 00:00 0
7fd37369a000-7fd37379a000 rw-p 00000000 00:00 0
7fd37379a000-7fd37379b000 ---p 00000000 00:00 0
7fd37379b000-7fd37389b000 rw-p 00000000 00:00 0
7fd37399c000-7fd37399d000 ---p 00000000 00:00 0
7fd37399d000-7fd373a9d000 rw-p 00000000 00:00 0
7fd373a9d000-7fd373a9e000 ---p 00000000 00:00 0
7fd373a9e000-7fd373b9e000 rw-p 00000000 00:00 0
7fd373c9f000-7fd373ca0000 ---p 00000000 00:00 0
7fd373ca0000-7fd373da0000 rw-p 00000000 00:00 0
7fd373ea1000-7fd373ea2000 ---p 00000000 00:00 0
7fd373ea2000-7fd373fa2000 rw-p 00000000 00:00 0
7fd373fa2000-7fd373fa3000 ---p 00000000 00:00 0
7fd373fa3000-7fd3740a3000 rw-p 00000000 00:00 0
7fd3742a5000-7fd3742a6000 ---p 00000000 00:00 0
7fd3742a6000-7fd3743a6000 rw-p 00000000 00:00 0
7fd3743a6000-7fd3743a7000 ---p 00000000 00:00 0
7fd3743a7000-7fd3744a7000 rw-p 00000000 00:00 0
7fd3744a7000-7fd3744a8000 ---p 00000000 00:00 0
7fd3744a8000-7fd3745a8000 rw-p 00000000 00:00 0
7fd3745a8000-7fd3745a9000 ---p 00000000 00:00 0
7fd3745a9000-7fd3746a9000 rw-p 00000000 00:00 0
7fd3747aa000-7fd3747ab000 ---p 00000000 00:00 0
7fd3747ab000-7fd3748ab000 rw-p 00000000 00:00 0
7fd3748ab000-7fd3748ac000 ---p 00000000 00:00 0
7fd3748ac000-7fd3749ac000 rw-p 00000000 00:00 0
7fd3749ac000-7fd3749ad000 ---p 00000000 00:00 0
7fd3749ad000-7fd374aad000 rw-p 00000000 00:00 0
7fd374bae000-7fd374baf000 ---p 00000000 00:00 0
7fd374baf000-7fd374caf000 rw-p 00000000 00:00 0
7fd374caf000-7fd374cb0000 ---p 00000000 00:00 0
7fd374cb0000-7fd374db0000 rw-p 00000000 00:00 0
7fd374eb1000-7fd374eb2000 ---p 00000000 00:00 0
7fd374eb2000-7fd374fb2000 rw-p 00000000 00:00 0
7fd3750b3000-7fd3750b4000 ---p 00000000 00:00 0
7fd3750b4000-7fd3751b4000 rw-p 00000000 00:00 0
7fd3751b4000-7fd3751b5000 ---p 00000000 00:00 0
7fd3751b5000-7fd3752b5000 rw-p 00000000 00:00 0
7fd3752b5000-7fd3752b6000 ---p 00000000 00:00 0
7fd3752b6000-7fd3753b6000 rw-p 00000000 00:00 0
7fd3754b7000-7fd3754b8000 ---p 00000000 00:00 0
7fd3754b8000-7fd3755b8000 rw-p 00000000 00:00 0
7fd3755b8000-7fd3755b9000 ---p 00000000 00:00 0
7fd3755b9000-7fd3756b9000 rw-p 00000000 00:00 0
7fd3757ba000-7fd3757bb000 ---p 00000000 00:00 0
7fd3757bb000-7fd3758bb000 rw-p 00000000 00:00 0
7fd3758bb000-7fd3758bc000 ---p 00000000 00:00 0
7fd3758bc000-7fd3759bc000 rw-p 00000000 00:00 0
7fd375abd000-7fd375abe000 ---p 00000000 00:00 0
7fd375abe000-7fd375bbe000 rw-p 00000000 00:00 0
7fd375bbe000-7fd375bbf000 ---p 00000000 00:00 0
7fd375bbf000-7fd375cbf000 rw-p 00000000 00:00 0
7fd375dc0000-7fd375dc1000 ---p 00000000 00:00 0
7fd375dc1000-7fd375ec1000 rw-p 00000000 00:00 0
7fd375ec1000-7fd375ec2000 ---p 00000000 00:00 0
7fd375ec2000-7fd375fc2000 rw-p 00000000 00:00 0
7fd3760c3000-7fd3760c4000 ---p 00000000 00:00 0
7fd3760c4000-7fd3761c4000 rw-p 00000000 00:00 0
7fd3761c4000-7fd3761c5000 ---p 00000000 00:00 0
7fd3761c5000-7fd3762c5000 rw-p 00000000 00:00 0
7fd3763c6000-7fd3763c7000 ---p 00000000 00:00 0
7fd3763c7000-7fd3764c7000 rw-p 00000000 00:00 0
7fd3764c7000-7fd3764c8000 ---p 00000000 00:00 0
7fd3764c8000-7fd3765c8000 rw-p 00000000 00:00 0
7fd3766c9000-7fd3766ca000 ---p 00000000 00:00 0
7fd3766ca000-7fd3767ca000 rw-p 00000000 00:00 0
7fd3767ca000-7fd3767cb000 ---p 00000000 00:00 0
7fd3767cb000-7fd3768cb000 rw-p 00000000 00:00 0
7fd3769cc000-7fd3769cd000 ---p 00000000 00:00 0
7fd3769cd000-7fd376acd000 rw-p 00000000 00:00 0
7fd376acd000-7fd376ace000 ---p 00000000 00:00 0
7fd376ace000-7fd376bce000 rw-p 00000000 00:00 0
7fd376ccf000-7fd376cd0000 ---p 00000000 00:00 0
7fd376cd0000-7fd376dd0000 rw-p 00000000 00:00 0
7fd376dd0000-7fd376dd1000 ---p 00000000 00:00 0
7fd376dd1000-7fd376ed1000 rw-p 00000000 00:00 0
7fd376fd2000-7fd376fd3000 ---p 00000000 00:00 0
7fd376fd3000-7fd3770d3000 rw-p 00000000 00:00 0
7fd3770d3000-7fd3770d4000 ---p 00000000 00:00 0
7fd3770d4000-7fd3771d4000 rw-p 00000000 00:00 0
7fd3772d5000-7fd3772d6000 ---p 00000000 00:00 0
7fd3772d6000-7fd3773d6000 rw-p 00000000 00:00 0
7fd3773d6000-7fd3773d7000 ---p 00000000 00:00 0
7fd3773d7000-7fd3774d7000 rw-p 00000000 00:00 0
7fd3775d8000-7fd3775d9000 ---p 00000000 00:00 0
7fd3775d9000-7fd3776d9000 rw-p 00000000 00:00 0
7fd3776d9000-7fd3776da000 ---p 00000000 00:00 0
7fd3776da000-7fd3777da000 rw-p 00000000 00:00 0
7fd3778db000-7fd3778dc000 ---p 00000000 00:00 0
7fd3778dc000-7fd3779dc000 rw-p 00000000 00:00 0
7fd3779dc000-7fd3779dd000 ---p 00000000 00:00 0
7fd3779dd000-7fd377add000 rw-p 00000000 00:00 0
7fd377bde000-7fd377bdf000 ---p 00000000 00:00 0
7fd377bdf000-7fd377cdf000 rw-p 00000000 00:00 0
7fd377cdf000-7fd377ce0000 ---p 00000000 00:00 0
7fd377ce0000-7fd377de0000 rw-p 00000000 00:00 0
7fd377ee1000-7fd377ee2000 ---p 00000000 00:00 0
7fd377ee2000-7fd377fe2000 rw-p 00000000 00:00 0
7fd377fe2000-7fd377fe3000 ---p 00000000 00:00 0
7fd377fe3000-7fd3780e3000 rw-p 00000000 00:00 0
7fd3780e3000-7fd3780e4000 ---p 00000000 00:00 0
7fd3780e4000-7fd3781e4000 rw-p 00000000 00:00 0
7fd3781e4000-7fd3781e5000 ---p 00000000 00:00 0
7fd3781e5000-7fd3782e5000 rw-p 00000000 00:00 0
7fd3782e5000-7fd3782e6000 ---p 00000000 00:00 0
7fd3782e6000-7fd3783e6000 rw-p 00000000 00:00 0
7fd3783e6000-7fd3783e7000 ---p 00000000 00:00 0
7fd3783e7000-7fd3784e7000 rw-p 00000000 00:00 0
7fd3784e7000-7fd3784e8000 ---p 00000000 00:00 0
7fd3784e8000-7fd3785e8000 rw-p 00000000 00:00 0
7fd3785e8000-7fd3785e9000 ---p 00000000 00:00 0
7fd3785e9000-7fd3786e9000 rw-p 00000000 00:00 0
7fd3786e9000-7fd3786ea000 ---p 00000000 00:00 0
7fd3786ea000-7fd3787ea000 rw-p 00000000 00:00 0
7fd3787ea000-7fd3787eb000 ---p 00000000 00:00 0
7fd3787eb000-7fd3788eb000 rw-p 00000000 00:00 0
7fd3789ec000-7fd3789ed000 ---p 00000000 00:00 0
7fd3789ed000-7fd378aed000 rw-p 00000000 00:00 0
7fd378aed000-7fd378aee000 ---p 00000000 00:00 0
7fd378aee000-7fd378bee000 rw-p 00000000 00:00 0
7fd378bee000-7fd378bef000 ---p 00000000 00:00 0
7fd378bef000-7fd378cef000 rw-p 00000000 00:00 0
7fd378cef000-7fd378cf0000 ---p 00000000 00:00 0
7fd378cf0000-7fd378df0000 rw-p 00000000 00:00 0
7fd378df0000-7fd378df1000 ---p 00000000 00:00 0
7fd378df1000-7fd378ef1000 rw-p 00000000 00:00 0
7fd378ef1000-7fd378ef2000 ---p 00000000 00:00 0
7fd378ef2000-7fd378ff2000 rw-p 00000000 00:00 0
7fd378ff2000-7fd378ff3000 ---p 00000000 00:00 0
7fd378ff3000-7fd3790f3000 rw-p 00000000 00:00 0
7fd3791f4000-7fd3791f5000 ---p 00000000 00:00 0
7fd3791f5000-7fd3792f5000 rw-p 00000000 00:00 0
7fd3792f5000-7fd3792f6000 ---p 00000000 00:00 0
7fd3792f6000-7fd3793f6000 rw-p 00000000 00:00 0
7fd379408000-7fd379409000 ---p 00000000 00:00 0
7fd379409000-7fd379509000 rw-p 00000000 00:00 0
7fd379509000-7fd37950a000 ---p 00000000 00:00 0
7fd37950a000-7fd37960a000 rw-p 00000000 00:00 0
7fd37960a000-7fd37960b000 ---p 00000000 00:00 0
7fd37960b000-7fd37970b000 rw-p 00000000 00:00 0
7fd37982d000-7fd37982e000 ---p 00000000 00:00 0
7fd37982e000-7fd37992e000 rw-p 00000000 00:00 0
7fd37992e000-7fd37992f000 ---p 00000000 00:00 0
7fd37992f000-7fd379a2f000 rw-p 00000000 00:00 0
7fd379a2f000-7fd379a30000 ---p 00000000 00:00 0
7fd379a30000-7fd379b30000 rw-p 00000000 00:00 0
7fd379b30000-7fd379b31000 ---p 00000000 00:00 0
7fd379b31000-7fd379c31000 rw-p 00000000 00:00 0
7fd379c31000-7fd379c32000 ---p 00000000 00:00 0
7fd379c32000-7fd379d32000 rw-p 00000000 00:00 0
7fd379d32000-7fd379d33000 ---p 00000000 00:00 0
7fd379d33000-7fd379e33000 rw-p 00000000 00:00 0
7fd379e33000-7fd379e34000 ---p 00000000 00:00 0
7fd379e34000-7fd379f34000 rw-p 00000000 00:00 0
7fd379f34000-7fd379f35000 ---p 00000000 00:00 0
7fd379f35000-7fd37a035000 rw-p 00000000 00:00 0
7fd37a035000-7fd37a036000 ---p 00000000 00:00 0
7fd37a036000-7fd37a136000 rw-p 00000000 00:00 0
7fd37a136000-7fd37a137000 ---p 00000000 00:00 0
7fd37a137000-7fd37a237000 rw-p 00000000 00:00 0
7fd37a237000-7fd37a238000 ---p 00000000 00:00 0
7fd37a238000-7fd37a338000 rw-p 00000000 00:00 0
7fd37a338000-7fd37a339000 ---p 00000000 00:00 0
7fd37a339000-7fd37a439000 rw-p 00000000 00:00 0
7fd37a439000-7fd37a43a000 ---p 00000000 00:00 0
7fd37a43a000-7fd37a53a000 rw-p 00000000 00:00 0
7fd37a53a000-7fd37a53b000 ---p 00000000 00:00 0
7fd37a53b000-7fd37a63b000 rw-p 00000000 00:00 0
7fd37a63b000-7fd37a63c000 ---p 00000000 00:00 0
7fd37a63c000-7fd37a73c000 rw-p 00000000 00:00 0
7fd37a73c000-7fd37a73d000 ---p 00000000 00:00 0
7fd37a73d000-7fd37a83d000 rw-p 00000000 00:00 0
7fd37a83d000-7fd37a83e000 ---p 00000000 00:00 0
7fd37a83e000-7fd37a93e000 rw-p 00000000 00:00 0
7fd37a93e000-7fd37a93f000 ---p 00000000 00:00 0
7fd37a93f000-7fd37aa3f000 rw-p 00000000 00:00 0
7fd37aa3f000-7fd37aa40000 ---p 00000000 00:00 0
7fd37aa40000-7fd37ab40000 rw-p 00000000 00:00 0
7fd37ab40000-7fd37ab41000 ---p 00000000 00:00 0
7fd37ab41000-7fd37ac41000 rw-p 00000000 00:00 0
7fd37ac41000-7fd37ac42000 ---p 00000000 00:00 0
7fd37ac42000-7fd37ad42000 rw-p 00000000 00:00 0
7fd37ad42000-7fd37ad43000 ---p 00000000 00:00 0
7fd37ad43000-7fd37ae43000 rw-p 00000000 00:00 0
7fd37ae43000-7fd37ae44000 ---p 00000000 00:00 0
7fd37ae44000-7fd37af44000 rw-p 00000000 00:00 0
7fd37af44000-7fd37af45000 ---p 00000000 00:00 0
7fd37af45000-7fd37b045000 rw-p 00000000 00:00 0
7fd37b045000-7fd37b046000 ---p 00000000 00:00 0
7fd37b046000-7fd37b146000 rw-p 00000000 00:00 0
7fd37b146000-7fd37b147000 ---p 00000000 00:00 0
7fd37b147000-7fd37b247000 rw-p 00000000 00:00 0
7fd37b247000-7fd37b248000 ---p 00000000 00:00 0
7fd37b248000-7fd37b348000 rw-p 00000000 00:00 0
7fd37b348000-7fd37b349000 ---p 00000000 00:00 0
7fd37b349000-7fd37b449000 rw-p 00000000 00:00 0
7fd37b449000-7fd37b44a000 ---p 00000000 00:00 0
7fd37b44a000-7fd37b54a000 rw-p 00000000 00:00 0
7fd37b54a000-7fd37b54b000 ---p 00000000 00:00 0
7fd37b54b000-7fd37b64b000 rw-p 00000000 00:00 0
7fd37b64b000-7fd37b64c000 ---p 00000000 00:00 0
7fd37b64c000-7fd37b74c000 rw-p 00000000 00:00 0
7fd37b74c000-7fd37b74d000 ---p 00000000 00:00 0
7fd37b74d000-7fd37b84d000 rw-p 00000000 00:00 0
7fd37b84d000-7fd37b84e000 ---p 00000000 00:00 0
7fd37b84e000-7fd37b94e000 rw-p 00000000 00:00 0
7fd37b94e000-7fd37b94f000 ---p 00000000 00:00 0
7fd37b94f000-7fd37ba4f000 rw-p 00000000 00:00 0
7fd37ba4f000-7fd37ba50000 ---p 00000000 00:00 0
7fd37ba50000-7fd37bb50000 rw-p 00000000 00:00 0
7fd37bb50000-7fd37bb51000 ---p 00000000 00:00 0
7fd37bb51000-7fd37bc51000 rw-p 00000000 00:00 0
7fd37bc51000-7fd37bc52000 ---p 00000000 00:00 0
7fd37bc52000-7fd37bd52000 rw-p 00000000 00:00 0
7fd37be53000-7fd37be54000 ---p 00000000 00:00 0
7fd37be54000-7fd37bf54000 rw-p 00000000 00:00 0
7fd37bf54000-7fd37bf55000 ---p 00000000 00:00 0
7fd37bf55000-7fd37c055000 rw-p 00000000 00:00 0
7fd37c055000-7fd37c056000 ---p 00000000 00:00 0
7fd37c056000-7fd37c156000 rw-p 00000000 00:00 0
7fd37c156000-7fd37c157000 ---p 00000000 00:00 0
7fd37c157000-7fd37c257000 rw-p 00000000 00:00 0
7fd37c257000-7fd37c258000 ---p 00000000 00:00 0
7fd37c258000-7fd37c358000 rw-p 00000000 00:00 0
7fd37c358000-7fd37c359000 ---p 00000000 00:00 0
7fd37c359000-7fd37c459000 rw-p 00000000 00:00 0
7fd37c459000-7fd37c45a000 ---p 00000000 00:00 0
7fd37c45a000-7fd37c55a000 rw-p 00000000 00:00 0
7fd37c57e000-7fd37c57f000 ---p 00000000 00:00 0
7fd37c57f000-7fd37c67f000 rw-p 00000000 00:00 0
7fd37c7a4000-7fd37c7a5000 ---p 00000000 00:00 0
7fd37c7a5000-7fd37c8a5000 rw-p 00000000 00:00 0
7fd37c8b7000-7fd37c8b8000 ---p 00000000 00:00 0
7fd37c8b8000-7fd37c9b8000 rw-p 00000000 00:00 0
7fd37c9b8000-7fd37c9b9000 ---p 00000000 00:00 0
7fd37c9b9000-7fd37cab9000 rw-p 00000000 00:00 0
7fd37cab9000-7fd37caba000 ---p 00000000 00:00 0
7fd37caba000-7fd37cbba000 rw-p 00000000 00:00 0
7fd37cbba000-7fd37cbbb000 ---p 00000000 00:00 0
7fd37cbbb000-7fd37ccbb000 rw-p 00000000 00:00 0
7fd37ccbb000-7fd37ccbc000 ---p 00000000 00:00 0
7fd37ccbc000-7fd37cdbc000 rw-p 00000000 00:00 0
7fd37cdbc000-7fd37cdbd000 ---p 00000000 00:00 0
7fd37cdbd000-7fd37cebd000 rw-p 00000000 00:00 0
7fd37cebd000-7fd37cebe000 ---p 00000000 00:00 0
7fd37cebe000-7fd37cfbe000 rw-p 00000000 00:00 0
7fd37cfbe000-7fd37cfbf000 ---p 00000000 00:00 0
7fd37cfbf000-7fd37d0bf000 rw-p 00000000 00:00 0
7fd37d0bf000-7fd37d0c0000 ---p 00000000 00:00 0
7fd37d0c0000-7fd37d1c0000 rw-p 00000000 00:00 0
7fd37d1c0000-7fd37d1c1000 ---p 00000000 00:00 0
7fd37d1c1000-7fd37d2c1000 rw-p 00000000 00:00 0
7fd37d2c1000-7fd37d2c2000 ---p 00000000 00:00 0
7fd37d2c2000-7fd37d3c2000 rw-p 00000000 00:00 0
7fd37d3c2000-7fd37d3c3000 ---p 00000000 00:00 0
7fd37d3c3000-7fd37d4c3000 rw-p 00000000 00:00 0
7fd37d4c3000-7fd37d4c4000 ---p 00000000 00:00 0
7fd37d4c4000-7fd37d5c4000 rw-p 00000000 00:00 0
7fd37d5c4000-7fd37d5c5000 ---p 00000000 00:00 0
7fd37d5c5000-7fd37d6c5000 rw-p 00000000 00:00 0
7fd37d6c5000-7fd37d6c6000 ---p 00000000 00:00 0
7fd37d6c6000-7fd37d7c6000 rw-p 00000000 00:00 0
7fd37d7c6000-7fd37d7c7000 ---p 00000000 00:00 0
7fd37d7c7000-7fd37d8c7000 rw-p 00000000 00:00 0
7fd37d8c7000-7fd37d8c8000 ---p 00000000 00:00 0
7fd37d8c8000-7fd37d9c8000 rw-p 00000000 00:00 0
7fd37d9c8000-7fd37d9c9000 ---p 00000000 00:00 0
7fd37d9c9000-7fd37dac9000 rw-p 00000000 00:00 0
7fd37dac9000-7fd37daca000 ---p 00000000 00:00 0
7fd37daca000-7fd37dbca000 rw-p 00000000 00:00 0
7fd37dbca000-7fd37dbcb000 ---p 00000000 00:00 0
7fd37dbcb000-7fd37dccb000 rw-p 00000000 00:00 0
7fd37dccb000-7fd37dccc000 ---p 00000000 00:00 0
7fd37dccc000-7fd37ddcc000 rw-p 00000000 00:00 0
7fd37ddcc000-7fd37ddcd000 ---p 00000000 00:00 0
7fd37ddcd000-7fd37decd000 rw-p 00000000 00:00 0
7fd37decd000-7fd37dece000 ---p 00000000 00:00 0
7fd37dece000-7fd37dfce000 rw-p 00000000 00:00 0
7fd37dfce000-7fd37dfcf000 ---p 00000000 00:00 0
7fd37dfcf000-7fd37e0cf000 rw-p 00000000 00:00 0
7fd37e0cf000-7fd37e0d0000 ---p 00000000 00:00 0
7fd37e0d0000-7fd37e1d0000 rw-p 00000000 00:00 0
7fd37e1d0000-7fd37e1d1000 ---p 00000000 00:00 0
7fd37e1d1000-7fd37e2d1000 rw-p 00000000 00:00 0
7fd37e2d1000-7fd37e2d2000 ---p 00000000 00:00 0
7fd37e2d2000-7fd37e3d2000 rw-p 00000000 00:00 0
7fd37e3d2000-7fd37e3d3000 ---p 00000000 00:00 0
7fd37e3d3000-7fd37e4d3000 rw-p 00000000 00:00 0
7fd37e4d3000-7fd37e4d4000 ---p 00000000 00:00 0
7fd37e4d4000-7fd37e5d4000 rw-p 00000000 00:00 0
7fd37e5d4000-7fd37e5d5000 ---p 00000000 00:00 0
7fd37e5d5000-7fd37e6d5000 rw-p 00000000 00:00 0
7fd37e6d5000-7fd37e6d6000 ---p 00000000 00:00 0
7fd37e6d6000-7fd37e7d6000 rw-p 00000000 00:00 0
7fd37e7d6000-7fd37e7d7000 ---p 00000000 00:00 0
7fd37e7d7000-7fd37e8d7000 rw-p 00000000 00:00 0
7fd37e8d7000-7fd37e8d8000 ---p 00000000 00:00 0
7fd37e8d8000-7fd37e9d8000 rw-p 00000000 00:00 0
7fd37e9d8000-7fd37e9d9000 ---p 00000000 00:00 0
7fd37e9d9000-7fd37ead9000 rw-p 00000000 00:00 0
7fd37ead9000-7fd37eada000 ---p 00000000 00:00 0
7fd37eada000-7fd37ebda000 rw-p 00000000 00:00 0
7fd37ebda000-7fd37ebdb000 ---p 00000000 00:00 0
7fd37ebdb000-7fd37ecdb000 rw-p 00000000 00:00 0
7fd37ecdb000-7fd37ecdc000 ---p 00000000 00:00 0
7fd37ecdc000-7fd37eddc000 rw-p 00000000 00:00 0
7fd37eddc000-7fd37eddd000 ---p 00000000 00:00 0
7fd37eddd000-7fd37eedd000 rw-p 00000000 00:00 0
7fd37eedd000-7fd37eede000 ---p 00000000 00:00 0
7fd37eede000-7fd37efde000 rw-p 00000000 00:00 0
7fd37efde000-7fd37efdf000 ---p 00000000 00:00 0
7fd37efdf000-7fd37f0df000 rw-p 00000000 00:00 0
7fd37f0df000-7fd37f0e0000 ---p 00000000 00:00 0
7fd37f0e0000-7fd37f1e0000 rw-p 00000000 00:00 0
7fd37f1e0000-7fd37f1e1000 ---p 00000000 00:00 0
7fd37f1e1000-7fd37f2e1000 rw-p 00000000 00:00 0
7fd37f2e1000-7fd37f2e2000 ---p 00000000 00:00 0
7fd37f2e2000-7fd37f3e2000 rw-p 00000000 00:00 0
7fd37f3e2000-7fd37f3e3000 ---p 00000000 00:00 0
7fd37f3e3000-7fd37f4e3000 rw-p 00000000 00:00 0
7fd37f4e3000-7fd37f4e4000 ---p 00000000 00:00 0
7fd37f4e4000-7fd37f5e4000 rw-p 00000000 00:00 0
7fd37f5e4000-7fd37f5e5000 ---p 00000000 00:00 0
7fd37f5e5000-7fd37f6e5000 rw-p 00000000 00:00 0
7fd37f7e6000-7fd37f7e7000 ---p 00000000 00:00 0
7fd37f7e7000-7fd37f8e7000 rw-p 00000000 00:00 0
7fd37f91a000-7fd37f91b000 ---p 00000000 00:00 0
7fd37f91b000-7fd37fa1b000 rw-p 00000000 00:00 0
7fd37fa1b000-7fd37fa1c000 ---p 00000000 00:00 0
7fd37fa1c000-7fd37fb1c000 rw-p 00000000 00:00 0
7fd37fb1c000-7fd37fb1d000 ---p 00000000 00:00 0
7fd37fb1d000-7fd37fc1d000 rw-p 00000000 00:00 0
7fd37fc1d000-7fd37fc1e000 ---p 00000000 00:00 0
7fd37fc1e000-7fd37fd1e000 rw-p 00000000 00:00 0
7fd37fd1e000-7fd37fd1f000 ---p 00000000 00:00 0
7fd37fd1f000-7fd37fe1f000 rw-p 00000000 00:00 0
7fd37fe1f000-7fd37fe20000 ---p 00000000 00:00 0
7fd37fe20000-7fd37ff20000 rw-p 00000000 00:00 0
7fd37ff20000-7fd37ff21000 ---p 00000000 00:00 0
7fd37ff21000-7fd380021000 rw-p 00000000 00:00 0
7fd380021000-7fd380022000 ---p 00000000 00:00 0
7fd380022000-7fd380122000 rw-p 00000000 00:00 0
7fd380122000-7fd380123000 ---p 00000000 00:00 0
7fd380123000-7fd380223000 rw-p 00000000 00:00 0
7fd380223000-7fd380224000 ---p 00000000 00:00 0
7fd380224000-7fd380324000 rw-p 00000000 00:00 0
7fd380324000-7fd380325000 ---p 00000000 00:00 0
7fd380325000-7fd380425000 rw-p 00000000 00:00 0
7fd380425000-7fd380426000 ---p 00000000 00:00 0
7fd380426000-7fd380526000 rw-p 00000000 00:00 0
7fd380526000-7fd380527000 ---p 00000000 00:00 0
7fd380527000-7fd380627000 rw-p 00000000 00:00 0
7fd380627000-7fd380628000 ---p 00000000 00:00 0
7fd380628000-7fd380728000 rw-p 00000000 00:00 0
7fd380728000-7fd380729000 ---p 00000000 00:00 0
7fd380729000-7fd380829000 rw-p 00000000 00:00 0
7fd380829000-7fd38082a000 ---p 00000000 00:00 0
7fd38082a000-7fd38092a000 rw-p 00000000 00:00 0
7fd38092a000-7fd38092b000 ---p 00000000 00:00 0
7fd38092b000-7fd380a2b000 rw-p 00000000 00:00 0
7fd380a2b000-7fd380a2c000 ---p 00000000 00:00 0
7fd380a2c000-7fd380b2c000 rw-p 00000000 00:00 0
7fd380b2c000-7fd380b2d000 ---p 00000000 00:00 0
7fd380b2d000-7fd380c2d000 rw-p 00000000 00:00 0
7fd380c2d000-7fd380c2e000 ---p 00000000 00:00 0
7fd380c2e000-7fd380d2e000 rw-p 00000000 00:00 0
7fd380d2e000-7fd380d2f000 ---p 00000000 00:00 0
7fd380d2f000-7fd380e2f000 rw-p 00000000 00:00 0
7fd380e2f000-7fd380e30000 ---p 00000000 00:00 0
7fd380e30000-7fd380f30000 rw-p 00000000 00:00 0
7fd380f30000-7fd380f31000 ---p 00000000 00:00 0
7fd380f31000-7fd381031000 rw-p 00000000 00:00 0
7fd381132000-7fd381133000 ---p 00000000 00:00 0
7fd381133000-7fd381233000 rw-p 00000000 00:00 0
7fd381233000-7fd381234000 ---p 00000000 00:00 0
7fd381234000-7fd381334000 rw-p 00000000 00:00 0
7fd381334000-7fd381335000 ---p 00000000 00:00 0
7fd381335000-7fd381435000 rw-p 00000000 00:00 0
7fd381435000-7fd381436000 ---p 00000000 00:00 0
7fd381436000-7fd381536000 rw-p 00000000 00:00 0
7fd381536000-7fd381537000 ---p 00000000 00:00 0
7fd381537000-7fd381637000 rw-p 00000000 00:00 0
7fd381637000-7fd381638000 ---p 00000000 00:00 0
7fd381638000-7fd381738000 rw-p 00000000 00:00 0
7fd381738000-7fd381739000 ---p 00000000 00:00 0
7fd381739000-7fd381839000 rw-p 00000000 00:00 0
7fd381839000-7fd38183a000 ---p 00000000 00:00 0
7fd38183a000-7fd38193a000 rw-p 00000000 00:00 0
7fd38193a000-7fd38193b000 ---p 00000000 00:00 0
7fd38193b000-7fd381a3b000 rw-p 00000000 00:00 0
7fd381a3b000-7fd381a3c000 ---p 00000000 00:00 0
7fd381a3c000-7fd381b3c000 rw-p 00000000 00:00 0
7fd381b3c000-7fd381b3d000 ---p 00000000 00:00 0
7fd381b3d000-7fd381c3d000 rw-p 00000000 00:00 0
7fd381c3d000-7fd381c3e000 ---p 00000000 00:00 0
7fd381c3e000-7fd381d3e000 rw-p 00000000 00:00 0
7fd381d3e000-7fd381d3f000 ---p 00000000 00:00 0
7fd381d3f000-7fd381e3f000 rw-p 00000000 00:00 0
7fd381e3f000-7fd381e40000 ---p 00000000 00:00 0
7fd381e40000-7fd381f40000 rw-p 00000000 00:00 0
7fd381f40000-7fd381f41000 ---p 00000000 00:00 0
7fd381f41000-7fd382041000 rw-p 00000000 00:00 0
7fd382041000-7fd382042000 ---p 00000000 00:00 0
7fd382042000-7fd382142000 rw-p 00000000 00:00 0
7fd382142000-7fd382143000 ---p 00000000 00:00 0
7fd382143000-7fd382243000 rw-p 00000000 00:00 0
7fd382344000-7fd382345000 ---p 00000000 00:00 0
7fd382345000-7fd382445000 rw-p 00000000 00:00 0
7fd382445000-7fd382446000 ---p 00000000 00:00 0
7fd382446000-7fd382546000 rw-p 00000000 00:00 0
7fd382647000-7fd382648000 ---p 00000000 00:00 0
7fd382648000-7fd382748000 rw-p 00000000 00:00 0
7fd382748000-7fd382749000 ---p 00000000 00:00 0
7fd382749000-7fd382849000 rw-p 00000000 00:00 0
7fd382849000-7fd38284a000 ---p 00000000 00:00 0
7fd38284a000-7fd38294a000 rw-p 00000000 00:00 0
7fd38294a000-7fd38294b000 ---p 00000000 00:00 0
7fd38294b000-7fd382a4b000 rw-p 00000000 00:00 0
7fd382a4b000-7fd382a4c000 ---p 00000000 00:00 0
7fd382a4c000-7fd382b4c000 rw-p 00000000 00:00 0
7fd382c4d000-7fd382c4e000 ---p 00000000 00:00 0
7fd382c4e000-7fd382d4e000 rw-p 00000000 00:00 0
7fd382d4e000-7fd382d4f000 ---p 00000000 00:00 0
7fd382d4f000-7fd382e4f000 rw-p 00000000 00:00 0
7fd382f50000-7fd382f51000 ---p 00000000 00:00 0
7fd382f51000-7fd383051000 rw-p 00000000 00:00 0
7fd383051000-7fd383052000 ---p 00000000 00:00 0
7fd383052000-7fd383152000 rw-p 00000000 00:00 0
7fd383253000-7fd383254000 ---p 00000000 00:00 0
7fd383254000-7fd383354000 rw-p 00000000 00:00 0
7fd383354000-7fd383355000 ---p 00000000 00:00 0
7fd383355000-7fd383455000 rw-p 00000000 00:00 0
7fd383455000-7fd383456000 ---p 00000000 00:00 0
7fd383456000-7fd383556000 rw-p 00000000 00:00 0
7fd383556000-7fd383557000 ---p 00000000 00:00 0
7fd383557000-7fd383657000 rw-p 00000000 00:00 0
7fd383657000-7fd383658000 ---p 00000000 00:00 0
7fd383658000-7fd383758000 rw-p 00000000 00:00 0
7fd383859000-7fd38385a000 ---p 00000000 00:00 0
7fd38385a000-7fd38395a000 rw-p 00000000 00:00 0
7fd38395a000-7fd38395b000 ---p 00000000 00:00 0
7fd38395b000-7fd383a5b000 rw-p 00000000 00:00 0
7fd383b5c000-7fd383b5d000 ---p 00000000 00:00 0
7fd383b5d000-7fd383c5d000 rw-p 00000000 00:00 0
7fd383c5d000-7fd383c5e000 ---p 00000000 00:00 0
7fd383c5e000-7fd383d5e000 rw-p 00000000 00:00 0
7fd383d5e000-7fd383d5f000 ---p 00000000 00:00 0
7fd383d5f000-7fd383e5f000 rw-p 00000000 00:00 0
7fd383e5f000-7fd383e60000 ---p 00000000 00:00 0
7fd383e60000-7fd383f60000 rw-p 00000000 00:00 0
7fd383f60000-7fd383f61000 ---p 00000000 00:00 0
7fd383f61000-7fd384061000 rw-p 00000000 00:00 0
7fd384162000-7fd384163000 ---p 00000000 00:00 0
7fd384163000-7fd384263000 rw-p 00000000 00:00 0
7fd384364000-7fd384365000 ---p 00000000 00:00 0
7fd384365000-7fd384465000 rw-p 00000000 00:00 0
7fd384465000-7fd384466000 ---p 00000000 00:00 0
7fd384466000-7fd384566000 rw-p 00000000 00:00 0
7fd384667000-7fd384668000 ---p 00000000 00:00 0
7fd384668000-7fd384768000 rw-p 00000000 00:00 0
7fd384768000-7fd384769000 ---p 00000000 00:00 0
7fd384769000-7fd384869000 rw-p 00000000 00:00 0
7fd384869000-7fd38486a000 ---p 00000000 00:00 0
7fd38486a000-7fd38496a000 rw-p 00000000 00:00 0
7fd38496a000-7fd38496b000 ---p 00000000 00:00 0
7fd38496b000-7fd384a6b000 rw-p 00000000 00:00 0
7fd384a6b000-7fd384a6c000 ---p 00000000 00:00 0
7fd384a6c000-7fd384b6c000 rw-p 00000000 00:00 0
7fd384b6c000-7fd384b6d000 ---p 00000000 00:00 0
7fd384b6d000-7fd384c6d000 rw-p 00000000 00:00 0
7fd384c6d000-7fd384c6e000 ---p 00000000 00:00 0
7fd384c6e000-7fd384d6e000 rw-p 00000000 00:00 0
7fd384d6e000-7fd384d6f000 ---p 00000000 00:00 0
7fd384d6f000-7fd384e6f000 rw-p 00000000 00:00 0
7fd384e6f000-7fd384e70000 ---p 00000000 00:00 0
7fd384e70000-7fd384f70000 rw-p 00000000 00:00 0
7fd384f70000-7fd384f71000 ---p 00000000 00:00 0
7fd384f71000-7fd385071000 rw-p 00000000 00:00 0
7fd385071000-7fd385072000 ---p 00000000 00:00 0
7fd385072000-7fd385172000 rw-p 00000000 00:00 0
7fd385172000-7fd385173000 ---p 00000000 00:00 0
7fd385173000-7fd385273000 rw-p 00000000 00:00 0
7fd385273000-7fd385274000 ---p 00000000 00:00 0
7fd385274000-7fd385374000 rw-p 00000000 00:00 0
7fd385374000-7fd385375000 ---p 00000000 00:00 0
7fd385375000-7fd385475000 rw-p 00000000 00:00 0
7fd385475000-7fd385476000 ---p 00000000 00:00 0
7fd385476000-7fd385576000 rw-p 00000000 00:00 0
7fd385576000-7fd385577000 ---p 00000000 00:00 0
7fd385577000-7fd385677000 rw-p 00000000 00:00 0
7fd385677000-7fd385678000 ---p 00000000 00:00 0
7fd385678000-7fd385778000 rw-p 00000000 00:00 0
7fd385778000-7fd385779000 ---p 00000000 00:00 0
7fd385779000-7fd385879000 rw-p 00000000 00:00 0
7fd385879000-7fd38587a000 ---p 00000000 00:00 0
7fd38587a000-7fd38597a000 rw-p 00000000 00:00 0
7fd38597a000-7fd38597b000 ---p 00000000 00:00 0
7fd38597b000-7fd385a7b000 rw-p 00000000 00:00 0
7fd385a7b000-7fd385a7c000 ---p 00000000 00:00 0
7fd385a7c000-7fd385b7c000 rw-p 00000000 00:00 0
7fd385b7c000-7fd385b7d000 ---p 00000000 00:00 0
7fd385b7d000-7fd385c7d000 rw-p 00000000 00:00 0
7fd385c7d000-7fd385c7e000 ---p 00000000 00:00 0
7fd385c7e000-7fd385d7e000 rw-p 00000000 00:00 0
7fd385d7e000-7fd385d7f000 ---p 00000000 00:00 0
7fd385d7f000-7fd385e7f000 rw-p 00000000 00:00 0
7fd385e7f000-7fd385e80000 ---p 00000000 00:00 0
7fd385e80000-7fd385f80000 rw-p 00000000 00:00 0
7fd385f80000-7fd385f81000 ---p 00000000 00:00 0
7fd385f81000-7fd386081000 rw-p 00000000 00:00 0
7fd386081000-7fd386082000 ---p 00000000 00:00 0
7fd386082000-7fd386182000 rw-p 00000000 00:00 0
7fd386182000-7fd386183000 ---p 00000000 00:00 0
7fd386183000-7fd386283000 rw-p 00000000 00:00 0
7fd386283000-7fd386284000 ---p 00000000 00:00 0
7fd386284000-7fd386384000 rw-p 00000000 00:00 0
7fd386384000-7fd386385000 ---p 00000000 00:00 0
7fd386385000-7fd386485000 rw-p 00000000 00:00 0
7fd386485000-7fd386486000 ---p 00000000 00:00 0
7fd386486000-7fd386586000 rw-p 00000000 00:00 0
7fd386586000-7fd386587000 ---p 00000000 00:00 0
7fd386587000-7fd386687000 rw-p 00000000 00:00 0
7fd3866ab000-7fd3866ac000 ---p 00000000 00:00 0
7fd3866ac000-7fd3867ac000 rw-p 00000000 00:00 0
7fd3867ac000-7fd3867ad000 ---p 00000000 00:00 0
7fd3867ad000-7fd3868ad000 rw-p 00000000 00:00 0
7fd3868bf000-7fd3868c0000 ---p 00000000 00:00 0
7fd3868c0000-7fd3869c0000 rw-p 00000000 00:00 0
7fd3869c0000-7fd3869c1000 ---p 00000000 00:00 0
7fd3869c1000-7fd386ac1000 rw-p 00000000 00:00 0
7fd386ac1000-7fd386ac2000 ---p 00000000 00:00 0
7fd386ac2000-7fd386bc2000 rw-p 00000000 00:00 0
7fd386be6000-7fd386be7000 ---p 00000000 00:00 0
7fd386be7000-7fd386ce7000 rw-p 00000000 00:00 0
7fd386ce7000-7fd386ce8000 ---p 00000000 00:00 0
7fd386ce8000-7fd386de8000 rw-p 00000000 00:00 0
7fd386de8000-7fd386de9000 ---p 00000000 00:00 0
7fd386de9000-7fd386ee9000 rw-p 00000000 00:00 0
7fd386ee9000-7fd386eea000 ---p 00000000 00:00 0
7fd386eea000-7fd386fea000 rw-p 00000000 00:00 0
7fd386fea000-7fd386feb000 ---p 00000000 00:00 0
7fd386feb000-7fd3870eb000 rw-p 00000000 00:00 0
7fd3870eb000-7fd3870ec000 ---p 00000000 00:00 0
7fd3870ec000-7fd3871ec000 rw-p 00000000 00:00 0
7fd3871ec000-7fd3871ed000 ---p 00000000 00:00 0
7fd3871ed000-7fd3872ed000 rw-p 00000000 00:00 0
7fd3872ed000-7fd3872ee000 ---p 00000000 00:00 0
7fd3872ee000-7fd3873ee000 rw-p 00000000 00:00 0
7fd3873ee000-7fd3873ef000 ---p 00000000 00:00 0
7fd3873ef000-7fd3874ef000 rw-p 00000000 00:00 0
7fd3874ef000-7fd3874f0000 ---p 00000000 00:00 0
7fd3874f0000-7fd3875f0000 rw-p 00000000 00:00 0
7fd3875f0000-7fd3875f1000 ---p 00000000 00:00 0
7fd3875f1000-7fd3876f1000 rw-p 00000000 00:00 0
7fd3876f1000-7fd3876f2000 ---p 00000000 00:00 0
7fd3876f2000-7fd3877f2000 rw-p 00000000 00:00 0
7fd38794a000-7fd38794b000 ---p 00000000 00:00 0
7fd38794b000-7fd387a4b000 rw-p 00000000 00:00 0
7fd387a4b000-7fd387a4c000 ---p 00000000 00:00 0
7fd387a4c000-7fd387b4c000 rw-p 00000000 00:00 0
7fd387b82000-7fd387b83000 ---p 00000000 00:00 0
7fd387b83000-7fd387c83000 rw-p 00000000 00:00 0
7fd387c83000-7fd387c84000 ---p 00000000 00:00 0
7fd387c84000-7fd387d84000 rw-p 00000000 00:00 0
7fd387d84000-7fd387d85000 ---p 00000000 00:00 0
7fd387d85000-7fd387e85000 rw-p 00000000 00:00 0
7fd387e85000-7fd387e86000 ---p 00000000 00:00 0
7fd387e86000-7fd387f86000 rw-p 00000000 00:00 0
7fd3880de000-7fd3880df000 ---p 00000000 00:00 0
7fd3880df000-7fd3881df000 rw-p 00000000 00:00 0
7fd3881df000-7fd3881e0000 ---p 00000000 00:00 0
7fd3881e0000-7fd3882e0000 rw-p 00000000 00:00 0
7fd388316000-7fd388317000 ---p 00000000 00:00 0
7fd388317000-7fd388417000 rw-p 00000000 00:00 0
7fd388417000-7fd388418000 ---p 00000000 00:00 0
7fd388418000-7fd388518000 rw-p 00000000 00:00 0
7fd388518000-7fd388519000 ---p 00000000 00:00 0
7fd388519000-7fd388619000 rw-p 00000000 00:00 0
7fd388619000-7fd38861a000 ---p 00000000 00:00 0
7fd38861a000-7fd38871a000 rw-p 00000000 00:00 0
7fd388872000-7fd388873000 ---p 00000000 00:00 0
7fd388873000-7fd388973000 rw-p 00000000 00:00 0
7fd388973000-7fd388974000 ---p 00000000 00:00 0
7fd388974000-7fd388a74000 rw-p 00000000 00:00 0
7fd388aaa000-7fd388aab000 ---p 00000000 00:00 0
7fd388aab000-7fd388bab000 rw-p 00000000 00:00 0
7fd388bab000-7fd388bac000 ---p 00000000 00:00 0
7fd388bac000-7fd388cac000 rw-p 00000000 00:00 0
7fd388cac000-7fd388cad000 ---p 00000000 00:00 0
7fd388cad000-7fd388dad000 rw-p 00000000 00:00 0
7fd388dad000-7fd388dae000 ---p 00000000 00:00 0
7fd388dae000-7fd388eae000 rw-p 00000000 00:00 0
7fd389006000-7fd389007000 ---p 00000000 00:00 0
7fd389007000-7fd389107000 rw-p 00000000 00:00 0
7fd389107000-7fd389108000 ---p 00000000 00:00 0
7fd389108000-7fd389208000 rw-p 00000000 00:00 0
7fd38922c000-7fd38922d000 ---p 00000000 00:00 0
7fd38922d000-7fd38932d000 rw-p 00000000 00:00 0
7fd38933f000-7fd389340000 ---p 00000000 00:00 0
7fd389340000-7fd389440000 rw-p 00000000 00:00 0
7fd389440000-7fd389441000 ---p 00000000 00:00 0
7fd389441000-7fd389541000 rw-p 00000000 00:00 0
7fd389541000-7fd389542000 ---p 00000000 00:00 0
7fd389542000-7fd389642000 rw-p 00000000 00:00 0
7fd38979a000-7fd38979b000 ---p 00000000 00:00 0
7fd38979b000-7fd38989b000 rw-p 00000000 00:00 0
7fd38989b000-7fd38989c000 ---p 00000000 00:00 0
7fd38989c000-7fd38999c000 rw-p 00000000 00:00 0
7fd3899d2000-7fd3899d3000 ---p 00000000 00:00 0
7fd3899d3000-7fd389ad3000 rw-p 00000000 00:00 0
7fd389ad3000-7fd389ad4000 ---p 00000000 00:00 0
7fd389ad4000-7fd389bd4000 rw-p 00000000 00:00 0
7fd389bd4000-7fd389bd5000 ---p 00000000 00:00 0
7fd389bd5000-7fd389cd5000 rw-p 00000000 00:00 0
7fd389cd5000-7fd389cd6000 ---p 00000000 00:00 0
7fd389cd6000-7fd389dd6000 rw-p 00000000 00:00 0
7fd389f2e000-7fd389f2f000 ---p 00000000 00:00 0
7fd389f2f000-7fd38a02f000 rw-p 00000000 00:00 0
7fd38a02f000-7fd38a030000 ---p 00000000 00:00 0
7fd38a030000-7fd38a130000 rw-p 00000000 00:00 0
7fd38a166000-7fd38a167000 ---p 00000000 00:00 0
7fd38a167000-7fd38a267000 rw-p 00000000 00:00 0
7fd38a267000-7fd38a268000 ---p 00000000 00:00 0
7fd38a268000-7fd38a368000 rw-p 00000000 00:00 0
7fd38a368000-7fd38a369000 ---p 00000000 00:00 0
7fd38a369000-7fd38a469000 rw-p 00000000 00:00 0
7fd38a469000-7fd38a46a000 ---p 00000000 00:00 0
7fd38a46a000-7fd38a56a000 rw-p 00000000 00:00 0
7fd38a6c2000-7fd38a6c3000 ---p 00000000 00:00 0
7fd38a6c3000-7fd38a7c3000 rw-p 00000000 00:00 0
7fd38a7c3000-7fd38a7c4000 ---p 00000000 00:00 0
7fd38a7c4000-7fd38a8c4000 rw-p 00000000 00:00 0
7fd38a8fa000-7fd38a8fb000 ---p 00000000 00:00 0
7fd38a8fb000-7fd38a9fb000 rw-p 00000000 00:00 0
7fd38a9fb000-7fd38a9fc000 ---p 00000000 00:00 0
7fd38a9fc000-7fd38aafc000 rw-p 00000000 00:00 0
7fd38aafc000-7fd38aafd000 ---p 00000000 00:00 0
7fd38aafd000-7fd38abfd000 rw-p 00000000 00:00 0
7fd38abfd000-7fd38abfe000 ---p 00000000 00:00 0
7fd38abfe000-7fd38acfe000 rw-p 00000000 00:00 0
7fd38ad31000-7fd38ad32000 ---p 00000000 00:00 0
7fd38ad32000-7fd38ae32000 rw-p 00000000 00:00 0
7fd38ae56000-7fd38ae57000 ---p 00000000 00:00 0
7fd38ae57000-7fd38af57000 rw-p 00000000 00:00 0
7fd38af57000-7fd38af58000 ---p 00000000 00:00 0
7fd38af58000-7fd38b058000 rw-p 00000000 00:00 0
7fd38b08e000-7fd38b08f000 ---p 00000000 00:00 0
7fd38b08f000-7fd38b18f000 rw-p 00000000 00:00 0
7fd38b18f000-7fd38b190000 ---p 00000000 00:00 0
7fd38b190000-7fd38b290000 rw-p 00000000 00:00 0
7fd38b290000-7fd38b291000 ---p 00000000 00:00 0
7fd38b291000-7fd38b391000 rw-p 00000000 00:00 0
7fd38b391000-7fd38b392000 ---p 00000000 00:00 0
7fd38b392000-7fd38b492000 rw-p 00000000 00:00 0
7fd38b4c5000-7fd38b4c6000 ---p 00000000 00:00 0
7fd38b4c6000-7fd38b5c6000 rw-p 00000000 00:00 0
7fd38b5ea000-7fd38b5eb000 ---p 00000000 00:00 0
7fd38b5eb000-7fd38b6eb000 rw-p 00000000 00:00 0
7fd38b6eb000-7fd38b6ec000 ---p 00000000 00:00 0
7fd38b6ec000-7fd38b7ec000 rw-p 00000000 00:00 0
7fd38b822000-7fd38b823000 ---p 00000000 00:00 0
7fd38b823000-7fd38b923000 rw-p 00000000 00:00 0
7fd38b923000-7fd38b924000 ---p 00000000 00:00 0
7fd38b924000-7fd38ba24000 rw-p 00000000 00:00 0
7fd38ba24000-7fd38ba25000 ---p 00000000 00:00 0
7fd38ba25000-7fd38bb25000 rw-p 00000000 00:00 0
7fd38bb25000-7fd38bb26000 ---p 00000000 00:00 0
7fd38bb26000-7fd38bc26000 rw-p 00000000 00:00 0
7fd38bc47000-7fd38bc48000 ---p 00000000 00:00 0
7fd38bc48000-7fd38bd48000 rw-p 00000000 00:00 0
7fd38bd6c000-7fd38bd6d000 ---p 00000000 00:00 0
7fd38bd6d000-7fd38be6d000 rw-p 00000000 00:00 0
7fd38be6d000-7fd38be6e000 ---p 00000000 00:00 0
7fd38be6e000-7fd38bf6e000 rw-p 00000000 00:00 0
7fd38bfa4000-7fd38bfa5000 ---p 00000000 00:00 0
7fd38bfa5000-7fd38c0a5000 rw-p 00000000 00:00 0
7fd38c0a5000-7fd38c0a6000 ---p 00000000 00:00 0
7fd38c0a6000-7fd38c1a6000 rw-p 00000000 00:00 0
7fd38c1a6000-7fd38c1a7000 ---p 00000000 00:00 0
7fd38c1a7000-7fd38c2a7000 rw-p 00000000 00:00 0
7fd38c2b9000-7fd38c2ba000 ---p 00000000 00:00 0
7fd38c2ba000-7fd38c3ba000 rw-p 00000000 00:00 0
7fd38c3db000-7fd38c3dc000 ---p 00000000 00:00 0
7fd38c3dc000-7fd38c4dc000 rw-p 00000000 00:00 0
7fd38c500000-7fd38c501000 ---p 00000000 00:00 0
7fd38c501000-7fd38c601000 rw-p 00000000 00:00 0
7fd38c601000-7fd38c602000 ---p 00000000 00:00 0
7fd38c602000-7fd38c702000 rw-p 00000000 00:00 0
7fd38c738000-7fd38c739000 ---p 00000000 00:00 0
7fd38c739000-7fd38c839000 rw-p 00000000 00:00 0
7fd38c839000-7fd38c83a000 ---p 00000000 00:00 0
7fd38c83a000-7fd38c93a000 rw-p 00000000 00:00 0
7fd38c93a000-7fd38c93b000 ---p 00000000 00:00 0
7fd38c93b000-7fd38ca3b000 rw-p 00000000 00:00 0
7fd38ca4d000-7fd38ca4e000 ---p 00000000 00:00 0
7fd38ca4e000-7fd38cb4e000 rw-p 00000000 00:00 0
7fd38cb6f000-7fd38cb70000 ---p 00000000 00:00 0
7fd38cb70000-7fd38cc70000 rw-p 00000000 00:00 0
7fd38cc94000-7fd38cc95000 ---p 00000000 00:00 0
7fd38cc95000-7fd38cd95000 rw-p 00000000 00:00 0
7fd38cd95000-7fd38cd96000 ---p 00000000 00:00 0
7fd38cd96000-7fd38ce96000 rw-p 00000000 00:00 0
7fd38cecc000-7fd38cecd000 ---p 00000000 00:00 0
7fd38cecd000-7fd38cfcd000 rw-p 00000000 00:00 0
7fd38cfcd000-7fd38cfce000 ---p 00000000 00:00 0
7fd38cfce000-7fd38d0ce000 rw-p 00000000 00:00 0
7fd38d0ce000-7fd38d0cf000 ---p 00000000 00:00 0
7fd38d0cf000-7fd38d1cf000 rw-p 00000000 00:00 0
7fd38d1e1000-7fd38d1e2000 ---p 00000000 00:00 0
7fd38d1e2000-7fd38d2e2000 rw-p 00000000 00:00 0
7fd38d315000-7fd38d316000 ---p 00000000 00:00 0
7fd38d316000-7fd38d416000 rw-p 00000000 00:00 0
7fd38d43a000-7fd38d43b000 ---p 00000000 00:00 0
7fd38d43b000-7fd38d53b000 rw-p 00000000 00:00 0
7fd38d53b000-7fd38d53c000 ---p 00000000 00:00 0
7fd38d53c000-7fd38d63c000 rw-p 00000000 00:00 0
7fd38d672000-7fd38d673000 ---p 00000000 00:00 0
7fd38d673000-7fd38d773000 rw-p 00000000 00:00 0
7fd38d773000-7fd38d774000 ---p 00000000 00:00 0
7fd38d774000-7fd38d874000 rw-p 00000000 00:00 0
7fd38d874000-7fd38d875000 ---p 00000000 00:00 0
7fd38d875000-7fd38d975000 rw-p 00000000 00:00 0
7fd38d975000-7fd38d976000 ---p 00000000 00:00 0
7fd38d976000-7fd38da76000 rw-p 00000000 00:00 0
7fd38daa9000-7fd38daaa000 ---p 00000000 00:00 0
7fd38daaa000-7fd38dbaa000 rw-p 00000000 00:00 0
7fd38dbce000-7fd38dbcf000 ---p 00000000 00:00 0
7fd38dbcf000-7fd38dccf000 rw-p 00000000 00:00 0
7fd38dccf000-7fd38dcd0000 ---p 00000000 00:00 0
7fd38dcd0000-7fd38ddd0000 rw-p 00000000 00:00 0
7fd38ddf4000-7fd38ddf5000 ---p 00000000 00:00 0
7fd38ddf5000-7fd38def5000 rw-p 00000000 00:00 0
7fd38df07000-7fd38df08000 ---p 00000000 00:00 0
7fd38df08000-7fd38e008000 rw-p 00000000 00:00 0
7fd38e008000-7fd38e009000 ---p 00000000 00:00 0
7fd38e009000-7fd38e109000 rw-p 00000000 00:00 0
7fd38e109000-7fd38e10a000 ---p 00000000 00:00 0
7fd38e10a000-7fd38e20a000 rw-p 00000000 00:00 0
7fd38e23d000-7fd38e23e000 ---p 00000000 00:00 0
7fd38e23e000-7fd38e33e000 rw-p 00000000 00:00 0
7fd38e362000-7fd38e363000 ---p 00000000 00:00 0
7fd38e363000-7fd38e463000 rw-p 00000000 00:00 0
7fd38e463000-7fd38e464000 ---p 00000000 00:00 0
7fd38e464000-7fd38e564000 rw-p 00000000 00:00 0
7fd38e59a000-7fd38e59b000 ---p 00000000 00:00 0
7fd38e59b000-7fd38e69b000 rw-p 00000000 00:00 0
7fd38e69b000-7fd38e69c000 ---p 00000000 00:00 0
7fd38e69c000-7fd38e79c000 rw-p 00000000 00:00 0
7fd38e79c000-7fd38e79d000 ---p 00000000 00:00 0
7fd38e79d000-7fd38e89d000 rw-p 00000000 00:00 0
7fd38e89d000-7fd38e89e000 ---p 00000000 00:00 0
7fd38e89e000-7fd38e99e000 rw-p 00000000 00:00 0
7fd38e9d1000-7fd38e9d2000 ---p 00000000 00:00 0
7fd38e9d2000-7fd38ead2000 rw-p 00000000 00:00 0
7fd38eaf6000-7fd38eaf7000 ---p 00000000 00:00 0
7fd38eaf7000-7fd38ebf7000 rw-p 00000000 00:00 0
7fd38ebf7000-7fd38ebf8000 ---p 00000000 00:00 0
7fd38ebf8000-7fd38ecf8000 rw-p 00000000 00:00 0
7fd38ed2e000-7fd38ed2f000 ---p 00000000 00:00 0
7fd38ed2f000-7fd38ee2f000 rw-p 00000000 00:00 0
7fd38ee2f000-7fd38ee30000 ---p 00000000 00:00 0
7fd38ee30000-7fd38ef30000 rw-p 00000000 00:00 0
7fd38ef30000-7fd38ef31000 ---p 00000000 00:00 0
7fd38ef31000-7fd38f031000 rw-p 00000000 00:00 0
7fd38f031000-7fd38f032000 ---p 00000000 00:00 0
7fd38f032000-7fd38f132000 rw-p 00000000 00:00 0
7fd38f165000-7fd38f166000 ---p 00000000 00:00 0
7fd38f166000-7fd38f266000 rw-p 00000000 00:00 0
7fd38f28a000-7fd38f28b000 ---p 00000000 00:00 0
7fd38f28b000-7fd38f38b000 rw-p 00000000 00:00 0
7fd38f38b000-7fd38f38c000 ---p 00000000 00:00 0
7fd38f38c000-7fd38f48c000 rw-p 00000000 00:00 0
7fd38f4b0000-7fd38f4b1000 ---p 00000000 00:00 0
7fd38f4b1000-7fd38f5b1000 rw-p 00000000 00:00 0
7fd38f5c3000-7fd38f5c4000 ---p 00000000 00:00 0
7fd38f5c4000-7fd38f6c4000 rw-p 00000000 00:00 0
7fd38f6c4000-7fd38f6c5000 ---p 00000000 00:00 0
7fd38f6c5000-7fd38f7c5000 rw-p 00000000 00:00 0
7fd38f7c5000-7fd38f7c6000 ---p 00000000 00:00 0
7fd38f7c6000-7fd38f8c6000 rw-p 00000000 00:00 0
7fd38f8f9000-7fd38f8fa000 ---p 00000000 00:00 0
7fd38f8fa000-7fd38f9fa000 rw-p 00000000 00:00 0
7fd38fa1e000-7fd38fa1f000 ---p 00000000 00:00 0
7fd38fa1f000-7fd38fb1f000 rw-p 00000000 00:00 0
7fd38fb1f000-7fd38fb20000 ---p 00000000 00:00 0
7fd38fb20000-7fd38fc20000 rw-p 00000000 00:00 0
7fd38fc56000-7fd38fc57000 ---p 00000000 00:00 0
7fd38fc57000-7fd38fd57000 rw-p 00000000 00:00 0
7fd38fd57000-7fd38fd58000 ---p 00000000 00:00 0
7fd38fd58000-7fd38fe58000 rw-p 00000000 00:00 0
7fd38fe58000-7fd38fe59000 ---p 00000000 00:00 0
7fd38fe59000-7fd38ff59000 rw-p 00000000 00:00 0
7fd38ff59000-7fd38ff5a000 ---p 00000000 00:00 0
7fd38ff5a000-7fd39005a000 rw-p 00000000 00:00 0
7fd39008d000-7fd39008e000 ---p 00000000 00:00 0
7fd39008e000-7fd39018e000 rw-p 00000000 00:00 0
7fd3901b2000-7fd3901b3000 ---p 00000000 00:00 0
7fd3901b3000-7fd3902b3000 rw-p 00000000 00:00 0
7fd3902b3000-7fd3902b4000 ---p 00000000 00:00 0
7fd3902b4000-7fd3903b4000 rw-p 00000000 00:00 0
7fd3903ea000-7fd3903eb000 ---p 00000000 00:00 0
7fd3903eb000-7fd3904eb000 rw-p 00000000 00:00 0
7fd3904eb000-7fd3904ec000 ---p 00000000 00:00 0
7fd3904ec000-7fd3905ec000 rw-p 00000000 00:00 0
7fd3905ec000-7fd3905ed000 ---p 00000000 00:00 0
7fd3905ed000-7fd3906ed000 rw-p 00000000 00:00 0
7fd3906ff000-7fd390700000 ---p 00000000 00:00 0
7fd390700000-7fd390800000 rw-p 00000000 00:00 0
7fd390821000-7fd390822000 ---p 00000000 00:00 0
7fd390822000-7fd390922000 rw-p 00000000 00:00 0
7fd390946000-7fd390947000 ---p 00000000 00:00 0
7fd390947000-7fd390a47000 rw-p 00000000 00:00 0
7fd390a47000-7fd390a48000 ---p 00000000 00:00 0
7fd390a48000-7fd390b48000 rw-p 00000000 00:00 0
7fd390b6c000-7fd390b6d000 ---p 00000000 00:00 0
7fd390b6d000-7fd390c6d000 rw-p 00000000 00:00 0
7fd390c7f000-7fd390c80000 ---p 00000000 00:00 0
7fd390c80000-7fd390d80000 rw-p 00000000 00:00 0
7fd390d80000-7fd390d81000 ---p 00000000 00:00 0
7fd390d81000-7fd390e81000 rw-p 00000000 00:00 0
7fd390e93000-7fd390e94000 ---p 00000000 00:00 0
7fd390e94000-7fd390f94000 rw-p 00000000 00:00 0
7fd390fb5000-7fd390fb6000 ---p 00000000 00:00 0
7fd390fb6000-7fd3910b6000 rw-p 00000000 00:00 0
7fd3910da000-7fd3910db000 ---p 00000000 00:00 0
7fd3910db000-7fd3911db000 rw-p 00000000 00:00 0
7fd3911db000-7fd3911dc000 ---p 00000000 00:00 0
7fd3911dc000-7fd3912dc000 rw-p 00000000 00:00 0
7fd391312000-7fd391313000 ---p 00000000 00:00 0
7fd391313000-7fd391413000 rw-p 00000000 00:00 0
7fd391413000-7fd391414000 ---p 00000000 00:00 0
7fd391414000-7fd391514000 rw-p 00000000 00:00 0
7fd391514000-7fd391515000 ---p 00000000 00:00 0
7fd391515000-7fd391615000 rw-p 00000000 00:00 0
7fd391627000-7fd391628000 ---p 00000000 00:00 0
7fd391628000-7fd391728000 rw-p 00000000 00:00 0
7fd391749000-7fd39174a000 ---p 00000000 00:00 0
7fd39174a000-7fd39184a000 rw-p 00000000 00:00 0
7fd39186e000-7fd39186f000 ---p 00000000 00:00 0
7fd39186f000-7fd39196f000 rw-p 00000000 00:00 0
7fd39196f000-7fd391970000 ---p 00000000 00:00 0
7fd391970000-7fd391a70000 rw-p 00000000 00:00 0
7fd391a94000-7fd391a95000 ---p 00000000 00:00 0
7fd391a95000-7fd391b95000 rw-p 00000000 00:00 0
7fd391ba7000-7fd391ba8000 ---p 00000000 00:00 0
7fd391ba8000-7fd391ca8000 rw-p 00000000 00:00 0
7fd391ca8000-7fd391ca9000 ---p 00000000 00:00 0
7fd391ca9000-7fd391da9000 rw-p 00000000 00:00 0
7fd391dbb000-7fd391dbc000 ---p 00000000 00:00 0
7fd391dbc000-7fd391ebc000 rw-p 00000000 00:00 0
7fd391edd000-7fd391ede000 ---p 00000000 00:00 0
7fd391ede000-7fd391fde000 rw-p 00000000 00:00 0
7fd392002000-7fd392003000 ---p 00000000 00:00 0
7fd392003000-7fd392103000 rw-p 00000000 00:00 0
7fd392103000-7fd392104000 ---p 00000000 00:00 0
7fd392104000-7fd392204000 rw-p 00000000 00:00 0
7fd39223a000-7fd39223b000 ---p 00000000 00:00 0
7fd39223b000-7fd39233b000 rw-p 00000000 00:00 0
7fd39233b000-7fd39233c000 ---p 00000000 00:00 0
7fd39233c000-7fd39243c000 rw-p 00000000 00:00 0
7fd39243c000-7fd39243d000 ---p 00000000 00:00 0
7fd39243d000-7fd39253d000 rw-p 00000000 00:00 0
7fd39254f000-7fd392550000 ---p 00000000 00:00 0
7fd392550000-7fd392650000 rw-p 00000000 00:00 0
7fd392671000-7fd392672000 ---p 00000000 00:00 0
7fd392672000-7fd392772000 rw-p 00000000 00:00 0
7fd392796000-7fd392797000 ---p 00000000 00:00 0
7fd392797000-7fd392897000 rw-p 00000000 00:00 0
7fd392897000-7fd392898000 ---p 00000000 00:00 0
7fd392898000-7fd392998000 rw-p 00000000 00:00 0
7fd3929ce000-7fd3929cf000 ---p 00000000 00:00 0
7fd3929cf000-7fd392acf000 rw-p 00000000 00:00 0
7fd392acf000-7fd392ad0000 ---p 00000000 00:00 0
7fd392ad0000-7fd392bd0000 rw-p 00000000 00:00 0
7fd392bd0000-7fd392bd1000 ---p 00000000 00:00 0
7fd392bd1000-7fd392cd1000 rw-p 00000000 00:00 0
7fd392ce3000-7fd392ce4000 ---p 00000000 00:00 0
7fd392ce4000-7fd392de4000 rw-p 00000000 00:00 0
7fd392e05000-7fd392e06000 ---p 00000000 00:00 0
7fd392e06000-7fd392f06000 rw-p 00000000 00:00 0
7fd392f2a000-7fd392f2b000 ---p 00000000 00:00 0
7fd392f2b000-7fd39302b000 rw-p 00000000 00:00 0
7fd39302b000-7fd39302c000 ---p 00000000 00:00 0
7fd39302c000-7fd39312c000 rw-p 00000000 00:00 0
7fd393162000-7fd393163000 ---p 00000000 00:00 0
7fd393163000-7fd393263000 rw-p 00000000 00:00 0
7fd393263000-7fd393264000 ---p 00000000 00:00 0
7fd393264000-7fd393364000 rw-p 00000000 00:00 0
7fd393364000-7fd393365000 ---p 00000000 00:00 0
7fd393365000-7fd393465000 rw-p 00000000 00:00 0
7fd393465000-7fd393466000 ---p 00000000 00:00 0
7fd393466000-7fd393566000 rw-p 00000000 00:00 0
7fd393599000-7fd39359a000 ---p 00000000 00:00 0
7fd39359a000-7fd39369a000 rw-p 00000000 00:00 0
7fd3936be000-7fd3936bf000 ---p 00000000 00:00 0
7fd3936bf000-7fd3937bf000 rw-p 00000000 00:00 0
7fd3937bf000-7fd3937c0000 ---p 00000000 00:00 0
7fd3937c0000-7fd3938c0000 rw-p 00000000 00:00 0
7fd3938f6000-7fd3938f7000 ---p 00000000 00:00 0
7fd3938f7000-7fd3939f7000 rw-p 00000000 00:00 0
7fd3939f7000-7fd3939f8000 ---p 00000000 00:00 0
7fd3939f8000-7fd393af8000 rw-p 00000000 00:00 0
7fd393af8000-7fd393af9000 ---p 00000000 00:00 0
7fd393af9000-7fd393bf9000 rw-p 00000000 00:00 0
7fd393c0b000-7fd393c0c000 ---p 00000000 00:00 0
7fd393c0c000-7fd393d0c000 rw-p 00000000 00:00 0
7fd393d2d000-7fd393d2e000 ---p 00000000 00:00 0
7fd393d2e000-7fd393e2e000 rw-p 00000000 00:00 0
7fd393e52000-7fd393e53000 ---p 00000000 00:00 0
7fd393e53000-7fd393f53000 rw-p 00000000 00:00 0
7fd393f53000-7fd393f54000 ---p 00000000 00:00 0
7fd393f54000-7fd394054000 rw-p 00000000 00:00 0
7fd39408a000-7fd39408b000 ---p 00000000 00:00 0
7fd39408b000-7fd39418b000 rw-p 00000000 00:00 0
7fd39418b000-7fd39418c000 ---p 00000000 00:00 0
7fd39418c000-7fd39428c000 rw-p 00000000 00:00 0
7fd39428c000-7fd39428d000 ---p 00000000 00:00 0
7fd39428d000-7fd39438d000 rw-p 00000000 00:00 0
7fd39439f000-7fd3943a0000 ---p 00000000 00:00 0
7fd3943a0000-7fd3944a0000 rw-p 00000000 00:00 0
7fd3944c1000-7fd3944c2000 ---p 00000000 00:00 0
7fd3944c2000-7fd3945c2000 rw-p 00000000 00:00 0
7fd3945e6000-7fd3945e7000 ---p 00000000 00:00 0
7fd3945e7000-7fd3946e7000 rw-p 00000000 00:00 0
7fd3946e7000-7fd3946e8000 ---p 00000000 00:00 0
7fd3946e8000-7fd3947e8000 rw-p 00000000 00:00 0
7fd39481e000-7fd39481f000 ---p 00000000 00:00 0
7fd39481f000-7fd39491f000 rw-p 00000000 00:00 0
7fd39491f000-7fd394920000 ---p 00000000 00:00 0
7fd394920000-7fd394a20000 rw-p 00000000 00:00 0
7fd394a20000-7fd394a21000 ---p 00000000 00:00 0
7fd394a21000-7fd394b21000 rw-p 00000000 00:00 0
7fd394b33000-7fd394b34000 ---p 00000000 00:00 0
7fd394b34000-7fd394c34000 rw-p 00000000 00:00 0
7fd394c55000-7fd394c56000 ---p 00000000 00:00 0
7fd394c56000-7fd394d56000 rw-p 00000000 00:00 0
7fd394d7a000-7fd394d7b000 ---p 00000000 00:00 0
7fd394d7b000-7fd394e7b000 rw-p 00000000 00:00 0
7fd394e7b000-7fd394e7c000 ---p 00000000 00:00 0
7fd394e7c000-7fd394f7c000 rw-p 00000000 00:00 0
7fd394fb2000-7fd394fb3000 ---p 00000000 00:00 0
7fd394fb3000-7fd3950b3000 rw-p 00000000 00:00 0
7fd3950b3000-7fd3950b4000 ---p 00000000 00:00 0
7fd3950b4000-7fd3951b4000 rw-p 00000000 00:00 0
7fd3951b4000-7fd3951b5000 ---p 00000000 00:00 0
7fd3951b5000-7fd3952b5000 rw-p 00000000 00:00 0
7fd3952b5000-7fd3952b6000 ---p 00000000 00:00 0
7fd3952b6000-7fd3953b6000 rw-p 00000000 00:00 0
7fd3953e9000-7fd3953ea000 ---p 00000000 00:00 0
7fd3953ea000-7fd3954ea000 rw-p 00000000 00:00 0
7fd39550e000-7fd39550f000 ---p 00000000 00:00 0
7fd39550f000-7fd39560f000 rw-p 00000000 00:00 0
7fd39560f000-7fd395610000 ---p 00000000 00:00 0
7fd395610000-7fd395710000 rw-p 00000000 00:00 0
7fd395746000-7fd395747000 ---p 00000000 00:00 0
7fd395747000-7fd395847000 rw-p 00000000 00:00 0
7fd395847000-7fd395848000 ---p 00000000 00:00 0
7fd395848000-7fd395948000 rw-p 00000000 00:00 0
7fd395948000-7fd395949000 ---p 00000000 00:00 0
7fd395949000-7fd395a49000 rw-p 00000000 00:00 0
7fd395a49000-7fd395a4a000 ---p 00000000 00:00 0
7fd395a4a000-7fd395b4a000 rw-p 00000000 00:00 0
7fd395b7d000-7fd395b7e000 ---p 00000000 00:00 0
7fd395b7e000-7fd395c7e000 rw-p 00000000 00:00 0
7fd395ca2000-7fd395ca3000 ---p 00000000 00:00 0
7fd395ca3000-7fd395da3000 rw-p 00000000 00:00 0
7fd395da3000-7fd395da4000 ---p 00000000 00:00 0
7fd395da4000-7fd395ea4000 rw-p 00000000 00:00 0
7fd395eda000-7fd395edb000 ---p 00000000 00:00 0
7fd395edb000-7fd395fdb000 rw-p 00000000 00:00 0
7fd395fdb000-7fd395fdc000 ---p 00000000 00:00 0
7fd395fdc000-7fd3960dc000 rw-p 00000000 00:00 0
7fd3960dc000-7fd3960dd000 ---p 00000000 00:00 0
7fd3960dd000-7fd3961dd000 rw-p 00000000 00:00 0
7fd3961dd000-7fd3961de000 ---p 00000000 00:00 0
7fd3961de000-7fd3962de000 rw-p 00000000 00:00 0
7fd396311000-7fd396312000 ---p 00000000 00:00 0
7fd396312000-7fd396412000 rw-p 00000000 00:00 0
7fd396436000-7fd396437000 ---p 00000000 00:00 0
7fd396437000-7fd396537000 rw-p 00000000 00:00 0
7fd396537000-7fd396538000 ---p 00000000 00:00 0
7fd396538000-7fd396638000 rw-p 00000000 00:00 0
7fd39665c000-7fd39665d000 ---p 00000000 00:00 0
7fd39665d000-7fd39675d000 rw-p 00000000 00:00 0
7fd39676f000-7fd396770000 ---p 00000000 00:00 0
7fd396770000-7fd396870000 rw-p 00000000 00:00 0
7fd396870000-7fd396871000 ---p 00000000 00:00 0
7fd396871000-7fd396971000 rw-p 00000000 00:00 0
7fd396971000-7fd396972000 ---p 00000000 00:00 0
7fd396972000-7fd396a72000 rw-p 00000000 00:00 0
7fd396a93000-7fd396a94000 ---p 00000000 00:00 0
7fd396a94000-7fd396b94000 rw-p 00000000 00:00 0
7fd396bb8000-7fd396bb9000 ---p 00000000 00:00 0
7fd396bb9000-7fd396cb9000 rw-p 00000000 00:00 0
7fd396cb9000-7fd396cba000 ---p 00000000 00:00 0
7fd396cba000-7fd396dba000 rw-p 00000000 00:00 0
7fd396dde000-7fd396ddf000 ---p 00000000 00:00 0
7fd396ddf000-7fd396edf000 rw-p 00000000 00:00 0
7fd396ef1000-7fd396ef2000 ---p 00000000 00:00 0
7fd396ef2000-7fd396ff2000 rw-p 00000000 00:00 0
7fd396ff2000-7fd396ff3000 ---p 00000000 00:00 0
7fd396ff3000-7fd3970f3000 rw-p 00000000 00:00 0
7fd397105000-7fd397106000 ---p 00000000 00:00 0
7fd397106000-7fd397206000 rw-p 00000000 00:00 0
7fd397227000-7fd397228000 ---p 00000000 00:00 0
7fd397228000-7fd397328000 rw-p 00000000 00:00 0
7fd39734c000-7fd39734d000 ---p 00000000 00:00 0
7fd39734d000-7fd39744d000 rw-p 00000000 00:00 0
7fd39744d000-7fd39744e000 ---p 00000000 00:00 0
7fd39744e000-7fd39754e000 rw-p 00000000 00:00 0
7fd397572000-7fd397573000 ---p 00000000 00:00 0
7fd397573000-7fd397673000 rw-p 00000000 00:00 0
7fd397685000-7fd397686000 ---p 00000000 00:00 0
7fd397686000-7fd397786000 rw-p 00000000 00:00 0
7fd397786000-7fd397787000 ---p 00000000 00:00 0
7fd397787000-7fd397887000 rw-p 00000000 00:00 0
7fd397899000-7fd39789a000 ---p 00000000 00:00 0
7fd39789a000-7fd39799a000 rw-p 00000000 00:00 0
7fd3979bb000-7fd3979bc000 ---p 00000000 00:00 0
7fd3979bc000-7fd397abc000 rw-p 00000000 00:00 0
7fd397ae0000-7fd397ae1000 ---p 00000000 00:00 0
7fd397ae1000-7fd397be1000 rw-p 00000000 00:00 0
7fd397be1000-7fd397be2000 ---p 00000000 00:00 0
7fd397be2000-7fd397ce2000 rw-p 00000000 00:00 0
7fd397d18000-7fd397d19000 ---p 00000000 00:00 0
7fd397d19000-7fd397e19000 rw-p 00000000 00:00 0
7fd397e19000-7fd397e1a000 ---p 00000000 00:00 0
7fd397e1a000-7fd397f1a000 rw-p 00000000 00:00 0
7fd397f1a000-7fd397f1b000 ---p 00000000 00:00 0
7fd397f1b000-7fd39801b000 rw-p 00000000 00:00 0
7fd39802d000-7fd39802e000 ---p 00000000 00:00 0
7fd39802e000-7fd39812e000 rw-p 00000000 00:00 0
7fd39814f000-7fd398150000 ---p 00000000 00:00 0
7fd398150000-7fd398250000 rw-p 00000000 00:00 0
7fd398274000-7fd398275000 ---p 00000000 00:00 0
7fd398275000-7fd398375000 rw-p 00000000 00:00 0
7fd398375000-7fd398376000 ---p 00000000 00:00 0
7fd398376000-7fd398476000 rw-p 00000000 00:00 0
7fd39849a000-7fd39849b000 ---p 00000000 00:00 0
7fd39849b000-7fd39859b000 rw-p 00000000 00:00 0
7fd3985ad000-7fd3985ae000 ---p 00000000 00:00 0
7fd3985ae000-7fd3986ae000 rw-p 00000000 00:00 0
7fd3986ae000-7fd3986af000 ---p 00000000 00:00 0
7fd3986af000-7fd3987af000 rw-p 00000000 00:00 0
7fd3987c1000-7fd3987c2000 ---p 00000000 00:00 0
7fd3987c2000-7fd3988c2000 rw-p 00000000 00:00 0
7fd3988e3000-7fd3988e4000 ---p 00000000 00:00 0
7fd3988e4000-7fd3989e4000 rw-p 00000000 00:00 0
7fd398a08000-7fd398a09000 ---p 00000000 00:00 0
7fd398a09000-7fd398b09000 rw-p 00000000 00:00 0
7fd398b09000-7fd398b0a000 ---p 00000000 00:00 0
7fd398b0a000-7fd398c0a000 rw-p 00000000 00:00 0
7fd398c2e000-7fd398c2f000 ---p 00000000 00:00 0
7fd398c2f000-7fd398d2f000 rw-p 00000000 00:00 0
7fd398d41000-7fd398d42000 ---p 00000000 00:00 0
7fd398d42000-7fd398e42000 rw-p 00000000 00:00 0
7fd398e42000-7fd398e43000 ---p 00000000 00:00 0
7fd398e43000-7fd398f43000 rw-p 00000000 00:00 0
7fd398f55000-7fd398f56000 ---p 00000000 00:00 0
7fd398f56000-7fd399056000 rw-p 00000000 00:00 0
7fd399077000-7fd399078000 ---p 00000000 00:00 0
7fd399078000-7fd399178000 rw-p 00000000 00:00 0
7fd39919c000-7fd39919d000 ---p 00000000 00:00 0
7fd39919d000-7fd39929d000 rw-p 00000000 00:00 0
7fd3993c2000-7fd3993c3000 ---p 00000000 00:00 0
7fd3993c3000-7fd3994c3000 rw-p 00000000 00:00 0
7fd3994d5000-7fd3994d6000 ---p 00000000 00:00 0
7fd3994d6000-7fd3995d6000 rw-p 00000000 00:00 0
7fd3995d6000-7fd3995d7000 ---p 00000000 00:00 0
7fd3995d7000-7fd3996d7000 rw-p 00000000 00:00 0
7fd3996e9000-7fd3996ea000 ---p 00000000 00:00 0
7fd3996ea000-7fd3997ea000 rw-p 00000000 00:00 0
7fd39980b000-7fd39980c000 ---p 00000000 00:00 0
7fd39980c000-7fd39990c000 rw-p 00000000 00:00 0
7fd399930000-7fd399931000 ---p 00000000 00:00 0
7fd399931000-7fd399a31000 rw-p 00000000 00:00 0
7fd399a31000-7fd399a32000 ---p 00000000 00:00 0
7fd399a32000-7fd399b32000 rw-p 00000000 00:00 0
7fd399b68000-7fd399b69000 ---p 00000000 00:00 0
7fd399b69000-7fd399c69000 rw-p 00000000 00:00 0
7fd399c69000-7fd399c6a000 ---p 00000000 00:00 0
7fd399c6a000-7fd399d6a000 rw-p 00000000 00:00 0
7fd399d6a000-7fd399d6b000 ---p 00000000 00:00 0
7fd399d6b000-7fd399e6b000 rw-p 00000000 00:00 0
7fd399e7d000-7fd399e7e000 ---p 00000000 00:00 0
7fd399e7e000-7fd399f7e000 rw-p 00000000 00:00 0
7fd399f9f000-7fd399fa0000 ---p 00000000 00:00 0
7fd399fa0000-7fd39a0a0000 rw-p 00000000 00:00 0
7fd39a0c4000-7fd39a0c5000 ---p 00000000 00:00 0
7fd39a0c5000-7fd39a1c5000 rw-p 00000000 00:00 0
7fd39a1c5000-7fd39a1c6000 ---p 00000000 00:00 0
7fd39a1c6000-7fd39a2c6000 rw-p 00000000 00:00 0
7fd39a2ea000-7fd39a2eb000 ---p 00000000 00:00 0
7fd39a2eb000-7fd39a3eb000 rw-p 00000000 00:00 0
7fd39a3fd000-7fd39a3fe000 ---p 00000000 00:00 0
7fd39a3fe000-7fd39a4fe000 rw-p 00000000 00:00 0
7fd39a4fe000-7fd39a4ff000 ---p 00000000 00:00 0
7fd39a4ff000-7fd39a5ff000 rw-p 00000000 00:00 0
7fd39a611000-7fd39a612000 ---p 00000000 00:00 0
7fd39a612000-7fd39a712000 rw-p 00000000 00:00 0
7fd39a733000-7fd39a734000 ---p 00000000 00:00 0
7fd39a734000-7fd39a834000 rw-p 00000000 00:00 0
7fd39a858000-7fd39a859000 ---p 00000000 00:00 0
7fd39a859000-7fd39a959000 rw-p 00000000 00:00 0
7fd39a959000-7fd39a95a000 ---p 00000000 00:00 0
7fd39a95a000-7fd39aa5a000 rw-p 00000000 00:00 0
7fd39aa90000-7fd39aa91000 ---p 00000000 00:00 0
7fd39aa91000-7fd39ab91000 rw-p 00000000 00:00 0
7fd39ab91000-7fd39ab92000 ---p 00000000 00:00 0
7fd39ab92000-7fd39ac92000 rw-p 00000000 00:00 0
7fd39ac92000-7fd39ac93000 ---p 00000000 00:00 0
7fd39ac93000-7fd39ad93000 rw-p 00000000 00:00 0
7fd39ada5000-7fd39ada6000 ---p 00000000 00:00 0
7fd39ada6000-7fd39aea6000 rw-p 00000000 00:00 0
7fd39aec7000-7fd39aec8000 ---p 00000000 00:00 0
7fd39aec8000-7fd39afc8000 rw-p 00000000 00:00 0
7fd39b0ed000-7fd39b0ee000 ---p 00000000 00:00 0
7fd39b0ee000-7fd39b1ee000 rw-p 00000000 00:00 0
7fd39b224000-7fd39b225000 ---p 00000000 00:00 0
7fd39b225000-7fd39b325000 rw-p 00000000 00:00 0
7fd39b325000-7fd39b326000 ---p 00000000 00:00 0
7fd39b326000-7fd39b426000 rw-p 00000000 00:00 0
7fd39b426000-7fd39b427000 ---p 00000000 00:00 0
7fd39b427000-7fd39b527000 rw-p 00000000 00:00 0
7fd39b539000-7fd39b53a000 ---p 00000000 00:00 0
7fd39b53a000-7fd39b63a000 rw-p 00000000 00:00 0
7fd39b65b000-7fd39b65c000 ---p 00000000 00:00 0
7fd39b65c000-7fd39b75c000 rw-p 00000000 00:00 0
7fd39b780000-7fd39b781000 ---p 00000000 00:00 0
7fd39b781000-7fd39b881000 rw-p 00000000 00:00 0
7fd39b881000-7fd39b882000 ---p 00000000 00:00 0
7fd39b882000-7fd39b982000 rw-p 00000000 00:00 0
7fd39b9b8000-7fd39b9b9000 ---p 00000000 00:00 0
7fd39b9b9000-7fd39bab9000 rw-p 00000000 00:00 0
7fd39bab9000-7fd39baba000 ---p 00000000 00:00 0
7fd39baba000-7fd39bbba000 rw-p 00000000 00:00 0
7fd39bbba000-7fd39bbbb000 ---p 00000000 00:00 0
7fd39bbbb000-7fd39bcbb000 rw-p 00000000 00:00 0
7fd39bccd000-7fd39bcce000 ---p 00000000 00:00 0
7fd39bcce000-7fd39bdce000 rw-p 00000000 00:00 0
7fd39bdef000-7fd39bdf0000 ---p 00000000 00:00 0
7fd39bdf0000-7fd39bef0000 rw-p 00000000 00:00 0
7fd39bf14000-7fd39bf15000 ---p 00000000 00:00 0
7fd39bf15000-7fd39c015000 rw-p 00000000 00:00 0
7fd39c015000-7fd39c016000 ---p 00000000 00:00 0
7fd39c016000-7fd39c116000 rw-p 00000000 00:00 0
7fd39c14c000-7fd39c14d000 ---p 00000000 00:00 0
7fd39c14d000-7fd39c24d000 rw-p 00000000 00:00 0
7fd39c24d000-7fd39c24e000 ---p 00000000 00:00 0
7fd39c24e000-7fd39c34e000 rw-p 00000000 00:00 0
7fd39c34e000-7fd39c34f000 ---p 00000000 00:00 0
7fd39c34f000-7fd39c44f000 rw-p 00000000 00:00 0
7fd39c461000-7fd39c462000 ---p 00000000 00:00 0
7fd39c462000-7fd39c562000 rw-p 00000000 00:00 0
7fd39c583000-7fd39c584000 ---p 00000000 00:00 0
7fd39c584000-7fd39c684000 rw-p 00000000 00:00 0
7fd39c7a9000-7fd39c7aa000 ---p 00000000 00:00 0
7fd39c7aa000-7fd39c8aa000 rw-p 00000000 00:00 0
7fd39c8e0000-7fd39c8e1000 ---p 00000000 00:00 0
7fd39c8e1000-7fd39c9e1000 rw-p 00000000 00:00 0
7fd39c9e1000-7fd39c9e2000 ---p 00000000 00:00 0
7fd39c9e2000-7fd39cae2000 rw-p 00000000 00:00 0
7fd39cae2000-7fd39cae3000 ---p 00000000 00:00 0
7fd39cae3000-7fd39cbe3000 rw-p 00000000 00:00 0
7fd39cbf5000-7fd39cbf6000 ---p 00000000 00:00 0
7fd39cbf6000-7fd39ccf6000 rw-p 00000000 00:00 0
7fd39ce3c000-7fd39ce3d000 ---p 00000000 00:00 0
7fd39ce3d000-7fd39cf3d000 rw-p 00000000 00:00 0
7fd39cf3d000-7fd39cf3e000 ---p 00000000 00:00 0
7fd39cf3e000-7fd39d03e000 rw-p 00000000 00:00 0
7fd39d074000-7fd39d075000 ---p 00000000 00:00 0
7fd39d075000-7fd39d175000 rw-p 00000000 00:00 0
7fd39d175000-7fd39d176000 ---p 00000000 00:00 0
7fd39d176000-7fd39d276000 rw-p 00000000 00:00 0
7fd39d276000-7fd39d277000 ---p 00000000 00:00 0
7fd39d277000-7fd39d377000 rw-p 00000000 00:00 0
7fd39d389000-7fd39d38a000 ---p 00000000 00:00 0
7fd39d38a000-7fd39d48a000 rw-p 00000000 00:00 0
7fd39d5e2000-7fd39d5e3000 ---p 00000000 00:00 0
7fd39d5e3000-7fd39d6e3000 rw-p 00000000 00:00 0
7fd39d6e3000-7fd39d6e4000 ---p 00000000 00:00 0
7fd39d6e4000-7fd39d7e4000 rw-p 00000000 00:00 0
7fd39d808000-7fd39d809000 ---p 00000000 00:00 0
7fd39d809000-7fd39d909000 rw-p 00000000 00:00 0
7fd39d91b000-7fd39d91c000 ---p 00000000 00:00 0
7fd39d91c000-7fd39da1c000 rw-p 00000000 00:00 0
7fd39da1c000-7fd39da1d000 ---p 00000000 00:00 0
7fd39da1d000-7fd39db1d000 rw-p 00000000 00:00 0
7fd39db1d000-7fd39db1e000 ---p 00000000 00:00 0
7fd39db1e000-7fd39dc1e000 rw-p 00000000 00:00 0
7fd39dd76000-7fd39dd77000 ---p 00000000 00:00 0
7fd39dd77000-7fd39de77000 rw-p 00000000 00:00 0
7fd39de77000-7fd39de78000 ---p 00000000 00:00 0
7fd39de78000-7fd39df78000 rw-p 00000000 00:00 0
7fd39dfae000-7fd39dfaf000 ---p 00000000 00:00 0
7fd39dfaf000-7fd39e0af000 rw-p 00000000 00:00 0
7fd39e0af000-7fd39e0b0000 ---p 00000000 00:00 0
7fd39e0b0000-7fd39e1b0000 rw-p 00000000 00:00 0
7fd39e1b0000-7fd39e1b1000 ---p 00000000 00:00 0
7fd39e1b1000-7fd39e2b1000 rw-p 00000000 00:00 0
7fd39e2b1000-7fd39e2b2000 ---p 00000000 00:00 0
7fd39e2b2000-7fd39e3b2000 rw-p 00000000 00:00 0
7fd39e60b000-7fd39e60c000 ---p 00000000 00:00 0
7fd39e60c000-7fd39e70c000 rw-p 00000000 00:00 0
7fd39e742000-7fd39e743000 ---p 00000000 00:00 0
7fd39e743000-7fd39e843000 rw-p 00000000 00:00 0
7fd39e843000-7fd39e844000 ---p 00000000 00:00 0
7fd39e844000-7fd39e944000 rw-p 00000000 00:00 0
7fd39e944000-7fd39e945000 ---p 00000000 00:00 0
7fd39e945000-7fd39ea45000 rw-p 00000000 00:00 0
7fd39ea57000-7fd39ea58000 ---p 00000000 00:00 0
7fd39ea58000-7fd39eb58000 rw-p 00000000 00:00 0
7fd39ed9f000-7fd39eda0000 ---p 00000000 00:00 0
7fd39eda0000-7fd39eea0000 rw-p 00000000 00:00 0
7fd39eed6000-7fd39eed7000 ---p 00000000 00:00 0
7fd39eed7000-7fd39efd7000 rw-p 00000000 00:00 0
7fd39efd7000-7fd39efd8000 ---p 00000000 00:00 0
7fd39efd8000-7fd39f0d8000 rw-p 00000000 00:00 0
7fd39f0d8000-7fd39f0d9000 ---p 00000000 00:00 0
7fd39f0d9000-7fd39f1d9000 rw-p 00000000 00:00 0
7fd39f1eb000-7fd39f1ec000 ---p 00000000 00:00 0
7fd39f1ec000-7fd39f2ec000 rw-p 00000000 00:00 0
7fd39f432000-7fd39f433000 ---p 00000000 00:00 0
7fd39f433000-7fd39f533000 rw-p 00000000 00:00 0
7fd39f658000-7fd39f659000 ---p 00000000 00:00 0
7fd39f659000-7fd39f759000 rw-p 00000000 00:00 0
7fd39f76b000-7fd39f76c000 ---p 00000000 00:00 0
7fd39f76c000-7fd39f86c000 rw-p 00000000 00:00 0
7fd39f86c000-7fd39f86d000 ---p 00000000 00:00 0
7fd39f86d000-7fd39f96d000 rw-p 00000000 00:00 0
7fd39f97f000-7fd39f980000 ---p 00000000 00:00 0
7fd39f980000-7fd39fa80000 rw-p 00000000 00:00 0
7fd39fbc6000-7fd39fbc7000 ---p 00000000 00:00 0
7fd39fbc7000-7fd39fcc7000 rw-p 00000000 00:00 0
7fd39fcc7000-7fd39fcc8000 ---p 00000000 00:00 0
7fd39fcc8000-7fd39fdc8000 rw-p 00000000 00:00 0
7fd39fdfe000-7fd39fdff000 ---p 00000000 00:00 0
7fd39fdff000-7fd39feff000 rw-p 00000000 00:00 0
7fd39feff000-7fd39ff00000 ---p 00000000 00:00 0
7fd39ff00000-7fd3a0000000 rw-p 00000000 00:00 0
7fd3a0000000-7fd3a021e000 rw-p 00000000 00:00 0
7fd3a021e000-7fd3a4000000 ---p 00000000 00:00 0
7fd3a4000000-7fd3a41f3000 rw-p 00000000 00:00 0
7fd3a41f3000-7fd3a8000000 ---p 00000000 00:00 0
7fd3a8000000-7fd3a8142000 rw-p 00000000 00:00 0
7fd3a8142000-7fd3ac000000 ---p 00000000 00:00 0
7fd3ac000000-7fd3ac22d000 rw-p 00000000 00:00 0
7fd3ac22d000-7fd3b0000000 ---p 00000000 00:00 0
7fd3b0000000-7fd3b0314000 rw-p 00000000 00:00 0
7fd3b0314000-7fd3b4000000 ---p 00000000 00:00 0
7fd3b4000000-7fd3b4214000 rw-p 00000000 00:00 0
7fd3b4214000-7fd3b8000000 ---p 00000000 00:00 0
7fd3b8003000-7fd3b8004000 ---p 00000000 00:00 0
7fd3b8004000-7fd3b8104000 rw-p 00000000 00:00 0
7fd3b8116000-7fd3b8117000 ---p 00000000 00:00 0
7fd3b8117000-7fd3b8217000 rw-p 00000000 00:00 0
7fd3b836f000-7fd3b8370000 ---p 00000000 00:00 0
7fd3b8370000-7fd3b8470000 rw-p 00000000 00:00 0
7fd3b8470000-7fd3b8471000 ---p 00000000 00:00 0
7fd3b8471000-7fd3b8571000 rw-p 00000000 00:00 0
7fd3b8595000-7fd3b8596000 ---p 00000000 00:00 0
7fd3b8596000-7fd3b8696000 rw-p 00000000 00:00 0
7fd3b86a8000-7fd3b86a9000 ---p 00000000 00:00 0
7fd3b86a9000-7fd3b87a9000 rw-p 00000000 00:00 0
7fd3b87a9000-7fd3b87aa000 ---p 00000000 00:00 0
7fd3b87aa000-7fd3b88aa000 rw-p 00000000 00:00 0
7fd3b88bc000-7fd3b88bd000 ---p 00000000 00:00 0
7fd3b88bd000-7fd3b89bd000 rw-p 00000000 00:00 0
7fd3b8b15000-7fd3b8b16000 ---p 00000000 00:00 0
7fd3b8b16000-7fd3b8c16000 rw-p 00000000 00:00 0
7fd3b8c16000-7fd3b8c17000 ---p 00000000 00:00 0
7fd3b8c17000-7fd3b8d17000 rw-p 00000000 00:00 0
7fd3b8d3b000-7fd3b8d3c000 ---p 00000000 00:00 0
7fd3b8d3c000-7fd3b8e3c000 rw-p 00000000 00:00 0
7fd3b8e4e000-7fd3b8e4f000 ---p 00000000 00:00 0
7fd3b8e4f000-7fd3b8f4f000 rw-p 00000000 00:00 0
7fd3b8f4f000-7fd3b8f50000 ---p 00000000 00:00 0
7fd3b8f50000-7fd3b9050000 rw-p 00000000 00:00 0
7fd3b9062000-7fd3b9063000 ---p 00000000 00:00 0
7fd3b9063000-7fd3b9163000 rw-p 00000000 00:00 0
7fd3b92bb000-7fd3b92bc000 ---p 00000000 00:00 0
7fd3b92bc000-7fd3b93bc000 rw-p 00000000 00:00 0
7fd3b93bc000-7fd3b93bd000 ---p 00000000 00:00 0
7fd3b93bd000-7fd3b94bd000 rw-p 00000000 00:00 0
7fd3b94f3000-7fd3b94f4000 ---p 00000000 00:00 0
7fd3b94f4000-7fd3b95f4000 rw-p 00000000 00:00 0
7fd3b95f4000-7fd3b95f5000 ---p 00000000 00:00 0
7fd3b95f5000-7fd3b96f5000 rw-p 00000000 00:00 0
7fd3b96f5000-7fd3b96f6000 ---p 00000000 00:00 0
7fd3b96f6000-7fd3b97f6000 rw-p 00000000 00:00 0
7fd3b9808000-7fd3b9809000 ---p 00000000 00:00 0
7fd3b9809000-7fd3b9909000 rw-p 00000000 00:00 0
7fd3b9a61000-7fd3b9a62000 ---p 00000000 00:00 0
7fd3b9a62000-7fd3b9b62000 rw-p 00000000 00:00 0
7fd3b9b62000-7fd3b9b63000 ---p 00000000 00:00 0
7fd3b9b63000-7fd3b9c63000 rw-p 00000000 00:00 0
7fd3b9c87000-7fd3b9c88000 ---p 00000000 00:00 0
7fd3b9c88000-7fd3b9d88000 rw-p 00000000 00:00 0
7fd3b9d9a000-7fd3b9d9b000 ---p 00000000 00:00 0
7fd3b9d9b000-7fd3b9e9b000 rw-p 00000000 00:00 0
7fd3b9e9b000-7fd3b9e9c000 ---p 00000000 00:00 0
7fd3b9e9c000-7fd3b9f9c000 rw-p 00000000 00:00 0
7fd3b9fae000-7fd3b9faf000 ---p 00000000 00:00 0
7fd3b9faf000-7fd3ba0af000 rw-p 00000000 00:00 0
7fd3ba207000-7fd3ba208000 ---p 00000000 00:00 0
7fd3ba208000-7fd3ba308000 rw-p 00000000 00:00 0
7fd3ba308000-7fd3ba309000 ---p 00000000 00:00 0
7fd3ba309000-7fd3ba409000 rw-p 00000000 00:00 0
7fd3ba42d000-7fd3ba42e000 ---p 00000000 00:00 0
7fd3ba42e000-7fd3ba52e000 rw-p 00000000 00:00 0
7fd3ba52e000-7fd3ba52f000 ---p 00000000 00:00 0
7fd3ba52f000-7fd3ba62f000 rw-p 00000000 00:00 0
7fd3ba641000-7fd3ba642000 ---p 00000000 00:00 0
7fd3ba642000-7fd3ba742000 rw-p 00000000 00:00 0
7fd3ba754000-7fd3ba755000 ---p 00000000 00:00 0
7fd3ba755000-7fd3ba855000 rw-p 00000000 00:00 0
7fd3ba9ad000-7fd3ba9ae000 ---p 00000000 00:00 0
7fd3ba9ae000-7fd3baaae000 rw-p 00000000 00:00 0
7fd3baaae000-7fd3baaaf000 ---p 00000000 00:00 0
7fd3baaaf000-7fd3babaf000 rw-p 00000000 00:00 0
7fd3babd3000-7fd3babd4000 ---p 00000000 00:00 0
7fd3babd4000-7fd3bacd4000 rw-p 00000000 00:00 0
7fd3bace6000-7fd3bace7000 ---p 00000000 00:00 0
7fd3bace7000-7fd3bade7000 rw-p 00000000 00:00 0
7fd3bade7000-7fd3bade8000 ---p 00000000 00:00 0
7fd3bade8000-7fd3baee8000 rw-p 00000000 00:00 0
7fd3baefa000-7fd3baefb000 ---p 00000000 00:00 0
7fd3baefb000-7fd3baffb000 rw-p 00000000 00:00 0
7fd3bb153000-7fd3bb154000 ---p 00000000 00:00 0
7fd3bb154000-7fd3bb254000 rw-p 00000000 00:00 0
7fd3bb254000-7fd3bb255000 ---p 00000000 00:00 0
7fd3bb255000-7fd3bb355000 rw-p 00000000 00:00 0
7fd3bb379000-7fd3bb37a000 ---p 00000000 00:00 0
7fd3bb37a000-7fd3bb47a000 rw-p 00000000 00:00 0
7fd3bb48c000-7fd3bb48d000 ---p 00000000 00:00 0
7fd3bb48d000-7fd3bb58d000 rw-p 00000000 00:00 0
7fd3bb58d000-7fd3bb58e000 ---p 00000000 00:00 0
7fd3bb58e000-7fd3bb68e000 rw-p 00000000 00:00 0
7fd3bb6a0000-7fd3bb6a1000 ---p 00000000 00:00 0
7fd3bb6a1000-7fd3bb7a1000 rw-p 00000000 00:00 0
7fd3bb8f9000-7fd3bb8fa000 ---p 00000000 00:00 0
7fd3bb8fa000-7fd3bb9fa000 rw-p 00000000 00:00 0
7fd3bb9fa000-7fd3bb9fb000 ---p 00000000 00:00 0
7fd3bb9fb000-7fd3bbafb000 rw-p 00000000 00:00 0
7fd3bbafb000-7fd3bbafc000 ---p 00000000 00:00 0
7fd3bbafc000-7fd3bbbfc000 rw-p 00000000 00:00 0
7fd3bbbfc000-7fd3bbbfd000 ---p 00000000 00:00 0
7fd3bbbfd000-7fd3bbcfd000 rw-p 00000000 00:00 0
7fd3bbcfd000-7fd3bbcfe000 ---p 00000000 00:00 0
7fd3bbcfe000-7fd3bbdfe000 rw-p 00000000 00:00 0
7fd3bbdfe000-7fd3bbdff000 ---p 00000000 00:00 0
7fd3bbdff000-7fd3bbeff000 rw-p 00000000 00:00 0
7fd3bc000000-7fd3bc1e7000 rw-p 00000000 00:00 0
7fd3bc1e7000-7fd3c0000000 ---p 00000000 00:00 0
7fd3c0000000-7fd3c0227000 rw-p 00000000 00:00 0
7fd3c0227000-7fd3c4000000 ---p 00000000 00:00 0
7fd3c4000000-7fd3c4222000 rw-p 00000000 00:00 0
7fd3c4222000-7fd3c8000000 ---p 00000000 00:00 0
7fd3c8000000-7fd3c822c000 rw-p 00000000 00:00 0
7fd3c822c000-7fd3cc000000 ---p 00000000 00:00 0
7fd3cc000000-7fd3cc215000 rw-p 00000000 00:00 0
7fd3cc215000-7fd3d0000000 ---p 00000000 00:00 0
7fd3d0000000-7fd3d0234000 rw-p 00000000 00:00 0
7fd3d0234000-7fd3d4000000 ---p 00000000 00:00 0
7fd3d4000000-7fd3d42b9000 rw-p 00000000 00:00 0
7fd3d42b9000-7fd3d8000000 ---p 00000000 00:00 0
7fd3d8000000-7fd3d8232000 rw-p 00000000 00:00 0
7fd3d8232000-7fd3dc000000 ---p 00000000 00:00 0
7fd3dc000000-7fd3dc221000 rw-p 00000000 00:00 0
7fd3dc221000-7fd3e0000000 ---p 00000000 00:00 0
7fd3e0000000-7fd3e0267000 rw-p 00000000 00:00 0
7fd3e0267000-7fd3e4000000 ---p 00000000 00:00 0
7fd3e4000000-7fd3e4141000 rw-p 00000000 00:00 0
7fd3e4141000-7fd3e8000000 ---p 00000000 00:00 0
7fd3e8000000-7fd3e830a000 rw-p 00000000 00:00 0
7fd3e830a000-7fd3ec000000 ---p 00000000 00:00 0
7fd3ec000000-7fd3ec208000 rw-p 00000000 00:00 0
7fd3ec208000-7fd3f0000000 ---p 00000000 00:00 0
7fd3f0000000-7fd3f01fb000 rw-p 00000000 00:00 0
7fd3f01fb000-7fd3f4000000 ---p 00000000 00:00 0
7fd3f4000000-7fd3f41e8000 rw-p 00000000 00:00 0
7fd3f41e8000-7fd3f8000000 ---p 00000000 00:00 0
7fd3f8000000-7fd3f822a000 rw-p 00000000 00:00 0
7fd3f822a000-7fd3fc000000 ---p 00000000 00:00 0
7fd3fc000000-7fd3fc216000 rw-p 00000000 00:00 0
7fd3fc216000-7fd400000000 ---p 00000000 00:00 0
7fd400000000-7fd40030b000 rw-p 00000000 00:00 0
7fd40030b000-7fd404000000 ---p 00000000 00:00 0
7fd404000000-7fd4041e5000 rw-p 00000000 00:00 0
7fd4041e5000-7fd408000000 ---p 00000000 00:00 0
7fd408000000-7fd408154000 rw-p 00000000 00:00 0
7fd408154000-7fd40c000000 ---p 00000000 00:00 0
7fd40c000000-7fd40c220000 rw-p 00000000 00:00 0
7fd40c220000-7fd410000000 ---p 00000000 00:00 0
7fd410000000-7fd410229000 rw-p 00000000 00:00 0
7fd410229000-7fd414000000 ---p 00000000 00:00 0
7fd414000000-7fd414322000 rw-p 00000000 00:00 0
7fd414322000-7fd418000000 ---p 00000000 00:00 0
7fd418000000-7fd4181f6000 rw-p 00000000 00:00 0
7fd4181f6000-7fd41c000000 ---p 00000000 00:00 0
7fd41c000000-7fd41c324000 rw-p 00000000 00:00 0
7fd41c324000-7fd420000000 ---p 00000000 00:00 0
7fd4200a2000-7fd4200a3000 ---p 00000000 00:00 0
7fd4200a3000-7fd4201a3000 rw-p 00000000 00:00 0
7fd4201a3000-7fd4201a4000 ---p 00000000 00:00 0
7fd4201a4000-7fd4202a4000 rw-p 00000000 00:00 0
7fd4202da000-7fd4202db000 ---p 00000000 00:00 0
7fd4202db000-7fd4203db000 rw-p 00000000 00:00 0
7fd4203db000-7fd4203dc000 ---p 00000000 00:00 0
7fd4203dc000-7fd4204dc000 rw-p 00000000 00:00 0
7fd4204dc000-7fd4204dd000 ---p 00000000 00:00 0
7fd4204dd000-7fd4205dd000 rw-p 00000000 00:00 0
7fd4205ef000-7fd4205f0000 ---p 00000000 00:00 0
7fd4205f0000-7fd4206f0000 rw-p 00000000 00:00 0
7fd420848000-7fd420849000 ---p 00000000 00:00 0
7fd420849000-7fd420949000 rw-p 00000000 00:00 0
7fd420949000-7fd42094a000 ---p 00000000 00:00 0
7fd42094a000-7fd420a4a000 rw-p 00000000 00:00 0
7fd420a80000-7fd420a81000 ---p 00000000 00:00 0
7fd420a81000-7fd420b81000 rw-p 00000000 00:00 0
7fd420b81000-7fd420b82000 ---p 00000000 00:00 0
7fd420b82000-7fd420c82000 rw-p 00000000 00:00 0
7fd420c82000-7fd420c83000 ---p 00000000 00:00 0
7fd420c83000-7fd420d83000 rw-p 00000000 00:00 0
7fd420d95000-7fd420d96000 ---p 00000000 00:00 0
7fd420d96000-7fd420e96000 rw-p 00000000 00:00 0
7fd420fee000-7fd420fef000 ---p 00000000 00:00 0
7fd420fef000-7fd4210ef000 rw-p 00000000 00:00 0
7fd4210ef000-7fd4210f0000 ---p 00000000 00:00 0
7fd4210f0000-7fd4211f0000 rw-p 00000000 00:00 0
7fd421226000-7fd421227000 ---p 00000000 00:00 0
7fd421227000-7fd421327000 rw-p 00000000 00:00 0
7fd421327000-7fd421328000 ---p 00000000 00:00 0
7fd421328000-7fd421428000 rw-p 00000000 00:00 0
7fd421428000-7fd421429000 ---p 00000000 00:00 0
7fd421429000-7fd421529000 rw-p 00000000 00:00 0
7fd42153b000-7fd42153c000 ---p 00000000 00:00 0
7fd42153c000-7fd42163c000 rw-p 00000000 00:00 0
7fd421782000-7fd421783000 ---p 00000000 00:00 0
7fd421783000-7fd421b83000 rw-p 00000000 00:00 0
7fd421ba7000-7fd421ba8000 ---p 00000000 00:00 0
7fd421ba8000-7fd421ca8000 rw-p 00000000 00:00 0
7fd421ca8000-7fd421ca9000 ---p 00000000 00:00 0
7fd421ca9000-7fd421da9000 rw-p 00000000 00:00 0
7fd421da9000-7fd421daa000 ---p 00000000 00:00 0
7fd421daa000-7fd421eaa000 rw-p 00000000 00:00 0
7fd421eaa000-7fd421eab000 ---p 00000000 00:00 0
7fd421eab000-7fd421fab000 rw-p 00000000 00:00 0
7fd421fab000-7fd421fac000 ---p 00000000 00:00 0
7fd421fac000-7fd4220ac000 rw-p 00000000 00:00 0
7fd4220ac000-7fd4220c3000 r-xp 00000000 fd:00 90863 /usr/lib64/libnsl-2.17.so
7fd4220c3000-7fd4222c2000 ---p 00017000 fd:00 90863 /usr/lib64/libnsl-2.17.so
7fd4222c2000-7fd4222c3000 r--p 00016000 fd:00 90863 /usr/lib64/libnsl-2.17.so
7fd4222c3000-7fd4222c4000 rw-p 00017000 fd:00 90863 /usr/lib64/libnsl-2.17.so
7fd4222c4000-7fd4222c6000 rw-p 00000000 00:00 0
7fd4222c6000-7fd4222d1000 r-xp 00000000 fd:00 90874 /usr/lib64/libnss_nis-2.17.so
7fd4222d1000-7fd4224d0000 ---p 0000b000 fd:00 90874 /usr/lib64/libnss_nis-2.17.so
7fd4224d0000-7fd4224d1000 r--p 0000a000 fd:00 90874 /usr/lib64/libnss_nis-2.17.so
7fd4224d1000-7fd4224d2000 rw-p 0000b000 fd:00 90874 /usr/lib64/libnss_nis-2.17.so
7fd4224d2000-7fd4224de000 r-xp 00000000 fd:00 1065586 /usr/lib64/libnss_files-2.17.so
7fd4224de000-7fd4226dd000 ---p 0000c000 fd:00 1065586 /usr/lib64/libnss_files-2.17.so
7fd4226dd000-7fd4226de000 r--p 0000b000 fd:00 1065586 /usr/lib64/libnss_files-2.17.so
7fd4226de000-7fd4226df000 rw-p 0000c000 fd:00 1065586 /usr/lib64/libnss_files-2.17.so
7fd4226df000-7fd4226e5000 rw-p 00000000 00:00 0
7fd4226e5000-7fd4228a9000 r-xp 00000000 fd:00 13198 /usr/lib64/libc-2.17.so
7fd4228a9000-7fd422aa8000 ---p 001c4000 fd:00 13198 /usr/lib64/libc-2.17.so
7fd422aa8000-7fd422aac000 r--p 001c3000 fd:00 13198 /usr/lib64/libc-2.17.so
7fd422aac000-7fd422aae000 rw-p 001c7000 fd:00 13198 /usr/lib64/libc-2.17.so
7fd422aae000-7fd422ab3000 rw-p 00000000 00:00 0
7fd422ab3000-7fd422aca000 r-xp 00000000 fd:00 90878 /usr/lib64/libpthread-2.17.so
7fd422aca000-7fd422cc9000 ---p 00017000 fd:00 90878 /usr/lib64/libpthread-2.17.so
7fd422cc9000-7fd422cca000 r--p 00016000 fd:00 90878 /usr/lib64/libpthread-2.17.so
7fd422cca000-7fd422ccb000 rw-p 00017000 fd:00 90878 /usr/lib64/libpthread-2.17.so
7fd422ccb000-7fd422ccf000 rw-p 00000000 00:00 0
7fd422ccf000-7fd422ce4000 r-xp 00000000 fd:00 10587 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fd422ce4000-7fd422ee3000 ---p 00015000 fd:00 10587 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fd422ee3000-7fd422ee4000 r--p 00014000 fd:00 10587 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fd422ee4000-7fd422ee5000 rw-p 00015000 fd:00 10587 /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fd422ee5000-7fd422fe6000 r-xp 00000000 fd:00 90860 /usr/lib64/libm-2.17.so
7fd422fe6000-7fd4231e5000 ---p 00101000 fd:00 90860 /usr/lib64/libm-2.17.so
7fd4231e5000-7fd4231e6000 r--p 00100000 fd:00 90860 /usr/lib64/libm-2.17.so
7fd4231e6000-7fd4231e7000 rw-p 00101000 fd:00 90860 /usr/lib64/libm-2.17.so
7fd4231e7000-7fd4232d2000 r-xp 00000000 fd:02 338049191 <..trimmed..>/lib/libstdc++.so.6.0.20
7fd4232d2000-7fd4234d2000 ---p 000eb000 fd:02 338049191 <..trimmed..>/lib/libstdc++.so.6.0.20
7fd4234d2000-7fd4234da000 r--p 000eb000 fd:02 338049191 <..trimmed..>/lib/libstdc++.so.6.0.20
7fd4234da000-7fd4234dc000 rw-p 000f3000 fd:02 338049191 <..trimmed..>/lib/libstdc++.so.6.0.20
7fd4234dc000-7fd4234f1000 rw-p 00000000 00:00 0
7fd4234f1000-7fd4234f3000 r-xp 00000000 fd:00 1065700 /usr/lib64/libutil-2.17.so
7fd4234f3000-7fd4236f2000 ---p 00002000 fd:00 1065700 /usr/lib64/libutil-2.17.so
7fd4236f2000-7fd4236f3000 r--p 00001000 fd:00 1065700 /usr/lib64/libutil-2.17.so
7fd4236f3000-7fd4236f4000 rw-p 00002000 fd:00 1065700 /usr/lib64/libutil-2.17.so
7fd4236f4000-7fd4236f6000 r-xp 00000000 fd:00 12002 /usr/lib64/libdl-2.17.so
7fd4236f6000-7fd4238f6000 ---p 00002000 fd:00 12002 /usr/lib64/libdl-2.17.so
7fd4238f6000-7fd4238f7000 r--p 00002000 fd:00 12002 /usr/lib64/libdl-2.17.so
7fd4238f7000-7fd4238f8000 rw-p 00003000 fd:00 12002 /usr/lib64/libdl-2.17.so
7fd4238f8000-7fd4238ff000 r-xp 00000000 fd:00 1065594 /usr/lib64/librt-2.17.so
7fd4238ff000-7fd423afe000 ---p 00007000 fd:00 1065594 /usr/lib64/librt-2.17.so
7fd423afe000-7fd423aff000 r--p 00006000 fd:00 1065594 /usr/lib64/librt-2.17.so
7fd423aff000-7fd423b00000 rw-p 00007000 fd:00 1065594 /usr/lib64/librt-2.17.so
7fd423b00000-7fd423c2a000 r-xp 00000000 fd:02 338050204 <..trimmed..>
7fd423c2a000-7fd423e29000 ---p 0012a000 fd:02 338050204 <..trimmed..>
7fd423e29000-7fd423e2c000 r--p 00129000 fd:02 338050204 <..trimmed..>
7fd423e2c000-7fd423e32000 rw-p 0012c000 fd:02 338050204 <..trimmed..>
7fd423e32000-7fd4240dc000 r-xp 00000000 fd:02 337483017 <..trimmed..>
7fd4240dc000-7fd4242db000 ---p 002aa000 fd:02 337483017 <..trimmed..>
7fd4242db000-7fd4242f8000 r--p 002a9000 fd:02 337483017 <..trimmed..>
7fd4242f8000-7fd424304000 rw-p 002c6000 fd:02 337483017 <..trimmed..>
7fd424304000-7fd424ff0000 r-xp 00000000 fd:02 335960173 <..trimmed..>
7fd424ff0000-7fd4251f0000 ---p 00cec000 fd:02 335960173 <..trimmed..>
7fd4251f0000-7fd42524e000 r--p 00cec000 fd:02 335960173 <..trimmed..>
7fd42524e000-7fd425271000 rw-p 00d4a000 fd:02 335960173 <..trimmed..>
7fd425271000-7fd425276000 rw-p 00000000 00:00 0
7fd425276000-7fd425ffc000 r-xp 00000000 fd:02 338049189 <..trimmed..>
7fd425ffc000-7fd4261fc000 ---p 00d86000 fd:02 338049189 <..trimmed..>
7fd4261fc000-7fd426221000 r--p 00d86000 fd:02 338049189 <..trimmed..>
7fd426221000-7fd42624a000 rw-p 00dab000 fd:02 338049189 <..trimmed..>
7fd42624a000-7fd4262bd000 rw-p 00000000 00:00 0
7fd4262bd000-7fd4262f6000 r-xp 00000000 fd:02 336402206 <..trimmed..>
7fd4262f6000-7fd4264f5000 ---p 00039000 fd:02 336402206 <..trimmed..>
7fd4264f5000-7fd4264f6000 r--p 00038000 fd:02 336402206 <..trimmed..>
7fd4264f6000-7fd4264f8000 rw-p 00039000 fd:02 336402206 <..trimmed..>
7fd4264f8000-7fd4264f9000 rw-p 00000000 00:00 0
7fd4264f9000-7fd42652f000 r-xp 00000000 fd:02 335872889 <..trimmed..>
7fd42652f000-7fd42672e000 ---p 00036000 fd:02 335872889 <..trimmed..>
7fd42672e000-7fd426731000 r--p 00035000 fd:02 335872889 <..trimmed..>
7fd426731000-7fd426732000 rw-p 00038000 fd:02 335872889 <..trimmed..>
7fd426732000-7fd426b93000 r-xp 00000000 fd:02 335942741 <..trimmed..>
7fd426b93000-7fd426d92000 ---p 00461000 fd:02 335942741 <..trimmed..>
7fd426d92000-7fd426dae000 r--p 00460000 fd:02 335942741 <..trimmed..>
7fd426dae000-7fd426ddb000 rw-p 0047c000 fd:02 335942741 <..trimmed..>
7fd426ddb000-7fd426ddc000 rw-p 00000000 00:00 0
7fd426ddc000-7fd426e1d000 r-xp 00000000 fd:02 339133453 <..trimmed..>
7fd426e1d000-7fd42701c000 ---p 00041000 fd:02 339133453 <..trimmed..>
7fd42701c000-7fd42701e000 r--p 00040000 fd:02 339133453 <..trimmed..>
7fd42701e000-7fd427020000 rw-p 00042000 fd:02 339133453 <..trimmed..>
7fd427020000-7fd427042000 r-xp 00000000 fd:00 13191 /usr/lib64/ld-2.17.so
7fd42708d000-7fd42708e000 ---p 00000000 00:00 0
7fd42708e000-7fd42718e000 rw-p 00000000 00:00 0
7fd427214000-7fd427215000 ---p 00000000 00:00 0
7fd427215000-7fd427225000 rw-p 00000000 00:00 0
7fd427225000-7fd427226000 ---p 00000000 00:00 0
7fd427226000-7fd42722e000 rw-p 00000000 00:00 0
7fd42723c000-7fd427241000 rw-p 00000000 00:00 0
7fd427241000-7fd427242000 r--p 00021000 fd:00 13191 /usr/lib64/ld-2.17.so
7fd427242000-7fd427243000 rw-p 00022000 fd:00 13191 /usr/lib64/ld-2.17.so
7fd427243000-7fd427244000 rw-p 00000000 00:00 0
7ffe3eb9b000-7ffe3ebbc000 rw-p 00000000 00:00 0 [stack]
7ffe3ebbf000-7ffe3ebc1000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
[-- Attachment #5: valgrind.log --]
[-- Type: text/x-log, Size: 274 bytes --]
==9052== Massif, a heap profiler
==9052== Copyright (C) 2003-2017, and GNU GPL'd, by Nicholas Nethercote
==9052== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info
==9052== Command: mt-multiconnect-server 65000
==9052== Parent PID: 1481
==9052==
==9052==
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: GLIBC malloc behavior question
2023-02-07 15:06 GLIBC malloc behavior question Nikolay Shustov
@ 2023-02-07 16:16 ` Paulo César Pereira de Andrade
2023-02-07 16:55 ` Nikolay Shustov
0 siblings, 1 reply; 8+ messages in thread
From: Paulo César Pereira de Andrade @ 2023-02-07 16:16 UTC (permalink / raw)
To: Nikolay.Shustov; +Cc: libc-alpha
Em ter., 7 de fev. de 2023 às 12:07, Nikolay Shustov via Libc-alpha
<libc-alpha@sourceware.org> escreveu:
>
> Hi,
> I have a question about the malloc() behavior which I observe.
> The synopsis is that the during the stress load, the application
> aggressively allocates virtual memory without any upper limit.
> Just to note, after application is loaded just with the peak of activity
> and goes idle, its virtual memory doesn't scale back (I do not expect
> much of that though - should I?).
There is no garbage collector thread or something similar in some
worker thread. But maybe something similar could be done in your
code.
> The application is heavily multithreaded; at its peak of its activitiy
> it creates new threads and destroys them at a pace of approx. 100/second.
> After the long and tedious investigation I dare to say that there are no
> memory leaks involved.
> (Well, there were memory leaks and I first went after those; found and
> fixed - but the result did not change much.)
You might experiment with a tradeoff speed vs memory usage. The
minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
see 'man mallopt' for other options.
> The application is cross-platform and runs on Windows and some other
> platforms too.
> There is an OS abstraction layer that provides the unified thread and
> memory allocation API for business logic, but the business logic that
> triggers memory allocations is platform-independent.
> There are no progressive memory allocations in OS abstraction layer
> which could be blamed for the memory growth.
>
> The thing is, on Windows, for the same activity there is no such
> application memory growth at all.
> It allocates memory moderately and scales back after peak of activity.
> This makes me think it is not the business logic to be blamed (to the
> extent of that it does not leak memory).
>
> I used valigrind to profile for memory leaks and heap usage.
> Please see massif outputs attached (some callstacks had to be trimmed out).
> I am also attaching the memory map for the application (run without
> valgrind); snapshot is taken after all the threads but main were
> destroyed and application is idle.
>
> The pace of the virtual memory growth is not quite linear.
Most likely there are long lived objects doing contention and also
probably memory fragmentation, preventing returning memory to
the system after a free call.
> From my observation, it allocates a big hunk in the beginning of the
> peak loading, then in some time starts to grow in steps of ~80Mb / 10
> seconds, then after some times starts to steadily grow it at pace of
> ~2Mb/second.
>
> Some stats from the host:
>
> OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
>
> ldd -version
>
> ldd (GNU libc) 2.17
> Copyright (C) 2012 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions. There
> is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> PURPOSE.
> Written by Roland McGrath and Ulrich Drepper.
>
> uname -a
>
> Linux <skipped> 3.10.0-1160.53.1.el7.x86_64 #1 SMP Thu Dec 16
> 10:19:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>
>
> At a peak load, the number of application threads is ~180.
> If application is left running, I did not observe it would hit any max
> virtual memory threshold and eventually ends up with hitting ulimit.
>
> My questions are:
>
> - Is this memory growth an expected behavior?
It should eventually stabilize. But it is possible that some allocation
pattern is causing both, fragmentation and long lived objects preventing
consolidation of memory chunks.
> - What can be done to prevent it from happening?
First approach is MALLOC_ARENA_MAX. After that some coding
patterns might help, for example, have large long lived objects allocated
from the same thread, preferably at startup.
Can also attempt to cache some memory, but note that caching is also
an easy way to get contention. To avoid this, you could use memory from
buffers from mmap.
Depending on your code, you can also experiment with jemalloc or
tcmalloc. I would suggest tcmalloc, as its main feature is to work
in multithreaded environments:
https://gperftools.github.io/gperftools/tcmalloc.html
Glibc newer than 2.17 has a per thread cache, but the issue you
are experimenting is not it being slow, but memory usage. AFAIK tcmalloc
has a kind of garbage collector, but it should not be much different than
glibc consolidation logic; it should only run during free, and if there is
some contention, it might not be able to release memory.
> Thanks in advance,
> - Nikolay
Thanks!
Paulo
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: GLIBC malloc behavior question
2023-02-07 16:16 ` Paulo César Pereira de Andrade
@ 2023-02-07 16:55 ` Nikolay Shustov
2023-02-07 17:26 ` Ben Woodard
0 siblings, 1 reply; 8+ messages in thread
From: Nikolay Shustov @ 2023-02-07 16:55 UTC (permalink / raw)
To: Paulo César Pereira de Andrade; +Cc: libc-alpha
[-- Attachment #1: Type: text/plain, Size: 5938 bytes --]
> There is no garbage collector thread or something similar in some
> worker thread. But maybe something similar could be done in your
> code.
No, there is nothing of the kind in the application.
> You might experiment with a tradeoff speed vs memory usage. The
> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
> see 'man mallopt' for other options.
MALLOC_ARENA_MAX=1 made a huge difference.
The initial memory allocations went done on the scale of magnitude.
In fact, I do not see that much of the application slowdown but this
will need more profiling.
The stable allocations growth is still ~2Mb/second.
I am going to investigate your idea of long living objects
contention/memory fragmentation.
This sounds very probably, even though I do not see real memory leaks
even after all the aux threads died.
I have TLS instances in use, maybe those really get in the way.
Thanks a lot for your help.
If/when I find something new or interesting, I will send an update -
hope it will help someone else, too.
Regards,
- Nikolay
On 2/7/23 11:16, Paulo César Pereira de Andrade wrote:
> Em ter., 7 de fev. de 2023 às 12:07, Nikolay Shustov via Libc-alpha
> <libc-alpha@sourceware.org> escreveu:
>> Hi,
>> I have a question about the malloc() behavior which I observe.
>> The synopsis is that the during the stress load, the application
>> aggressively allocates virtual memory without any upper limit.
>> Just to note, after application is loaded just with the peak of activity
>> and goes idle, its virtual memory doesn't scale back (I do not expect
>> much of that though - should I?).
> There is no garbage collector thread or something similar in some
> worker thread. But maybe something similar could be done in your
> code.
>
>> The application is heavily multithreaded; at its peak of its activitiy
>> it creates new threads and destroys them at a pace of approx. 100/second.
>> After the long and tedious investigation I dare to say that there are no
>> memory leaks involved.
>> (Well, there were memory leaks and I first went after those; found and
>> fixed - but the result did not change much.)
> You might experiment with a tradeoff speed vs memory usage. The
> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
> see 'man mallopt' for other options.
>
>> The application is cross-platform and runs on Windows and some other
>> platforms too.
>> There is an OS abstraction layer that provides the unified thread and
>> memory allocation API for business logic, but the business logic that
>> triggers memory allocations is platform-independent.
>> There are no progressive memory allocations in OS abstraction layer
>> which could be blamed for the memory growth.
>>
>> The thing is, on Windows, for the same activity there is no such
>> application memory growth at all.
>> It allocates memory moderately and scales back after peak of activity.
>> This makes me think it is not the business logic to be blamed (to the
>> extent of that it does not leak memory).
>>
>> I used valigrind to profile for memory leaks and heap usage.
>> Please see massif outputs attached (some callstacks had to be trimmed out).
>> I am also attaching the memory map for the application (run without
>> valgrind); snapshot is taken after all the threads but main were
>> destroyed and application is idle.
>>
>> The pace of the virtual memory growth is not quite linear.
> Most likely there are long lived objects doing contention and also
> probably memory fragmentation, preventing returning memory to
> the system after a free call.
>
>> From my observation, it allocates a big hunk in the beginning of the
>> peak loading, then in some time starts to grow in steps of ~80Mb / 10
>> seconds, then after some times starts to steadily grow it at pace of
>> ~2Mb/second.
>>
>> Some stats from the host:
>>
>> OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
>>
>> ldd -version
>>
>> ldd (GNU libc) 2.17
>> Copyright (C) 2012 Free Software Foundation, Inc.
>> This is free software; see the source for copying conditions. There
>> is NO
>> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
>> PURPOSE.
>> Written by Roland McGrath and Ulrich Drepper.
>>
>> uname -a
>>
>> Linux <skipped> 3.10.0-1160.53.1.el7.x86_64 #1 SMP Thu Dec 16
>> 10:19:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>>
>>
>> At a peak load, the number of application threads is ~180.
>> If application is left running, I did not observe it would hit any max
>> virtual memory threshold and eventually ends up with hitting ulimit.
>>
>> My questions are:
>>
>> - Is this memory growth an expected behavior?
> It should eventually stabilize. But it is possible that some allocation
> pattern is causing both, fragmentation and long lived objects preventing
> consolidation of memory chunks.
>
>> - What can be done to prevent it from happening?
> First approach is MALLOC_ARENA_MAX. After that some coding
> patterns might help, for example, have large long lived objects allocated
> from the same thread, preferably at startup.
> Can also attempt to cache some memory, but note that caching is also
> an easy way to get contention. To avoid this, you could use memory from
> buffers from mmap.
>
> Depending on your code, you can also experiment with jemalloc or
> tcmalloc. I would suggest tcmalloc, as its main feature is to work
> in multithreaded environments:
>
> https://gperftools.github.io/gperftools/tcmalloc.html
>
> Glibc newer than 2.17 has a per thread cache, but the issue you
> are experimenting is not it being slow, but memory usage. AFAIK tcmalloc
> has a kind of garbage collector, but it should not be much different than
> glibc consolidation logic; it should only run during free, and if there is
> some contention, it might not be able to release memory.
>
>> Thanks in advance,
>> - Nikolay
> Thanks!
>
> Paulo
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: GLIBC malloc behavior question
2023-02-07 16:55 ` Nikolay Shustov
@ 2023-02-07 17:26 ` Ben Woodard
2023-02-07 18:01 ` Nikolay Shustov
0 siblings, 1 reply; 8+ messages in thread
From: Ben Woodard @ 2023-02-07 17:26 UTC (permalink / raw)
To: Nikolay.Shustov, Paulo César Pereira de Andrade; +Cc: libc-alpha
On 2/7/23 08:55, Nikolay Shustov via Libc-alpha wrote:
>> There is no garbage collector thread or something similar in some
>> worker thread. But maybe something similar could be done in your
>> code.
>
> No, there is nothing of the kind in the application.
>
>> You might experiment with a tradeoff speed vs memory usage. The
>> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
>> see 'man mallopt' for other options.
>
> MALLOC_ARENA_MAX=1 made a huge difference.
I just wanted to point out that it isn't 1 or the default. That was most
likely a simple test to test a hypothesis about what could be going
wrong. This is a tunable knob and your application could have a sweet
spot. For some of the applications that I help support, we have
empirically found that a good number is slightly lower than the number
of processors that they system has. e.g. if their are 16 cores giving it
12 arenas doesn't impact speed but makes the memory footprint more compact.
> The initial memory allocations went done on the scale of magnitude.
> In fact, I do not see that much of the application slowdown but this
> will need more profiling.
> The stable allocations growth is still ~2Mb/second.
>
> I am going to investigate your idea of long living objects
> contention/memory fragmentation.
>
> This sounds very probably, even though I do not see real memory leaks
> even after all the aux threads died.
> I have TLS instances in use, maybe those really get in the way.
>
> Thanks a lot for your help.
> If/when I find something new or interesting, I will send an update -
> hope it will help someone else, too.
>
> Regards,
> - Nikolay
>
> On 2/7/23 11:16, Paulo César Pereira de Andrade wrote:
>> Em ter., 7 de fev. de 2023 às 12:07, Nikolay Shustov via Libc-alpha
>> <libc-alpha@sourceware.org> escreveu:
>>> Hi,
>>> I have a question about the malloc() behavior which I observe.
>>> The synopsis is that the during the stress load, the application
>>> aggressively allocates virtual memory without any upper limit.
>>> Just to note, after application is loaded just with the peak of
>>> activity
>>> and goes idle, its virtual memory doesn't scale back (I do not expect
>>> much of that though - should I?).
>> There is no garbage collector thread or something similar in some
>> worker thread. But maybe something similar could be done in your
>> code.
>>
>>> The application is heavily multithreaded; at its peak of its activitiy
>>> it creates new threads and destroys them at a pace of approx.
>>> 100/second.
>>> After the long and tedious investigation I dare to say that there
>>> are no
>>> memory leaks involved.
>>> (Well, there were memory leaks and I first went after those; found and
>>> fixed - but the result did not change much.)
>> You might experiment with a tradeoff speed vs memory usage. The
>> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
>> see 'man mallopt' for other options.
>>
>>> The application is cross-platform and runs on Windows and some other
>>> platforms too.
>>> There is an OS abstraction layer that provides the unified thread and
>>> memory allocation API for business logic, but the business logic that
>>> triggers memory allocations is platform-independent.
>>> There are no progressive memory allocations in OS abstraction layer
>>> which could be blamed for the memory growth.
>>>
>>> The thing is, on Windows, for the same activity there is no such
>>> application memory growth at all.
>>> It allocates memory moderately and scales back after peak of activity.
>>> This makes me think it is not the business logic to be blamed (to the
>>> extent of that it does not leak memory).
>>>
>>> I used valigrind to profile for memory leaks and heap usage.
>>> Please see massif outputs attached (some callstacks had to be
>>> trimmed out).
>>> I am also attaching the memory map for the application (run without
>>> valgrind); snapshot is taken after all the threads but main were
>>> destroyed and application is idle.
>>>
>>> The pace of the virtual memory growth is not quite linear.
>> Most likely there are long lived objects doing contention and also
>> probably memory fragmentation, preventing returning memory to
>> the system after a free call.
>>
>>> From my observation, it allocates a big hunk in the beginning of the
>>> peak loading, then in some time starts to grow in steps of ~80Mb / 10
>>> seconds, then after some times starts to steadily grow it at pace of
>>> ~2Mb/second.
>>>
>>> Some stats from the host:
>>>
>>> OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
>>>
>>> ldd -version
>>>
>>> ldd (GNU libc) 2.17
>>> Copyright (C) 2012 Free Software Foundation, Inc.
>>> This is free software; see the source for copying conditions.
>>> There
>>> is NO
>>> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
>>> PURPOSE.
>>> Written by Roland McGrath and Ulrich Drepper.
>>>
>>> uname -a
>>>
>>> Linux <skipped> 3.10.0-1160.53.1.el7.x86_64 #1 SMP Thu Dec 16
>>> 10:19:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>>>
>>>
>>> At a peak load, the number of application threads is ~180.
>>> If application is left running, I did not observe it would hit any max
>>> virtual memory threshold and eventually ends up with hitting ulimit.
>>>
>>> My questions are:
>>>
>>> - Is this memory growth an expected behavior?
>> It should eventually stabilize. But it is possible that some
>> allocation
>> pattern is causing both, fragmentation and long lived objects preventing
>> consolidation of memory chunks.
>>
>>> - What can be done to prevent it from happening?
>> First approach is MALLOC_ARENA_MAX. After that some coding
>> patterns might help, for example, have large long lived objects
>> allocated
>> from the same thread, preferably at startup.
>> Can also attempt to cache some memory, but note that caching is also
>> an easy way to get contention. To avoid this, you could use memory from
>> buffers from mmap.
>>
>> Depending on your code, you can also experiment with jemalloc or
>> tcmalloc. I would suggest tcmalloc, as its main feature is to work
>> in multithreaded environments:
>>
>> https://gperftools.github.io/gperftools/tcmalloc.html
>>
>> Glibc newer than 2.17 has a per thread cache, but the issue you
>> are experimenting is not it being slow, but memory usage. AFAIK tcmalloc
>> has a kind of garbage collector, but it should not be much different
>> than
>> glibc consolidation logic; it should only run during free, and if
>> there is
>> some contention, it might not be able to release memory.
>>
>>> Thanks in advance,
>>> - Nikolay
>> Thanks!
>>
>> Paulo
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: GLIBC malloc behavior question
2023-02-07 17:26 ` Ben Woodard
@ 2023-02-07 18:01 ` Nikolay Shustov
2023-02-07 20:56 ` Nikolay Shustov
0 siblings, 1 reply; 8+ messages in thread
From: Nikolay Shustov @ 2023-02-07 18:01 UTC (permalink / raw)
To: Ben Woodard, Paulo César Pereira de Andrade; +Cc: libc-alpha
[-- Attachment #1: Type: text/plain, Size: 7254 bytes --]
Got it, thanks.
For now, I am using this tunable merely to hep to surface whatever I
might have missed in terms of long living objects and heap fragmentation.
But I am definitely going to play with it when I am reasonably sure that
it is what does the major impact.
On 2/7/23 12:26, Ben Woodard wrote:
>
> On 2/7/23 08:55, Nikolay Shustov via Libc-alpha wrote:
>>> There is no garbage collector thread or something similar in some
>>> worker thread. But maybe something similar could be done in your
>>> code.
>>
>> No, there is nothing of the kind in the application.
>>
>>> You might experiment with a tradeoff speed vs memory usage. The
>>> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
>>> see 'man mallopt' for other options.
>>
>> MALLOC_ARENA_MAX=1 made a huge difference.
> I just wanted to point out that it isn't 1 or the default. That was
> most likely a simple test to test a hypothesis about what could be
> going wrong. This is a tunable knob and your application could have a
> sweet spot. For some of the applications that I help support, we have
> empirically found that a good number is slightly lower than the number
> of processors that they system has. e.g. if their are 16 cores giving
> it 12 arenas doesn't impact speed but makes the memory footprint more
> compact.
>> The initial memory allocations went done on the scale of magnitude.
>> In fact, I do not see that much of the application slowdown but this
>> will need more profiling.
>> The stable allocations growth is still ~2Mb/second.
>>
>> I am going to investigate your idea of long living objects
>> contention/memory fragmentation.
>>
>> This sounds very probably, even though I do not see real memory leaks
>> even after all the aux threads died.
>> I have TLS instances in use, maybe those really get in the way.
>>
>> Thanks a lot for your help.
>> If/when I find something new or interesting, I will send an update -
>> hope it will help someone else, too.
>>
>> Regards,
>> - Nikolay
>>
>> On 2/7/23 11:16, Paulo César Pereira de Andrade wrote:
>>> Em ter., 7 de fev. de 2023 às 12:07, Nikolay Shustov via Libc-alpha
>>> <libc-alpha@sourceware.org> escreveu:
>>>> Hi,
>>>> I have a question about the malloc() behavior which I observe.
>>>> The synopsis is that the during the stress load, the application
>>>> aggressively allocates virtual memory without any upper limit.
>>>> Just to note, after application is loaded just with the peak of
>>>> activity
>>>> and goes idle, its virtual memory doesn't scale back (I do not expect
>>>> much of that though - should I?).
>>> There is no garbage collector thread or something similar in some
>>> worker thread. But maybe something similar could be done in your
>>> code.
>>>
>>>> The application is heavily multithreaded; at its peak of its activitiy
>>>> it creates new threads and destroys them at a pace of approx.
>>>> 100/second.
>>>> After the long and tedious investigation I dare to say that there
>>>> are no
>>>> memory leaks involved.
>>>> (Well, there were memory leaks and I first went after those; found and
>>>> fixed - but the result did not change much.)
>>> You might experiment with a tradeoff speed vs memory usage. The
>>> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
>>> see 'man mallopt' for other options.
>>>
>>>> The application is cross-platform and runs on Windows and some other
>>>> platforms too.
>>>> There is an OS abstraction layer that provides the unified thread and
>>>> memory allocation API for business logic, but the business logic that
>>>> triggers memory allocations is platform-independent.
>>>> There are no progressive memory allocations in OS abstraction layer
>>>> which could be blamed for the memory growth.
>>>>
>>>> The thing is, on Windows, for the same activity there is no such
>>>> application memory growth at all.
>>>> It allocates memory moderately and scales back after peak of activity.
>>>> This makes me think it is not the business logic to be blamed (to the
>>>> extent of that it does not leak memory).
>>>>
>>>> I used valigrind to profile for memory leaks and heap usage.
>>>> Please see massif outputs attached (some callstacks had to be
>>>> trimmed out).
>>>> I am also attaching the memory map for the application (run without
>>>> valgrind); snapshot is taken after all the threads but main were
>>>> destroyed and application is idle.
>>>>
>>>> The pace of the virtual memory growth is not quite linear.
>>> Most likely there are long lived objects doing contention and also
>>> probably memory fragmentation, preventing returning memory to
>>> the system after a free call.
>>>
>>>> From my observation, it allocates a big hunk in the beginning of the
>>>> peak loading, then in some time starts to grow in steps of ~80Mb / 10
>>>> seconds, then after some times starts to steadily grow it at pace of
>>>> ~2Mb/second.
>>>>
>>>> Some stats from the host:
>>>>
>>>> OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
>>>>
>>>> ldd -version
>>>>
>>>> ldd (GNU libc) 2.17
>>>> Copyright (C) 2012 Free Software Foundation, Inc.
>>>> This is free software; see the source for copying conditions.
>>>> There
>>>> is NO
>>>> warranty; not even for MERCHANTABILITY or FITNESS FOR A
>>>> PARTICULAR
>>>> PURPOSE.
>>>> Written by Roland McGrath and Ulrich Drepper.
>>>>
>>>> uname -a
>>>>
>>>> Linux <skipped> 3.10.0-1160.53.1.el7.x86_64 #1 SMP Thu Dec 16
>>>> 10:19:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>>>>
>>>>
>>>> At a peak load, the number of application threads is ~180.
>>>> If application is left running, I did not observe it would hit any
>>>> max
>>>> virtual memory threshold and eventually ends up with hitting ulimit.
>>>>
>>>> My questions are:
>>>>
>>>> - Is this memory growth an expected behavior?
>>> It should eventually stabilize. But it is possible that some
>>> allocation
>>> pattern is causing both, fragmentation and long lived objects
>>> preventing
>>> consolidation of memory chunks.
>>>
>>>> - What can be done to prevent it from happening?
>>> First approach is MALLOC_ARENA_MAX. After that some coding
>>> patterns might help, for example, have large long lived objects
>>> allocated
>>> from the same thread, preferably at startup.
>>> Can also attempt to cache some memory, but note that caching is also
>>> an easy way to get contention. To avoid this, you could use memory from
>>> buffers from mmap.
>>>
>>> Depending on your code, you can also experiment with jemalloc or
>>> tcmalloc. I would suggest tcmalloc, as its main feature is to work
>>> in multithreaded environments:
>>>
>>> https://gperftools.github.io/gperftools/tcmalloc.html
>>>
>>> Glibc newer than 2.17 has a per thread cache, but the issue you
>>> are experimenting is not it being slow, but memory usage. AFAIK
>>> tcmalloc
>>> has a kind of garbage collector, but it should not be much different
>>> than
>>> glibc consolidation logic; it should only run during free, and if
>>> there is
>>> some contention, it might not be able to release memory.
>>>
>>>> Thanks in advance,
>>>> - Nikolay
>>> Thanks!
>>>
>>> Paulo
>>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: GLIBC malloc behavior question
2023-02-07 18:01 ` Nikolay Shustov
@ 2023-02-07 20:56 ` Nikolay Shustov
2023-02-07 21:38 ` Paulo César Pereira de Andrade
0 siblings, 1 reply; 8+ messages in thread
From: Nikolay Shustov @ 2023-02-07 20:56 UTC (permalink / raw)
To: Ben Woodard, Paulo César Pereira de Andrade; +Cc: libc-alpha
[-- Attachment #1: Type: text/plain, Size: 8497 bytes --]
I was able to increase ratio of threads reuse in application (i.e. reuse
running thread instead of destroying it and creating new) and it seemed
to have a positive effect on the amount of allocated memory.
Valgrind reports some small memory leaks - and I know from experience
these reports are bogus as they deal with its inability to detect thread
local storage key destructor function invocations.
I was able to ensure these are invoked as it supposed to be. The amount
of the reported leaks is really small and in no way could contribute to
gigabytes of VIRT been held in allocations.
I experimented more and at this stage most of my suspicions are towards
heap fragmentation.
However, if all the used memory has been released (as I expect it do be
when all but main threads exited), should have I expected mmaped regions
to be released and process virtual memory size shrink?
Or GLIBC would do it upon some special event? (can it be forced somehow?)
Thanks,
- Nikolay
On 2/7/23 13:01, Nikolay Shustov wrote:
> Got it, thanks.
> For now, I am using this tunable merely to hep to surface whatever I
> might have missed in terms of long living objects and heap fragmentation.
> But I am definitely going to play with it when I am reasonably sure
> that it is what does the major impact.
>
> On 2/7/23 12:26, Ben Woodard wrote:
>>
>> On 2/7/23 08:55, Nikolay Shustov via Libc-alpha wrote:
>>>> There is no garbage collector thread or something similar in some
>>>> worker thread. But maybe something similar could be done in your
>>>> code.
>>>
>>> No, there is nothing of the kind in the application.
>>>
>>>> You might experiment with a tradeoff speed vs memory usage. The
>>>> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
>>>> see 'man mallopt' for other options.
>>>
>>> MALLOC_ARENA_MAX=1 made a huge difference.
>> I just wanted to point out that it isn't 1 or the default. That was
>> most likely a simple test to test a hypothesis about what could be
>> going wrong. This is a tunable knob and your application could have a
>> sweet spot. For some of the applications that I help support, we have
>> empirically found that a good number is slightly lower than the
>> number of processors that they system has. e.g. if their are 16 cores
>> giving it 12 arenas doesn't impact speed but makes the memory
>> footprint more compact.
>>> The initial memory allocations went done on the scale of magnitude.
>>> In fact, I do not see that much of the application slowdown but this
>>> will need more profiling.
>>> The stable allocations growth is still ~2Mb/second.
>>>
>>> I am going to investigate your idea of long living objects
>>> contention/memory fragmentation.
>>>
>>> This sounds very probably, even though I do not see real memory
>>> leaks even after all the aux threads died.
>>> I have TLS instances in use, maybe those really get in the way.
>>>
>>> Thanks a lot for your help.
>>> If/when I find something new or interesting, I will send an update -
>>> hope it will help someone else, too.
>>>
>>> Regards,
>>> - Nikolay
>>>
>>> On 2/7/23 11:16, Paulo César Pereira de Andrade wrote:
>>>> Em ter., 7 de fev. de 2023 às 12:07, Nikolay Shustov via Libc-alpha
>>>> <libc-alpha@sourceware.org> escreveu:
>>>>> Hi,
>>>>> I have a question about the malloc() behavior which I observe.
>>>>> The synopsis is that the during the stress load, the application
>>>>> aggressively allocates virtual memory without any upper limit.
>>>>> Just to note, after application is loaded just with the peak of
>>>>> activity
>>>>> and goes idle, its virtual memory doesn't scale back (I do not expect
>>>>> much of that though - should I?).
>>>> There is no garbage collector thread or something similar in some
>>>> worker thread. But maybe something similar could be done in your
>>>> code.
>>>>
>>>>> The application is heavily multithreaded; at its peak of its
>>>>> activitiy
>>>>> it creates new threads and destroys them at a pace of approx.
>>>>> 100/second.
>>>>> After the long and tedious investigation I dare to say that there
>>>>> are no
>>>>> memory leaks involved.
>>>>> (Well, there were memory leaks and I first went after those; found
>>>>> and
>>>>> fixed - but the result did not change much.)
>>>> You might experiment with a tradeoff speed vs memory usage. The
>>>> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
>>>> see 'man mallopt' for other options.
>>>>
>>>>> The application is cross-platform and runs on Windows and some other
>>>>> platforms too.
>>>>> There is an OS abstraction layer that provides the unified thread and
>>>>> memory allocation API for business logic, but the business logic that
>>>>> triggers memory allocations is platform-independent.
>>>>> There are no progressive memory allocations in OS abstraction layer
>>>>> which could be blamed for the memory growth.
>>>>>
>>>>> The thing is, on Windows, for the same activity there is no such
>>>>> application memory growth at all.
>>>>> It allocates memory moderately and scales back after peak of
>>>>> activity.
>>>>> This makes me think it is not the business logic to be blamed (to the
>>>>> extent of that it does not leak memory).
>>>>>
>>>>> I used valigrind to profile for memory leaks and heap usage.
>>>>> Please see massif outputs attached (some callstacks had to be
>>>>> trimmed out).
>>>>> I am also attaching the memory map for the application (run without
>>>>> valgrind); snapshot is taken after all the threads but main were
>>>>> destroyed and application is idle.
>>>>>
>>>>> The pace of the virtual memory growth is not quite linear.
>>>> Most likely there are long lived objects doing contention and also
>>>> probably memory fragmentation, preventing returning memory to
>>>> the system after a free call.
>>>>
>>>>> From my observation, it allocates a big hunk in the beginning of
>>>>> the
>>>>> peak loading, then in some time starts to grow in steps of ~80Mb / 10
>>>>> seconds, then after some times starts to steadily grow it at pace of
>>>>> ~2Mb/second.
>>>>>
>>>>> Some stats from the host:
>>>>>
>>>>> OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
>>>>>
>>>>> ldd -version
>>>>>
>>>>> ldd (GNU libc) 2.17
>>>>> Copyright (C) 2012 Free Software Foundation, Inc.
>>>>> This is free software; see the source for copying conditions.
>>>>> There
>>>>> is NO
>>>>> warranty; not even for MERCHANTABILITY or FITNESS FOR A
>>>>> PARTICULAR
>>>>> PURPOSE.
>>>>> Written by Roland McGrath and Ulrich Drepper.
>>>>>
>>>>> uname -a
>>>>>
>>>>> Linux <skipped> 3.10.0-1160.53.1.el7.x86_64 #1 SMP Thu Dec 16
>>>>> 10:19:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>>>>>
>>>>>
>>>>> At a peak load, the number of application threads is ~180.
>>>>> If application is left running, I did not observe it would hit
>>>>> any max
>>>>> virtual memory threshold and eventually ends up with hitting ulimit.
>>>>>
>>>>> My questions are:
>>>>>
>>>>> - Is this memory growth an expected behavior?
>>>> It should eventually stabilize. But it is possible that some
>>>> allocation
>>>> pattern is causing both, fragmentation and long lived objects
>>>> preventing
>>>> consolidation of memory chunks.
>>>>
>>>>> - What can be done to prevent it from happening?
>>>> First approach is MALLOC_ARENA_MAX. After that some coding
>>>> patterns might help, for example, have large long lived objects
>>>> allocated
>>>> from the same thread, preferably at startup.
>>>> Can also attempt to cache some memory, but note that caching is
>>>> also
>>>> an easy way to get contention. To avoid this, you could use memory
>>>> from
>>>> buffers from mmap.
>>>>
>>>> Depending on your code, you can also experiment with jemalloc or
>>>> tcmalloc. I would suggest tcmalloc, as its main feature is to work
>>>> in multithreaded environments:
>>>>
>>>> https://gperftools.github.io/gperftools/tcmalloc.html
>>>>
>>>> Glibc newer than 2.17 has a per thread cache, but the issue you
>>>> are experimenting is not it being slow, but memory usage. AFAIK
>>>> tcmalloc
>>>> has a kind of garbage collector, but it should not be much
>>>> different than
>>>> glibc consolidation logic; it should only run during free, and if
>>>> there is
>>>> some contention, it might not be able to release memory.
>>>>
>>>>> Thanks in advance,
>>>>> - Nikolay
>>>> Thanks!
>>>>
>>>> Paulo
>>>
>>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: GLIBC malloc behavior question
2023-02-07 20:56 ` Nikolay Shustov
@ 2023-02-07 21:38 ` Paulo César Pereira de Andrade
2023-02-07 23:41 ` Nikolay Shustov
0 siblings, 1 reply; 8+ messages in thread
From: Paulo César Pereira de Andrade @ 2023-02-07 21:38 UTC (permalink / raw)
To: Nikolay.Shustov; +Cc: Ben Woodard, libc-alpha
Em ter., 7 de fev. de 2023 às 17:56, Nikolay Shustov
<nikolay.shustov@gmail.com> escreveu:
>
> I was able to increase ratio of threads reuse in application (i.e. reuse running thread instead of destroying it and creating new) and it seemed to have a positive effect on the amount of allocated memory.
> Valgrind reports some small memory leaks - and I know from experience these reports are bogus as they deal with its inability to detect thread local storage key destructor function invocations.
> I was able to ensure these are invoked as it supposed to be. The amount of the reported leaks is really small and in no way could contribute to gigabytes of VIRT been held in allocations.
>
> I experimented more and at this stage most of my suspicions are towards heap fragmentation.
>
> However, if all the used memory has been released (as I expect it do be when all but main threads exited), should have I expected mmaped regions to be released and process virtual memory size shrink?
For glibc-2.17, what I see is, when free is called:
1. if not the main arena, it will munmap if it is a large chunk
allocated with mmap.
2. If the main arena, it can trim the memory with sbrk and a negative argument
if the top free memory when merged, is 128K or larger.
3. If not the main arena, it will call madvise(addr, size, MADV_DONTNEED) if
it finds 64K of contiguous unused memory.
In either case, if releasing with sbrk or mdavise, the process virtual size
should decrease.
> Or GLIBC would do it upon some special event? (can it be forced somehow?)
You can call malloc_trim().
You can also use a smaller value for MALLOC_MMAP_THRESHOLD_ to
have more memory allocated/released with mmap. This value should not be
too small. Basically, tell it to use mmap for large blocks. The default is 128k.
Fragmentation usually happens when allocating different sized objects, and
due to memory layout, with too many small objects not released with free,
it cannot find contiguous free blocks.
> Thanks,
> - Nikolay
>
>
> On 2/7/23 13:01, Nikolay Shustov wrote:
>
> Got it, thanks.
> For now, I am using this tunable merely to hep to surface whatever I might have missed in terms of long living objects and heap fragmentation.
> But I am definitely going to play with it when I am reasonably sure that it is what does the major impact.
>
> On 2/7/23 12:26, Ben Woodard wrote:
>
>
> On 2/7/23 08:55, Nikolay Shustov via Libc-alpha wrote:
>
> There is no garbage collector thread or something similar in some
> worker thread. But maybe something similar could be done in your
> code.
>
>
> No, there is nothing of the kind in the application.
>
> You might experiment with a tradeoff speed vs memory usage. The
> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
> see 'man mallopt' for other options.
>
>
> MALLOC_ARENA_MAX=1 made a huge difference.
>
> I just wanted to point out that it isn't 1 or the default. That was most likely a simple test to test a hypothesis about what could be going wrong. This is a tunable knob and your application could have a sweet spot. For some of the applications that I help support, we have empirically found that a good number is slightly lower than the number of processors that they system has. e.g. if their are 16 cores giving it 12 arenas doesn't impact speed but makes the memory footprint more compact.
>
> The initial memory allocations went done on the scale of magnitude.
> In fact, I do not see that much of the application slowdown but this will need more profiling.
> The stable allocations growth is still ~2Mb/second.
>
> I am going to investigate your idea of long living objects contention/memory fragmentation.
>
> This sounds very probably, even though I do not see real memory leaks even after all the aux threads died.
> I have TLS instances in use, maybe those really get in the way.
>
> Thanks a lot for your help.
> If/when I find something new or interesting, I will send an update - hope it will help someone else, too.
>
> Regards,
> - Nikolay
>
> On 2/7/23 11:16, Paulo César Pereira de Andrade wrote:
>
> Em ter., 7 de fev. de 2023 às 12:07, Nikolay Shustov via Libc-alpha
> <libc-alpha@sourceware.org> escreveu:
>
> Hi,
> I have a question about the malloc() behavior which I observe.
> The synopsis is that the during the stress load, the application
> aggressively allocates virtual memory without any upper limit.
> Just to note, after application is loaded just with the peak of activity
> and goes idle, its virtual memory doesn't scale back (I do not expect
> much of that though - should I?).
>
> There is no garbage collector thread or something similar in some
> worker thread. But maybe something similar could be done in your
> code.
>
> The application is heavily multithreaded; at its peak of its activitiy
> it creates new threads and destroys them at a pace of approx. 100/second.
> After the long and tedious investigation I dare to say that there are no
> memory leaks involved.
> (Well, there were memory leaks and I first went after those; found and
> fixed - but the result did not change much.)
>
> You might experiment with a tradeoff speed vs memory usage. The
> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
> see 'man mallopt' for other options.
>
> The application is cross-platform and runs on Windows and some other
> platforms too.
> There is an OS abstraction layer that provides the unified thread and
> memory allocation API for business logic, but the business logic that
> triggers memory allocations is platform-independent.
> There are no progressive memory allocations in OS abstraction layer
> which could be blamed for the memory growth.
>
> The thing is, on Windows, for the same activity there is no such
> application memory growth at all.
> It allocates memory moderately and scales back after peak of activity.
> This makes me think it is not the business logic to be blamed (to the
> extent of that it does not leak memory).
>
> I used valigrind to profile for memory leaks and heap usage.
> Please see massif outputs attached (some callstacks had to be trimmed out).
> I am also attaching the memory map for the application (run without
> valgrind); snapshot is taken after all the threads but main were
> destroyed and application is idle.
>
> The pace of the virtual memory growth is not quite linear.
>
> Most likely there are long lived objects doing contention and also
> probably memory fragmentation, preventing returning memory to
> the system after a free call.
>
> From my observation, it allocates a big hunk in the beginning of the
> peak loading, then in some time starts to grow in steps of ~80Mb / 10
> seconds, then after some times starts to steadily grow it at pace of
> ~2Mb/second.
>
> Some stats from the host:
>
> OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
>
> ldd -version
>
> ldd (GNU libc) 2.17
> Copyright (C) 2012 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions. There
> is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> PURPOSE.
> Written by Roland McGrath and Ulrich Drepper.
>
> uname -a
>
> Linux <skipped> 3.10.0-1160.53.1.el7.x86_64 #1 SMP Thu Dec 16
> 10:19:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>
>
> At a peak load, the number of application threads is ~180.
> If application is left running, I did not observe it would hit any max
> virtual memory threshold and eventually ends up with hitting ulimit.
>
> My questions are:
>
> - Is this memory growth an expected behavior?
>
> It should eventually stabilize. But it is possible that some allocation
> pattern is causing both, fragmentation and long lived objects preventing
> consolidation of memory chunks.
>
> - What can be done to prevent it from happening?
>
> First approach is MALLOC_ARENA_MAX. After that some coding
> patterns might help, for example, have large long lived objects allocated
> from the same thread, preferably at startup.
> Can also attempt to cache some memory, but note that caching is also
> an easy way to get contention. To avoid this, you could use memory from
> buffers from mmap.
>
> Depending on your code, you can also experiment with jemalloc or
> tcmalloc. I would suggest tcmalloc, as its main feature is to work
> in multithreaded environments:
>
> https://gperftools.github.io/gperftools/tcmalloc.html
>
> Glibc newer than 2.17 has a per thread cache, but the issue you
> are experimenting is not it being slow, but memory usage. AFAIK tcmalloc
> has a kind of garbage collector, but it should not be much different than
> glibc consolidation logic; it should only run during free, and if there is
> some contention, it might not be able to release memory.
>
> Thanks in advance,
> - Nikolay
>
> Thanks!
>
> Paulo
>
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: GLIBC malloc behavior question
2023-02-07 21:38 ` Paulo César Pereira de Andrade
@ 2023-02-07 23:41 ` Nikolay Shustov
0 siblings, 0 replies; 8+ messages in thread
From: Nikolay Shustov @ 2023-02-07 23:41 UTC (permalink / raw)
To: Paulo César Pereira de Andrade; +Cc: Ben Woodard, libc-alpha
[-- Attachment #1: Type: text/plain, Size: 9196 bytes --]
Makes sense, thank you.
On 2/7/23 16:38, Paulo César Pereira de Andrade wrote:
> Em ter., 7 de fev. de 2023 às 17:56, Nikolay Shustov
> <nikolay.shustov@gmail.com> escreveu:
>> I was able to increase ratio of threads reuse in application (i.e. reuse running thread instead of destroying it and creating new) and it seemed to have a positive effect on the amount of allocated memory.
>> Valgrind reports some small memory leaks - and I know from experience these reports are bogus as they deal with its inability to detect thread local storage key destructor function invocations.
>> I was able to ensure these are invoked as it supposed to be. The amount of the reported leaks is really small and in no way could contribute to gigabytes of VIRT been held in allocations.
>>
>> I experimented more and at this stage most of my suspicions are towards heap fragmentation.
>>
>> However, if all the used memory has been released (as I expect it do be when all but main threads exited), should have I expected mmaped regions to be released and process virtual memory size shrink?
> For glibc-2.17, what I see is, when free is called:
> 1. if not the main arena, it will munmap if it is a large chunk
> allocated with mmap.
> 2. If the main arena, it can trim the memory with sbrk and a negative argument
> if the top free memory when merged, is 128K or larger.
> 3. If not the main arena, it will call madvise(addr, size, MADV_DONTNEED) if
> it finds 64K of contiguous unused memory.
>
> In either case, if releasing with sbrk or mdavise, the process virtual size
> should decrease.
>
>> Or GLIBC would do it upon some special event? (can it be forced somehow?)
> You can call malloc_trim().
>
> You can also use a smaller value for MALLOC_MMAP_THRESHOLD_ to
> have more memory allocated/released with mmap. This value should not be
> too small. Basically, tell it to use mmap for large blocks. The default is 128k.
>
> Fragmentation usually happens when allocating different sized objects, and
> due to memory layout, with too many small objects not released with free,
> it cannot find contiguous free blocks.
>
>> Thanks,
>> - Nikolay
>>
>>
>> On 2/7/23 13:01, Nikolay Shustov wrote:
>>
>> Got it, thanks.
>> For now, I am using this tunable merely to hep to surface whatever I might have missed in terms of long living objects and heap fragmentation.
>> But I am definitely going to play with it when I am reasonably sure that it is what does the major impact.
>>
>> On 2/7/23 12:26, Ben Woodard wrote:
>>
>>
>> On 2/7/23 08:55, Nikolay Shustov via Libc-alpha wrote:
>>
>> There is no garbage collector thread or something similar in some
>> worker thread. But maybe something similar could be done in your
>> code.
>>
>>
>> No, there is nothing of the kind in the application.
>>
>> You might experiment with a tradeoff speed vs memory usage. The
>> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
>> see 'man mallopt' for other options.
>>
>>
>> MALLOC_ARENA_MAX=1 made a huge difference.
>>
>> I just wanted to point out that it isn't 1 or the default. That was most likely a simple test to test a hypothesis about what could be going wrong. This is a tunable knob and your application could have a sweet spot. For some of the applications that I help support, we have empirically found that a good number is slightly lower than the number of processors that they system has. e.g. if their are 16 cores giving it 12 arenas doesn't impact speed but makes the memory footprint more compact.
>>
>> The initial memory allocations went done on the scale of magnitude.
>> In fact, I do not see that much of the application slowdown but this will need more profiling.
>> The stable allocations growth is still ~2Mb/second.
>>
>> I am going to investigate your idea of long living objects contention/memory fragmentation.
>>
>> This sounds very probably, even though I do not see real memory leaks even after all the aux threads died.
>> I have TLS instances in use, maybe those really get in the way.
>>
>> Thanks a lot for your help.
>> If/when I find something new or interesting, I will send an update - hope it will help someone else, too.
>>
>> Regards,
>> - Nikolay
>>
>> On 2/7/23 11:16, Paulo César Pereira de Andrade wrote:
>>
>> Em ter., 7 de fev. de 2023 às 12:07, Nikolay Shustov via Libc-alpha
>> <libc-alpha@sourceware.org> escreveu:
>>
>> Hi,
>> I have a question about the malloc() behavior which I observe.
>> The synopsis is that the during the stress load, the application
>> aggressively allocates virtual memory without any upper limit.
>> Just to note, after application is loaded just with the peak of activity
>> and goes idle, its virtual memory doesn't scale back (I do not expect
>> much of that though - should I?).
>>
>> There is no garbage collector thread or something similar in some
>> worker thread. But maybe something similar could be done in your
>> code.
>>
>> The application is heavily multithreaded; at its peak of its activitiy
>> it creates new threads and destroys them at a pace of approx. 100/second.
>> After the long and tedious investigation I dare to say that there are no
>> memory leaks involved.
>> (Well, there were memory leaks and I first went after those; found and
>> fixed - but the result did not change much.)
>>
>> You might experiment with a tradeoff speed vs memory usage. The
>> minimum memory usage should be achieved with MALLOC_ARENA_MAX=1
>> see 'man mallopt' for other options.
>>
>> The application is cross-platform and runs on Windows and some other
>> platforms too.
>> There is an OS abstraction layer that provides the unified thread and
>> memory allocation API for business logic, but the business logic that
>> triggers memory allocations is platform-independent.
>> There are no progressive memory allocations in OS abstraction layer
>> which could be blamed for the memory growth.
>>
>> The thing is, on Windows, for the same activity there is no such
>> application memory growth at all.
>> It allocates memory moderately and scales back after peak of activity.
>> This makes me think it is not the business logic to be blamed (to the
>> extent of that it does not leak memory).
>>
>> I used valigrind to profile for memory leaks and heap usage.
>> Please see massif outputs attached (some callstacks had to be trimmed out).
>> I am also attaching the memory map for the application (run without
>> valgrind); snapshot is taken after all the threads but main were
>> destroyed and application is idle.
>>
>> The pace of the virtual memory growth is not quite linear.
>>
>> Most likely there are long lived objects doing contention and also
>> probably memory fragmentation, preventing returning memory to
>> the system after a free call.
>>
>> From my observation, it allocates a big hunk in the beginning of the
>> peak loading, then in some time starts to grow in steps of ~80Mb / 10
>> seconds, then after some times starts to steadily grow it at pace of
>> ~2Mb/second.
>>
>> Some stats from the host:
>>
>> OS: Red Hat Enterprise Linux Server release 7.9 (Maipo)
>>
>> ldd -version
>>
>> ldd (GNU libc) 2.17
>> Copyright (C) 2012 Free Software Foundation, Inc.
>> This is free software; see the source for copying conditions. There
>> is NO
>> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
>> PURPOSE.
>> Written by Roland McGrath and Ulrich Drepper.
>>
>> uname -a
>>
>> Linux <skipped> 3.10.0-1160.53.1.el7.x86_64 #1 SMP Thu Dec 16
>> 10:19:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>>
>>
>> At a peak load, the number of application threads is ~180.
>> If application is left running, I did not observe it would hit any max
>> virtual memory threshold and eventually ends up with hitting ulimit.
>>
>> My questions are:
>>
>> - Is this memory growth an expected behavior?
>>
>> It should eventually stabilize. But it is possible that some allocation
>> pattern is causing both, fragmentation and long lived objects preventing
>> consolidation of memory chunks.
>>
>> - What can be done to prevent it from happening?
>>
>> First approach is MALLOC_ARENA_MAX. After that some coding
>> patterns might help, for example, have large long lived objects allocated
>> from the same thread, preferably at startup.
>> Can also attempt to cache some memory, but note that caching is also
>> an easy way to get contention. To avoid this, you could use memory from
>> buffers from mmap.
>>
>> Depending on your code, you can also experiment with jemalloc or
>> tcmalloc. I would suggest tcmalloc, as its main feature is to work
>> in multithreaded environments:
>>
>> https://gperftools.github.io/gperftools/tcmalloc.html
>>
>> Glibc newer than 2.17 has a per thread cache, but the issue you
>> are experimenting is not it being slow, but memory usage. AFAIK tcmalloc
>> has a kind of garbage collector, but it should not be much different than
>> glibc consolidation logic; it should only run during free, and if there is
>> some contention, it might not be able to release memory.
>>
>> Thanks in advance,
>> - Nikolay
>>
>> Thanks!
>>
>> Paulo
>>
>>
>>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-02-07 23:41 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-07 15:06 GLIBC malloc behavior question Nikolay Shustov
2023-02-07 16:16 ` Paulo César Pereira de Andrade
2023-02-07 16:55 ` Nikolay Shustov
2023-02-07 17:26 ` Ben Woodard
2023-02-07 18:01 ` Nikolay Shustov
2023-02-07 20:56 ` Nikolay Shustov
2023-02-07 21:38 ` Paulo César Pereira de Andrade
2023-02-07 23:41 ` Nikolay Shustov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).