mmap&munmap syscall and malloc&free function call mismatch

mmap&munmap syscall and malloc&free function call mismatch

[Zhengyu Zhang]

Hi list!

As I run the code shown below, mmap&munmap syscall doesn’t match
malloc&free in some cases.

```
// g++ test_memcpy.c -std=c++11 -lpthread -O0
#include <stdlib.h>
#include <stdio.h>
#include <malloc.h>
#include <string.h>
#include <unistd.h>
#include <thread>
#include <iostream>

#define THREAD_NUM 40

void foo()
{
        char *src_buf = (char *)malloc(256*1024);
        char *dst_buf = NULL;
        long loop = 80000;
        memset(src_buf, 1, 256*1024);
        while (--loop) {
                posix_memalign((void**)&dst_buf, 4096, 256 * 1024);
                memcpy(dst_buf, src_buf, 256 * 1024);
                free(dst_buf);
                //usleep(100);
        }
}

int main() {
    int i = 0;
    std::thread *t[THREAD_NUM];
    for (i = 0; i < THREAD_NUM; ++i)
        t[i] = new std::thread(foo);

    for (i = 0; i < THREAD_NUM; ++i)
        t[i]->join();
    return 0;
}
```
The binary is linked against glibc 2.17.

When THREAD_NUM=4, strace results show mmap is called 320,000 times,
as expected (4 * 80,000)
When THREAD_NUM=40, mmap is only called about 430,0000 times. I am
expecting 3,200,000 (40 * 80,000)
Then I add `usleep(100)` in the loop, it becomes 3,200,000 as expected.

I do not know how to explain this. Since I investigate the glibc
malloc code, free() will munmap the mmapped memory immediately so
there should be no chance for caching stuff to save syscall.

So how can mmap&munmap be less than malloc&free?


All the best!
Zhengyu