Re: Optimization question

Re: Optimization question

[Florian Weimer]

* Nikolaus Dunn:

> I've run into an issue where turning on compiler optimization using
> -O1 or higher causes an issue in my program. Specifically, I am
> replacing the new and delete operators globally to perform some real
> time allocation tracking. When I compile with -O1 or -O2, my
> implementation of new is not being called by STL classes, via the
> std::allocator. My version of delete IS being called.

This matter is appropriate for the gcc-help list, redirecting there.

> I realize I may be doing something wrong and I've read many posts from
> people saying not to replace new and delete, but I'm hacking on a 15
> year old baseline. We've just stepped up to GCC 5.3.0 and are
> compiling with the -std=c++14 option (to give a little context). I'm
> also compiling using MinGW, but I'm figuring MinGW would be using the
> same optimization logic at the compiler level as pure GCC.

It's unclear whether this is an optimization issue, or a different
build issue.  We can't really comment on what's going wrong until you
have actual source code to show, I'm afraid.

Do you define your version of the global operators new and delete in
the same translation unit?

Re: Optimization question

[Jonathan Wakely]

* Nikolaus Dunn:
> I've run into an issue where turning on compiler optimization using
> -O1 or higher causes an issue in my program. Specifically, I am
> replacing the new and delete operators globally to perform some real
> time allocation tracking. When I compile with -O1 or -O2, my
> implementation of new is not being called by STL classes, via the
> std::allocator. My version of delete IS being called.

That doesn't make much sense, because std::allocator<T> doesn't use
new or delete for objects of type T, so neither should be called.

Instead std::allocator<T> allocates untyped memory (e.g. via malloc)
and then constructs objects into it with placement new-expressions.

What do you replacement new and delete operators look like?

Re: Optimization question

[Nikolaus Dunn]

Maybe I'm using the wrong terminology then or am possibly confused. I am 
under the impression that std::allocator points to std::new_allocator by 
default. In fact, in debugging with gdb, I've gotten into 
new_allocator.h. At any rate, in trying to contrive the smallest 
possible test program, I learned some new things that in my original 
pass, I mistook.

It turns out that std::vector seems to work fine with -O2 or without it.

std::ostringstream however does not call my new OR delete with no 
optimization. With -O2, it calls only my delete.

If I do not attempt to wrap malloc and free, I get the same result. 
std::vector calls my new and delete, ostringstream calls neither.

The command line I used to compile it is:
g++ -g -O2 --std=c++14 -Wl,-wrap,malloc -Wl,-wrap,free Test.c -o test.exe

Test.c:

#include <sstream>
#include <iostream>
#include <vector>

extern "C" void *__real_malloc(size_t);
extern "C" void __real_free(void *);

extern "C" void *__wrap_malloc(size_t nbytes) {
     printf("wrap_malloc: %ld bytes\n", nbytes);
     void * foo = __real_malloc(nbytes);
     printf("wrap_malloc address: %ld\n", foo);
     return foo;
}

extern "C" void __wrap_free(void *ptr) {
     printf("wrap_free: %ld\n", ptr);
     __real_free(ptr);
}

void* operator new (std::size_t size) throw (std::bad_alloc) {
     printf("new: %ld bytes\n", size);
     void *p = __wrap_malloc(size);
     printf("new address: %ld\n", p);
     return p;
}

void operator delete (void *p) {
     printf("delete: %ld\n", p);
     __wrap_free(p);
}

void operator delete (void *p, std::size_t) {
     printf("delete with size: %ld\n", p);
     __wrap_free(p);
}

int main(int argc, char ** argv) {
    printf("Creating new vector\n\n");
    std::vector<int> foo;

    printf("pushing an element\n\n");
    foo.push_back(5);

    printf("Creating new ostringstream\n\n");
    std::ostringstream msg2;

    printf("pushing once\n\n");
    msg2 << "Blah blah blah ";

    printf("pushing twice\n\n");
    msg2 << "boo dee doo" << std::endl;

    printf("printing it out\n\n");
    std::cout << msg2.str();
}


On 09/27/2016 01:10 PM, Jonathan Wakely wrote:
> * Nikolaus Dunn:
>> I've run into an issue where turning on compiler optimization using
>> -O1 or higher causes an issue in my program. Specifically, I am
>> replacing the new and delete operators globally to perform some real
>> time allocation tracking. When I compile with -O1 or -O2, my
>> implementation of new is not being called by STL classes, via the
>> std::allocator. My version of delete IS being called.
> That doesn't make much sense, because std::allocator<T> doesn't use
> new or delete for objects of type T, so neither should be called.
>
> Instead std::allocator<T> allocates untyped memory (e.g. via malloc)
> and then constructs objects into it with placement new-expressions.
>
> What do you replacement new and delete operators look like?

Re: Optimization question

[Jonathan Wakely]

On 27 September 2016 at 18:39, Nikolaus Dunn wrote:
> Maybe I'm using the wrong terminology then or am possibly confused. I am
> under the impression that std::allocator points to std::new_allocator by
> default. In fact, in debugging with gdb, I've gotten into new_allocator.h.
> At any rate, in trying to contrive the smallest possible test program, I
> learned some new things that in my original pass, I mistook.

Sorry, I completely misread your original mail, I thought you meant
new and delete operators for specific classes, not the global ones.

You're right that std::allocator uses the global new and delete operators.

> It turns out that std::vector seems to work fine with -O2 or without it.
>
> std::ostringstream however does not call my new OR delete with no
> optimization. With -O2, it calls only my delete.
>
> If I do not attempt to wrap malloc and free, I get the same result.
> std::vector calls my new and delete, ostringstream calls neither.

I can't reproduce this, I see stringstream calling both.


Creating new vector

pushing an element

new: 4 bytes
wrap_malloc: 4 bytes
wrap_malloc address: 0x1528c20
new address: 0x1528c20
Creating new ostringstream

pushing once

new: 513 bytes
wrap_malloc: 513 bytes
wrap_malloc address: 0x1528c40
new address: 0x1528c40
pushing twice

printing it out

new: 28 bytes
wrap_malloc: 28 bytes
wrap_malloc address: 0x1528e50
new address: 0x1528e50
Blah blah blah boo dee doo
delete: 0x1528e50
wrap_free: 0x1528e50
delete: 0x1528c40
wrap_free: 0x1528c40
delete: 0x1528c20
wrap_free: 0x1528c20



> The command line I used to compile it is:
> g++ -g -O2 --std=c++14 -Wl,-wrap,malloc -Wl,-wrap,free Test.c -o test.exe
>
> Test.c:
>
> #include <sstream>
> #include <iostream>
> #include <vector>
>
> extern "C" void *__real_malloc(size_t);
> extern "C" void __real_free(void *);
>
> extern "C" void *__wrap_malloc(size_t nbytes) {
>     printf("wrap_malloc: %ld bytes\n", nbytes);
>     void * foo = __real_malloc(nbytes);
>     printf("wrap_malloc address: %ld\n", foo);

N.B. compiling your code produces loads of warnings, the correct
format specifiers for printf are %zu and %p.

Re: Optimization question

[Florian Weimer]

* Nikolaus Dunn:

> std::ostringstream however does not call my new OR delete with no
> optimization. With -O2, it calls only my delete.
>
> If I do not attempt to wrap malloc and free, I get the same
> result. std::vector calls my new and delete, ostringstream calls
> neither.
>
> The command line I used to compile it is:
> g++ -g -O2 --std=c++14 -Wl,-wrap,malloc -Wl,-wrap,free Test.c -o test.exe

“-Wl,-wrap,malloc -Wl,-wrap,free” is only effective for newly-compiled
code.  Your example probably uses some template instantiations which
are supplied by libstdc++, and these will keep calling the unwrapped
malloc/free implementations.

I don't know if there are mechanisms on mingw which are comparable to
ELF symbol interposition.

Re: Optimization question

[Jonathan Wakely]

On 27 September 2016 at 19:32, Florian Weimer wrote:
> * Nikolaus Dunn:
>
>> std::ostringstream however does not call my new OR delete with no
>> optimization. With -O2, it calls only my delete.
>>
>> If I do not attempt to wrap malloc and free, I get the same
>> result. std::vector calls my new and delete, ostringstream calls
>> neither.
>>
>> The command line I used to compile it is:
>> g++ -g -O2 --std=c++14 -Wl,-wrap,malloc -Wl,-wrap,free Test.c -o test.exe
>
> “-Wl,-wrap,malloc -Wl,-wrap,free” is only effective for newly-compiled
> code.  Your example probably uses some template instantiations which
> are supplied by libstdc++, and these will keep calling the unwrapped
> malloc/free implementations.

Right, std::stringstream doesn't allocate any memory itself, but
std::string does, and that is instantiated in the libstdc++.so
library.

> I don't know if there are mechanisms on mingw which are comparable to
> ELF symbol interposition.

optimization question

[Igor Schein]

Hi,

let's say I don't trust gcc info pages because they
might be outdated.  Is there a way to determine which exactly
optimizations are chosen at any -On level?  Also, what
is the maximum level of optimization?  info pages only talk
about -O3 and below, but I see people use -O6 and even -O9.

Thanks

Igor

optimization question

[Igor Schein]

Hi,

let's say I don't trust gcc info pages because they
might be outdated.  Is there a way to determine which exactly
optimizations are chosen at any -On level?  Also, what
is the maximum level of optimization?  info pages only talk
about -O3 and below, but I see people use -O6 and even -O9.

Thanks

Igor