Re: Differences between C++ 'new' operator and 'malloc()' (NOT a C/C++ question)

[Ryan Johnson <ryan.johnson@cs.utoronto.ca>]

On 12/07/2012 8:12 AM, Claude SIMON wrote:
> Ryan Johnson wrote:
>> On 10/07/2012 12:46 PM, Claude SIMON wrote:
>>> Ryan Johnson wrote:
>>>> On 05/07/2012 9:36 AM, Claude SIMON wrote:
>>>>> Ryan Johnson wrote:
>>>>>> On 04/07/2012 5:45 AM, Claude SIMON wrote:
>>>>>>> When I compile the component with Visual C++, it works. When I
>>>>>>> compile
>>>>>>> the
>>>>>>> component with g++... it crashes.
>>>>>>>
>>>>>>> With 'gdb', I found that the problem happens when calling the
>>>>>>> 'malloc'
>>>>>>> function (as soon as the function is called, NOT when the returned
>>>>>>> allocated memory is used). When I replace the 'malloc' by a the C++
>>>>>>> 'new'
>>>>>>> operator, the component compiled with Cygwin g++ doesn't crash
>>>>>>> anymore.
>>>>>>> I thought that the C++ 'new' operator calls the 'malloc' function,
>>>>>>> but
>>>>>>> this seems not to be the case. As I want to use 'malloc'-like
>>>>>>> functions
>>>>>>> rather than the C++ 'new' operator, I wonder which functions are
>>>>>>> used
>>>>>>> in
>>>>>>> the C++ 'new' operator to allocate memory (and naturally which
>>>>>>> functions
>>>>>>> are used in the C++ 'delete' operator to free the memory).
>>>>>> Operator new() and malloc() are explicitly *not* interchangeable (for
>>>>>> many reasons, not least of which that the Standard says so). If you
>>>>>> were
>>>>>> to free new'ed memory, or delete malloc'ed memory, the resulting heap
>>>>>> corruption could easily manifest as a crash the next time you tried
>>>>>> to
>>>>>> allocate something... or it might just silently clobber data and lead
>>>>>> to
>>>>>> "spooky action at a distance."
>>>>>>
>>>>> Thank you for the answer, but I am aware of this and my problem has
>>>>> nothing to do with it, nor, as stated in the subject, with having some
>>>>> lacuna in C/C++ programming.
>>>>>
>>>>> Let's try to be a little more explicit despite my poor English.
>>>>>
>>>>> Let's consider a Java native component which only calls a 'malloc(1)'.
>>>>> It
>>>>> doesn't even test the returned value (it is usually not a good idea,
>>>>> but
>>>>> it doesn't matter here).
>>>>>
>>>>> This component :
>>>>> - compiled with g++ under Linux : works,
>>>>> - compiled with g++ under Mac OS : works,
>>>>> - compiled with Visual C++ under Windows : works,
>>>>> - compiled with g++ under Cygwin : CRASHES !
>>>>>
>>>>> It crashes as soon the 'malloc(1)' function is called. You don't even
>>>>> have
>>>>> the opportunity to test the returned value, nor to use it. It's
>>>>> perhaps
>>>>> a
>>>>> Cygwin bug, or perhaps a JVM/JRE/JDK bug ; I don't know and I don't
>>>>> bother
>>>>> (but if someone will make some investigation about that, I'm ready to
>>>>> help
>>>>> him or her if I can).
>>>>>
>>>>> When you replace the 'malloc()' by the 'new' operator, then the
>>>>> component
>>>>> compiled with g++ under Cygwin works too.
>>>>> The 'new' operator, among other things, allocates memory, as
>>>>> 'malloc()'
>>>>> does, but obviously it doesn't use 'malloc()' as it doesn't crash. So,
>>>>> because I can't use 'malloc()' in my Java native components, and
>>>>> because
>>>>> I
>>>>> doesn't want to use the 'new' operator, I wish to know which functions
>>>>> the
>>>>> 'new' operator uses to allocate memory, so I can use them in my Java
>>>>> native component so they would no more crash when compiled with g++
>>>>> under
>>>>> Cygwin.
>>>> A crash inside malloc is 99.99% likely due to a bug in user code (wild
>>>> pointer, double-free, smashed stack, etc). The fact that your code
>>>> doesn't crash under other circumstances does precisely *nothing* to
>>>> rule
>>>> out a bug in your code if bad has been observed anywhere (it just
>>>> proves
>>>> the platforms really are different). The buggy code may have nothing to
>>>> do with malloc, other than having the bad luck of clobbering a data
>>>> structure the latter needs. Even a single mix-up of new/malloc usage
>>>> (perhaps due to losing track of a pointer's provenance) is also enough.
>>> Indeed. The problem is... the crash happens even when there is no other
>>> code which could be buggy.
>> #include <stdlib.h>
>> int main() { return (int) malloc(10); }
>>
>> Does not crash. There must be some other code which is buggy.
>>
> The test case I provided is EXACTLY the code your wrote above, only
> applied to a Java native component (and without the 'return', which is not
> needed for the test case), and it DOES crash. So ?
Sorry, I should have actually looked at the repo before assuming the 
test case was a monstrosity. By way of penance, I've now looked, 
downloaded, tweaked, and tested it.

Tweaks:
- wrap the .cpp in `extern "C"' and remove the stdcall_alias linker flag 
(could have led to ABI issues)
- made jcmc.h a makefile target that calls my local javah (I'm paranoid)

My setup:
cygwin 1.7.14
jdk 1.6.0_24 (64-bit native Windows)
jre ? (32-bit native Windows)
g++ 4.7.1 (cygwin), 4.5.2 (64-bit mingw)

Attempt 1:
> $ cat jcmc.java
> class jcmc {
>         static private native void bug();
>         static public void main ( String[] args ) {
>                 System.out.print( "Loading library 'jcmc'..." );
>                 System.out.flush();
>                 System.loadLibrary("jcmc");
>                 System.out.println( " 'jcmc' library loaded !" );
>                 System.out.print( "Calling 'malloc' function from a 
> native component..." );
>                 System.out.flush();
>                 bug();
>                 System.out.println( " 'malloc' calling succeed !" );
>         }
> }
>
> $ cat jcmc.cpp
> #include "jcmc.h"
> #include <stdlib.h>
> extern "C" {
>         JNIEXPORT void JNICALL Java_jcmc_bug( JNIEnv *, jclass ) { 
> new( char[10] ); }
> }
>
> $ make
> rm -f jcmc.h jcmc.class jcmc.dll jcmc.o
> jdk/bin/javac jcmc.java
> jdk/bin/javah jcmc
> g++ -c -g -Ijdk/include -Ijdk/include/win32 "-D__int64=long long" jcmc.cpp
> g++ -shared -o jcmc.dll jcmc.o
> java -cp . jcmc
> Loading library 'jcmc'... 'jcmc' library loaded !
> Calling 'malloc' function from a native component...Exception in 
> thread "main" java.lang.UnsatisfiedLinkError: jcmc.bug()V
>         at jcmc.bug(Native Method)
>         at jcmc.main(jcmc.java:16)
This more or less was what I think you tried. I don't seg fault, but the 
malloc() function is not available at runtime. Since JNI is by 
definition not portable, and the java in my path is from some 32-bit 
JRE, I decided to try again with the JDK's 64-bit java.

Attempt 2:
> $ make
> rm -f jcmc.h jcmc.class jcmc.dll jcmc.o
> jdk/bin/javac jcmc.java
> jdk/bin/javah jcmc
> g++ -c -g -Ijdk/include -Ijdk/include/win32 "-D__int64=long long" jcmc.cpp
> g++ -shared -o jcmc.dll jcmc.o
> ./jdk/bin/java -cp . jcmc
> Loading library 'jcmc'...Exception in thread "main" 
> java.lang.UnsatisfiedLinkError: 
> C:\cygwin\home\Ryan\experiments\epeios\bugs\jcmc\jcmc.dll: Can't load 
> IA 32-bit .dll on a AMD 64-bit platform
>         at java.lang.ClassLoader$NativeLibrary.load(Native Method)
>         at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1803)
>         at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1728)
>         at java.lang.Runtime.loadLibrary0(Runtime.java:823)
>         at java.lang.System.loadLibrary(System.java:1028)
>         at jcmc.main(jcmc.java:9)
>
That's clear enough... time to switch to 64-bit mingw.

Attempt 3:
> $ make
> rm -f jcmc.h jcmc.class jcmc.dll jcmc.o
> jdk/bin/javac jcmc.java
> jdk/bin/javah jcmc
> x86_64-w64-mingw32-g++ -static-libgcc -static-libstdc++ -c -g 
> -Ijdk/include -Ijdk/include/win32 "-D__int64=long long"  jcmc.cpp
> x86_64-w64-mingw32-g++ -static-libgcc -static-libstdc++  -shared -o 
> jcmc.dll jcmc.o
> ./jdk/bin/java -cp . jcmc
> Loading library 'jcmc'... 'jcmc' library loaded !
> Calling 'malloc' function from a native component... 'malloc' calling 
> succeed !
Et voilà . Don't mix 32/64 bit (if applicable), don't try to call JNI 
stuff with a different JVM than was used to create it, and life is good. 
This has nothing to do with either cygwin or new vs. malloc AFAICT. Note 
that I used new char[10] in all the examples above, so I don't know why 
you were able to run with operator new and not malloc. I don't know why 
you got a seg fault either; I assume you haven't seen such clear error 
messages as I got, or you could have solved this yourself in about five 
minutes. Maybe you have a different (older?) JVM that's less careful 
about checking runtime linker dependencies? A broken dll trampoline 
could easily cause a seg fault.

Note that cygwin has no 64-bit capabilities for the time being. Mingw is 
a windows-targeted cross compiler that runs under cygwin but produces 
native windows binaries, so it doesn't have any posix functions 
available; the -static flags tell mingw to make a truly stand-alone 
executable that has only standard windows runtime dependencies.

Ryan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple