GCC vs MSVC compiler issues in a port

GCC vs MSVC compiler issues in a port

[John Cotton Ericson]

Hello,

For a little over a year I have worked on porting the a voxel engine 
called Voxlap to work on Linux. My code is here: 
https://github.com/Ericson2314/Voxlap

I now can compile the same code with both MSVC and gcc. The MSVC code 
runs well enough, but the gcc version does not. Specifically when I run 
the MSVC version, everything in-game renders, and a bit of text is 
thrown on top as a watermark. When I run  the gcc version, all I get is 
a black window and that water mark. The problem is exactly whether I 
compile on windows with mingw or compile on linux. In both cases I am 
compiling the same branch with most of the inline assembly converted to 
C, and in both cases I compile against the SDL backend, not the direct X 
backend.

Because I have done my best to hold everything else constant, I am 
pretty convinced this error is either due to some subtle difference's in 
the two compile's understanding of the C semantics, or an error in the 
gcc version of what inline assembly remains (inline assembly being the 
only code I can think of which is #ifdefed based on the compiler).

After a couple of months of sporadically trying to solve this issue, I 
am at a loss what to do next. As I did not write the original renderer, 
and thus I am only so familiar with it's inner workings. I was hoping 
somebody more experience with porting or more knowledge of subtle 
differences between gcc and MSVC would be able to help me solve this 
issue, which should be the last before I have a basic gcc version ready.

Thanks,
John

Notes:

Here is a list of the only "leads" I can think of:

* If I do not use -ffast-math, the gcc version crashes almost 
immediately due to an arithmetic overflow. I do not think the MSVC 
version with any flags I tried ever had this problem

* I think I onced compared the symbol tables of the same object file 
compiled with each compiler and found some differences. However, I have 
no idea what differences there are acceptable (besides different 
name-mangling schemes)  and what ones are not.

* What inline assembly assembly remains is #ifdef-d based on the 
compiler. This is the only code I can think of that is compiler-specific.

While the code is saved as a C++ file it contains almost no c++-specific 
syntax. A couple of people have asked me about this from time to time, 
so I'd include the answer as an addendum.

I wrote a pretty decent (I hope?) history of the port which may be found 
in the readme. It may be of use.

Re: GCC vs MSVC compiler issues in a port

[Oleg Endo]

Hello,

On Sat, 2012-12-22 at 04:07 -0500, John Cotton Ericson wrote:
> Hello,
> 
> For a little over a year I have worked on porting the a voxel engine 
> called Voxlap to work on Linux. My code is here: 
> https://github.com/Ericson2314/Voxlap
> 
> I now can compile the same code with both MSVC and gcc. The MSVC code 
> runs well enough, but the gcc version does not. Specifically when I run 
> the MSVC version, everything in-game renders, and a bit of text is 
> thrown on top as a watermark. When I run  the gcc version, all I get is 
> a black window and that water mark. The problem is exactly whether I 
> compile on windows with mingw or compile on linux. In both cases I am 
> compiling the same branch with most of the inline assembly converted to 
> C, and in both cases I compile against the SDL backend, not the direct X 
> backend.
> 
> Because I have done my best to hold everything else constant, I am 
> pretty convinced this error is either due to some subtle difference's in 
> the two compile's understanding of the C semantics, or an error in the 
> gcc version of what inline assembly remains (inline assembly being the 
> only code I can think of which is #ifdefed based on the compiler).
> 
> After a couple of months of sporadically trying to solve this issue, I 
> am at a loss what to do next. As I did not write the original renderer, 
> and thus I am only so familiar with it's inner workings. I was hoping 
> somebody more experience with porting or more knowledge of subtle 
> differences between gcc and MSVC would be able to help me solve this 
> issue, which should be the last before I have a basic gcc version ready.

This is difficult to answer.
I guess the best thing to do in this case would be a full analysis of
the code and see how it works and what it's trying to do.
I'd suggest to create a pure standard C/C++ version of the code with
MSVC, step by step, verifying after each modification that everything
still works as expected.  For that version it might be simpler to just
remove all the ifdef stuff, just to keep it simple.
Having a bunch of test cases would be certainly helpful here, too.
Then, once the pure C/C++ version works, try compiling it with GCC,
initially without optimizations such as -ffast-math.

Quickly looking over the code, I'd suspect that there are bugs in the
inline asm codes.  Also, "optimizations" such as found in the functions
"static inline void clearbufbyte (void *d, long c, long a)"
or "dmulshr0" are ... from the past, I guess.

Another thing that sometimes can be a pain is intentional or
unintentional usage of undefined behavior.
For example:

some_type* p = ...;
*p++ = *(p - 1);

I'm not sure whether the code you mention has any of these, but that
kind of stuff could also be a bug source.

Last but not least, you haven't mentioned which GCC version you're
using.  It would be good to know that.

Cheers,
Oleg

Re: GCC vs MSVC compiler issues in a port

[Andrew Haley]

On 12/22/2012 09:07 AM, John Cotton Ericson wrote:
> Hello,
> 
> For a little over a year I have worked on porting the a voxel engine 
> called Voxlap to work on Linux. My code is here: 
> https://github.com/Ericson2314/Voxlap
> 
> I now can compile the same code with both MSVC and gcc. The MSVC code 
> runs well enough, but the gcc version does not. Specifically when I run 
> the MSVC version, everything in-game renders, and a bit of text is 
> thrown on top as a watermark. When I run  the gcc version, all I get is 
> a black window and that water mark. The problem is exactly whether I 
> compile on windows with mingw or compile on linux. In both cases I am 
> compiling the same branch with most of the inline assembly converted to 
> C, and in both cases I compile against the SDL backend, not the direct X 
> backend.
> 
> Because I have done my best to hold everything else constant, I am 
> pretty convinced this error is either due to some subtle difference's in 
> the two compile's understanding of the C semantics, or an error in the 
> gcc version of what inline assembly remains (inline assembly being the 
> only code I can think of which is #ifdefed based on the compiler).
> 
> After a couple of months of sporadically trying to solve this issue, I 
> am at a loss what to do next. As I did not write the original renderer, 
> and thus I am only so familiar with it's inner workings. I was hoping 
> somebody more experience with porting or more knowledge of subtle 
> differences between gcc and MSVC would be able to help me solve this 
> issue, which should be the last before I have a basic gcc version ready.

You need to start with an unoptimized build; does it crash in the
same way?  What about building with -Wall ?

Andrew.

Re: GCC vs MSVC compiler issues in a port

[John Cotton Ericson]

On 12/22/2012 11:30 AM, Andrew Haley wrote:
> You need to start with an unoptimized build; does it crash in the same 
> way? What about building with -Wall ?
I have been using no optimization other than -ffast-math. Whether or not 
I used -msse and -march=native or -mi386 I get arithmetic overflows 
without -ffast-math.

I am looking at the warnings with -Wall. Nothing too interesting but 
I'll try to resolve them.

On 12/22/2012 07:10 AM, Oleg Endo wrote:
> This is difficult to answer. I guess the best thing to do in this case 
> would be a full analysis of the code and see how it works and what 
> it's trying to do. I'd suggest to create a pure standard C/C++ version 
> of the code with MSVC, step by step, verifying after each modification 
> that everything still works as expected. For that version it might be 
> simpler to just remove all the ifdef stuff, just to keep it simple. 
> Having a bunch of test cases would be certainly helpful here, too. 
> Then, once the pure C/C++ version works, try compiling it with GCC, 
> initially without optimizations such as -ffast-math.
I agree, I tried to make my inline assembly as match the original as 
much is possible. But you just can't rule it out while it's still there. 
For testing would you recommend something like 
http://check.sourceforge.net/ ?
> Quickly looking over the code, I'd suspect that there are bugs in the 
> inline asm codes. Also, "optimizations" such as found in the functions 
> "static inline void clearbufbyte (void *d, long c, long a)" or 
> "dmulshr0" are ... from the past, I guess.
Yes, voxlap is quite the time capsule. Whatever version of MSVC the 
original author used must have been perfectly awful.
> Another thing that sometimes can be a pain is intentional or 
> unintentional usage of undefined behavior. For example: some_type* p = 
> ...; *p++ = *(p - 1); I'm not sure whether the code you mention has 
> any of these, but that kind of stuff could also be a bug source.
Hopefully with -Wall I will catch things like this.
> Last but not least, you haven't mentioned which GCC version you're 
> using. It would be good to know that.
On Linux I have 4.7.2.

Another possible lead is that I have heard that switching between Intel 
and at&t syntax in inline assembly blocks messes with gcc extended asm. 
I often have whole blocks Intel and  noprefix to make comparing the MSVC 
inline asm easier.

Thanks, both of you,

John

Re: GCC vs MSVC compiler issues in a port

[David Brown]

On 23/12/12 06:39, John Cotton Ericson wrote:
> On 12/22/2012 11:30 AM, Andrew Haley wrote:
>> You need to start with an unoptimized build; does it crash in the same
>> way? What about building with -Wall ?
> I have been using no optimization other than -ffast-math. Whether or not
> I used -msse and -march=native or -mi386 I get arithmetic overflows
> without -ffast-math.
> 
> I am looking at the warnings with -Wall. Nothing too interesting but
> I'll try to resolve them.
> 

There are more warning flags in gcc than just -Wall - perhaps some
others will give you new clues (though there might be a lot of "noise",
especially with old code).  Also note that many of the gcc warnings only
work well when you have optimisation enabled, as they depend on some of
the analysis passes in the optimiser.  So try at least "-O2 -Wall -Wextra".

Re: GCC vs MSVC compiler issues in a port

[Oleg Endo]

On Sun, 2012-12-23 at 00:39 -0500, John Cotton Ericson wrote:

> I agree, I tried to make my inline assembly as match the original as 
> much is possible. But you just can't rule it out while it's still there. 

Well, as a first step, I'd replace all inline asm stuff with C/C++
implementations in the original version using MSVC.  Performance might
become worse than the original, or it might not.  Either way, I'd leave
those optimizations for later.  Moreover, it would give you a CPU
independent version of the code, which is often a good thing :)

> For testing would you recommend something like 
> http://check.sourceforge.net/ ?

A test_main with a bunch of assert checks should suffice as a start.

Cheers,
Oleg

Re: GCC vs MSVC compiler issues in a port

[John Cotton Ericson]

After replacing more inline assembly with C/C++ I was indeed able to get 
something to render so I could at least visually debug. That means I can 
now work backwards: adding blocks of inline assembly back in and 
carefully verifying that the gcc inline assembly matches the original 
through disassembly.

In addition, I also found some documentation which should help me make 
proper run-time tests, as such tests are the only real way to verify the 
C/C++ and make sure this works on architectures besides x86-32.

Between those two methods I think I'll be able to smoothly finish this. 
Thank you both very much!

On 12/28/2012 06:46 PM, Oleg Endo wrote:
> On Sun, 2012-12-23 at 00:39 -0500, John Cotton Ericson wrote:
>
>> I agree, I tried to make my inline assembly as match the original as
>> much is possible. But you just can't rule it out while it's still there.
> Well, as a first step, I'd replace all inline asm stuff with C/C++
> implementations in the original version using MSVC.  Performance might
> become worse than the original, or it might not.  Either way, I'd leave
> those optimizations for later.  Moreover, it would give you a CPU
> independent version of the code, which is often a good thing :)
>
>> For testing would you recommend something like
>> http://check.sourceforge.net/ ?
> A test_main with a bunch of assert checks should suffice as a start.
>
> Cheers,
> Oleg
>