[Bug fortran/53015] New: free_pi_tree(): Unresolved fixup is back

[Bug fortran/53015] New: free_pi_tree(): Unresolved fixup is back

[brainschrat at gmx dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53015

             Bug #: 53015
           Summary: free_pi_tree(): Unresolved fixup is back
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: brainschrat@gmx.de


Bug 47546 was fixed some time ago, but I get "Internal error - free_pi_tree():
Unresolved fixup" all over the place again, with versions 4.6.2, 4.7.0 and
4.8.0 of gfortran from svn.

I have a project that uses many Fortran 2003 features, like polymorphism,
type-bound procedures etc. Some sub-classes are in different modules, which did
not cause problems before and is supported by the standard. I also use OpenMPI
and PETSc, so there are some preprocessor macros. The code compiles and runs
fine withe the Intel Fortran compiler, which is the only other Fortran compiler
I have access to.

Like in 47546 , the issue is very elusive: Sometimes it happens when compiling
the modules themselves, sometimes when compiling a program that uses the
modules. Sometimes commenting out one arbitrary line of code fixes the error,
and it remains fixed after putting the line in again. Sometimes removing all
modules and object files and re-compiling (even files that remain unchanged)
fixes the error. On some days, I cannot compile my code at all, but using an
different version of gfortran works. But then the issue pops up again after
adding a new line of code.

Because of this, I am not able to provide a test case as I have no clue what
the error means and the rest of the case seems non-deterministic. Maybe someone
can assist me in narrowing down the possibilities.

[Bug fortran/53015] free_pi_tree(): Unresolved fixup is back

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53015

Tobias Burnus <burnus at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |burnus at gcc dot gnu.org

--- Comment #1 from Tobias Burnus <burnus at gcc dot gnu.org> 2012-04-17 09:08:51 UTC ---
(In reply to comment #0)
> I get "Internal error - free_pi_tree(): Unresolved fixup" all over the place again, with versions 4.6.2, 4.7.0 and 4.8.0 of gfortran from svn. [...]
> the issue is very elusive: Sometimes it happens when compiling the modules
> themselves, sometimes when compiling a program that uses the modules.
> I have no clue what the error means and the rest of the case seems
> non-deterministic.

The pi_tree is used when reading a module file (.mod). Often, when reading a
symbol from the .mod file, not all pieces of information are available. Thus,
an incomplete symbol is created - and later the missing bits are added. The
information is storied in a binary tree ("pi_tree" as it stores a pointer and
an integer as id). And the ICE (= internal compiler error) occurs if that tree
has a bad status.

There are two kinds of possible problems, leading to such a failure:
(a) The tree is not properly walked (during fixup resolution); that typically
occurs when it is modified while walking the tree - as the modification might
cause a tree rebalancing. 
(b) Freed memory is accessed - which either shouldn't be accessed or shouldn't
be freed. Often the memory (still) contains the correct or at least somewhat
sensible data. [Or some variants of that.]

Both type of problems depend on many details - and not only on the compiler
input but also the hardware, libraries and how the compiler has been compiled.
Tiny differences can cause the program to fail or not to fail. Unfortunately,
it does not make it simple to find the bug.

In this case, my suspicion would be that there is a bug when reading a .mod
file. However, I remember a problem* where symbols where not properly resolved,
leading to writing insufficient data to the .mod file, such that at the end the
ICE occurred when reading the .mod file - but the bug was much earlier. (*
http://gcc.gnu.org/ml/fortran/2011-11/msg00000.html) Unfortunately, debugging
the pi_tree is very difficult as the pointers it contains lack type
information.


I hope that your problem is reproducible, i.e., if you find a test case, it
will more or less reliably also fail on, e.g., my computer. That will make
debugging tremendously easier. You could also try valgrind; there should be no
failure (except of leaking memory). However, I fear that valgrind won't help in
this case.

Other than that, I can only suggest to create a small rest case. For instance
using http://gcc.gnu.org/wiki/A_guide_to_testcase_reduction [please use GCC 4.8
for that]. I do not have any other idea and I fear that even with a small test
case, debugging won't be easy.

[Bug fortran/53015] free_pi_tree(): Unresolved fixup is back

[brainschrat at gmx dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53015

--- Comment #2 from brainschrat at gmx dot de 2012-04-17 13:46:29 UTC ---
Ok, thanks for your explanation! 

Currently, I have one case that always fails and will try the delta approach.
Manually trying to narrow it down would drive me nuts, but delta seems
feasible.

[Bug fortran/53015] free_pi_tree(): Unresolved fixup is back

[brainschrat at gmx dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53015

--- Comment #3 from brainschrat at gmx dot de 2012-04-18 15:22:01 UTC ---
Maybe this is related to using both -I and -J to the same directory.

As I wanted to use delta, I tried to simplify my folder layout for the test
case:

project
 - src
 - lib
 - testsuite
    o test_1
    o test_2 etc

Usually, make is called in project, compiles .f90 files in src and puts all
.mod and .o files there as well via -J switch. The bug happens if I compile
file B in test_2, but points to file A in src (or rather, A.mod).

When I put all files in test_2 for using delta, the bug disappeared. After some
tinkering I found out that putting the .mod files anywhere else than src (via
-J) works. There is both a -I and a -J to src, directly after one another.

If I -I and -J point to the same directory, I get the free pi_tree stuff.
Removing the -I (as it is not needed for this purpose) solves the problem. The
module files produced by the two different calls have the same number of lines,
the same size but the ordering in the file.

Should the generation of the module files be deterministic? Are there any known
problems with -I an d -J? (I could not find anything in bugzilla.)

[Bug fortran/53015] free_pi_tree(): Unresolved fixup is back

[janus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53015

janus at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |janus at gcc dot gnu.org

--- Comment #4 from janus at gcc dot gnu.org 2012-04-19 12:15:16 UTC ---
(In reply to comment #3)
> If I -I and -J point to the same directory, I get the free pi_tree stuff.
> Removing the -I (as it is not needed for this purpose) solves the problem.

I have to admit that I have never used the -J option myself, but the
documentation at
http://gcc.gnu.org/onlinedocs/gcc-4.7.0/gfortran/Directory-Options.html seems
to suggest that using "-Jdir" sort of implies "-Idir" (in the sense that this
directory is searched for .mod files).

Is there a particular reason that you use both of them? The problem may indeed
be due to the two interfering (which of course would be a bug). What happens if
you use neither -I nor -J?

In any case it would be helpful to have some sort of test case (it doesn't
necessarily need to be extremely reduced, but the smaller the better, of
course).

[Bug fortran/53015] free_pi_tree(): Unresolved fixup is back

[brainschrat at gmx dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53015

--- Comment #5 from brainschrat at gmx dot de 2012-04-23 11:48:20 UTC ---
Yes, -I and -J are totally unneccessary together. I made use of -I long before
I started putting the modules in other directories, and together they produce
this mess, but not reliably.

I have six modules (the problem is matlib.f90, the most complex one) and about
seven test cases that use them. Six of seven test cases build fine, and the
seventh can be reduced to an empty subroutine with a couple of use statements,
which are identical to the ones from a working case. The problem occurs on "use
matlib"

I tried to use the Fortran-aware delta to reduce the case, but I get some
confusing errors. The error disappears if I put the f90 files for the modules
in the same directory as the source file for the test case, even if I move the
modules with -J. My testscript for delta returns 0 if I call it, but it returns
1 if called by delta:

cat deltascript.sh
#! /bin/sh
make -C /home/staff/paul/noo DEBUG=1 lib
mpif90 -I/home/staff/paul/noo/include -c $1 2>&1| grep -q fixup
if ! test $? = 0; then
    exit 1
fi
exit 0

The first call is to make sure the library (made up of the object files related
to the six modules) is there. This returns 0 if I call it myself with
./deltascript.sh gugv_ass_test.f90, but using

 ~/skripte/delta -test=deltascript.sh -suffix=.f90 -cp_minimal=bla.f90
../../../src/matlib.f90

gives 

[0,2108] 000.f90make: Entering directory `/home/staff/paul/noo'
make: Nothing to be done for `lib'.
make: Leaving directory `/home/staff/paul/noo'


    **************** FAIL: The initial input does not pass the test.

So appearently I am doing something wrong or the whole problem is also related
to the directory from which make (or gfortran?) is called, as delta copies the
files to tmp**/arena. I am sort of stuck at this issue, but maybe I am not
using delta correctly?

[Bug fortran/53015] free_pi_tree(): Unresolved fixup is back

[burnus at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53015

--- Comment #6 from Tobias Burnus <burnus at gcc dot gnu.org> 2012-04-23 12:46:20 UTC ---
(In reply to comment #5)
> My testscript for delta returns 0 if I call it, but it returns
> 1 if called by delta

Delta generates a new directory, copies the file there and runs then the
script. You probably miss some .mod files. You could add a
  cp /full/path/{a,b,c}.mod .
at the beginning of the delta script for those .mod files you need.

[Bug fortran/53015] free_pi_tree(): Unresolved fixup is back

[brainschrat at gmx dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53015

--- Comment #7 from brainschrat at gmx dot de 2012-04-30 13:01:21 UTC ---
I fixed up my delta script as I now finaly understood what it did:

#! /bin/sh
cp input.f90 /home/staff/paul/noo/src
make -C /home/staff/paul/noo DEBUG=1 lib
cp /home/staff/paul/noo/include/*.mod ./
cp /home/staff/paul/noo/testsuite/bugs/delta/gugv_ass_test_noo.f90 ./
mpif90 -c gugv_ass_test_noo.f90 2>&1 | grep -q fixup
if ! test $? = 0; then
        exit 1
fi
exit 0

The point is that the problematic file (matlib.f90) has to be compiled in the
original source directory using the original Makefile, because the error cannot
be reproduced reliably. Then the generated modules are moved to the current
directory and the problematic test case is compiled.

Now delta proceeds to remove ALL lines of code, so something is still wrong.
What are temporary source files in delta iterations called? (This is the file I
have to cp to src/matlib.f90.)

[Bug fortran/53015] free_pi_tree(): Unresolved fixup is back

[brainschrat at gmx dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53015

brainschrat at gmx dot de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |INVALID

--- Comment #8 from brainschrat at gmx dot de 2012-06-01 10:00:29 UTC ---
As 
1) I cannot reproduce this error reliably,
2) I am not able to reduce the test case with delta and
3) there is a known workaround (or rather there is no need to use the compiler
switches that trigger this),

I changed the status to resolved: invalid.