[Bug fortran/35063] New: write statement leaks handles/memory on windows xp

[Bug fortran/35063] New: write statement leaks handles/memory on windows xp

[mikko dot lyly at csc dot fi]

Hi,

The follwing small sample program seems to produce exsessive handle/memory leak
on Windows XP SP2. This affects programs, where a high number of write
statements are used to manipulate character strings.

$ less test.f90
program test
  implicit none
  character(len=10000) :: string
  integer :: m,n
  do m = 1,10000
     print *,'Check memory and handles and press ENTER...'
     call flush
     read(*,*)
     do n = 1,10000
        write(string,*) 'some characters'
     end do
  end do
end program test
$ gfortran test.f90 -O -o test.exe
$ ./test.exe

Regards,
ML


-- 
           Summary: write statement leaks handles/memory on windows xp
           Product: gcc
           Version: 4.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: fortran
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: mikko dot lyly at csc dot fi


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug fortran/35063] write statement leaks handles/memory on windows xp

[jpr at csc dot fi]

------- Comment #1 from jpr at csc dot fi  2008-02-03 14:38 -------
*** Bug 35064 has been marked as a duplicate of this bug. ***


-- 

jpr at csc dot fi changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jpr at csc dot fi


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug fortran/35063] write statement leaks handles/memory on windows xp

[jvdelisle at gcc dot gnu dot org]

------- Comment #2 from jvdelisle at gcc dot gnu dot org  2008-02-03 17:36 -------
I have confirmed this under cygwin. Taking the READ statement out so the
program can run unabated, leads to a system failure trying to allocate memory. 
This is platform specific, the program runs to completion on linux with no
problems.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2008-02-03 17:36:06
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] write statement leaks handles/memory on windows xp

[jvdelisle at gcc dot gnu dot org]

------- Comment #3 from jvdelisle at gcc dot gnu dot org  2008-02-04 04:35 -------
I did not see this on my first run because it scrolls by too fast on my
terminal.

==25454== Conditional jump or move depends on uninitialised value(s)
==25454==    at 0x4333F1: strlen (in /home/jerry/prs/pr35063/a.out)
==25454==    by 0x45DE74: fillin_rpath (in /home/jerry/prs/pr35063/a.out)
==25454==    by 0x45FC35: _dl_init_paths (in /home/jerry/prs/pr35063/a.out)
==25454==    by 0x439CDE: _dl_non_dynamic_init (in
/home/jerry/prs/pr35063/a.out)
==25454==    by 0x43A3DA: __libc_init_first (in /home/jerry/prs/pr35063/a.out)
==25454==    by 0x414CF5: (below main) (in /home/jerry/prs/pr35063/a.out)
==25454== 
==25454== Conditional jump or move depends on uninitialised value(s)
==25454==    at 0x4333BD: strlen (in /home/jerry/prs/pr35063/a.out)
==25454==    by 0x408087: write_integer (write.c:612)
==25454==    by 0x40AEDD: _gfortrani_list_formatted_write (write.c:785)
==25454==    by 0x400330: MAIN__ (in /home/jerry/prs/pr35063/a.out)
==25454==    by 0x40043B: main (fmain.c:21)

The end of the run shows:

==25454== Use of uninitialised value of size 8
==25454==    at 0x41AB57: exit (in /home/jerry/prs/pr35063/a.out)
==25454== 
==25454== ERROR SUMMARY: 12 errors from 3 contexts (suppressed: 0 from 0)
==25454== malloc/free: in use at exit: 0 bytes in 0 blocks.
==25454== malloc/free: 0 allocs, 0 frees, 0 bytes allocated.
==25454== For counts of detected errors, rerun with: -v
==25454== All heap blocks were freed -- no leaks are possible.

But no memory leaks reported.  Still, its suspicious.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] write statement leaks handles/memory on windows xp

[tkoenig at gcc dot gnu dot org]

------- Comment #4 from tkoenig at gcc dot gnu dot org  2008-02-04 07:17 -------
(In reply to comment #3)

> ==25454== Conditional jump or move depends on uninitialised value(s)
> ==25454==    at 0x4333BD: strlen (in /home/jerry/prs/pr35063/a.out)
> ==25454==    by 0x408087: write_integer (write.c:612)
> ==25454==    by 0x40AEDD: _gfortrani_list_formatted_write (write.c:785)
> ==25454==    by 0x400330: MAIN__ (in /home/jerry/prs/pr35063/a.out)
> ==25454==    by 0x40043B: main (fmain.c:21)

Strange.  How did the code ever reach write_integer?  There
is no integer I/O there.  Could you maybe set a breakpoint and
check how it is reached?

Unfortunately, I can't reproduce a problem on i686-pc-linux-gnu.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] write statement leaks handles/memory on windows xp

[jvdelisle at gcc dot gnu dot org]

------- Comment #5 from jvdelisle at gcc dot gnu dot org  2008-02-04 21:05 -------
Sorry for the confusion.  I used:

print *,'Check memory and handles and press ENTER...', m

so I could track the progress.  Without the m, I see no errors.  Also notice
that he two upfront errors are in calls to strlen.   One of those has nothing
to do with libgfortran.. I suspect a libc error there.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] write statement leaks handles/memory on windows xp

[fxcoudert at gcc dot gnu dot org]

------- Comment #6 from fxcoudert at gcc dot gnu dot org  2008-02-04 23:40 -------
(In reply to comment #2)
> I have confirmed this under cygwin. Taking the READ statement out so the
> program can run unabated, leads to a system failure trying to allocate memory. 
> This is platform specific, the program runs to completion on linux with no
> problems.

What's weird is that the number of "Handles" get bigger even though the "Mem
usage" of the application actually doesn't grow (for me, it's fixed at 860 K).
I've debugged it a bit: of malloc/calloc/realloc, only malloc is called, and
always from libgfortran's get_mem. Having adding output statements to get_mem()
and free_mem(), I've checked that all allocations are deallocated soon after.
Maybe that's the result of some alloca'ed memory? (shooting in the dark,
really)


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fxcoudert at gcc dot gnu dot
                   |                            |org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] I/O leaks handles/memory on Windows XP

[fxcoudert at gcc dot gnu dot org]

------- Comment #7 from fxcoudert at gcc dot gnu dot org  2008-02-05 10:55 -------
OK, found it by commenting out various parts of the I/O library until it
disappeared. The leak is the following line in data_transfer_init (transfer.c):

    dtp->u.p.current_unit = get_unit (dtp, 1);

The memory allocated for the unit is never freed. This also happens for
external units:

  program test
    do
      open(10,file="foo",status="replace")
      write(10,'()')
      close(10)
    end do
  end program test

(under Windows, the number of handles will grow indefinitely, like for the
internal unit case).


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|write statement leaks       |I/O leaks handles/memory on
                   |handles/memory on windows xp|Windows XP


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #8 from jvdelisle at gcc dot gnu dot org  2008-02-06 01:27 -------
I will work this one.  I think the problem is not in freeing, but in not
reusing a UNIT structure already open.


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|unassigned at gcc dot gnu   |jvdelisle at gcc dot gnu dot
                   |dot org                     |org
             Status|NEW                         |ASSIGNED
   Last reconfirmed|2008-02-03 17:36:06         |2008-02-06 01:27:05
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] I/O leaks handles/memory on Windows XP

[jpr at csc dot fi]

------- Comment #9 from jpr at csc dot fi  2008-02-06 11:31 -------
Subject: Re:  I/O leaks handles/memory on Windows XP


just an observation: if i build gcc with "thread model=single", all seems 
well.

Juha

>
>
> ------- Comment #8 from jvdelisle at gcc dot gnu dot org  2008-02-06 01:27 -------
> I will work this one.  I think the problem is not in freeing, but in not
> reusing a UNIT structure already open.
>
>
> -- 
>
> jvdelisle at gcc dot gnu dot org changed:
>
>           What    |Removed                     |Added
> ----------------------------------------------------------------------------
>         AssignedTo|unassigned at gcc dot gnu   |jvdelisle at gcc dot gnu dot
>                   |dot org                     |org
>             Status|NEW                         |ASSIGNED
>   Last reconfirmed|2008-02-03 17:36:06         |2008-02-06 01:27:05
>               date|                            |
>
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063
>
> ------- You are receiving this mail because: -------
> You are on the CC list for the bug, or are watching someone who is.
>


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] I/O leaks handles/memory on Windows XP

[Jerry_V_DeLisle at rl dot gov]

------- Comment #10 from Jerry_V_DeLisle at rl dot gov  2008-02-06 20:25 -------
Reply to comment #9:

This is a very important observation.  I have checked the code and we have code
that free's the internal unit (free_internal_unit) and I have confirmed that it
is called correctly after every write.

My current thinking now is that we are not locking/unlocking the unit correctly
and maybe it is this lock that is leaving behind a windows handle.  Stay tuned,
and thanks for reporting this observation.  


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] I/O leaks handles/memory on Windows XP

[jpr at csc dot fi]

------- Comment #11 from jpr at csc dot fi  2008-02-06 20:54 -------
Subject: Re:  I/O leaks handles/memory on Windows XP


I had a look: the problem seems to be with the logic of  mutex->counter in
gcc/config/i386/gthr-win32.c. Libgfortran makes the  following calls 
during an internal write

1) __gthr_win32_mutex_init_function()
   - sets counter=-1
   - creates a semaphore (the windows handle!)

2)__gthr_win32_mutex_lock()
    - increments counter (now 0)

3) __gthr_win32_mutex_unlock()
   - decrements counter (now -1 again)
   - would release the semaphore, if counter >= 0 but doesn't...!

not a libfortran bug after all(?)


>
>
> ------- Comment #10 from Jerry_V_DeLisle at rl dot gov  2008-02-06 20:25 -------
> Reply to comment #9:
>
> This is a very important observation.  I have checked the code and we have code
> that free's the internal unit (free_internal_unit) and I have confirmed that it
> is called correctly after every write.
>
> My current thinking now is that we are not locking/unlocking the unit correctly
> and maybe it is this lock that is leaving behind a windows handle.  Stay tuned,
> and thanks for reporting this observation.
>
>
> -- 
>
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063
>
> ------- You are receiving this mail because: -------
> You are on the CC list for the bug, or are watching someone who is.
>


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] I/O leaks handles/memory on Windows XP

[tkoenig at gcc dot gnu dot org]

------- Comment #12 from tkoenig at gcc dot gnu dot org  2008-02-06 21:49 -------
Just wondering... is this a regression?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] I/O leaks handles/memory on Windows XP

[jpr at csc dot fi]

------- Comment #13 from jpr at csc dot fi  2008-02-07 05:07 -------
Subject: Re:  I/O leaks handles/memory on Windows XP


>
> ------- Comment #12 from tkoenig at gcc dot gnu dot org  2008-02-06 
21:49 -------
> Just wondering... is this a regression?

At least g77 runs the two examples (in comments #1 & #7) without 
problems...

Reading  windows  docs: somewhere windows CloseHandle() should be called 
to close the Semaphore created by CreateSemaphore() called by 
__gthr_win32_mutex_init_function().

Otherwise the __gthr_win32.c code might be OK (ReleaseSemaphore() only 
modifies the semaphore usage count... i thought something else.)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #14 from jvdelisle at gcc dot gnu dot org  2008-02-07 06:07 -------
Yes this is a regression wrt g77 and I have noted this in the summary.  I have
attempted to insert a mutex_unlock in the right place and it has no effect.  I
think jpr's assessment is correct, there needs to be a close_handle call
somewhere in the code path.  There is no close_handle call in gthr-win32.c


-- 

jvdelisle at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |wrong-code
            Summary|I/O leaks handles/memory on |[Regression wrt g77] I/O
                   |Windows XP                  |leaks handles/memory on
                   |                            |Windows XP


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[kargl at gcc dot gnu dot org]

------- Comment #15 from kargl at gcc dot gnu dot org  2008-02-07 06:26 -------
(In reply to comment #14)
> Yes this is a regression wrt g77 and I have noted this in the summary.  I have
> attempted to insert a mutex_unlock in the right place and it has no effect.  I
> think jpr's assessment is correct, there needs to be a close_handle call
> somewhere in the code path.  There is no close_handle call in gthr-win32.c
> 

It's not a regression wrt to g77 because g77 isn't thread safe.
In fact, if comment #11 is accurate, this isn't a libgfortran
bug. It's a target bug.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jpr at csc dot fi]

------- Comment #16 from jpr at csc dot fi  2008-02-07 09:19 -------
Subject: Re:  [Regression wrt g77] I/O leaks handles/memory
 on Windows XP


Well, the internal write case is anyway a regression introduced by this 
patch

http://gcc.gnu.org/ml/gcc-patches/2006-06/msg00717.html


>
>
> ------- Comment #15 from kargl at gcc dot gnu dot org  2008-02-07 06:26 -------
> (In reply to comment #14)
>> Yes this is a regression wrt g77 and I have noted this in the summary.  I have
>> attempted to insert a mutex_unlock in the right place and it has no effect.  I
>> think jpr's assessment is correct, there needs to be a close_handle call
>> somewhere in the code path.  There is no close_handle call in gthr-win32.c
>>
>
> It's not a regression wrt to g77 because g77 isn't thread safe.
> In fact, if comment #11 is accurate, this isn't a libgfortran
> bug. It's a target bug.
>
>
> -- 
>
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063
>
> ------- You are receiving this mail because: -------
> You are on the CC list for the bug, or are watching someone who is.
>


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #17 from jvdelisle at gcc dot gnu dot org  2008-02-07 15:01 -------
Once again, good detective work by jpr.  This then does make this a regression
wrt 4.1.  I do not have a 4.1 build on my windows machine to check.  Regardless
...

This begs a question.  Why do we even want locking on internal units?  The
string is local to the thread and allocated on the stack most likely.  I am not
sure that another thread can even see it.  The unit structure is created on a
per READ/WRITE basis and is freed after the operation is complete.  Anyone else
have any thoughts on this.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[fxcoudert at gcc dot gnu dot org]

------- Comment #18 from fxcoudert at gcc dot gnu dot org  2008-02-07 15:20 -------
(In reply to comment #17)

> This begs a question.  Why do we even want locking on internal units?  The
> string is local to the thread and allocated on the stack most likely.

What about the case of two threads writing to the same string (maybe to
overlapping parts of the same string)? It's not good programing, but if we
support it for files, why not for internal units?

I don't see why the string should be local to the thread, and it might not
always be allocated on the stack. "Most likely" just doesn't seem good enough
:)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[Jerry_V_DeLisle at rl dot gov]

------- Comment #19 from Jerry_V_DeLisle at rl dot gov  2008-02-07 15:54 -------
Writing to an internal unit is nothing more than a fancy (formatted)
assignment.  Each thread allocates its own unit structure.  Its not like file
I/O where threads are accessing a common shared resource.

I should mention that Danny Smith has sent me a potential patch that I will
test tonight or tomorrow.  I would not mind seeing a threaded example where
this could be tested and that confirms we need to lock internal units. :)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[brian at dessent dot net]

------- Comment #20 from brian at dessent dot net  2008-02-07 21:17 -------
Subject: Re:  I/O leaks handles/memory on Windows XP


> 1) __gthr_win32_mutex_init_function()
>    - sets counter=-1
>    - creates a semaphore (the windows handle!)
> 
> 2)__gthr_win32_mutex_lock()
>     - increments counter (now 0)
> 
> 3) __gthr_win32_mutex_unlock()
>    - decrements counter (now -1 again)
>    - would release the semaphore, if counter >= 0 but doesn't...!

This gthr-win32 stuff should only be run if --enable-threads=win32. 
That is not the case with Cygwin which should be using pthreads
(--enable-threads=posix).  So if the above is really at fault then the
leak should not manifest on Cygwin at all.

Brian


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #21 from jvdelisle at gcc dot gnu dot org  2008-02-08 03:14 -------
The problem manifests with:

$ gfc -v
Using built-in specs.
Target: i686-pc-cygwin
Configured with: ../gcc43/configure --prefix=/usr/local/gfortran
--enable-languages=c,fortran --disable-bootstrap --enable-threads=posix
--enable-sjlj-exceptions --enable-version-specific-runtime-libs --enable-nls
--disable-libmudflap --disable-shared --disable-win32-registry
--with-system-zlib --enable-checking=release --enable-werror
--without-included-gettext --without-x --enable-libgomp
Thread model: posix
gcc version 4.3.0 20080128 (experimental) [trunk revision 131899] (GCC) 

I removed the mutex_lock code from the relevant sections of
libgfortran/io/unit.c for internal units and the handle count growth goes away.
This confirms the problem.  I am still working the solution and have two
approaches.  One is to modify the mutex_unlock to delete the semaphore created
and thus release the handle (Danny Smith idea).  The other is to keep track of
units assigned as internal and not create a new unit structure, but reuse the
old. Stay tuned.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jpr at csc dot fi]

------- Comment #22 from jpr at csc dot fi  2008-02-08 05:29 -------
Subject: Re:  [Regression wrt g77] I/O leaks handles/memory
 on Windows XP



> So if the above is really at fault then the
> leak should not manifest on Cygwin at all.

The logic there is ok i guess, as i already noted later in the thread.
As the handles also leak using posix threads under cygwin,  i guess the 
posix thread implementation does something similiar. The  problem is that 
when the mutex is done with pthread_mutex_destroy() should be called. 
gcc's gthread interface used in libfortran however does not however have 
this functionality as far as i can tell from the sources.

Anyway, Jerry idea of reusing the internal unit structure would 
effectively solve this problem, being also a good thing otherwise IMO.

Juha

> This gthr-win32 stuff should only be run if --enable-threads=win32.
> That is not the case with Cygwin which should be using pthreads
> (--enable-threads=posix).  So if the above is really at fault then the
> leak should not manifest on Cygwin at all.
>
> Brian
>
>
> -- 
>
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063
>
> ------- You are receiving this mail because: -------
> You are on the CC list for the bug, or are watching someone who is.
>


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[fxcoudert at gcc dot gnu dot org]

------- Comment #23 from fxcoudert at gcc dot gnu dot org  2008-02-08 08:58 -------
(In reply to comment #19)
>  I would not mind seeing a threaded example where
> this could be tested and that confirms we need to lock internal units. :)

Me too :)  I think I got confused between locking the file/string and locking
the unit. I agree that internal units don't need to be locked.

(In reply to comment #21)
> The other is to keep track of
> units assigned as internal and not create a new unit structure, but reuse the
> old. Stay tuned.

This might be desirable for other reasons, but it will not fix the problem
completely. My comment #7 shows that this problem is also present for exernal
units:

  program test
    do
      open(10,file="foo",status="replace")
      write(10,'()')
      close(10)
    end do
  end program test

(and watch the Handle count grow). 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jpr at csc dot fi]

------- Comment #24 from jpr at csc dot fi  2008-02-08 12:12 -------
Subject: Re:  [Regression wrt g77] I/O leaks handles/memory
 on Windows XP


> This might be desirable for other reasons, but it will not fix the problem
> completely.

Yeah well, the easy way out here would be to include 
"__gthread_mutex_destroy" in the gthread-api. This should call 
CloseHandle() (or a new __gthr_win32_mutex_destroy() with a call to 
CloseHandle()) for the win32 threads and pthread_mutex_destroy() for the 
posix threads,  etc... and call the new  function in a few relevant places 
in libgfortran...


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #25 from jvdelisle at gcc dot gnu dot org  2008-02-08 14:26 -------
Reply to comment #23.  With external units we have a treap structure which
holds pointers to the unit structures so that we reuse them as needed  The
treap is searched first for an existing unit and returns that if it exists. 
This keys off of the unit number.  This one handle per unit.

With internal units we do not have a unit number to key off.  I could use the
address of the string itself. I have not pursued that avebue yet because it is
not the simplest approach.

I have successfully tested a patch that uses a single static unit structure for
all internal units, relying on the locking mechanism to assure that it is used
only by one operation at a time.  It works great and improves performance by
avoiding the memory allocation overhead. It uses at most only one handle.
Unfortunately, it breaks write_recursive.f90.  Recursive I/O is permitted on
internal units in F2003.

Reply to comment #24
The patch from Danny adds a new function to gthr-win32.c that does a
CloseHandle.  I will be testing this approach soon.

Regardless, I am leaning toward not locking the internal units because of some
performance gain and the need to retain recursive IO.  I will do some
comparative tests between the CloseHandle approach and the no lock approach
before proposing to the list the final patch.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[fxcoudert at gcc dot gnu dot org]

------- Comment #26 from fxcoudert at gcc dot gnu dot org  2008-02-08 14:39 -------
(In reply to comment #25)
> With external units we have a treap structure which
> holds pointers to the unit structures so that we reuse them as needed.

I haven't looked at the source code, but I can assure you that the following
code clearly shows on Windows that we leak handles for external units. It is
fact.

   character(len=1) :: string
   do
     open(10,file="foo",status="replace")
     write(10,'()')
    close(10)
  end do
  end

Compile it, run it with Task manager open, and see handle count increase.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #27 from jvdelisle at gcc dot gnu dot org  2008-02-08 14:58 -------
OK OK I believe you.  :)  What the heck was I looking at before?  :)  I am just
working up the closehandle patch now.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #28 from jvdelisle at gcc dot gnu dot org  2008-02-08 16:33 -------
pthread_mutex_destroy() fixes the problem on Cygwin including the case in
Comment #26.  It regression tests OK on Linux-x86-64.  I am now just reviewing
to make sure we don't try to lock after a destroy, which can lead to undefined
behavior.

I will submit the patch here first since it will need to be tested on mingw.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #29 from jvdelisle at gcc dot gnu dot org  2008-02-08 18:10 -------
Created an attachment (id=15118)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15118&action=view)
Proposed patch, needs testing on mingw

Here is the patch.  I have tested this on Cygwin, but not mingw.  It is
regression testing on x86-64.  A previous version passed regression testing,
but I am retesting after clean-up.

I would appreciate if others would review/test this patch.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[mikko dot lyly at csc dot fi]

------- Comment #30 from mikko dot lyly at csc dot fi  2008-02-08 22:31 -------
> I would appreciate if others would review/test this patch.

I have applied this patch against rev 132188 on mingw32.

The problem described in comments #1 and #7 for internal and external units,
rspectively, seems to be resolved.

$ /c/gcc-patched/mingw/bin/gfortran-patched.exe -v                       
Using built-in specs.
Target: mingw32
Configured with: ../gcc/configure --with-gcc --host=mingw32 --build=mingw32
--target=mingw32 --with-arch=i486 --with-tune=generic --program-suffix=-patched
--disable-werror --prefix=/mingw --with-local-prefix=/mingw --disable-nls
--enable-languages=c,fortran --disable-win32-registry --enable-threads
--enable-sjlj-exceptions --enable-libstdcxx-debug
--enable-cxx-flags='-fno-function-sections -fno-data-sections'
--enable-version-specific-runtime-libs --disable-bootstrap
Thread model: win32
gcc version 4.3.0 20080208 (experimental) (GCC) 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at verizon dot net]

------- Comment #31 from jvdelisle at verizon dot net  2008-02-08 23:00 -------
Subject: Re:  [Regression wrt g77] I/O leaks
 handles/memory on Windows XP

mikko dot lyly at csc dot fi wrote:
> ------- Comment #30 from mikko dot lyly at csc dot fi  2008-02-08 22:31 -------
>> I would appreciate if others would review/test this patch.
> 
> I have applied this patch against rev 132188 on mingw32.
> 
> The problem described in comments #1 and #7 for internal and external units,
> rspectively, seems to be resolved.
> 

Thanks for testing and the report.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jpr at csc dot fi]

------- Comment #32 from jpr at csc dot fi  2008-02-09 06:46 -------
Subject: Re:  [Regression wrt g77] I/O leaks handles/memory
 on Windows XP


hi,

the patch looks good to me, although from the clarity point of view and 
to avoid potential problem with other thread models, i'd go all the way 
and introduce the __gthread_mutex_destroy() function in the  gcc/gthr*.h 
files.

juha

>
>
> ------- Comment #29 from jvdelisle at gcc dot gnu dot org  2008-02-08 18:10 -------
> Created an attachment (id=15118)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15118&action=view)
> --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15118&action=view)
> Proposed patch, needs testing on mingw
>
> Here is the patch.  I have tested this on Cygwin, but not mingw.  It is
> regression testing on x86-64.  A previous version passed regression testing,
> but I am retesting after clean-up.
>
> I would appreciate if others would review/test this patch.
>
>
> -- 
>
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063
>
> ------- You are receiving this mail because: -------
> You are on the CC list for the bug, or are watching someone who is.
>


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[tkoenig at gcc dot gnu dot org]

------- Comment #33 from tkoenig at gcc dot gnu dot org  2008-02-09 21:49 -------
We should fix this in 4.3.0 (and it's a regression, too).


-- 

tkoenig at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |4.3.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #34 from jvdelisle at gcc dot gnu dot org  2008-02-10 00:25 -------
Just waiting for someone who can approve it to actually approve it.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jb at gcc dot gnu dot org]

------- Comment #35 from jb at gcc dot gnu dot org  2008-02-10 16:55 -------
Patch with __gthread_mutex_destroy for all implementations:

http://gcc.gnu.org/ml/fortran/2008-02/msg00082.html


-- 

jb at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jb at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #36 from jvdelisle at gcc dot gnu dot org  2008-02-10 18:07 -------
Subject: Bug 35063

Author: jvdelisle
Date: Sun Feb 10 18:06:34 2008
New Revision: 132217

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=132217
Log:
2008-02-10  Danny Smith  <dannysmith@users.sourceforge.net>

        PR gcc/35063
        * gthr-win32.h (__gthread_mutex_destroy_function): New function
        to CloseHandle after unlocking to prevent accumulation of handle
        count. 

2008-02-10  Jerry DeLisle  <jvdelisle@gcc.gnu.org>

        PR libfortran/35063
        * io/unit.c (destroy_unit_mutex): New function that uses
        __gthread_mutex_destroy_function or pthread_mutex_destroy after
        unlocking and before free_mem for final closure of I/O unit.
        (delete_root): Use new function.
        (free_internal_unit): Likewise.
        (close_unit_1): Likewise.

Modified:
    trunk/ChangeLog
    trunk/gcc/gthr-win32.h
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/unit.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] [Regression wrt g77] I/O leaks handles/memory on Windows XP

[jvdelisle at gcc dot gnu dot org]

------- Comment #37 from jvdelisle at gcc dot gnu dot org  2008-02-10 18:09 -------
Interim windows only patch committed.  Followup on Comment #35 for 4.4. 
Keeping this PR open until then.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] Clean up use of destroy_unit_mutex()

[fxcoudert at gcc dot gnu dot org]

-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |minor
           Keywords|wrong-code                  |
            Summary|[Regression wrt g77] I/O    |Clean up use of
                   |leaks handles/memory on     |destroy_unit_mutex()
                   |Windows XP                  |
   Target Milestone|4.3.0                       |4.4.0


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] Clean up use of destroy_unit_mutex()

[jvdelisle at gcc dot gnu dot org]

------- Comment #38 from jvdelisle at gcc dot gnu dot org  2008-02-23 16:40 -------
Janne, do you want to bring your patch from comment #35 up to date and submit
for approval to the lists?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] Clean up use of destroy_unit_mutex()

[jb at gcc dot gnu dot org]

------- Comment #39 from jb at gcc dot gnu dot org  2008-02-23 18:46 -------
(In reply to comment #38)
> Janne, do you want to bring your patch from comment #35 up to date and submit
> for approval to the lists?

Sure, I hope to get it done withing a few days; assigning to myself.


-- 

jb at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|jvdelisle at gcc dot gnu dot|jb at gcc dot gnu dot org
                   |org                         |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] Clean up use of destroy_unit_mutex()

[jb at gcc dot gnu dot org]

------- Comment #40 from jb at gcc dot gnu dot org  2008-02-25 20:47 -------
Updated version of the patch in #35 here:
http://gcc.gnu.org/ml/gcc-patches/2008-02/msg01222.html


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] Clean up use of destroy_unit_mutex()

[jb at gcc dot gnu dot org]

------- Comment #41 from jb at gcc dot gnu dot org  2008-03-01 10:52 -------
Subject: Bug 35063

Author: jb
Date: Sat Mar  1 10:46:31 2008
New Revision: 132800

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=132800
Log:
gcc:

2008-03-01  Janne Blomqvist  <jb@gcc.gnu.org>

        PR gcc/35063
        * gthr.h: Add __gthread_mutex_destroy as a function that must be
        implemented.
        * gthr-vxworks.h (__gthread_mutex_destroy): Null implementation.
        * gthr-single.h (__gthread_mutex_destroy): Likewise.
        * gthr-rtems.h (__gthread_mutex_destroy): Likewise.
        * gthr-mipssde.h (__gthread_mutex_destroy): Likewise.
        * gthr-nks.h (__gthread_mutex_destroy): Likewise.
        * gthr-solaris.h (__gthread_mutex_destroy): Call mutex_destroy.
        * gthr-win32.h (__GTHREAD_MUTEX_DESTROY_FUNCTION): Remove.
        (__gthread_mutex_destroy_function): Rename to
        __gthread_mutex_destroy.
        * gthr-dce.h (__gthread_mutex_destroy): Call
        pthread_mutex_destroy.
        * gthr-tpf.h (__gthread_mutex_destroy): Likewise.
        * gthr-posix.h (__gthread_mutex_destroy): Likewise.
        * gthr-posix95.h (__gthread_mutex_destroy): Likewise.


libgfortran:

2008-03-01  Janne Blomqvist  <jb@gcc.gnu.org>

        PR libfortran/35063
        * io/unit.c (destroy_unit_mutex): Call __gthread_mutex_destroy
        instead of macro kludge.


Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/gthr-dce.h
    trunk/gcc/gthr-mipssde.h
    trunk/gcc/gthr-nks.h
    trunk/gcc/gthr-posix.h
    trunk/gcc/gthr-posix95.h
    trunk/gcc/gthr-rtems.h
    trunk/gcc/gthr-single.h
    trunk/gcc/gthr-solaris.h
    trunk/gcc/gthr-tpf.h
    trunk/gcc/gthr-vxworks.h
    trunk/gcc/gthr-win32.h
    trunk/gcc/gthr.h
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/io/unit.c


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] Clean up use of destroy_unit_mutex()

[fxcoudert at gcc dot gnu dot org]

------- Comment #42 from fxcoudert at gcc dot gnu dot org  2008-03-01 12:22 -------
Now fixed.


-- 

fxcoudert at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063

[Bug libfortran/35063] Clean up use of destroy_unit_mutex()

[jvdelisle at gcc dot gnu dot org]

------- Comment #43 from jvdelisle at gcc dot gnu dot org  2008-03-01 17:13 -------
Subject: Bug 35063

Author: jvdelisle
Date: Sat Mar  1 17:12:52 2008
New Revision: 132802

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=132802
Log:
2008-03-01  Janne Blomqvist  <jb@gcc.gnu.org>

        PR gcc/35063
        * gthr-posix.h (__gthread_mutex_destroy): Remove extra declarations.
        * gthr-posix95.h (__gthread_mutex_destroy): Likewise. Note this fixes
        regression from previous patch.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/gthr-posix.h
    trunk/gcc/gthr-posix95.h


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35063