Fork and Windows Heap

Fork and Windows Heap

[Bill Zissimopoulos]

I am the creator of WinFsp [1], a user mode file system solution for
Windows (i.e. FUSE for Windows). WinFsp has its own API, but I have been
working on adding a FUSE (high-level) API using SSHFS as my primary test
case. This has proven to work very well, except for one important problem
which is the subject of my post.

FUSE provides an API fuse_daemonize() that allows a FUSE file system to
become a daemon, which SSHFS uses. I have provided a simple implementation
using daemon(3), which of course uses fork. Unfortunately SSHFS crashes
whenever it is daemonized and I have traced the failure to an invalid
Windows heap pointer.

WinFsp consists of an FSD (file system driver) and a DLL component. The
WinFsp DLL uses the Windows heap functions to manage memory
(HeapAlloc/HeapFree). It seems that the Windows heap is not properly
cloned after a fork, which leads to the crash I am experiencing. I have
the following questions:

(1) Is my assumption that Windows heaps are not properly cloned after a
fork correct? A 2011 post [2] seems to suggest so.
(2) Is there any workaround that the WinFsp DLL can use to get around this
limitation and work properly after a fork? The obvious answer would be to
have the DLL use Cygwin's malloc/free and this is indeed possible within
the DLL's FUSE layer, but not workable elsewhere.

Any insights welcome.

Bill Zissimopoulos

[1] http://www.secfs.net/winfsp/
[2] https://www.cygwin.com/ml/cygwin-developers/2011-04/msg00035.html

Re: Fork and Windows Heap

[René Berber]

On 6/15/2016 7:42 PM, Bill Zissimopoulos wrote:

[snip]
> WinFsp consists of an FSD (file system driver) and a DLL component. The
> WinFsp DLL uses the Windows heap functions to manage memory
> (HeapAlloc/HeapFree). It seems that the Windows heap is not properly
> cloned after a fork, which leads to the crash I am experiencing. I have
> the following questions:
> 
> (1) Is my assumption that Windows heaps are not properly cloned after a
> fork correct? A 2011 post [2] seems to suggest so.
> (2) Is there any workaround that the WinFsp DLL can use to get around this
> limitation and work properly after a fork? The obvious answer would be to
> have the DLL use Cygwin's malloc/free and this is indeed possible within
> the DLL's FUSE layer, but not workable elsewhere.

Those are the wrong questions: you shouldn't be mixing Windows and Unix
(SuSv4, Posix) APIs, and by your description you are trying a Windows
port, mixing it with probably the base, and expecting it to work on
Cygwin, that's a no-go from the start.

Unix software usually requires no porting, or very little to account for
libc implementations (glibc in Linux, newlib in Cygwin), and the Windows
environment.

I think you have 2 options:

1. Keep the Windows port and use the cross-compiler tools to create
Windows native binaries.  That means: forget about fork, start using the
spawn family of functions, i.e. _wspawnvp.  Of course the result will
work independently of Cygwin, daemons are called services (and created
differently).

2. Keep the base Unix version for everything.  Fork/exec works fine,
daemon works fine but you'll have to use cygrunsrv to install the daemon
as service.

You should read the documentation:

https://cygwin.com/cygwin-ug-net/highlights.html#ov-hi-ansiclib

Hope this helps.
-- 
René Berber




--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Fork and Windows Heap

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2653 bytes --]

Hi Bill,

On Jun 16 00:42, Bill Zissimopoulos wrote:
> I am the creator of WinFsp [1], a user mode file system solution for
> Windows (i.e. FUSE for Windows). WinFsp has its own API, but I have been
> working on adding a FUSE (high-level) API using SSHFS as my primary test
> case. This has proven to work very well, except for one important problem
> which is the subject of my post.
> 
> FUSE provides an API fuse_daemonize() that allows a FUSE file system to
> become a daemon, which SSHFS uses. I have provided a simple implementation
> using daemon(3), which of course uses fork. Unfortunately SSHFS crashes
> whenever it is daemonized and I have traced the failure to an invalid
> Windows heap pointer.
> 
> WinFsp consists of an FSD (file system driver) and a DLL component. The
> WinFsp DLL uses the Windows heap functions to manage memory
> (HeapAlloc/HeapFree). It seems that the Windows heap is not properly
> cloned after a fork, which leads to the crash I am experiencing. I have
> the following questions:
> 
> (1) Is my assumption that Windows heaps are not properly cloned after a
> fork correct? A 2011 post [2] seems to suggest so.

You are correct.  Cygwin fork only clones the datastructures explicitely
set up by Cygwin and stuff allocated using Cygwin's POSIX API.

You can't simply clone a Windows heap for various reasons:

- There's no information given how Windows keeps track over its heaps
  so we don't know how to clone this information.

- A child process will get its own default heap at startup, in another
  memory location than its parent (ASLR).

- Data from non-Cygwin DLLs is inherently not fork-safe.  Especially
  data from DLLs like user32.dll is not simply clonable.  Given that
  cloning is unprovided for, may Windows DLLs will run their own
  initilization code at DLL starttime and creat their completely
  disjunct set of data.  Stuff like connecting to the current
  Windows console fails, etc.

> (2) Is there any workaround that the WinFsp DLL can use to get around this
> limitation and work properly after a fork? The obvious answer would be to
> have the DLL use Cygwin's malloc/free and this is indeed possible within
> the DLL's FUSE layer, but not workable elsewhere.

Exactly.  Ideally, you would provide two DLLs, one for native Windows
using Windows functions, the other for Cygwin, linked against Cygwin and
using POSIX functions only.

Granted, that's a bit more work :}


HTH,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Fork and Windows Heap

[Bill Zissimopoulos]

Hi, Corinna:

> You are correct.  Cygwin fork only clones the datastructures explicitely
> set up by Cygwin and stuff allocated using Cygwin's POSIX API.
>
> You can't simply clone a Windows heap for various reasons...

Thank you for your detailed response and explanation.

Bill

Re: Fork and Windows Heap

[Bill Zissimopoulos]

Renà Berber wrote:

> On 6/15/2016 7:42 PM, Bill Zissimopoulos wrote:

> > (1) Is my assumption that Windows heaps are not properly cloned after a
> > fork correct? A 2011 post [2] seems to suggest so.
> > (2) Is there any workaround that the WinFsp DLL can use to get around
>this
> > limitation and work properly after a fork? The obvious answer would be
>to
> > have the DLL use Cygwin's malloc/free and this is indeed possible
>within
> > the DLL's FUSE layer, but not workable elsewhere.
> 
> Those are the wrong questions: you shouldn't be mixing Windows and Unix
> (SuSv4, Posix) APIs, and by your description you are trying a Windows
> port, mixing it with probably the base, and expecting it to work on
> Cygwin, that's a no-go from the start.

Thank you for your response. With all due respect however I think you have
given me the wrong answers/pointers.

I am not porting anything from UNIX. I have a Windows solution developed
using Visual Studio and the Windows Driver Kit that I am building a FUSE
compatibility layer for. My DLL is not a Cygwin DLL. It is a native
Windows DLL that also has a FUSE compatibility layer. I am taking pains to
make that FUSE compatibility layer available to both Win32 and Cygwin apps.

The answers to my questions are:

(1) Cygwin does not clone Windows heaps after a fork. [At least my brief
perusal of winsup/cygwin/fork.cc seems to indicate so.]

(2) The workaround is to avoid allocating resources that Cygwin cannot
account for (e.g. from the Windows heap) prior to daemon/fork. Luckily
this is possible in a FUSE file system design, because in general it looks
like this:

    fuse_new
    fuse_daemonize		// do not allocate any non-Cygwin resources prior to
this
    fuse_loop/fuse_loop_mt	// safe to allocate non-Cygwin resources
    fuse_destroy

I have now modified the WinFsp FUSE layer accordingly and tested it
against SSHFS. I am able to daemonize SSHFS and everything works
correctly. I therefore consider this problem solved for me.

Bill