From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail1.pdinc.us (mail.pdinc.us [67.90.184.27]) by sourceware.org (Postfix) with ESMTPS id D87153857807 for ; Tue, 25 Aug 2020 15:25:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org D87153857807 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=pdinc.us Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=jpyeron@pdinc.us Received: from blackfat (nsa1.pdinc.us [67.90.184.2]) (authenticated bits=0) by mail1.pdinc.us (8.14.4/8.14.4) with ESMTP id 07PFP7OF022466 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 25 Aug 2020 11:25:07 -0400 DKIM-Filter: OpenDKIM Filter v2.11.0 mail1.pdinc.us 07PFP7OF022466 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pdinc.us; s=default; t=1598369107; bh=Hz2UzlXajkqFdPjqJZxiZtchD9V1my81rZUJ3IdeQD8=; h=From:To:References:In-Reply-To:Subject:Date:From; b=ryxiHderUAuAZvBFGsfzFfoR6x/agCzXhReBFR9RAICTzU3w+aoIewqlxo3gXfrg0 ChJDP6VFAqZrP/NOcAl2sUzIAdqQ9dXa8tBj/6zgph/KKqnQUFXi6MVX18zp1IHAvZ 8+txwDSpy2hJnvYwJziBrlK05uJnmfP8Z2lFCcEdbX7DyHYHScjLBAQhOlZkdLMAV2 0n0JQESLMvRuX3tGiEYJONIrC1K7dcpBwn30IbSpbI5OUEtzNfDji0SSMmUMghO787 smIibsmmSXw+8RRaCmtIOPDy5/wOebRHuZgma0LriB80SA/muqDLFPDC8rXWXUYBDs gJX83UGxDyhuA== From: "Jason Pyeron" To: References: <4AA035EB-1325-4C1B-B399-28FC9176F203@roc.cs.umass.edu> <006c01d67aed$2f7f0660$8e7d1320$@linuxandria.com> In-Reply-To: Subject: RE: [cygwin] Re: Mandatory ASLR breaks Cygwin - Windows 10 Date: Tue, 25 Aug 2020 11:25:16 -0400 Organization: PD Inc Message-ID: <1b2501d67af3$efcd9e80$cf68db80$@pdinc.us> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Content-Language: en-us Thread-Index: AQKHdN2I7RjeEs9XHVIuOgXE8fUDuQI8fs8GAaDF4ycCbRNWXae0qsVg X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_INFOUSMEBIZ, KAM_NUMSUBJECT, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Aug 2020 15:25:11 -0000 > -----Original Message----- > From: Brian Inglis > Sent: Tuesday, August 25, 2020 11:13 AM >=20 > On 2020-08-25 08:36, Alexandria Cortez wrote: > > On Tuesday, August 25, 2020 10:35 AM, Eliot Moss wrote:>> On Aug 25, = 2020, at 10:17 AM, Alexandria > Cortez wrote: > >>> =EF=BB=BFI was experimenting with security settings this morning = on windows, and > >>> after changing Mandatory ASLR (Windows Security -> App and Browser = Control > >>> -> Exploit Protection) to default on, no Cygwin programs that rely = on the > >>> Cygwin dll would start, stating that a resource was temporarily = unavailable > >>> and could not fork. Rebasell, bash, you name it crashed and would = not start. > >>> After some investigation, turning off that setting allows Cygwin = to work. > >>> > >>> Now the next question: why does this not work? Is this intended = behavior or > >>> a bug? Having that setting turned on seems like a good idea from a = security > >>> standpoint, and who knows it may eventually become default. >=20 > >> It=E2=80=99s intentional; too long to explain in detail on phone, = but fork > >> requires each dll to load in the child at the same address as in = the > >> parent, and ASLR interferes with achieving that. > > Is there any plans to implement a workaround in the future? Seeing = as Cygwin > > is only one of two programs I've noticed that are broken with it on, = it > > would be nice to be able to have it on from a security perspective. > Cygwin is an all-volunteer project - Someone Has To Do It! > Feel free to submit patches to support that in Cygwin under Windows. > A low level understanding of details of both is required. Some relevant reading=20 https://stackoverflow.com/questions/8171298/is-address-space-layout-rando= mization-applied-on-a-forked-process Is Address Space Layout Randomization applied on a forked process Q: Say I fork a process from another process. Will Address Space Layout = Randomization (ASLR) be applied on it in an OS which has ASLR set? Note that I am talking about the case where I don't call execve function = after doing fork. A: 12 Yes. However note that after fork both parent and child have the same = randomization applied to them (they are copies of each other after = all!). If the child and parent are to call mmap(NULL, ...), then their address = maps will start to diverge. Update: > Isn't your statement contradictory? Not at all. Immediately after fork, the parent and child address spaces = are identical (that's the definition of what fork does). But the ASLR is = still in effect for both the parent and the child. The randomization = can't "go back in time" and randomize decisions that have already been = made, but any future decisions (such as where to place next mmap) will = be randomized, and will likely result in different outcome for parent = and child. > Does it have to do with basic mmap or OS writers introduce randomness = in mmap as well for security? Perhaps you don't understand what ASLR is? In short, with ASLR on, the OS will randomize placement of main stack, = and placement of any non-MAP_FIXED mmaps. By the time you fork, the main stack placement has long been determined, = so parent and child will have the same. The future mmap are the only = things that can (and will be) affected by ASLR going forward. https://cygwin.com/cygwin-ug-net/highlights.html Process Creation The fork call in Cygwin is particularly interesting because it does not = map well on top of the Win32 API. This makes it very difficult to = implement correctly. Currently, the Cygwin fork is a non-copy-on-write = implementation similar to what was present in early flavors of UNIX. As the child process is created as new process, both the main executable = and all the dlls loaded either statically or dynamically have to be = identical as to when the parent process has started or loaded a dll. = While Windows does not allow to remove binaries in use from the file = system, they still can be renamed or moved into the recycle bin, as = outlined for unlink(2) in the section called =E2=80=9CFile Access = related changes=E2=80=9D. To allow an existing process to fork, the = original binary files need to be available via their original file = names, but they may reside in a different directory when using the = DotLocal (.local) Dll Redirection feature. Since NTFS does support = hardlinks, when the fork fails we try again, but create a private = directory containing hardlinks to the original files as well as the = .local file now. The base directory for the private hardlink directory = is /var/run/cygfork/, which you have to create manually for now if you = need to protect fork against updates to executables and dlls on your = Cygwin instance. As hardlinks cannot be used across multiple NTFS file = systems, please make sure your executable and dll replacing operations = operate on the same single NTFS file system as your Cygwin instance and = the /var/run/cygfork/ directory. Note that this private hardlink = directory also does help for when a wrong dll is found in the parent = process' current working directory. To enable creating the hardlinks, = you need to stop all currently running Cygwin processes after creating = this directory, once per Cygwin installation: $ mkdir --mode=3Da=3Drwxt /var/run/cygfork We create one hardlink directory per user, application and application = age, and remove it when no more processes use that directory. To = indicate whether a directory still is in use, we define a mutex name = similar to the directory name. As mutexes are destroyed when no process = holds a handle open any more, we can clean up even after power loss or = similar: Both the parent and child process, at exit they lock the mutex = with almost no timeout and close it, to get the closure promoted = synchronously. If the lock succeeded before closing, directory cleanup = is started: For each directory found, the corresponding mutex is created = with lock. If that succeeds, the directory is removed, as it is unused = now, and the corresponding mutex handle is closed. Before fork, when about to create hardlinks for the first time, the = mutex is opened and locked with infinite timeout, to wait for the = cleanup that may run at the same time. Once locked, the mutex is = unlocked immediately, but the mutex handle stays open until exit, and = the hardlinks are created. It is fine for multiple processes to = concurrently create the same hardlinks, as the result really should be = identical. Once the mutex is open, we can create more hardlinks within = this one directory without the need to lock the mutex again. The first thing that happens when a parent process forks a child process = is that the parent initializes a space in the Cygwin process table for = the child. It then creates a suspended child process using the Win32 = CreateProcess call. Next, the parent process calls setjmp to save its = own context and sets a pointer to this in a Cygwin shared memory area = (shared among all Cygwin tasks). It then fills in the child's .data and = .bss sections by copying from its own address space into the suspended = child's address space. After the child's address space is initialized, = the child is run while the parent waits on a mutex. The child discovers = it has been forked and longjumps using the saved jump buffer. The child = then sets the mutex the parent is waiting on and blocks on another = mutex. This is the signal for the parent to copy its stack and heap into = the child, after which it releases the mutex the child is waiting on and = returns from the fork call. Finally, the child wakes from blocking on = the last mutex, recreates any memory-mapped areas passed to it via the = shared area, and returns from fork itself. While we have some ideas as to how to speed up our fork implementation = by reducing the number of context switches between the parent and child = process, fork will almost certainly always be inefficient under Win32. = Fortunately, in most circumstances the spawn family of calls provided by = Cygwin can be substituted for a fork/exec pair with only a little = effort. These calls map cleanly on top of the Win32 API. As a result, = they are much more efficient. Changing the compiler's driver program to = call spawn instead of fork was a trivial change and increased = compilation speeds by twenty to thirty percent in our tests. However, spawn and exec present their own set of difficulties. Because = there is no way to do an actual exec under Win32, Cygwin has to invent = its own Process IDs (PIDs). As a result, when a process performs = multiple exec calls, there will be multiple Windows PIDs associated with = a single Cygwin PID. In some cases, stubs of each of these Win32 = processes may linger, waiting for their exec'd Cygwin process to exit. Problems with process creation The semantics of fork require that a forked child process have exactly = the same address space layout as its parent. However, Windows provides = no native support for cloning address space between processes and = several features actively undermine a reliable fork implementation. = Three issues are especially prevalent: DLL base address collisions. Unlike *nix shared libraries, which use = "position-independent code", Windows shared libraries assume a fixed = base address. Whenever the hard-wired address ranges of two DLLs collide = (which occurs quite often), the Windows loader must "rebase" one of them = to a different address. However, it may not resolve collisions = consistently, and may rebase a different dll and/or move it to a = different address every time. Cygwin can usually compensate for this = effect when it involves libraries opened dynamically, but collisions = among statically-linked dlls (dependencies known at compile time) are = resolved before cygwin1.dll initializes and cannot be fixed afterward. = This problem can only be solved by removing the base address conflicts = which cause the problem, usually using the rebaseall tool. Address space layout randomization (ASLR). Starting with Vista, Windows = implements ASLR, which means that thread stacks, heap, memory-mapped = files, and statically-linked dlls are placed at different (random) = locations in each process. This behaviour interferes with a proper fork, = and if an unmovable object (process heap or system dll) ends up at the = wrong location, Cygwin can do nothing to compensate (though it will = retry a few times automatically). DLL injection by BLODA. Badly-behaved applications which inject dlls = into other processes often manage to clobber important sections of the = child's address space, leading to base address collisions which rebasing = cannot fix. The only way to resolve this problem is to remove (usually = uninstall) the offending app. In summary, current Windows implementations make it impossible to = implement a perfectly reliable fork, and occasional fork failures are = inevitable.