From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 129915 invoked by alias); 8 Feb 2018 11:47:44 -0000 Mailing-List: contact cygwin-help@cygwin.com; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner@cygwin.com Mail-Followup-To: cygwin@cygwin.com Received: (qmail 129906 invoked by uid 89); 8 Feb 2018 11:47:44 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.5 required=5.0 tests=AWL,BAYES_00,KAM_NUMSUBJECT,SPF_HELO_PASS,SPF_PASS,T_RP_MATCHES_RCVD autolearn=no version=3.3.2 spammy=H*r:LOGIN, optin, opt-in, H*UA:Outlook X-HELO: ppsw-30.csi.cam.ac.uk Received: from ppsw-30.csi.cam.ac.uk (HELO ppsw-30.csi.cam.ac.uk) (131.111.8.130) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 08 Feb 2018 11:47:42 +0000 Received: from liber.sm.cl.cam.ac.uk ([128.232.60.117]:56865 helo=Liber) by ppsw-30.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:25) with esmtpsa (LOGIN:dra27) (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) id 1ejkg0-000No2-e2 (Exim 4.89_1) for cygwin@cygwin.com (return-path ); Thu, 08 Feb 2018 11:47:40 +0000 From: "David Allsopp" To: Subject: Regression for OCaml introduced by rebase 4.4.4 Date: Thu, 08 Feb 2018 11:47:00 -0000 Message-ID: <000001d3a0d2$9f604860$de20d920$@cl.cam.ac.uk> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-IsSubscribed: yes X-SW-Source: 2018-02/txt/msg00080.txt.bz2 TL;DR flexlink-compiled DLLs (i.e. ocaml libraries) are broken by the 0x200000000 base address requirement added in rebase 4.4.4. Possible fixes for this at the bottom. Commit bfd383 in the rebase sources introduces a new minimum base address requirement of 0x200000000 for Cygwin64. This is a problem for the correct operation of the flexdll package and affects ocaml. On a fresh up-to-date Cygwin64 installation, install the ocaml package: $ rebase -i /usr/lib/ocaml/stublibs/* /usr/lib/ocaml/stublibs/dllbigarray.so base 0x000000010000 size 0x00015000 * /usr/lib/ocaml/stublibs/dllcamlstr.so base 0x000000010000 size 0x00014000 * /usr/lib/ocaml/stublibs/dllgraphics.so base 0x000000010000 size 0x00038000 * /usr/lib/ocaml/stublibs/dllnums.so base 0x000000010000 size 0x00011000 * /usr/lib/ocaml/stublibs/dllthreads.so base 0x000000010000 size 0x00025000 * /usr/lib/ocaml/stublibs/dllunix.so base 0x000000010000 size 0x0004c000 * /usr/lib/ocaml/stublibs/dllvmthreads.so base 0x000000010000 size 0x0001f000 * Here you can see a problem we already know about with flexlink - all libraries have a base address of 0x10000 (https://github.com/alainfrisch/flexdll/issues/50). However, this allows you to load libraries dynamically: $ ocaml OCaml version 4.04.2 # #load "unix.cma";; # #directory "+threads";; # #load "threads.cma";; but not fork (we know about this problem): # Unix.fork ();; 0 [main] ocamlrun 5688 child_info_fork::abort: address space needed by 'dllunix.so' (0x400000) is already occupied Exception: Unix.Unix_error (Unix.EAGAIN, "fork", ""). Now do a rebaseall. $ rebase -i /usr/lib/ocaml/stublibs/* /usr/lib/ocaml/stublibs/dllvmthreads.so base 0x0003fec20000 size 0x0001f000 /usr/lib/ocaml/stublibs/dllunix.so base 0x0003fec40000 size 0x0004c000 /usr/lib/ocaml/stublibs/dllthreads.so base 0x0003fec90000 size 0x00025000 /usr/lib/ocaml/stublibs/dllnums.so base 0x0003fecc0000 size 0x00011000 /usr/lib/ocaml/stublibs/dllgraphics.so base 0x0003fece0000 size 0x00038000 /usr/lib/ocaml/stublibs/dllcamlstr.so base 0x0003fed20000 size 0x00014000 /usr/lib/ocaml/stublibs/dllbigarray.so base 0x0003fed40000 size 0x00015000 So forking should now be fine. However: $ ocaml OCaml version 4.04.2 # #load "unix.cma";; Cannot load required shared library dllunix. Reason: /usr/lib/ocaml/stublibs/dllunix.so: flexdll error: cannot relocate RELOC_REL32, target is too far: 0xfffffffc013d8b5f 0x13d8b5f. This is a known problem and fundamental limitation of flexdll (there is no RELOC_REL64 in COFF). On our CI, we have been using a workaround for the fork problem at https://github.com/ocaml/ocaml/blob/trunk/tools/ci-build#L230-L231 but that no longer works with rebase 4.4.4 because of the new minimum base address. It was already the case that rebaseall was breaking OCaml DLLs, but now with 4.4.4 they cannot even be fixed by hand, so it's clearly a good moment to put some effort into this (read as: I'm offering both coding and testing time!). For this to work at all, there needs to be some address space below 0x80000000 which DLLs may be permitted to opt-in to using and which rebase needs to respect. Assuming that's OK, I think something along the following lines is needed: 1. We (ab)use either a DLL characteristics flag or a section header flag to indicate that the DLL needs to be loaded below 0x8000000 2. The rebase utility warning for base addresses should take that flag into account (to the point of requiring < 0x80000000 if this new bit is set in the image) [2a. While we're changing validation for the image base, it'd be sensible to add a check that the supplied address is 64K aligned :$] 3. The flexlink utility should stop using 0x10000 all the time. Probably the best way to achieve this is if the rebase utility has a flag which *sets* the new bit so that flexlink calls rebase after compilation to assign an improved base address to the DLL. On x86, we don't force a given base address at all - I assume that Cygwin's binutils stuff is already rebase-aware and produces sensible base addresses for newly-compiled DLLs, as I don't recall having ever seen the fork conflict problem on x86 builds of OCaml? Comments on the proposed need for some DLLs to occupy memory below 0x80000000 and on the fixes much appreciated, thanks! David -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple