From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2128.outbound.protection.outlook.com [40.107.237.128]) by sourceware.org (Postfix) with ESMTPS id CB34C3857C6A for ; Wed, 22 Jul 2020 16:42:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org CB34C3857C6A ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gEQNAut0+ZRegFjPhN75HzWEHmcw4Tkt70bt8XO7xL2X96tkO61VPnc5s4DObgPL1SIYhNqSSKvzYzySZpAhqEzpY2BvyyAcQbwcpaIp7wnWjOnSZAn+hu06SC2LT7W9DnBlKf68XiIqEyj7F4PsorbJhMVl29O2IsYRh01hvxawAbKTBowU/hvRgacJWfqSR6/BpVHh6G3uq3aW8x6aWl2OVQbyQRHZ27WCsP5ptNAq++YhmxDIDSu1hbEYzu5dk1oExSAc6PXWFQaGcOcYk5e2N2jRTxgIV2V7AYZj2EX4Tpjr0fIxGPF81PyVSfpJdRW6KICvgIPufTHcVnU/ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wbKYGdHRoibdpoebprV8twjss5Z3BiAhuyUJOaBAuDw=; b=AWLF4aFxATxETT6Hm7/4pv5RNOwsS9t8iHTH7lerchOzRB2AUNoIxzkiMFctBIi7B3qay3kCj59SoHmioMgjNO5+guXXaGcP92wHvJHkbRrCXAZIvM/DMNLfKOlQeJOKK/GNh2ybwWqAcqom6wD4k4Ov5fQA9n4XtlPSO96HnrfxIOUzvNS58HZmvYW2duDAjJz2CXkuKogDYPjr/liuUAvdLDddGVoyz75dTt1tR6IKMekxibZdBPWyGvUm8SlG6dIweG9j3qk8Nn6EpJ3UNVD5rBp9KTko8cHXoAV41jVI2cMA0hSUZgnUl/wknQBLj9/zRF42sR0hn1Ymn/inUQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none Received: from MN2PR04MB6176.namprd04.prod.outlook.com (2603:10b6:208:e3::13) by MN2PR04MB6237.namprd04.prod.outlook.com (2603:10b6:208:d5::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.23; Wed, 22 Jul 2020 16:42:34 +0000 Received: from MN2PR04MB6176.namprd04.prod.outlook.com ([fe80::184d:a265:1d48:499a]) by MN2PR04MB6176.namprd04.prod.outlook.com ([fe80::184d:a265:1d48:499a%7]) with mapi id 15.20.3195.028; Wed, 22 Jul 2020 16:42:34 +0000 Subject: Re: page_size vs allocation_granularity To: cygwin-developers@cygwin.com References: <3ac8f341-1dbe-b407-de64-4a2d5c191e68@cornell.edu> <20200722083327.GR16360@calimero.vinschen.de> From: Ken Brown Message-ID: <3e8c9ae9-cfa7-f4db-d243-dc9593b252e9@cornell.edu> Date: Wed, 22 Jul 2020 12:42:31 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 In-Reply-To: <20200722083327.GR16360@calimero.vinschen.de> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH2PR17CA0027.namprd17.prod.outlook.com (2603:10b6:610:53::37) To MN2PR04MB6176.namprd04.prod.outlook.com (2603:10b6:208:e3::13) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [IPv6:2604:6000:b407:7f00:ad12:1782:80ed:dfca] (2604:6000:b407:7f00:ad12:1782:80ed:dfca) by CH2PR17CA0027.namprd17.prod.outlook.com (2603:10b6:610:53::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.21 via Frontend Transport; Wed, 22 Jul 2020 16:42:33 +0000 X-Originating-IP: [2604:6000:b407:7f00:ad12:1782:80ed:dfca] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f376fcff-4985-4897-edb1-08d82e5e3be4 X-MS-TrafficTypeDiagnostic: MN2PR04MB6237: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Ts7e9U3e1TYjIBFnG5Finod86DTAQ9zIu8l4cSVJVejyjUSn5rKP4eo/8l+EckeSoghfXMtPtkqL4D7rf73/Bn+joB2pfFHXOVXm2pwRwUMUw6HfyTJhDW21CU00eRJYkEy9bgVSP6svcCIxM33421ZrJQ7Kr++s612b7B2PY412Ct+ipNqrhVu7TrxjNUwot6s4il7tocLA0tISzPs3CUMnIEaqzygL6lRrJs50JveG3eWGz6YzoXb1C3kqpSN6jSXjQSoU5ubHnANMYFs4LiIptIUiwwhpsgZtzadkAUF/j6gOt6tZb6suBRIL6Mk8OJaD4hhNzMRhYtKcJGWLqlJVGFTQd2S0vu+t5d1TqgvNveomTtv5w9Lk5bvw1oRc9tLejDzs3wdVjrcX27V9feKfuJtEJqve9kE0krfR0YXIqmFDgz/kDKgai1N3fE98/+Jh3bxXUPd/RXfIPTgthA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR04MB6176.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(136003)(346002)(396003)(376002)(39860400002)(316002)(786003)(2616005)(7116003)(75432002)(2906002)(478600001)(966005)(5660300002)(66476007)(66556008)(186003)(31696002)(52116002)(66946007)(86362001)(31686004)(6916009)(53546011)(83380400001)(8676002)(8936002)(36756003)(6486002)(16526019)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: PemopkYTGBAK57yhxJpTIXpQhrSpJTQgWOxMAELY6DZpiVKqroTJFw3cwCGkw8XjSmvyYg1IxUbgDiQR7NFnbHRyhoTmJu3e/wDclG8m9E3zyZbjKTVk2RxDpEW2BCtDamQxr/v75H2/sdhjOITq3qAaP1J1/7AsuQC38eEYwd1GafCS9yuR+NOAzVmbWSEK28nOUMcQ71+I/lhD+a0yMJpHlonHZKUStXKEuRWGsWczM/vPoG7Y3oB7deJi8Sz8sBhZFIuA23MNulpqC7lOFuUJgMBmAPX3lg4D04KrZoAJZ0aw70G5Zy6Epz0vf98h6rJ+qJgXKkIctc5Dri3IsL60laA/WDvNk94v3JlFLS1sbLWPXMbnv7VaoTivqUb7DxwH3ExZAXm26rQrCWI0JM+c27JgSo8cKOerzS3idiwDYkaSt0d1rUVcIix1WwhlDJ7XaQYDWF20MHGq4L0vuIpR/Y5oVqZIEPKOsp8Uibubuh0P05QD8xhlQc33U5DPRRz/NSYJsTFPnSZzpOKfGLEWFZi++pesLiHjHkTXtZE= X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: f376fcff-4985-4897-edb1-08d82e5e3be4 X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB6176.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jul 2020 16:42:34.1776 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uUMP4RsjabFD7IZ0W04YgoWSeCC3INoLh7gSqSwveHwdsv3saDT0T+hQeNi2D4UYfph9OBB27k/HlbBwgbgxrA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6237 X-Spam-Status: No, score=-3.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jul 2020 16:42:38 -0000 On 7/22/2020 4:33 AM, Corinna Vinschen wrote: > On Jul 21 18:40, Ken Brown via Cygwin-developers wrote: >> Hi Corinna, >> >> I'm curious about the design decision that causes sysconf(_SC_PAGESIZE) to >> return wincap.allocation_granularity() rather than wincap.page_size(). >> Changing this would improve Linux compatibility, I think, but maybe it would >> have some bad consequences that I'm not aware of. > > It was a long and hard process to move from 4K to 64K pagesize, with > lots of loaded discussions. The Cygwin mailing list archives will > show a lot of this in the 200X years. > > It was the only way to make mmap 99% POSIX-conformant. Consider, for > instance this: > > pagesize = sysconf(_SC_PAGESIZE); > addr = mmap (NULL, pagesize, PROT_READ | PROT_WRITE, > MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); > addr2 = mmap (addr + pagesize, pagesize, PROT_READ | PROT_WRITE, > MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); > > On Windows, this fails with pagesize = 4K, but it works with pagesize = > 64K, because of that idiotic Windows allocation granularity. Almost > all POSIX expectations are automagically fixed by using the granularity > as pagesize in a POSIX sense. > > There's only one problem left: While you can only allocate usefully in > 64K steps, the size of the memory area allocated for a file is only 4K > aligned, thus leaving the remainder of the 64K block unmapped. > > This problem could be fixed back in 32 bit times by adding the > AT_ROUND_TO_PAGE mapping. Very unfortunately, the 64 bit Windows > designer decided to keep the braindead 64K allocation granularity > but to drop the AT_ROUND_TO_PAGE flag, thus removing the only chance > to make this single situation POSIX-compatible as well. > >> I'm asking because in my recent fooling around with php, I noticed that >> Yaakov had to apply the following Cygwin-specific patch to avoid a crash: > > It would be nice to learn what kind of crash that was. Here's a better reference than the one I gave in my previous reply, which actually explains what's going on: https://sourceware.org/pipermail/cygwin/2017-May/232562.html > If php reads or writes in the remainder of the block constituting EOF, > or tries to change page protection, shit happens. Every time, a process > stabs into the EOF block following the last valid 4K block, it results > in a STATUS_ACCESS_VIOLATION which in turn calls > mmap_is_attached_or_noreserve(). While this situation can be > recognized, I don't see a way to fix this from the processes POV. So that's exactly what happens when php maps a file whose size is a multiple of 4K but not a multiple of 64K. It expects that there is a zero-filled region beyond EOF that it can safely read from. Ken