From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-eopbgr760134.outbound.protection.outlook.com [40.107.76.134]) by sourceware.org (Postfix) with ESMTPS id 1D9F33857032 for ; Wed, 22 Jul 2020 18:35:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 1D9F33857032 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SP1PBUQT6Xv8UmKIiiuwlR3e/OcDqUjEaKNJ0BC0VrFOekFQtYqZ7iUps6jmcSrDMYNQmb4fKW2JzzDOfJDUKzK6bP7OO1NwCaDMBPqJqaVo5Tp9PyEtSdIc8KKaZcJ44HyvmdlsvBtc2/acOJGfpjFwZnZy1bGBh/F9aB32TFCAIo8YYPhAKv33n7br9TxXjepIw0z7089gfow3lMcX4gEo1kw690U9UKBAeygxUhlCq0+k5JTq40SL1aW+DNxVVtXgqgkAvC1YYGv0NsN46D5QER0NwAMY+JCvWHDQsuKQ/XInlDx4RRfJ6bJNOAZPZ4EdT7fhLKdf2ZxXjgVHpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zPFmZ6/SBL34T1mQQf0NQjrxywJlvU47rU3+dmdOgZU=; b=X0fynwYrsRP5JYQQODq/okElR0YyDcctUG6uQxjhDxrPbgSh5UuEl1JXgdoFFCXj6g3D2mShqi9yHvY0PyKgVwH9wtGBV4yhp1/2oZNzHZtcfLGl7PjgUpOGedNNgtA7uflWAelcPoy2jm3xYqRuvYUV3uvzLybEB2+kkdqiqXPnztfOyxT8isk4w1dL5TwqRYfw8L2DJmOfoJNNRZqRmyVxUXrk9iJHAg9jdTZigOJ0ZFlMplQw+ipl3x+/+r8WxYlapw61D8FO1cVQOfGCdPOEhDi0UKGG9VEbSQu5QB970AHuCDNhM1dvVn81OnWi5iNEa+2ZItdi5SX6z80BCw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cornell.edu; dmarc=pass action=none header.from=cornell.edu; dkim=pass header.d=cornell.edu; arc=none Received: from MN2PR04MB6176.namprd04.prod.outlook.com (2603:10b6:208:e3::13) by MN2PR04MB5760.namprd04.prod.outlook.com (2603:10b6:208:3e::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.23; Wed, 22 Jul 2020 18:35:07 +0000 Received: from MN2PR04MB6176.namprd04.prod.outlook.com ([fe80::184d:a265:1d48:499a]) by MN2PR04MB6176.namprd04.prod.outlook.com ([fe80::184d:a265:1d48:499a%7]) with mapi id 15.20.3195.028; Wed, 22 Jul 2020 18:35:07 +0000 Subject: Re: page_size vs allocation_granularity To: cygwin-developers@cygwin.com References: <3ac8f341-1dbe-b407-de64-4a2d5c191e68@cornell.edu> <20200722083327.GR16360@calimero.vinschen.de> <3e8c9ae9-cfa7-f4db-d243-dc9593b252e9@cornell.edu> From: Ken Brown Message-ID: <5eaa60f8-37d5-1d73-7ec7-679ca0cce3c9@cornell.edu> Date: Wed, 22 Jul 2020 14:35:05 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 In-Reply-To: <3e8c9ae9-cfa7-f4db-d243-dc9593b252e9@cornell.edu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-ClientProxiedBy: MN2PR03CA0017.namprd03.prod.outlook.com (2603:10b6:208:23a::22) To MN2PR04MB6176.namprd04.prod.outlook.com (2603:10b6:208:e3::13) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [IPv6:2604:6000:b407:7f00:ad12:1782:80ed:dfca] (2604:6000:b407:7f00:ad12:1782:80ed:dfca) by MN2PR03CA0017.namprd03.prod.outlook.com (2603:10b6:208:23a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.23 via Frontend Transport; Wed, 22 Jul 2020 18:35:07 +0000 X-Originating-IP: [2604:6000:b407:7f00:ad12:1782:80ed:dfca] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d6a70779-6e26-41fa-94cd-08d82e6df54e X-MS-TrafficTypeDiagnostic: MN2PR04MB5760: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3044; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: He9UPIisuW2aRNPiCOXWi5oE9UA7F/1gj65h5b20J+r5U0/5Nq/WLuX3rm6uw7as1dMuIme3xAwZC5+ZPHgKDB52pSEMuMi67lL1Zgcm0M6g5IJ61VWO8eamUvj6LA2ML/jIMM+vtdTdiosmgQ/gVUj6Q6pG/M8xP88XZH1zG0T+gJd4ldHQqxe/EdZTGIP/1qbySnLpf+E/nXCorXLI1wcg0OiiebUkRO9PYfYOpA8f7gqlL5YrdaBRC8+20SoNfHJW1L8W7SjV8H44EJLrHrLlPlNZXozzWZ5jxkEVQk1V2br4sUbHGjXzmjrjwWBxqVAwkM4NDlPhMF/0hl+KsW7oCT5K416DVv+Y0C8UILDw23R3Fo2s7gz4EBBBHtoud7VK7W5dMXKgaExcjG/P19uaRhLJ8NSD+eAFnJYz60nj/CgiC0u3sUFUjrFsEOePej1l2raRCohOSo/pYf1ZgA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR04MB6176.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(376002)(39860400002)(396003)(346002)(366004)(31696002)(8936002)(6916009)(8676002)(2906002)(786003)(7116003)(6486002)(316002)(86362001)(186003)(16526019)(53546011)(75432002)(31686004)(478600001)(83380400001)(66556008)(66476007)(66946007)(5660300002)(966005)(36756003)(2616005)(52116002)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: 9OkNqDGVlY4YYZ/4gFYrtoSQmBxfRnyF77H3/HuAAXGio1rp0OKsUcno6JH90FXxQUhJM+Cvy0NShX3EcvmXQX6hpYl/+G1JlnpaUs73KdebDe6ON7JCcgbYUG8X2LnkHrFwW5ffDYE5xHGWLk7QHbo0LTfh8u1erldBS2wW1fcpUClF5sM790P1BaFHJmKlE1wCxHbnUO9JpatTlNdG3EG8qfiVW13jT+ArQMd1Dj3+ZLHhAhW7EfZDSmarht0zDUww3A20HdR9zprrNdkdaATSqN5XXkSN4zhcCKR8r3Qmyh+tkQ2DviiRo2d/S3pXntoKmEcf/btgXyXecxFCuOI2VybxRPwN6F1QJIQOwXuJmSFmKSwI0GCUypTI3BnY8Bs5iin9rv7IrEVSbl0vLp2C9fiyri4mkTclbhAcyPfEW75JtlfGG5uvC/+0Wltau3Wg1+uZl1Kcie803hSVlKPYnXfecYns1cNaR1qnCKUbjy9rWEcQD9/sdsI42PWuC81jd526qUXRhVL8sNOvJA6lTBp9YIXIDSnwsa9Aay9ieirOPe5Jc8Ro0bBJKei6 X-OriginatorOrg: cornell.edu X-MS-Exchange-CrossTenant-Network-Message-Id: d6a70779-6e26-41fa-94cd-08d82e6df54e X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB6176.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jul 2020 18:35:07.7574 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PrDngU47/yh56GCXcBMSWhmdYhh0MLYbhFDlPKpudS82dmuhMKjrHFlhXZinaiTbv3jBILMGfjo9SB4N/w5hgQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB5760 X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00, BODY_8BITS, DKIM_INVALID, DKIM_SIGNED, KAM_DMARC_STATUS, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-developers@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin core component developers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jul 2020 18:35:14 -0000 On 7/22/2020 12:42 PM, Ken Brown via Cygwin-developers wrote: > On 7/22/2020 4:33 AM, Corinna Vinschen wrote: >> On Jul 21 18:40, Ken Brown via Cygwin-developers wrote: >>> Hi Corinna, >>> >>> I'm curious about the design decision that causes sysconf(_SC_PAGESIZE) to >>> return wincap.allocation_granularity() rather than wincap.page_size(). >>> Changing this would improve Linux compatibility, I think, but maybe it would >>> have some bad consequences that I'm not aware of. >> >> It was a long and hard process to move from 4K to 64K pagesize, with >> lots of loaded discussions.  The Cygwin mailing list archives will >> show a lot of this in the 200X years. >> >> It was the only way to make mmap 99% POSIX-conformant.  Consider, for >> instance this: >> >>    pagesize = sysconf(_SC_PAGESIZE); >>    addr = mmap (NULL, pagesize, PROT_READ | PROT_WRITE, >>            MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); >>    addr2 = mmap (addr + pagesize, pagesize, PROT_READ | PROT_WRITE, >>         MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); >> >> On Windows, this fails with pagesize = 4K, but it works with pagesize = >> 64K, because of that idiotic Windows allocation granularity.  Almost >> all POSIX expectations are automagically fixed by using the granularity >> as pagesize in a POSIX sense. >> >> There's only one problem left:  While you can only allocate usefully in >> 64K steps, the size of the memory area allocated for a file is only 4K >> aligned, thus leaving the remainder of the 64K block unmapped. >> >> This problem could be fixed back in 32 bit times by adding the >> AT_ROUND_TO_PAGE mapping.  Very unfortunately, the 64 bit Windows >> designer decided to keep the braindead 64K allocation granularity >> but to drop the AT_ROUND_TO_PAGE flag, thus removing the only chance >> to make this single situation POSIX-compatible as well. >> >>> I'm asking because in my recent fooling around with php, I noticed that >>> Yaakov had to apply the following Cygwin-specific patch to avoid a crash: >> >> It would be nice to learn what kind of crash that was. > > Here's a better reference than the one I gave in my previous reply, which > actually explains what's going on: > >   https://sourceware.org/pipermail/cygwin/2017-May/232562.html > >> If php reads or writes in the remainder of the block constituting EOF, >> or tries to change page protection, shit happens.  Every time, a process >> stabs into the EOF block following the last valid 4K block, it results >> in a STATUS_ACCESS_VIOLATION which in turn calls >> mmap_is_attached_or_noreserve().  While this situation can be >> recognized, I don't see a way to fix this from the processes POV. > > So that's exactly what happens when php maps a file whose size is a multiple of > 4K but not a multiple of 64K.  It expects that there is a zero-filled region > beyond EOF that it can safely read from. Interestingly, you mentioned exactly this scenario in 2002 as a reason for keeping the pagesize at 4K rather than 64K: https://cygwin.com/pipermail/cygwin/2002-January/068154.html I have nothing new to contribute, so we should probably just drop this. My curiosity has been satisfied. Ken