From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 3863F3858C1F for ; Wed, 22 Mar 2023 12:21:02 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3863F3858C1F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679487661; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZhNqBymavEKVIVjGkXcuzVQB9SmC27zG+q3r3Rr18TU=; b=aMPN9QdnUkkRjm29fy2BlbfbZC2ufbP/K/NYA30LJRxCvUk9RF3Pa+OTWNycL8m7M7h3IB MAudGhDKozIwLfVTQ29+FS1bDQxEym5GXmJVt9dY1V//5cZBrqJruyaOuWLEJn0XcU2cEU XnMkhBGGutR1iYCCkUfpKIjxentmevc= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-302-fQ1bTd_mMiG6v4-N8Fu81g-1; Wed, 22 Mar 2023 08:20:59 -0400 X-MC-Unique: fQ1bTd_mMiG6v4-N8Fu81g-1 Received: by mail-lj1-f200.google.com with SMTP id f4-20020a05651c160400b002959ea5bc51so4950447ljq.9 for ; Wed, 22 Mar 2023 05:20:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679487658; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dCy2+chgBMWOqpDcS1YJTTuDUSlJSxq9MP/KlmKOOgo=; b=Zy1deeKBR4U5WxfSr0SYLc1NocMGSpT3GlPTWC0o8sjy4L2oHjkXBsEmzUn91zYUWu 79CGMj2gGt2vP8+126mb8IoXT03dNUtAM/GSNyX0gzX7NXjUvKCh/uZpR6lcOaVJ9reV tNwJfNr0PGOS7yyLZVu4MAvO6DCVU6oiNQN+ZXGzZtkJ/IUy4NPRx9yPWjHRtbUTBRf3 8eyEyguSplmF/iHo5aW1uqHevXnNbohiIu8Z74LH2r0MKWp7uKKrjdX8L6rvxdL8xTRv Ucx2qDZI6Qtwt6+LHX6ptlo76EFIS3hD9kmsa3RRizqNPUDIcqHHEUkxVs3IY/RpImr5 AaqA== X-Gm-Message-State: AO0yUKV2z1dhL5p/FLoOKBOj9A9T/aie0d4PDyPM9dXgCPG/liD7vrJc Zh7oubZDq8IgwM5/XcnjlMZ2n26TNKpHVBGCIg0+gHyaf0U79as6xYx8jk65KgZXsq7gN3WtVIU hpDk0OjFrugzh7RZVZ3oLfMkrhiOhFTvkGa1lzUY+4g== X-Received: by 2002:a05:6512:90d:b0:4d5:ca32:6ed8 with SMTP id e13-20020a056512090d00b004d5ca326ed8mr629314lft.6.1679487658575; Wed, 22 Mar 2023 05:20:58 -0700 (PDT) X-Google-Smtp-Source: AK7set+BBLUryA6JePLC7fzfHR+RZZVUU0KX6slNYTPPdYIkfiTft1+bJ29LZ7lSXN7LXfL8ZhVaWUPXm2QzkgSaKzo= X-Received: by 2002:a05:6512:90d:b0:4d5:ca32:6ed8 with SMTP id e13-20020a056512090d00b004d5ca326ed8mr629310lft.6.1679487658303; Wed, 22 Mar 2023 05:20:58 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Jonathan Wakely Date: Wed, 22 Mar 2023 12:20:47 +0000 Message-ID: Subject: Re: [PATCH v2 1/2] libstdc++: also use sendfile for big files To: Jonathan Wakely Cc: =?UTF-8?Q?Jannik_Gl=C3=BCckert?= , libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/mixed; boundary="0000000000004746e405f77c31b0" X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,HTML_MESSAGE,KAM_SHORT,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --0000000000004746e405f77c31b0 Content-Type: multipart/alternative; boundary="0000000000004746e205f77c31ae" --0000000000004746e205f77c31ae Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, 22 Mar 2023 at 12:18, Jonathan Wakely wrote: > On Wed, 22 Mar 2023 at 12:14, Jonathan Wakely wrote: > >> >> >> On Mon, 20 Mar 2023 at 22:30, Jonathan Wakely via Libstdc++ < >> libstdc++@gcc.gnu.org> wrote: >> >>> On 20/03/23 22:27 +0000, Jonathan Wakely wrote: >>> >On 06/03/23 20:52 +0100, Jannik Gl=C3=BCckert wrote: >>> >>we were previously only using sendfile for files smaller than 2GB, as >>> >>sendfile needs to be called repeatedly for files bigger than that. >>> >> >>> >>some quick numbers, copying a 16GB file, average of 10 repetitions: >>> >> old: >>> >> real: 13.4s >>> >> user: 0.14s >>> >> sys : 7.43s >>> >> new: >>> >> real: 8.90s >>> >> user: 0.00s >>> >> sys : 3.68s >>> >> >>> >>Additionally, this fixes >>> >>https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D108178 >>> >> >>> >>libstdc++-v3/ChangeLog: >>> >> >>> >> * acinclude.m4 (_GLIBCXX_HAVE_LSEEK): define >>> >> * config.h.in: Regenerate. >>> >> * configure: Regenerate. >>> >> * src/filesystem/ops-common.h: enable sendfile for files >>> >> >2GB in std::filesystem::copy_file, skip zero-length files >>> >>> Also, the ChangeLog entry needs to be indented with tabs, name the >>> changed functions, and should be complete sentences, e.g. >>> >>> * acinclude.m4 (_GLIBCXX_HAVE_LSEEK): Define. >>> * config.h.in: Regenerate. >>> * configure: Regenerate. >>> * src/filesystem/ops-common.h (copy_file_sendfile): Define new >>> function for sendfile logic. Loop to support large files. Skip >>> zero-length files. >>> (do_copy_file): Use it. >>> >>> >> Here's what I plan to commit in a few weeks when GCC 14 Stage 1 opens. >> >> >> > And similarly for the copy_file_range change. > And finally, here's the fix for PR libstdc++/108178, replacing the zero-size check with checking for EOF in the source file --0000000000004746e205f77c31ae-- --0000000000004746e405f77c31b0 Content-Type: text/plain; charset="US-ASCII"; name="patch.txt" Content-Disposition: attachment; filename="patch.txt" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_lfjnkox40 Y29tbWl0IDNkOTk0ZjE5OThjOGYyZWZjMmM4ZjU3NDQ2MTVlOTI2NjFiZGU0 NmYKQXV0aG9yOiBKb25hdGhhbiBXYWtlbHkgPGp3YWtlbHlAcmVkaGF0LmNv bT4KRGF0ZTogICBUdWUgTWFyIDIxIDEyOjI5OjA4IDIwMjMKCiAgICBsaWJz dGRjKys6IE1ha2Ugc3RkOjpmaWxlc3lzdGVtOjpjb3B5X2ZpbGUgd29yayBm b3IgcHJvY2ZzIFtQUjEwODE3OF0KICAgIAogICAgVGhlIHNpemUgcmVwb3J0 ZWQgYnkgc3RhdCBpcyBhbHdheXMgemVybyBmb3Igc29tZSBzcGVjaWFsIGZp bGVzIHN1Y2ggYXMKICAgIHRob3NlIHVuZGVyIC9wcm9jLCB3aGljaCBtZWFu cyB0aGUgY3VycmVudCBjb3B5X2ZpbGUgaW1wbGVtZW50YXRpb24KICAgIHRo aW5rcyB0aGVyZSBpcyBub3RoaW5nIHRvIGNvcHkuIEluc3RlYWQgb2YgdHJ1 c3RpbmcgdGhlIHN0YXQgdmFsdWUsIHRyeQogICAgdG8gcmVhZCBhIGNoYXJh Y3RlciBmcm9tIGEgc3RyZWFtYnVmIGFuZCBjaGVjayBmb3IgRU9GLgogICAg CiAgICBsaWJzdGRjKystdjMvQ2hhbmdlTG9nOgogICAgCiAgICAgICAgICAg IFBSIGxpYnN0ZGMrKy8xMDgxNzgKICAgICAgICAgICAgKiBzcmMvZmlsZXN5 c3RlbS9vcHMtY29tbW9uLmggKGRvX2NvcHlfZmlsZSk6IENoZWNrIGZvciBl bXB0eQogICAgICAgICAgICBmaWxlcyBieSB0cnlpbmcgdG8gcmVhZCBhIGNo YXJhY3Rlci4KICAgICAgICAgICAgKiB0ZXN0c3VpdGUvMjdfaW8vZmlsZXN5 c3RlbS9vcGVyYXRpb25zL2NvcHlfZmlsZV8xMDgxNzguY2M6CiAgICAgICAg ICAgIE5ldyB0ZXN0LgoKZGlmZiAtLWdpdCBhL2xpYnN0ZGMrKy12My9zcmMv ZmlsZXN5c3RlbS9vcHMtY29tbW9uLmggYi9saWJzdGRjKystdjMvc3JjL2Zp bGVzeXN0ZW0vb3BzLWNvbW1vbi5oCmluZGV4IDkwNjQzNmI0ODRlLi5hMjhj YmViMTBiNSAxMDA2NDQKLS0tIGEvbGlic3RkYysrLXYzL3NyYy9maWxlc3lz dGVtL29wcy1jb21tb24uaAorKysgYi9saWJzdGRjKystdjMvc3JjL2ZpbGVz eXN0ZW0vb3BzLWNvbW1vbi5oCkBAIC02MTgsMTEgKzYxOCwxNiBAQCBfR0xJ QkNYWF9CRUdJTl9OQU1FU1BBQ0VfRklMRVNZU1RFTQogICAgIGlmIChzYm91 dC5pc19vcGVuKCkpCiAgICAgICBvdXQuZmQgPSAtMTsKIAotICAgIGlmIChm cm9tX3N0LT5zdF9zaXplICYmICEoc3RkOjpvc3RyZWFtKCZzYm91dCkgPDwg JnNiaW4pKQotICAgICAgewotCWVjID0gc3RkOjptYWtlX2Vycm9yX2NvZGUo c3RkOjplcnJjOjppb19lcnJvcik7Ci0JcmV0dXJuIGZhbHNlOwotICAgICAg fQorICAgIC8vIG9zdHJlYW06Om9wZXJhdG9yPDwoc3RyZWFtYnVmKikgZmFp bHMgaWYgaXQgZXh0cmFjdHMgbm8gY2hhcmFjdGVycywKKyAgICAvLyBzbyBk b24ndCB0cnkgdG8gdXNlIGl0IGZvciBlbXB0eSBmaWxlcy4gQnV0IGZyb21f c3QtPnN0X3NpemUgPT0gMCBmb3IKKyAgICAvLyBzb21lIHNwZWNpYWwgZmls ZXMgKGUuZy4gcHJvY2ZzLCBzZWUgUFIgbGlic3RkYysrLzEwODE3OCkgc28g anVzdCB0cnkKKyAgICAvLyB0byByZWFkIGEgY2hhcmFjdGVyIHRvIGRlY2lk ZSB3aGV0aGVyIHRoZXJlIGlzIGFueXRoaW5nIHRvIGNvcHkgb3Igbm90Lgor ICAgIGlmIChzYmluLnNnZXRjKCkgIT0gY2hhcl90cmFpdHM8Y2hhcj46OmVv ZigpKQorICAgICAgaWYgKCEoc3RkOjpvc3RyZWFtKCZzYm91dCkgPDwgJnNi aW4pKQorCXsKKwkgIGVjID0gc3RkOjptYWtlX2Vycm9yX2NvZGUoc3RkOjpl cnJjOjppb19lcnJvcik7CisJICByZXR1cm4gZmFsc2U7CisJfQogCiAgICAg aWYgKCFzYm91dC5jbG9zZSgpIHx8ICFzYmluLmNsb3NlKCkpCiAgICAgICB7 CmRpZmYgLS1naXQgYS9saWJzdGRjKystdjMvdGVzdHN1aXRlLzI3X2lvL2Zp bGVzeXN0ZW0vb3BlcmF0aW9ucy9jb3B5X2ZpbGVfMTA4MTc4LmNjIGIvbGli c3RkYysrLXYzL3Rlc3RzdWl0ZS8yN19pby9maWxlc3lzdGVtL29wZXJhdGlv bnMvY29weV9maWxlXzEwODE3OC5jYwpuZXcgZmlsZSBtb2RlIDEwMDY0NApp bmRleCAwMDAwMDAwMDAwMC4uMjUxMzU4MzRlMjEKLS0tIC9kZXYvbnVsbAor KysgYi9saWJzdGRjKystdjMvdGVzdHN1aXRlLzI3X2lvL2ZpbGVzeXN0ZW0v b3BlcmF0aW9ucy9jb3B5X2ZpbGVfMTA4MTc4LmNjCkBAIC0wLDAgKzEsMzMg QEAKKy8vIHsgZGctZG8gcnVuIHsgdGFyZ2V0IGMrKzE3IH0gfQorLy8geyBk Zy1yZXF1aXJlLWZpbGVzeXN0ZW0tdHMgIiIgfQorCisvLyBDKysxNyAzMC4x MC4xNS40IENvcHkgW2ZzLm9wLmNvcHlfZmlsZV0KKworI2luY2x1ZGUgPGZp bGVzeXN0ZW0+CisjaW5jbHVkZSA8ZnN0cmVhbT4KKyNpbmNsdWRlIDx1bmlz dGQuaD4gLy8gZ2V0cGlkCisjaW5jbHVkZSA8dGVzdHN1aXRlX2ZzLmg+Cisj aW5jbHVkZSA8dGVzdHN1aXRlX2hvb2tzLmg+CisKK25hbWVzcGFjZSBmcyA9 IHN0ZDo6ZmlsZXN5c3RlbTsKKwordm9pZAordGVzdF9wcm9jZnMoKSAvLyBQ UiBsaWJzdGRjKysvMTA4MTc4Cit7CisgIGF1dG8gcGlkID0gOjpnZXRwaWQo KTsKKyAgc3RkOjpzdHJpbmcgZnJvbSA9ICIvcHJvYy8iICsgc3RkOjp0b19z dHJpbmcocGlkKSArICIvc3RhdHVzIjsKKyAgaWYgKGZzOjpleGlzdHMoZnJv bSkpCisgIHsKKyAgICBhdXRvIHRvID0gX19nbnVfdGVzdDo6bm9uZXhpc3Rl bnRfcGF0aCgpOworICAgIGZzOjpjb3B5X2ZpbGUoZnJvbSwgdG8pOworICAg IHN0ZDo6aWZzdHJlYW0gZih0byk7CisgICAgVkVSSUZZKGYuaXNfb3Blbigp KTsKKyAgICBWRVJJRlkoZi5wZWVrKCkgIT0gc3RkOjpjaGFyX3RyYWl0czxj aGFyPjo6ZW9mKCkpOworICAgIGZzOjpyZW1vdmUodG8pOworICB9Cit9CisK K2ludCBtYWluKCkKK3sKKyAgdGVzdF9wcm9jZnMoKTsKK30K --0000000000004746e405f77c31b0--