From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 32D89385801E for ; Thu, 17 Mar 2022 19:28:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 32D89385801E Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-607--lzxIYnjPn6XX-k5eLwJ6g-1; Thu, 17 Mar 2022 15:28:05 -0400 X-MC-Unique: -lzxIYnjPn6XX-k5eLwJ6g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 63F0480159B for ; Thu, 17 Mar 2022 19:28:05 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9118C1400E73 for ; Thu, 17 Mar 2022 19:28:04 +0000 (UTC) From: Florian Weimer To: libc-alpha@sourceware.org Subject: [PATCH 00/26] vfprintf rework to remove vtables X-From-Line: 9bed5c50fe3e6c5872c94c451413ca50a2051ef5 Mon Sep 17 00:00:00 2001 Message-Id: Date: Thu, 17 Mar 2022 20:28:02 +0100 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Mar 2022 19:28:09 -0000 This started out as an attempt to fix bug 27124, the vtable hardening bypass involving obstack_printf. The new buffers are modeled after the write pointer/end pointer abstraction used for inline putc_unlocked. They have considerably lower setup overhead than the internal stdio-compatible streams used by sprintf. Performance seems to be at least on par with the old implementation, and a bit better for floating point formatting. The new internal interfaces are added in commit =E2=80=9Cstdio-common: Intr= oduce buffers for implementing printf=E2=80=9D. The bulk of the conversions happ= en in =E2=80=9Cstdio-common: Convert vfprintf and related functions to buffers=E2= =80=9D. Initially, I converted things piece by piece, but that required implementing additional bounce buffers for translation, so this one commit converts the majority of the functions. The commit message of this large commit has some suggestions for review order. Tested on i686-linux-gnu and x86_64-linux-gnu. Built with build-many-glibcs.py. Thanks, Florian Florian Weimer (26): libio: Convert tst_swprintf to the test framework libio: Flush-only _IO_str_overflow must not return EOF (bug 28949) stdio-common: Add wide stream coverage to tst-vfprintf-user-type stdio-common: Add tst-printf-width-i18n to cover numeric field width vfprintf: Move argument processing into vfprintf-process-arg.c vfprintf: Consolidate some multibyte/wide character processing __printf_fphex always uses LC_NUMERIC stdio-common: Add tst-memstream-string for open_memstream overflow stdio-common: Add printf specifier registry to stdio-common: Move union printf_arg int stdio-common: Simplify printf_unknown interface in vfprintf-internal.c locale: Call _nl_unload_locale from _nl_archive_subfreeres locale: Remove cleanup function pointer from struct __localedata locale: Remove private union from struct __locale_data locale: Add more cached data to LC_CTYPE locale: Implement struct grouping_iterator stdio-common: Introduce buffers for implementing printf stdio-common: Add __printf_function_invoke stdio-common: Add __translated_number_width stdio-common: Convert vfprintf and related functions to buffers stdio-common: Add lock optimization to vfprintf and vfwprintf libio: Convert __vsprintf_internal to buffers libio: Convert __vasprintf_internal to buffers libio: Convert __vdprintf_internal to buffers libio: Convert __obstack_vprintf_internal to buffers (bug 27124) libio: Convert __vswprintf_internal to buffers (bug 27857) include/printf.h | 79 +- include/printf_buffer.h | 332 +++++ libio/iovdprintf.c | 69 +- libio/iovsprintf.c | 70 +- libio/obprintf.c | 170 +-- libio/strops.c | 5 +- libio/tst-vtables-common.c | 9 +- libio/tst_swprintf.c | 110 +- libio/vasprintf.c | 141 +- libio/vsnprintf.c | 131 +- libio/vswprintf.c | 100 +- libio/wstrops.c | 5 +- locale/C-address.c | 2 +- locale/C-collate.c | 2 +- locale/C-ctype.c | 9 +- locale/C-identification.c | 2 +- locale/C-measurement.c | 2 +- locale/C-messages.c | 2 +- locale/C-monetary.c | 2 +- locale/C-name.c | 2 +- locale/C-numeric.c | 2 +- locale/C-paper.c | 2 +- locale/C-telephone.c | 2 +- locale/C-time.c | 2 +- locale/findlocale.c | 2 +- locale/loadarchive.c | 8 +- locale/loadlocale.c | 89 +- locale/localeinfo.h | 48 +- locale/setlocale.c | 2 +- manual/stdio.texi | 7 +- stdio-common/Makefile | 29 +- stdio-common/Xprintf_buffer_done.c | 40 + stdio-common/Xprintf_buffer_flush.c | 72 + stdio-common/Xprintf_buffer_pad_1.c | 44 + stdio-common/Xprintf_buffer_putc_1.c | 29 + stdio-common/Xprintf_buffer_puts_1.c | 37 + stdio-common/Xprintf_buffer_write.c | 43 + stdio-common/Xprintf_function_invoke.c | 42 + stdio-common/grouping_iterator.c | 125 ++ stdio-common/grouping_iterator.h | 65 + stdio-common/printf-parse.h | 23 - stdio-common/printf_buffer-char.h | 24 + stdio-common/printf_buffer-wchar_t.h | 24 + stdio-common/printf_buffer_as_file.c | 148 ++ stdio-common/printf_buffer_as_file.h | 87 ++ stdio-common/printf_buffer_done.c | 21 + stdio-common/printf_buffer_flush.c | 81 ++ stdio-common/printf_buffer_pad_1.c | 21 + stdio-common/printf_buffer_putc_1.c | 21 + stdio-common/printf_buffer_puts_1.c | 21 + stdio-common/printf_buffer_to_file.c | 122 ++ stdio-common/printf_buffer_to_file.h | 57 + stdio-common/printf_buffer_write.c | 21 + stdio-common/printf_fp.c | 736 +++++----- stdio-common/printf_fphex.c | 271 ++-- stdio-common/printf_function_invoke.c | 22 + stdio-common/reg-printf.c | 7 - stdio-common/translated_number_width.c | 42 + stdio-common/tst-grouping_iterator.c | 262 ++++ stdio-common/tst-memstream-string.c | 85 ++ stdio-common/tst-printf-width-i18n.c | 95 ++ stdio-common/tst-vfprintf-user-type.c | 128 +- stdio-common/vfprintf-internal.c | 1189 +++-------------- stdio-common/vfprintf-process-arg.c | 489 +++++++ stdio-common/wprintf_buffer_as_file.c | 153 +++ stdio-common/wprintf_buffer_done.c | 21 + stdio-common/wprintf_buffer_flush.c | 42 + stdio-common/wprintf_buffer_pad_1.c | 21 + stdio-common/wprintf_buffer_putc_1.c | 21 + stdio-common/wprintf_buffer_puts_1.c | 21 + stdio-common/wprintf_buffer_to_file.c | 55 + stdio-common/wprintf_buffer_write.c | 21 + stdio-common/wprintf_function_invoke.c | 22 + stdlib/strfmon_l.c | 196 ++- stdlib/strfrom-skeleton.c | 38 +- sysdeps/ia64/fpu/printf_fphex.c | 8 +- .../ieee754/ldbl-128/printf_fphex_macros.h | 36 +- sysdeps/ieee754/ldbl-128ibm/printf_fphex.c | 36 +- sysdeps/ieee754/ldbl-96/printf_fphex.c | 22 +- sysdeps/x86_64/fpu/printf_fphex.c | 21 +- time/alt_digit.c | 49 +- time/era.c | 49 +- time/lc-time-cleanup.c | 5 +- wcsmbs/wcsmbsload.c | 27 +- wcsmbs/wcsmbsload.h | 11 +- 85 files changed, 4275 insertions(+), 2531 deletions(-) create mode 100644 include/printf_buffer.h create mode 100644 stdio-common/Xprintf_buffer_done.c create mode 100644 stdio-common/Xprintf_buffer_flush.c create mode 100644 stdio-common/Xprintf_buffer_pad_1.c create mode 100644 stdio-common/Xprintf_buffer_putc_1.c create mode 100644 stdio-common/Xprintf_buffer_puts_1.c create mode 100644 stdio-common/Xprintf_buffer_write.c create mode 100644 stdio-common/Xprintf_function_invoke.c create mode 100644 stdio-common/grouping_iterator.c create mode 100644 stdio-common/grouping_iterator.h create mode 100644 stdio-common/printf_buffer-char.h create mode 100644 stdio-common/printf_buffer-wchar_t.h create mode 100644 stdio-common/printf_buffer_as_file.c create mode 100644 stdio-common/printf_buffer_as_file.h create mode 100644 stdio-common/printf_buffer_done.c create mode 100644 stdio-common/printf_buffer_flush.c create mode 100644 stdio-common/printf_buffer_pad_1.c create mode 100644 stdio-common/printf_buffer_putc_1.c create mode 100644 stdio-common/printf_buffer_puts_1.c create mode 100644 stdio-common/printf_buffer_to_file.c create mode 100644 stdio-common/printf_buffer_to_file.h create mode 100644 stdio-common/printf_buffer_write.c create mode 100644 stdio-common/printf_function_invoke.c create mode 100644 stdio-common/translated_number_width.c create mode 100644 stdio-common/tst-grouping_iterator.c create mode 100644 stdio-common/tst-memstream-string.c create mode 100644 stdio-common/tst-printf-width-i18n.c create mode 100644 stdio-common/vfprintf-process-arg.c create mode 100644 stdio-common/wprintf_buffer_as_file.c create mode 100644 stdio-common/wprintf_buffer_done.c create mode 100644 stdio-common/wprintf_buffer_flush.c create mode 100644 stdio-common/wprintf_buffer_pad_1.c create mode 100644 stdio-common/wprintf_buffer_putc_1.c create mode 100644 stdio-common/wprintf_buffer_puts_1.c create mode 100644 stdio-common/wprintf_buffer_to_file.c create mode 100644 stdio-common/wprintf_buffer_write.c create mode 100644 stdio-common/wprintf_function_invoke.c base-commit: e297d33c73aa22c86228170343c7b2f9bfac6f1e --=20 2.35.1