From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by sourceware.org (Postfix) with ESMTPS id 2D4323858C83 for ; Sun, 21 Aug 2022 10:44:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 2D4323858C83 Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id DB3AF5C00E6; Sun, 21 Aug 2022 06:44:53 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Sun, 21 Aug 2022 06:44:53 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeihedgfeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffkffoggfgsedtkeertd ertddtnecuhfhrohhmpefoihgthhgrvghlucfjuhgushhonhdqffhohihlvgcuoehmihgt hhgrvghlrdhhuhgushhonhestggrnhhonhhitggrlhdrtghomheqnecuggftrfgrthhtvg hrnhepgeeguedtieeugfegudevfeeiieeftdfhhfegkeelkedvhfeihfevjeejhefhgfek necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhifhh huughsohhnsehfrghsthhmrghilhdrfhhm X-ME-Proxy: Feedback-ID: i833146b3:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 21 Aug 2022 06:44:52 -0400 (EDT) From: Michael Hudson-Doyle To: libc-alpha@sourceware.org Subject: [PATCH] Avoid undefined behaviour in ibm128 implementation of llroundl Date: Sun, 21 Aug 2022 22:44:46 +1200 Message-Id: <20220821104446.46521-1-michael.hudson@canonical.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_LOW, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Aug 2022 10:44:58 -0000 Detecting an overflow edge case depended on signed overflow of a long long. Replace the signed long long with unsigned and cast it back to unsigned before comparisons (which is implementation defined behaviour, but I guess glibc does not support any one's complement architectures...). BZ #29488 --- sysdeps/ieee754/ldbl-128ibm/s_llroundl.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/sysdeps/ieee754/ldbl-128ibm/s_llroundl.c b/sysdeps/ieee754/ldbl-128ibm/s_llroundl.c index d85154e73a..e8117bfc2b 100644 --- a/sysdeps/ieee754/ldbl-128ibm/s_llroundl.c +++ b/sysdeps/ieee754/ldbl-128ibm/s_llroundl.c @@ -28,7 +28,8 @@ long long __llroundl (long double x) { double xh, xl; - long long res, hi, lo; + unsigned long long res; + long long hi, lo; ldbl_unpack (x, &xh, &xl); @@ -69,7 +70,7 @@ __llroundl (long double x) res = hi + lo; /* This is just sign(hi) == sign(lo) && sign(res) != sign(hi). */ - if (__glibc_unlikely (((~(hi ^ lo) & (res ^ hi)) < 0))) + if (__glibc_unlikely (((~(hi ^ lo) & (((long long)res) ^ hi)) < 0))) goto overflow; xh -= lo; @@ -95,7 +96,7 @@ __llroundl (long double x) res -= 1; } - if (__glibc_unlikely (((~(hi ^ (res - hi)) & (res ^ hi)) < 0))) + if (__glibc_unlikely (((~(hi ^ (((long long)res) - hi)) & (((long long)res) ^ hi)) < 0))) goto overflow; return res; -- 2.34.1