From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 27783 invoked by alias); 2 Aug 2002 11:57:31 -0000 Mailing-List: contact libc-hacker-help@sources.redhat.com; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-hacker-owner@sources.redhat.com Received: (qmail 27763 invoked from network); 2 Aug 2002 11:57:30 -0000 Received: from unknown (HELO md.dent.med.uni-muenchen.de) (138.245.179.2) by sources.redhat.com with SMTP; 2 Aug 2002 11:57:30 -0000 Received: (qmail 25577 invoked by uid 211); 2 Aug 2002 11:57:29 -0000 Date: Fri, 02 Aug 2002 04:57:00 -0000 Message-ID: <20020802115729.25576.qmail@md.dent.med.uni-muenchen.de> From: Wolfram Gloger To: jakub@redhat.com CC: schwab@suse.de, drepper@redhat.com, libc-hacker@sources.redhat.com In-reply-to: <20020802134512.D20867@sunsite.ms.mff.cuni.cz> (message from Jakub Jelinek on Fri, 2 Aug 2002 13:45:12 +0200) Subject: Re: [PATCH] xdr_array and calloc security fix References: <20020802004635.Y20867@sunsite.ms.mff.cuni.cz> <20020802092945.24679.qmail@md.dent.med.uni-muenchen.de> <3D4A5446.5030204@redhat.com> <3D4A55F0.5020007@redhat.com> <20020802115506.C20867@sunsite.ms.mff.cuni.cz> <20020802134512.D20867@sunsite.ms.mff.cuni.cz> X-SW-Source: 2002-08/txt/msg00019.txt.bz2 > > But (a > a * b || b > a * b) should work, shouldn't it? > > No. For a=1 b=2 this will give the correct answer (no overflow), but > for a=0x6000000 b=64 it will give incorrect one (no overflow, while > 0x180000000LL certainly doesn't fit into 32-bits (but 0x80000000 is > still bigger than any of the operands). Ok, if we're going to have two comparisions anyway, I'd suggest we assume at least 32bits and use a >= 46340 || b >= 46340 (46340 <= sqrt(2^31), if I did my math correctly) Of course this will detect some cases as overflow which actually aren't, but that is harmless. Regards, Wolfram.