From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 32659 invoked by alias); 2 Aug 2002 12:05:18 -0000 Mailing-List: contact libc-hacker-help@sources.redhat.com; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-hacker-owner@sources.redhat.com Received: (qmail 32640 invoked from network); 2 Aug 2002 12:05:16 -0000 Received: from unknown (HELO sunsite.mff.cuni.cz) (195.113.19.66) by sources.redhat.com with SMTP; 2 Aug 2002 12:05:16 -0000 Received: (from jakub@localhost) by sunsite.mff.cuni.cz (8.11.6/8.11.6) id g72C4CI17770; Fri, 2 Aug 2002 14:04:12 +0200 Date: Fri, 02 Aug 2002 05:05:00 -0000 From: Jakub Jelinek To: Wolfram Gloger Cc: schwab@suse.de, drepper@redhat.com, libc-hacker@sources.redhat.com Subject: Re: [PATCH] xdr_array and calloc security fix Message-ID: <20020802140412.E20867@sunsite.ms.mff.cuni.cz> Reply-To: Jakub Jelinek References: <20020802004635.Y20867@sunsite.ms.mff.cuni.cz> <20020802092945.24679.qmail@md.dent.med.uni-muenchen.de> <3D4A5446.5030204@redhat.com> <3D4A55F0.5020007@redhat.com> <20020802115506.C20867@sunsite.ms.mff.cuni.cz> <20020802134512.D20867@sunsite.ms.mff.cuni.cz> <20020802115729.25576.qmail@md.dent.med.uni-muenchen.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20020802115729.25576.qmail@md.dent.med.uni-muenchen.de>; from wmglo@dent.med.uni-muenchen.de on Fri, Aug 02, 2002 at 11:57:29AM -0000 X-SW-Source: 2002-08/txt/msg00020.txt.bz2 On Fri, Aug 02, 2002 at 11:57:29AM -0000, Wolfram Gloger wrote: > > > But (a > a * b || b > a * b) should work, shouldn't it? > > > > No. For a=1 b=2 this will give the correct answer (no overflow), but > > for a=0x6000000 b=64 it will give incorrect one (no overflow, while > > 0x180000000LL certainly doesn't fit into 32-bits (but 0x80000000 is > > still bigger than any of the operands). > > Ok, if we're going to have two comparisions anyway, I'd suggest we > assume at least 32bits and use > > a >= 46340 || b >= 46340 > > (46340 <= sqrt(2^31), if I did my math correctly) > Of course this will detect some cases as overflow which actually > aren't, but that is harmless. Why not 2^32? size_t is unsigned. So you mean something like: bytes = n * elem_size; if (__builtin_expect ((a | b) >= 65536, 0)) { if (bytes / elem_size != n) { MALLOC_FAILURE_ACTION; return 0; } } (ie. do the division only in the unlikely case)? Jakub