From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <libc-hacker-return-5363-listarch-libc-hacker=sourceware.cygnus.com@sources.redhat.com>
Received: (qmail 21245 invoked by alias); 2 Aug 2002 11:46:08 -0000
Mailing-List: contact libc-hacker-help@sources.redhat.com; run by ezmlm
Precedence: bulk
List-Subscribe: <mailto:libc-hacker-subscribe@sources.redhat.com>
List-Archive: <http://sources.redhat.com/ml/libc-hacker/>
List-Post: <mailto:libc-hacker@sources.redhat.com>
List-Help: <mailto:libc-hacker-help@sources.redhat.com>, <http://sources.redhat.com/ml/#faqs>
Sender: libc-hacker-owner@sources.redhat.com
Received: (qmail 21227 invoked from network); 2 Aug 2002 11:46:05 -0000
Received: from unknown (HELO sunsite.mff.cuni.cz) (195.113.19.66)
  by sources.redhat.com with SMTP; 2 Aug 2002 11:46:05 -0000
Received: (from jakub@localhost)
	by sunsite.mff.cuni.cz (8.11.6/8.11.6) id g72BjCf17128;
	Fri, 2 Aug 2002 13:45:12 +0200
Date: Fri, 02 Aug 2002 04:46:00 -0000
From: Jakub Jelinek <jakub@redhat.com>
To: Andreas Schwab <schwab@suse.de>
Cc: Ulrich Drepper <drepper@redhat.com>,
   Wolfram Gloger <wmglo@dent.med.uni-muenchen.de>,
   libc-hacker@sources.redhat.com
Subject: Re: [PATCH] xdr_array and calloc security fix
Message-ID: <20020802134512.D20867@sunsite.ms.mff.cuni.cz>
Reply-To: Jakub Jelinek <jakub@redhat.com>
References: <20020802004635.Y20867@sunsite.ms.mff.cuni.cz> <20020802092945.24679.qmail@md.dent.med.uni-muenchen.de> <3D4A5446.5030204@redhat.com> <3D4A55F0.5020007@redhat.com> <20020802115506.C20867@sunsite.ms.mff.cuni.cz> <jeit2twmba.fsf@sykes.suse.de>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <jeit2twmba.fsf@sykes.suse.de>; from schwab@suse.de on Fri, Aug 02, 2002 at 01:07:53PM +0200
X-SW-Source: 2002-08/txt/msg00018.txt.bz2

On Fri, Aug 02, 2002 at 01:07:53PM +0200, Andreas Schwab wrote:
> Jakub Jelinek <jakub@redhat.com> writes:
> 
> |> On Fri, Aug 02, 2002 at 02:50:40AM -0700, Ulrich Drepper wrote:
> |> > Ulrich Drepper wrote:
> |> > 
> |> > > It should be possible to have something like
> |> > > 
> |> > >   ((a | b) > (a * b))
> |> > > 
> |> > > for unsigned values.  I'm not 100% sure, though.
> |> > 
> |> > I mean, this is an approximation which lets us avoid the division in 
> |> > many (most?) cases.
> |> 
> |> Many. a=1 b=2 -> is this overflow?
> |> a=0x6000000 b=64 -> this would signal no overflow, while in fact
> 
> But (a > a * b || b > a * b) should work, shouldn't it?

No. For a=1 b=2 this will give the correct answer (no overflow), but
for a=0x6000000 b=64 it will give incorrect one (no overflow, while
0x180000000LL certainly doesn't fit into 32-bits (but 0x80000000 is
still bigger than any of the operands).

	Jakub