GCC 3.2.1 Core Tar Bzip2 ball MD5SUM mismatch (fwd)

GCC 3.2.1 Core Tar Bzip2 ball MD5SUM mismatch (fwd)

[Joseph S. Myers]

I don't see any documentation at <http://sources.redhat.com/sourceware/>
of how the md5.sum generation works, but I think problems such as this
have been reported before.  Is there some locking mechanism the
snapshot/release script can use to prevent generation of md5.sum while a
snapshot/release is incompletely copied to the ftp directory?

-- 
Joseph S. Myers
jsm28@cam.ac.uk

---------- Forwarded message ----------
Date: Thu, 09 Jan 2003 02:30:54 -0500
From: Gene Cumm <greenpuddy@stny.rr.com>
To: gcc@gcc.gnu.org
Subject: GCC 3.2.1 Core Tar Bzip2 ball MD5SUM mismatch

So far I've downloaded the gcc-core-3.2.1.tar.bz2 from 3 different 
locations. Every single one gives an MD5 of 
e87b2970e96c54b8e45d11bb41a3e915 and a file size of 10296249 rather than 
the MD5 listed in the md5.sum file of 26c533a010e65162f9e515bd7dc8dd0a.

I normally prefer the .bz2 files because they take less space to store 
and time to download but this makes me believe there is a problem somewhere.

I searched the mailing lists for about 30 minutes with no luck at 
finding any issues even nearly related to this. I also noticed that the 
gcc-g++, gcc-g77, gcc-java, gcc-objc, and gcc-testsuite -3.2.1.tar.bz2 
and -3.2-3.2.1.diff.bz2 files and gcc-core-3.2-3.2.1.diff.bz2 are not 
listed in the md5.sum file. If I were to guess, the md5.sum file was 
auto generated too early, ie in the middle of transfer of 
gcc-core-3.2.1.tar.bz2.

Anyone know the solution?

Re: GCC 3.2.1 Core Tar Bzip2 ball MD5SUM mismatch (fwd)

[Jason Molenda]

[-- Attachment #1: Type: text/plain, Size: 1471 bytes --]

On Thu, Jan 09, 2003 at 11:00:27AM +0000, Joseph S. Myers wrote:
> I don't see any documentation at <http://sources.redhat.com/sourceware/>
> of how the md5.sum generation works, but I think problems such as this
> have been reported before.  Is there some locking mechanism the
> snapshot/release script can use to prevent generation of md5.sum while a
> snapshot/release is incompletely copied to the ftp directory?

It's a little script that isn't under CVS or RCS or anything in 
/sourceware/ftp, run out of cron once an hour or something.
I'll attach it.

I've seen this once or twice before; the script makes the assumption
that mod-dates indicate when a file was placed in the directory.  The
gcc3 release process involves some step where mmitchell copies files
from his home system with the mod dates preserved (or something like
this), which tricks that mechanism.

The file is auto-regenerated if it doesn't exist, so I just blew away
the md5.sum in pub/gcc/releases/gcc-3.2.1.  A new one will appear soon.

The script would probably be more foolproof if it just based its decision
to regenerate the md5.sum on the mod time of the directory; that way any
file addition/removal/renaming would be caught correctly.  A little
reworking might be necessary at new-md5.sum-creation to get the mod date
order correctly (md5.sum vs its directory) and the script would have to
check these dates before it does any of its remove-unnecessary md5.sum
files or whatever.

J

[-- Attachment #2: create-md5-checksums.sh --]
[-- Type: text/plain, Size: 897 bytes --]

#! /bin/sh

FTPBASE=/sourceware/ftp/anonftp

BASEDIR=$FTPBASE/pub
cd $BASEDIR

find . -type d -print |
while read dirname
do
  cd $BASEDIR/$dirname
  files=`/bin/ls -1 | grep -v md5.sum`

  if [ -z "$files" ]
  then
    if [ -f md5.sum ]
    then
      rm -f md5.sum
    fi
    continue
  fi

  regenerate=0
  if [ ! -f md5.sum ]
  then
    regenerate=1
  else
    #  Any new files?
    for i in $files
    do
      if [ ! -d $i -a $i -nt md5.sum ]
      then
        regenerate=1
        break
      fi
    done

    #  Any files disappear?
    for i in `cat md5.sum | awk '{print $2}'`
    do
      if [ ! -f $i ]
      then
        regenerate=1
        break
      fi
    done
  fi

  if [ $regenerate = 1 ]
  then
    md5sum $files > md5.sum 2>/dev/null
  fi

  if [ -f md5.sum -a ! -s md5.sum ]
  then
    # Zero length md5.sum file -- not much point, eh?
    rm -f md5.sum
  fi
done

exit 0

Re: GCC 3.2.1 Core Tar Bzip2 ball MD5SUM mismatch (fwd)

[Christopher Faylor]

On Thu, Jan 09, 2003 at 11:00:27AM +0000, Joseph S. Myers wrote:
>I don't see any documentation at <http://sources.redhat.com/sourceware/>
>of how the md5.sum generation works,

Use the source: /sourceware/ftp/bin/create-md5-checksums.sh .

cgf

Re: GCC 3.2.1 Core Tar Bzip2 ball MD5SUM mismatch (fwd)

[Andrew Cagney]

> I don't see any documentation at <http://sources.redhat.com/sourceware/>
> of how the md5.sum generation works, but I think problems such as this
> have been reported before.  Is there some locking mechanism the
> snapshot/release script can use to prevent generation of md5.sum while a
> snapshot/release is incompletely copied to the ftp directory?

Ah, so this is why the GDB snapshot process puts everything in a 
directory within ~ftp and then uses `mv' to put it in place.

I'll update GDB's release process to indicate the same.  Regardless of 
the md5 problem, using `cp' is bad.

thanks!

Andrew