public inbox for gcc@gcc.gnu.org
 help / color / mirror / Atom feed
* git conversion of GCC wwwdocs repository
@ 2019-09-15  2:30 Joseph Myers
  2019-09-24 20:46 ` Joseph Myers
  0 siblings, 1 reply; 16+ messages in thread
From: Joseph Myers @ 2019-09-15  2:30 UTC (permalink / raw)
  To: gcc

[-- Attachment #1: Type: text/plain, Size: 2595 bytes --]

I've done a git conversion of the GCC wwwdocs repository (from CVS, using 
cvs-fast-export), plus written a git post-receive hook intended to 
correspond to what the CVS hook does.  (For the CVS hook, see loginfo in 
the CVSROOT, plus the scripts such as htdocs-checkout referenced from 
there.)

To access the test conversion (given SSH access to gcc.gnu.org), do:

git clone git+ssh://gcc.gnu.org/home/gccadmin/wwwdocs-test.git

(This only needs the usual restricted SSH access used to commit to GCC, 
not shell access.)

Note that the test repository does *not* have the post-receive hook 
enabled; pushing commits to it won't do anything useful; it's simply there 
to make it convenient to review the converted history.  In practice, I 
expect the post-receive hook may need bugs ironed out once in use in 
production, because it's hard to test when not hooked up to the real 
website.

I've attached the conversion scripts and proposed post-receive hook.  
do-download checks out cvs-fast-export and ESR's gcc-conversion repository 
(for the author map) and rsyncs the wwwdocs CVS repository.  do-convert 
builds cvs-fast-export and does the conversion (with a few extra authors 
in the author map, where e.g. new since the last update to the author map 
or only ever committed to wwwdocs not GCC sources; the names for htdigid 
and www are somewhat arbitrary, www commits would actually have been some 
human or humans committing as that user by accident); once cvs-fast-export 
has been built, this takes about 15 seconds.  All non-master tags and 
branches are removed, since they are clearly just accidents from tags / 
branches having been applied to the whole CVS repository rather than just 
to the GCC sources proper.

post-receive is the proposed hook.  It calls post-receive-email to send 
email (could no doubt be adjusted to use AdaCore hooks or any other 
email-sending system), then follows logic intended to replicate the old 
hooks to update a checkout of wwwdocs and call the preprocess script on 
new or modified files.

I'd expect the post-receive script to go in wwwdocs/bin (with the hook 
directory having a symlink to there).  There would of course be website 
changes needed to update the references to CVS, and the preprocess script 
would change to ignore .git when doing global preprocessing on the site.

I don't know whether either the system for updating www.gnu.org's copy of 
the site, or Gerald's HTML validation system, would need updating for a 
move of wwwdocs to git.

Any comments on the conversion or hook?

-- 
Joseph S. Myers
joseph@codesourcery.com

[-- Attachment #2: Type: text/plain, Size: 270 bytes --]

#!/bin/bash

set -e

TOP=$(cd $(dirname "$0") && pwd)

DOWNLOAD=$TOP/download

mkdir "$DOWNLOAD"
cd "$DOWNLOAD"
git clone https://gitlab.com/esr/cvs-fast-export.git
git clone https://gitlab.com/esr/gcc-conversion.git
rsync -a gcc.gnu.org::gcc-cvs/wwwdocs .

[-- Attachment #3: Type: text/plain, Size: 1465 bytes --]

#!/bin/bash

set -e

TOP=$(cd $(dirname "$0") && pwd)

REPO=$TOP/wwwdocs.git
mkdir $REPO
cd $REPO
git init --bare --shared

DOWNLOAD=$TOP/download

cd "$DOWNLOAD/cvs-fast-export"
make -j cvs-fast-export

grep -a = $DOWNLOAD/gcc-conversion/gcc.map | sed -e 's/>.*/>/' > $TOP/gcc.map
# dje specially handled for GCC conversion because of ambiguity with
# gcc2 repository.  dimitar, shorne postdate GCC conversion map; not
# needed once that is updated.  Others not needed in GCC conversion
# map, as only committed to wwwdocs not to main sources repository.
cat >> $TOP/gcc.map <<EOF
dje = David Edelsohn <dje.gcc@gmail.com>
dimitar = Dimitar Dimitrov <dimitar@dinux.eu>
shorne = Stafford Horne <shorne@gmail.com>
angela = Angela Marie Thomas <angela@releasedominatrix.com>
brads = Bradley Schatz <brads@gcc.gnu.org>
gumby = David Henkel-Wallace <gumby@gcc.gnu.org>
kingdon = Jim Kingdon <kingdon@gcc.gnu.org>
htdigid = htdigid automatic commit <htdigid@gcc.gnu.org>
www = www user <www@gcc.gnu.org>
EOF

cd "$DOWNLOAD/wwwdocs"
find . -name '*,v' | sort | ../cvs-fast-export/cvs-fast-export -A $TOP/gcc.map | (cd "$REPO" && git fast-import)

cd "$REPO"
# CVS branches and tags are accidents from whole-CVS-repository
# branching / tagging, remove.
for ref in $(git for-each-ref --format='%(refname)'); do
    if [ "$ref" != "refs/heads/master" ]; then
	git update-ref -d "$ref"
    fi
done
git gc --aggressive --prune=all

[-- Attachment #4: Type: text/plain, Size: 1398 bytes --]

#!/bin/bash

# The post-receive hook receives, on standard input, a series of lines
# of the form:
#
# <old-value> SP <new-value> SP <ref-name> LF
#
# describing ref updates that have just occurred.  In the case of this
# repository, only updates to refs/heads/master are expected, and only
# such updates should result in automatic website updates.

TOP_DIR=/www/gcc

exec >> "$TOP_DIR/updatelog" 2>&1

export QMAILHOST=gcc.gnu.org

tmp=$(mktemp)
cat > "$tmp"

# Send commit emails.  Appropriate config values should be set in the
# git repository for this.
/sourceware/libre/infra/bin/post-receive-email < "$tmp"

# Update web page checkouts, if applicable.
while read old_value new_value ref_name; do
    if [ "$ref_name" != "refs/heads/master" ]; then
	continue
    fi
    unset GIT_DIR
    unset GIT_WORK_TREE
    cd "$TOP_DIR/wwwdocs-checkout"
    git pull --quiet
    # $TOP_DIR/bin, $TOP_DIR/cgi-bin and $TOP_DIR/htdocs-preformatted
    # should be symlinks into wwwdocs-checkout.
    git diff --name-only "$old_value..$new_value" | while read file; do
	case "$file" in
	    (htdocs/*)
		;;
	    (*)
		continue
		;;
	esac
	dir="${file%/*}"
	if ! [ -d "$TOP_DIR/$dir" ]; then
	    mkdir "$TOP_DIR/$dir"
	    chmod 2775 "$TOP_DIR/$dir"
	fi
	if [ -f "$file" ]; then
	    /www/gcc/bin/preprocess "${file#htdocs/}"
	fi
    done
done < "$tmp"

^ permalink raw reply	[flat|nested] 16+ messages in thread
end of thread, other threads:[~2019-10-20 21:14 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-15  2:30 git conversion of GCC wwwdocs repository Joseph Myers
2019-09-24 20:46 ` Joseph Myers
2019-09-24 22:09   ` Jonathan Wakely
2019-09-24 22:15     ` Joseph Myers
2019-09-24 22:31       ` Jonathan Wakely
2019-09-25  2:51       ` Jason Merrill
2019-09-25 10:08         ` Thomas Schwinge
2019-09-25 11:03           ` Jonathan Wakely
2019-09-25 16:50             ` Joseph Myers
2019-09-25 17:22               ` Segher Boessenkool
2019-09-25 17:30                 ` Joseph Myers
2019-09-25 17:17             ` Segher Boessenkool
2019-10-20 19:17             ` Gerald Pfeifer
2019-10-20 21:14               ` Andreas Schwab
2019-09-30 19:56   ` Joseph Myers
2019-10-04  9:23     ` Thomas Schwinge

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).