GCC wwwdocs move to git done

GCC wwwdocs move to git done

[Joseph Myers]

I've done the move of GCC wwwdocs to git (using the previously posted and 
discussed scripts), including setting up the post-receive hook to do the 
same things previously covered by the old CVS hooks, and minimal updates 
to the web pages dealing with the CVS setup for wwwdocs.

Note 1: someone with the right access needs to create the symlink 
/sourceware/git/gcc-wwwdocs.git -> 
/sourceware/projects/gcc-home/wwwdocs.git (and anything else needed for 
anonymous git access to that repository).  Once that's done I'll simplify 
the path given for git checkouts in about.html and restore instructions 
for anonymous checkout.

Note 2: changes may be needed to the process for updating www.gnu.org and 
Gerald's validator.

Note 3: I don't see any reason the automated process (see 
maintainer-scripts/update_web_docs_svn in the main GCC sources repository) 
for generating install/ documentation should be broken by this, but that 
will still need watching carefully to see if it works as expected.

diff --git a/bin/post-receive b/bin/post-receive
new file mode 100755
index 00000000..1114efcb
--- /dev/null
+++ b/bin/post-receive
@@ -0,0 +1,55 @@
+#!/bin/bash
+
+# The post-receive hook receives, on standard input, a series of lines
+# of the form:
+#
+# <old-value> SP <new-value> SP <ref-name> LF
+#
+# describing ref updates that have just occurred.  In the case of this
+# repository, only updates to refs/heads/master are expected, and only
+# such updates should result in automatic website updates.
+
+TOP_DIR=/www/gcc
+
+exec >> "$TOP_DIR/updatelog" 2>&1
+
+export QMAILHOST=gcc.gnu.org
+
+tmp=$(mktemp)
+cat > "$tmp"
+
+# Send commit emails.  Appropriate config values should be set in the
+# git repository for this.
+/sourceware/libre/infra/bin/post-receive-email < "$tmp"
+
+# Update web page checkouts, if applicable.
+while read old_value new_value ref_name; do
+    if [ "$ref_name" != "refs/heads/master" ]; then
+	continue
+    fi
+    unset GIT_DIR
+    unset GIT_WORK_TREE
+    cd "$TOP_DIR/wwwdocs-checkout"
+    git pull --quiet
+    # $TOP_DIR/bin, $TOP_DIR/cgi-bin and $TOP_DIR/htdocs-preformatted
+    # should be symlinks into wwwdocs-checkout.
+    git diff --name-only "$old_value..$new_value" | while read file; do
+	case "$file" in
+	    (htdocs/*)
+		;;
+	    (*)
+		continue
+		;;
+	esac
+	dir="${file%/*}"
+	if ! [ -d "$TOP_DIR/$dir" ]; then
+	    mkdir "$TOP_DIR/$dir"
+	    chmod 2775 "$TOP_DIR/$dir"
+	fi
+	if [ -f "$file" ]; then
+	    /www/gcc/bin/preprocess "${file#htdocs/}"
+	fi
+    done
+done < "$tmp"
+
+rm "$tmp"
diff --git a/bin/preprocess b/bin/preprocess
index 2d09d548..56f83838 100755
--- a/bin/preprocess
+++ b/bin/preprocess
@@ -160,7 +160,7 @@ process_file()
     fi
 
     case $f in
-        */CVS|*\.cvsignore)
+        */.git|*\.gitignore)
             ;;
         *\.ihtml|*\.mhtml)
             ;;
@@ -248,8 +248,8 @@ if [ $# -gt 0 ]; then
     done
 else 
     # Process all files in the source tree, excluding files/directories
-    # called CVS.
-    for f in `find . \( -name CVS -prune \) -o -type f -print` ; do
+    # called .git.
+    for f in `find . \( -name .git -prune \) -o -type f -print` ; do
         process_file $f
     done
 fi
diff --git a/htdocs/about.html b/htdocs/about.html
index 19dd080c..30a5c943 100644
--- a/htdocs/about.html
+++ b/htdocs/about.html
@@ -19,7 +19,7 @@ many
 <a href="https://gcc.gnu.org/onlinedocs/gcc/Contributors.html">contributors
 </a>.</p>
 
-<p>The web pages are under <a href="#cvs">CVS control</a>.
+<p>The web pages are under <a href="#git">git control</a>.
 The pages on gcc.gnu.org are updated directly after a
 change has been committed. www.gnu.org is updated once a day at 4:00 -0700
 (PDT).</p>
@@ -48,20 +48,16 @@ a higher chance of being implemented soon. ;-)</p>
 
 
 <hr />
-<h2 id="cvs">Using the CVS repository</h2>
+<h2 id="git">Using the git repository</h2>
 
-<p>Assuming you have both CVS 
+<p>Assuming you have both git 
 and SSH installed, you can check out the web pages as follows:</p>
 
 <ol>
- <li>Set CVS_RSH in your environment to <code>ssh</code>.</li>
- <li><code>cvs -q -d :ext:<i>username</i>@gcc.gnu.org:/cvs/gcc checkout
--P wwwdocs</code> where <i>username</i> is your user name at gcc.gnu.org</li>
+ <li><code>git clone git+ssh://<i>username</i>@gcc.gnu.org/sourceware/projects/gcc-home/wwwdocs.git</code>
+ where <i>username</i> is your user name at gcc.gnu.org</li>
 </ol>
 
-<p>For anonymous access, use
-<code>-d :pserver:cvs@gcc.gnu.org:/cvs/gcc</code> instead.</p>
-
 
 <h3>Validating a change</h3>
 
@@ -76,14 +72,14 @@ the <a href="https://validator.w3.org">W3 Validator</a>. Just use the
 and prefer that each checkin be of a complete, single logical change.</p>
 
 <ol>
-<li>Sync your sources with the master repository via "<code>cvs
-update</code>".
+<li>Sync your sources with the master repository via "<code>git pull</code>".
 This will also identify any files in your local
 tree that you have modified.</li>
 
-<li>We recommend reviewing the output of "<code>cvs diff</code>".</li>
+<li>We recommend reviewing the output of "<code>git diff</code>".</li>
 
-<li>Use "<code>cvs commit</code>" to check in the patch.</li>
+<li>Use "<code>git commit</code>" and "<code>git push origin
+master</code>" to check in the patch.</li>
 
 <li>Upon checkin a message will be sent to the gcc-cvs-wwwdocs mailing
 list.</li>
diff --git a/htdocs/contribute.html b/htdocs/contribute.html
index 91fa65ad..499dfe86 100644
--- a/htdocs/contribute.html
+++ b/htdocs/contribute.html
@@ -152,7 +152,7 @@ validator.</p>
 
 <p>Please mark patches with the tag [wwwdocs] in the subject line.</p>
 
-<p>More <a href="about.html#cvs">about our web pages</a>.</p>
+<p>More <a href="about.html#git">about our web pages</a>.</p>
 
 
 <h2 id="patches">Submitting Patches</h2>
diff --git a/htdocs/svn.html b/htdocs/svn.html
index 5e93115e..2b90966b 100644
--- a/htdocs/svn.html
+++ b/htdocs/svn.html
@@ -22,7 +22,7 @@ GCC that is in our repository.</p>
 <p>In addition you can <a href="https://gcc.gnu.org/viewcvs/gcc/">browse
 our SVN history online</a>.</p>
 
-<p>(Our <a href="about.html#cvs">web pages are managed via CVS</a>.)</p>
+<p>(Our <a href="about.html#git">web pages are managed via git</a>.)</p>
 
 
 <h2>Using the SVN repository</h2>
diff --git a/htdocs/svnwrite.html b/htdocs/svnwrite.html
index 057a66a9..1a22133a 100644
--- a/htdocs/svnwrite.html
+++ b/htdocs/svnwrite.html
@@ -12,7 +12,7 @@
 <p>We have read/write access to the SVN repository available for
 maintainers and significant developers.</p>
 
-<p>Our <a href="about.html#cvs">web pages are managed via CVS</a>.</p>
+<p>Our <a href="about.html#git">web pages are managed via git</a>.</p>
 
 <hr />
 <h2>Contents</h2>

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: GCC wwwdocs move to git done

[Frank Ch. Eigler]

Hi -

Thanks - good job with moving this to git!

> Note 1: someone with the right access needs to create the symlink 
> /sourceware/git/gcc-wwwdocs.git -> 
> /sourceware/projects/gcc-home/wwwdocs.git (and anything else needed for 
> anonymous git access to that repository).

Done.

- FChE

Re: GCC wwwdocs move to git done

[Thomas Schwinge]

[-- Attachment #1: Type: text/plain, Size: 1241 bytes --]

Hi Joseph!

On 2019-10-09T00:27:38+0000, Joseph Myers <joseph@codesourcery.com> wrote:
> I've done the move of GCC wwwdocs to git

Thanks!  And now, ;-) as promised:

    # apt-get --purge remove cvs
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    The following packages will be REMOVED:
      cvs*
    0 upgraded, 0 newly installed, 1 to remove and 10 not upgraded.
    After this operation, 4706 kB disk space will be freed.
    Do you want to continue? [Y/n] 
    (Reading database ... 347879 files and directories currently installed.)
    Removing cvs (2:1.12.13+real-27) ...
    Processing triggers for install-info (6.5.0.dfsg.1-4+b1) ...
    Processing triggers for man-db (2.8.5-2) ...
    (Reading database ... 347817 files and directories currently installed.)
    Purging configuration files for cvs (2:1.12.13+real-27) ...
    Removing group `_cvsadmin' ...
    Done.


> --- a/htdocs/about.html
> +++ b/htdocs/about.html

> + <li><code>git clone git+ssh://<i>username</i>@gcc.gnu.org/sourceware/projects/gcc-home/wwwdocs.git</code>
> + where <i>username</i> is your user name at gcc.gnu.org</li>

Confirmed.


Grüße
 Thomas

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

Re: GCC wwwdocs move to git done

[Jonathan Wakely]

On Wed, 9 Oct 2019 at 01:28, Joseph Myers wrote:
>
> I've done the move of GCC wwwdocs to git (using the previously posted and
> discussed scripts), including setting up the post-receive hook to do the
> same things previously covered by the old CVS hooks, and minimal updates
> to the web pages dealing with the CVS setup for wwwdocs.

Thanks, Joseph.

I just pushed a change and although it worked and appeared on
gcc.gnu.org, I got an error message during the push:

remote: hooks/post-receive: line 14: /www/gcc/updatelog: Permission denied
remote:   Updating /www/gcc/htdocs/gcc-10/changes.html

Re: GCC wwwdocs move to git done

[Iain Sandoe]

Jonathan Wakely <jwakely.gcc@gmail.com> wrote:

> On Wed, 9 Oct 2019 at 01:28, Joseph Myers wrote:
>> I've done the move of GCC wwwdocs to git (using the previously posted and
>> discussed scripts), including setting up the post-receive hook to do the
>> same things previously covered by the old CVS hooks, and minimal updates
>> to the web pages dealing with the CVS setup for wwwdocs.
>
> Thanks, Joseph.

+1

I would like to be able to preview changes to the website by using it from  
a local
webserver.  I realise that individual pages can be viewed in a browser /  
validated
by uploading - but it would be nice to check connectivity etc.

At the moment, I can’t identify the “mhc” program that is used in preparing  
the text
(and too many unrelated hits from searches).

Is there any other “gotcha” known to prevent doing this?

thanks
iain

Re: GCC wwwdocs move to git done

[Mark Wielaard]

On Wed, 2019-10-09 at 10:44 +0100, Jonathan Wakely wrote:
> On Wed, 9 Oct 2019 at 01:28, Joseph Myers wrote:
> > 
> > I've done the move of GCC wwwdocs to git (using the previously
> > posted and
> > discussed scripts), including setting up the post-receive hook to
> > do the
> > same things previously covered by the old CVS hooks, and minimal
> > updates
> > to the web pages dealing with the CVS setup for wwwdocs.
> 
> Thanks, Joseph.
> 
> I just pushed a change and although it worked and appeared on
> gcc.gnu.org, I got an error message during the push:
> 
> remote: hooks/post-receive: line 14: /www/gcc/updatelog: Permission
> denied
> remote:   Updating /www/gcc/htdocs/gcc-10/changes.html

/www/gcc/updatelog is where the logs of the post-receive are written.
It was only readable by the gcc group, I did a chmod g+w.
Now things should work without warning.

Cheers,

Mark

Re: GCC wwwdocs move to git done

[Christopher Faylor]

On Wed, Oct 09, 2019 at 01:25:30PM +0100, Iain Sandoe wrote:
>Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>
>>On Wed, 9 Oct 2019 at 01:28, Joseph Myers wrote:
>>>I've done the move of GCC wwwdocs to git (using the previously posted and
>>>discussed scripts), including setting up the post-receive hook to do the
>>>same things previously covered by the old CVS hooks, and minimal updates
>>>to the web pages dealing with the CVS setup for wwwdocs.
>>
>>Thanks, Joseph.
>
>+1
>
>I would like to be able to preview changes to the website by using it
>from a local
>webserver.  I realise that individual pages can be viewed in a
>browser / validated
>by uploading - but it would be nice to check connectivity etc.
>
>At the moment, I can’t identify the “mhc” program that is used in
>preparing the text
>(and too many unrelated hits from searches).

I think it's the "metahtml" processor:

https://ftp.gnu.org/gnu/metahtml/

The binary is 20 years old and, somehow, the source code used to build
it seems to have disappeared.

Re: GCC wwwdocs move to git done

[Iain Sandoe]

Christopher Faylor <cgf-use-the-mailinglist-please@gnu.org> wrote:

> On Wed, Oct 09, 2019 at 01:25:30PM +0100, Iain Sandoe wrote:
>> Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>>
>>> On Wed, 9 Oct 2019 at 01:28, Joseph Myers wrote:
>>>> I've done the move of GCC wwwdocs to git (using the previously posted  
>>>> and
>>>> discussed scripts), including setting up the post-receive hook to do the
>>>> same things previously covered by the old CVS hooks, and minimal updates
>>>> to the web pages dealing with the CVS setup for wwwdocs.
>>>
>>> Thanks, Joseph.
>>
>> +1
>>
>> I would like to be able to preview changes to the website by using it
>> from a local
>> webserver.  I realise that individual pages can be viewed in a
>> browser / validated
>> by uploading - but it would be nice to check connectivity etc.
>>
>> At the moment, I can’t identify the “mhc” program that is used in
>> preparing the text
>> (and too many unrelated hits from searches).
>
> I think it's the "metahtml" processor:
>
> https://ftp.gnu.org/gnu/metahtml/
>
> The binary is 20 years old and, somehow, the source code used to build
> it seems to have disappeared.

Yeah - https://www.gnu.org/software/metahtml/

but the cvs co seems to hang on a lock :(

something to investigate later...
Iain

Re: GCC wwwdocs move to git done

[Christopher Faylor]

On Wed, Oct 09, 2019 at 09:37:54AM -0400, Christopher Faylor wrote:
>The binary is 20 years old and, somehow, the source code used to build
>it seems to have disappeared.

Sorry. Not 20 years:

-rwxr-xr-x. 1 root root 574221 Mar 22  2013 /usr/local/bin/mhc

cgf

Re: GCC wwwdocs move to git done

[David Malcolm]

On Wed, 2019-10-09 at 09:37 -0400, Christopher Faylor wrote:
> On Wed, Oct 09, 2019 at 01:25:30PM +0100, Iain Sandoe wrote:
> > Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
> > 
> > > On Wed, 9 Oct 2019 at 01:28, Joseph Myers wrote:
> > > > I've done the move of GCC wwwdocs to git (using the previously
> > > > posted and
> > > > discussed scripts), including setting up the post-receive hook
> > > > to do the
> > > > same things previously covered by the old CVS hooks, and
> > > > minimal updates
> > > > to the web pages dealing with the CVS setup for wwwdocs.
> > > 
> > > Thanks, Joseph.
> > 
> > +1
> > 
> > I would like to be able to preview changes to the website by using
> > it
> > from a local
> > webserver.  I realise that individual pages can be viewed in a
> > browser / validated
> > by uploading - but it would be nice to check connectivity etc.
> > 
> > At the moment, I can’t identify the “mhc” program that is used in
> > preparing the text
> > (and too many unrelated hits from searches).
> 
> I think it's the "metahtml" processor:
> 
> https://ftp.gnu.org/gnu/metahtml/
> 
> The binary is 20 years old and, somehow, the source code used to
> build
> it seems to have disappeared.

I wrote a replacement for this:
  "[PATCH] wwwdocs: port from MetaHTML to a Python 3 script"
     https://gcc.gnu.org/ml/gcc-patches/2018-06/msg00176.html

I had it in my mind that this had gone in, but I guess I misremembered.

[I'm currently frantically trying to finish a major new feature for GCC
stage 1, so my cycles are limited right now, but if someone else wants
to finish the patch in the meantime that would be great]

Dave

Re: GCC wwwdocs move to git done

[Joseph Myers]

On Tue, 8 Oct 2019, Frank Ch. Eigler wrote:

> Hi -
> 
> Thanks - good job with moving this to git!
> 
> > Note 1: someone with the right access needs to create the symlink 
> > /sourceware/git/gcc-wwwdocs.git -> 
> > /sourceware/projects/gcc-home/wwwdocs.git (and anything else needed for 
> > anonymous git access to that repository).
> 
> Done.

Thanks!  I've now updated the instructions to use the shorter path to the 
repository and restored the documentation of anonymous access.

diff --git a/htdocs/about.html b/htdocs/about.html
index 30a5c943..019b6fbd 100644
--- a/htdocs/about.html
+++ b/htdocs/about.html
@@ -54,10 +54,12 @@ a higher chance of being implemented soon. ;-)</p>
 and SSH installed, you can check out the web pages as follows:</p>
 
 <ol>
- <li><code>git clone git+ssh://<i>username</i>@gcc.gnu.org/sourceware/projects/gcc-home/wwwdocs.git</code>
+ <li><code>git clone git+ssh://<i>username</i>@gcc.gnu.org/git/gcc-wwwdocs.git</code>
  where <i>username</i> is your user name at gcc.gnu.org</li>
 </ol>
 
+<p>For anonymous access, use
+<code>git://gcc.gnu.org/git/gcc-wwwdocs.git</code> instead.</p>
 
 <h3>Validating a change</h3>
 

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: GCC wwwdocs move to git done

[David Malcolm]

On Wed, 2019-10-09 at 15:06 +0000, Joseph Myers wrote:
> On Tue, 8 Oct 2019, Frank Ch. Eigler wrote:
> 
> > Hi -
> > 
> > Thanks - good job with moving this to git!

+1

> > > Note 1: someone with the right access needs to create the
> > > symlink 
> > > /sourceware/git/gcc-wwwdocs.git -> 
> > > /sourceware/projects/gcc-home/wwwdocs.git (and anything else
> > > needed for 
> > > anonymous git access to that repository).
> > 
> > Done.
> 
> Thanks!  I've now updated the instructions to use the shorter path to
> the 
> repository and restored the documentation of anonymous access.
> 
> diff --git a/htdocs/about.html b/htdocs/about.html
> index 30a5c943..019b6fbd 100644
> --- a/htdocs/about.html
> +++ b/htdocs/about.html
> @@ -54,10 +54,12 @@ a higher chance of being implemented soon. ;-
> )</p>
>  and SSH installed, you can check out the web pages as follows:</p>
>  
>  <ol>
> - <li><code>git clone git+ssh://<i>username</i>@gcc.gnu.org/sourcewar
> e/projects/gcc-home/wwwdocs.git</code>;
> + <li><code>git clone git+ssh://<i>username</i>@gcc.gnu.org/git/gcc-w
> wwdocs.git</code>;
>   where <i>username</i> is your user name at gcc.gnu.org</li>
>  </ol>
>  
> +<p>For anonymous access, use
> +<code>git://gcc.gnu.org/git/gcc-wwwdocs.git</code> instead.</p>
>  
>  <h3>Validating a change</h3>

Is there a web UI for viewing this new git repo?

I don't see it listed at:
  https://gcc.gnu.org/git/
(or would adding it there complicate the migration of the main repo?)

Thanks.

Dave

Re: GCC wwwdocs move to git done

[Gerald Pfeifer]

On Wed, 9 Oct 2019, Joseph Myers wrote:
> I've done the move of GCC wwwdocs to git (using the previously posted and 
> discussed scripts), including setting up the post-receive hook to do the 
> same things previously covered by the old CVS hooks, and minimal updates 
> to the web pages dealing with the CVS setup for wwwdocs.

Really, really, cool.  Thanks a huge bunch, Joseph!

> Note 2: changes may be needed to the process for updating www.gnu.org 
> and Gerald's validator.

The validator *should* be working in this new world now.

And I am in contact with webmaster@gnu.org and will update this round
as things evolve.  (Not an urgency, just important.)

Gerald

Re: GCC wwwdocs move to git done

[Jose E. Marchesi]

Hi Joseph.

    I've done the move of GCC wwwdocs to git (using the previously posted and 
    discussed scripts), including setting up the post-receive hook to do the 
    same things previously covered by the old CVS hooks, and minimal updates 
    to the web pages dealing with the CVS setup for wwwdocs.

Belated thanks! :)

Re: GCC wwwdocs move to git done

[Georg-Johann Lay]

Am 09.10.19 um 02:27 schrieb Joseph Myers:
> I've done the move of GCC wwwdocs to git (using the previously posted and
> discussed scripts), including setting up the post-receive hook to do the
> same things previously covered by the old CVS hooks, and minimal updates
> to the web pages dealing with the CVS setup for wwwdocs.

Hi,

May it be the case that some parts are missing?  In particular, I cannot
find the source of

https://gcc.gnu.org/install/configure.html

Johann

Re: GCC wwwdocs move to git done

[Georg-Johann Lay]

Am 06.11.19 um 15:03 schrieb Georg-Johann Lay:
> Am 09.10.19 um 02:27 schrieb Joseph Myers:
>> I've done the move of GCC wwwdocs to git (using the previously posted and
>> discussed scripts), including setting up the post-receive hook to do the
>> same things previously covered by the old CVS hooks, and minimal updates
>> to the web pages dealing with the CVS setup for wwwdocs.
> 
> Hi,
> 
> May it be the case that some parts are missing?  In particular, I cannot
> find the source of
> 
> https://gcc.gnu.org/install/configure.html
> 
> Johann
> 

Ok, found it in install/README. knew it had something special about it...

Johann