Re: Updates of wwwdocs not working anymore

Re: Updates of wwwdocs not working anymore

[Gerald Pfeifer]

On Fri, 29 Nov 2019, Jakub Jelinek wrote:
> I've pushed this change some time ago, but it hasn't shown on the web yet,
> seems the last checkout in /www/gcc/ is from yesterday 18:05 or so.
> Your b22559c84001d6c75c64597f96ca119b488b9b1a change also doesn't show up.

Thanks for reporting this, Jakub!  Let me add overseers@, Joseph and
Richi and report what I found.


It appears commit 34bfcf1947c44e458af1b7ba201f25071c4d80a5

  Author: Gerald Pfeifer <gerald@pfeifer.com>
  Date:   Thu Nov 28 19:01:20 2019 +0100

    Push <meta http-equiv="Content-Type"> down into individual HTML files.
    
    Historically we have been adding <meta http-equiv="Content-Type"...> to
    all HTML files via our preprocessing machinery. With this change these
    files become more self contained and in particular easier to validate
    directly.

was applied to /www/gcc/htdocs-preformatted on gcc.gnu.org "somehow",
but appears as a local set of changes.

So any git pull now complains:

    $ git pull
    Updating f1a4881..1b74d49
    error: Your local changes to 'htdocs/about.html' would be overwritten 
    by merge.  Aborting.
    Please, commit your changes or stash them before you can merge.


I cleaned that up (as user gccadmin), but running git pull in
/www/gcc/htdocs-preformatted now complains as follows:

    $ git pull -v
    From /sourceware/projects/gcc-home/wwwdocs
     = [up to date]      master     -> origin/master
    Updating f1a4881..1b74d49
    error: unable to unlink old 'htdocs/onlinedocs/7.5.0/index.html' 
    (Permission denied)

And indeed:

    $ ls -lad onlinedocs/7.5.0/
    drwxr-sr-x. 2 rguenth gcc 4096 14. Nov 08:56 onlinedocs/7.5.0/
        ^^^       ^^^^^^^


I made this directory group writable, like the others there, only
to find that indeed git had applied the pull locally only to abort 
midstream.  Back to start, player 1.


After another

    $ git checkout .
    $ git pull -v

as gccadmin in /www/gcc/htdocs-preformatted things seem to be fine,
however I am worried there's a latent issue there.

Has something been changed recently, or is this simply an issue with
the move of wwwdocs to git that we did not run into before?

(Looking at /www/gcc/htdocs-preformatted.old/, the old version of that
tree from before the move to git, it appears different files/directories
are all owned by the one having done the last change, and group writable.)

Gerald

Re: Updates of wwwdocs not working anymore

[Joseph Myers]

On Fri, 29 Nov 2019, Gerald Pfeifer wrote:

> Has something been changed recently, or is this simply an issue with
> the move of wwwdocs to git that we did not run into before?

The post-receive hook explicitly sets permissions with chmod when creating 
directories in htdocs, for the *output* of preprocessing.

/www/gcc/wwwdocs-checkout/.git/config has "sharedrepository = 1".  That's 
meant to ensure files and directories are group-writable in the checkout 
itself (into which htdocs-preformatted is a symlink).  Maybe that only 
works in the .git part of the checkout rather than the whole checkout, in 
which case the post-receive hook (itself checked into the git repository, 
in bin/) would need at least to set the umask before doing git pull?

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Updates of wwwdocs not working anymore

[Gerald Pfeifer]

On Fri, 29 Nov 2019, Joseph Myers wrote:
>> Has something been changed recently, or is this simply an issue with
>> the move of wwwdocs to git that we did not run into before?
> The post-receive hook explicitly sets permissions with chmod when creating 
> directories in htdocs, for the *output* of preprocessing.
> 
> /www/gcc/wwwdocs-checkout/.git/config has "sharedrepository = 1".  That's 
> meant to ensure files and directories are group-writable in the checkout 
> itself (into which htdocs-preformatted is a symlink).  Maybe that only 
> works in the .git part of the checkout rather than the whole checkout, in 
> which case the post-receive hook (itself checked into the git repository, 
> in bin/) would need at least to set the umask before doing git pull?

I'll admit this is a bit over my head.  We just ran into this again,
though, so would you mind having a look?

To be clear, /www/gcc/htdocs-preformatted (the raw checkout from git)
features -rw-r--r-- for most files which seems to be fine, though, and
does not cause problems.  It's the pre-processed web tree under
/www/gcc/htdocs where this hits.  

Maybe setting a more permissive umask before invoking 
/www/gcc/bin/preprocess is the best approach?

Gerald

Re: Updates of wwwdocs not working anymore

[Joseph Myers]

On Sat, 11 Jan 2020, Gerald Pfeifer wrote:

> I'll admit this is a bit over my head.  We just ran into this again,
> though, so would you mind having a look?
> 
> To be clear, /www/gcc/htdocs-preformatted (the raw checkout from git)
> features -rw-r--r-- for most files which seems to be fine, though, and
> does not cause problems.  It's the pre-processed web tree under
> /www/gcc/htdocs where this hits.  
> 
> Maybe setting a more permissive umask before invoking 
> /www/gcc/bin/preprocess is the best approach?

I've applied a change that both sets a umask and explicitly removes 
destination files before copying over them (so as to be unaffected by 
their permissions).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Updates of wwwdocs not working anymore

[Gerald Pfeifer]

On Sat, 11 Jan 2020, Joseph Myers wrote:
> I've applied a change that both sets a umask and explicitly removes 
> destination files before copying over them (so as to be unaffected by 
> their permissions).

Thank you, Joseph! 

From the experience of the last week, where several of us updated 
the same file in wwwdocs it appears this resolved the issue.

Gerald