[Bug Infrastructure/29643] New: Release upload process improvements

[Bug Infrastructure/29643] New: Release upload process improvements

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29643

            Bug ID: 29643
           Summary: Release upload process improvements
           Product: sourceware
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Infrastructure
          Assignee: overseers at sourceware dot org
          Reporter: mark at klomp dot org
  Target Milestone: ---

Projects that use https://sourceware.org/pub/ for releases need a shell account
and are responsible for providing signatures themselves (although we do
generate md5 and sha512 sums).

It would be good to provide a release mechanism like:
https://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads
So no shell access is required and signatures are.

-- 
You are receiving this mail because:
You are the assignee for the bug.

Re: [Bug Infrastructure/29643] New: Release upload process improvements

[Ian Kelling]

mark at klomp dot org via Overseers <overseers@sourceware.org> writes:

> https://sourceware.org/bugzilla/show_bug.cgi?id=29643
>
>             Bug ID: 29643
>            Summary: Release upload process improvements
>            Product: sourceware
>            Version: unspecified
>             Status: NEW
>           Severity: normal
>           Priority: P2
>          Component: Infrastructure
>           Assignee: overseers at sourceware dot org
>           Reporter: mark at klomp dot org
>   Target Milestone: ---
>
> Projects that use https://sourceware.org/pub/ for releases need a shell account
> and are responsible for providing signatures themselves (although we do
> generate md5 and sha512 sums).
>
> It would be good to provide a release mechanism like:
> https://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads
> So no shell access is required and signatures are.

Is it better to comment on these in bugzilla? Apparently I never created
one, I just emailed, in the meantime, I'll reply here.

This is a great idea, and I suggest using the GNU FTP software. It is
unpublished currently, I think only because of momentum that it started
as a tiny script 16 years ago but we (FSF) could publish and/or share it
right away (just say the word). We recently had a volunteer audit the
code for security issues and have been working toward publish a release
that had all the documentation for other people to easily use it, but
that doesn't need to be perfect, we can just do it.

Right now, I see that GCC, GDB, GLIBC and Binutils are published both
places, so this seems like an obvious area for more
cooperation/coordination

For example, (and I would just open a separate bug for this, but I want
to get this off my chest): I notice that the sort order on
https://sourceware.org/pub/ is alphabetical instead of version ordering,
which is confusing. On the GNU FTP, we apparently get the right sort
ordering through debian's config,
/etc/apache2/mods-available/autoindex.conf, the line seems to be:

IndexOptions FancyIndexing VersionSort HTMLTable NameWidth=* DescriptionWidth=* Charset=UTF-8,

the debian upstream repo for this config is
https://salsa.debian.org/apache-team/apache2 in the path
./debian/config-dir/mods-available/autoindex.conf

I suggest adding that line to the sourceware apache config.

-- 
Ian Kelling | Senior Systems Administrator, Free Software Foundation
GPG Key: B125 F60B 7B28 7FF6 A2B7  DF8F 170A F0E2 9542 95DF
https://fsf.org | https://gnu.org

Re: [Bug Infrastructure/29643] New: Release upload process improvements

[Frank Ch. Eigler]

Hi -

> [...]  For example, (and I would just open a separate bug for this,
> but I want to get this off my chest): I notice that the sort order
> on https://sourceware.org/pub/ is alphabetical instead of version
> ordering, [...]

The systemwide default indexoptions already include versionsort.
Where do you see something contrary?  I'm seeing e.g. correct behaviour
in subdirectories such as https://sourceware.org/pub/gcc/releases/ .

- FChE

Re: [Bug Infrastructure/29643] New: Release upload process improvements

[Ian Kelling]

"Frank Ch. Eigler" <fche@redhat.com> writes:

> Hi -
>
>> [...]  For example, (and I would just open a separate bug for this,
>> but I want to get this off my chest): I notice that the sort order
>> on https://sourceware.org/pub/ is alphabetical instead of version
>> ordering, [...]
>
> The systemwide default indexoptions already include versionsort.
> Where do you see something contrary?  I'm seeing e.g. correct behaviour
> in subdirectories such as https://sourceware.org/pub/gcc/releases/ .

You are right, that is an embarrassing mistake, I chalk it up to working
while tired, thought I noticed something and didn't double check.

[Bug Infrastructure/29643] Release upload process improvements

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29643

--- Comment #1 from iank at fsf dot org ---
mark at klomp dot org via Overseers <overseers@sourceware.org> writes:

> https://sourceware.org/bugzilla/show_bug.cgi?id=29643
>
>             Bug ID: 29643
>            Summary: Release upload process improvements
>            Product: sourceware
>            Version: unspecified
>             Status: NEW
>           Severity: normal
>           Priority: P2
>          Component: Infrastructure
>           Assignee: overseers at sourceware dot org
>           Reporter: mark at klomp dot org
>   Target Milestone: ---
>
> Projects that use https://sourceware.org/pub/ for releases need a shell account
> and are responsible for providing signatures themselves (although we do
> generate md5 and sha512 sums).
>
> It would be good to provide a release mechanism like:
> https://www.gnu.org/prep/maintain/maintain.html#Automated-FTP-Uploads
> So no shell access is required and signatures are.

Is it better to comment on these in bugzilla? Apparently I never created
one, I just emailed, in the meantime, I'll reply here.

This is a great idea, and I suggest using the GNU FTP software. It is
unpublished currently, I think only because of momentum that it started
as a tiny script 16 years ago but we (FSF) could publish and/or share it
right away (just say the word). We recently had a volunteer audit the
code for security issues and have been working toward publish a release
that had all the documentation for other people to easily use it, but
that doesn't need to be perfect, we can just do it.

Right now, I see that GCC, GDB, GLIBC and Binutils are published both
places, so this seems like an obvious area for more
cooperation/coordination

For example, (and I would just open a separate bug for this, but I want
to get this off my chest): I notice that the sort order on
https://sourceware.org/pub/ is alphabetical instead of version ordering,
which is confusing. On the GNU FTP, we apparently get the right sort
ordering through debian's config,
/etc/apache2/mods-available/autoindex.conf, the line seems to be:

IndexOptions FancyIndexing VersionSort HTMLTable NameWidth=* DescriptionWidth=*
Charset=UTF-8,

the debian upstream repo for this config is
https://salsa.debian.org/apache-team/apache2 in the path
./debian/config-dir/mods-available/autoindex.conf

I suggest adding that line to the sourceware apache config.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29643] Release upload process improvements

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29643

--- Comment #2 from Mark Wielaard <mark at klomp dot org> ---
(In reply to iank from comment #1)
> This is a great idea, and I suggest using the GNU FTP software. It is
> unpublished currently, I think only because of momentum that it started
> as a tiny script 16 years ago but we (FSF) could publish and/or share it
> right away (just say the word). We recently had a volunteer audit the
> code for security issues and have been working toward publish a release
> that had all the documentation for other people to easily use it, but
> that doesn't need to be perfect, we can just do it.

If you do publish the scripts please add a pointer in this bug.

> For example, (and I would just open a separate bug for this, but I want
> to get this off my chest): I notice that the sort order on
> https://sourceware.org/pub/ is alphabetical instead of version ordering,
> which is confusing. On the GNU FTP, we apparently get the right sort
> ordering through debian's config,

It looks like we already are using VersionSort so this should be correct.
Do you have an directory where the sort oder is wrong?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29643] Release upload process improvements

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29643

iank at fsf dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |iank at fsf dot org

--- Comment #3 from iank at fsf dot org ---
Update: A volunteer, Jacob Bachmeyer, is working on doing some
documenting and refactoring so it is reasonable for anyone to adopt and
use instead of just the GNU release site. For example having a
documented config file instead of hard coded variables with path
locations. His estimate for this is a few weeks to a month or three, and
I prefer to wait to publish it until then. But as I said before, if you
want is sooner, just ask and I can provide it privately under GPLv3+ in
it's current state and then of course you have the freedom to share as
you wish.

I will definitely add a pointer on this bug when we publish it.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29643] Release upload process improvements

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29643

--- Comment #4 from Mark Wielaard <mark at klomp dot org> ---
Any update on publishing the scripts?

If the review isn't complete yet could you sent them privately so we can
inspect them ourselves and try them out on sourceware?

-- 
You are receiving this mail because:
You are the assignee for the bug.

Re: [Bug Infrastructure/29643] Release upload process improvements

[Ian Kelling]

mark at klomp dot org via Overseers <overseers@sourceware.org> writes:

> https://sourceware.org/bugzilla/show_bug.cgi?id=29643
>
> --- Comment #4 from Mark Wielaard <mark at klomp dot org> ---
> Any update on publishing the scripts?
>
> If the review isn't complete yet could you sent them privately so we can
> inspect them ourselves and try them out on sourceware?

Yes. Give me another day or so.

[Bug Infrastructure/29643] Release upload process improvements

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29643

--- Comment #5 from iank at fsf dot org ---
mark at klomp dot org via Overseers <overseers@sourceware.org> writes:

> https://sourceware.org/bugzilla/show_bug.cgi?id=29643
>
> --- Comment #4 from Mark Wielaard <mark at klomp dot org> ---
> Any update on publishing the scripts?
>
> If the review isn't complete yet could you sent them privately so we can
> inspect them ourselves and try them out on sourceware?

Yes. Give me another day or so.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29643] Release upload process improvements

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29643

--- Comment #6 from iank at fsf dot org ---
I understand this is public, we just aren't ready to advertise it yet:

git clone https://vcs.fsf.org/git/gatekeeper.git

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29643] Release upload process improvements

[iank at fsf dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29643

--- Comment #7 from iank at fsf dot org ---
As you will see from the repo, development got stalled over the winter
holiday but we are picking it back up again.

-- 
You are receiving this mail because:
You are the assignee for the bug.