Toolchain Infrastructure project statement of support

Toolchain Infrastructure project statement of support

[Carlos O'Donell]

The GNU Toolchain project leadership supports the proposal[1] to move the
services for the GNU Toolchain to the Linux Foundation IT under the auspices of
the Toolchain Infrastructure project (GTI) with fiscal sponsorship from the
OpenSSF and other major donors.

This support includes empowering the GTI Technical Advisory Committee to vote
on issues surrounding key infrastructure requirements. Additionally we welcome
any other projects at Sourceware.org that wish to join the effort.

The GNU Toolchain is the cornerstone of the GNU/Linux ecosystem, the Cloud, and
embedded environments. Open Source security best practices and mandates from
governments and industry on software supply chain security require a strong
effort to ensure compliance with policies, maintain the trust of the
distributors of software built by the GNU Toolchain, and the continued vitality
of the GNU Toolchain projects.

We believe that utilizing the experience and infrastructure of the LF IT team
that already is used by the Linux kernel community will provide the most
effective solution and best experience for the GNU Toolchain developer
community.

As with kernel.org, GTI has requested from the Linux Foundation IT that all
software implementing the infrastructure for the GNU Toolchain projects must
be Free Software.  If Free Software versions of necessary security tools are
missing, GTI will work with Free Software supporters to develop Free Software
alternatives.

Carlos O’Donell
David Edelsohn
Dave Miller
Jason Merrill
Jeff Law
Jim Wilson
Joel Brobecker
Joel Sherrill
Joseph Myers
Kaveh Ghazi
Martin Liska
Ramana Radhakrishnan
Richard Biener
Segher Boessenkool
Siddhesh Poyarekar
Toon Moene
Tulio Magno Quites Machado Filho

[1] https://sourceware.org/pipermail/overseers/2022q3/018896.html

Re: Toolchain Infrastructure project statement of support

[Andrew Pinski]

On Wed, Oct 12, 2022 at 9:43 AM Carlos O'Donell via Overseers
<overseers@sourceware.org> wrote:
>
> The GNU Toolchain project leadership supports the proposal[1] to move the
> services for the GNU Toolchain to the Linux Foundation IT under the auspices of
> the Toolchain Infrastructure project (GTI) with fiscal sponsorship from the
> OpenSSF and other major donors.

Who got to decide to be part of the leadership? Where are the bylaws?

>
> This support includes empowering the GTI Technical Advisory Committee to vote
> on issues surrounding key infrastructure requirements. Additionally we welcome
> any other projects at Sourceware.org that wish to join the effort.

Is there some kind of bylaws which describe the relationship between
GTI and the actual free source projects and the TAC?

>
> The GNU Toolchain is the cornerstone of the GNU/Linux ecosystem, the Cloud, and
> embedded environments. Open Source security best practices and mandates from
> governments and industry on software supply chain security require a strong
> effort to ensure compliance with policies, maintain the trust of the
> distributors of software built by the GNU Toolchain, and the continued vitality
> of the GNU Toolchain projects.

Trust is a heavy weighted word in general. Especially maintaining
trust of distributors is that really what we should be doing here,
rather than maintaining trust of the open/free sourceness of the
toolchain?

>
> We believe that utilizing the experience and infrastructure of the LF IT team
> that already is used by the Linux kernel community will provide the most
> effective solution and best experience for the GNU Toolchain developer
> community.

Can you expand on the voting mechanism that got to this decision? Is
there an email trail of all of emails between folks or notes from
meetings? Were the meetings held in private instead of allowing more
people to join the group? Who decided who was on the group and even
more how do people get to join the TAC and who gets to vote who is on
the TAC?

>
> As with kernel.org, GTI has requested from the Linux Foundation IT that all
> software implementing the infrastructure for the GNU Toolchain projects must
> be Free Software.  If Free Software versions of necessary security tools are
> missing, GTI will work with Free Software supporters to develop Free Software
> alternatives.

This is a stack backwards and even worse it is a fork.

Thanks,
Andrew Pinski

>
> Carlos O’Donell
> David Edelsohn
> Dave Miller
> Jason Merrill
> Jeff Law
> Jim Wilson
> Joel Brobecker
> Joel Sherrill
> Joseph Myers
> Kaveh Ghazi
> Martin Liska
> Ramana Radhakrishnan
> Richard Biener
> Segher Boessenkool
> Siddhesh Poyarekar
> Toon Moene
> Tulio Magno Quites Machado Filho
>
> [1] https://sourceware.org/pipermail/overseers/2022q3/018896.html
>

Re: Toolchain Infrastructure project statement of support

[Christopher Faylor]

Re: https://sourceware.org/pipermail/overseers/2022q4/018981.html

On Wed, Oct 12, 2022 at 12:43:09PM -0400, Carlos O'Donell wrote:
>The GNU Toolchain project leadership supports the proposal[1] to move the
>services for the GNU Toolchain to the Linux Foundation IT under the auspices of
>the Toolchain Infrastructure project (GTI) with fiscal sponsorship from the
>OpenSSF and other major donors.

Noted, however, a list of signatories does not automatically confer
authority over any particular project.  Any participation from 
overseers in moving projects to different infrastructure will require
clear approval from the individual projects themselves.

Also, the FSF, being the existing fiscal sponsor to these projects,
surely needs to review the formal agreements before we sunset our
infrastructural offerings to glibc, gcc, binutils, and gdb and hand
control of the projects' infrastructure over to a different entity.

We'd like to assure the communities that, when and if any individual
project formally expresses the decision of their developers to transfer
their services, we'll endeavor to make the move as smooth as possible. 
Those projects that wish to stay will continue to receive the best
services that the overseers can offer, with the ongoing assistance of
Red Hat, the SFC, and, when relevant, the FSF tech team.

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-13 14:25, Christopher Faylor via Overseers wrote:
> Also, the FSF, being the existing fiscal sponsor to these projects,
> surely needs to review the formal agreements before we sunset our
> infrastructural offerings to glibc, gcc, binutils, and gdb and hand
> control of the projects' infrastructure over to a different entity.

I believe Nick at least has said that for binutils they would like to 
use both, i.e. mirror some of the LF offerings with sourceware (or vice 
versa, I suppose it's Nick's call to take) and that'll probably be the 
case for glibc and gcc as well over the years.  So sunset, whenever that 
will be and whatever it'll look like, will likely be a while.

Sid

Re: Toolchain Infrastructure project statement of support

[Mark Wielaard]

Hi Carlos,

On Wed, Oct 12, 2022 at 12:43:09PM -0400, Carlos O'Donell wrote:
> The GNU Toolchain project leadership [...]

I must say I don't understand why you are communicating in this way.
Sending out "proclamations" about having support from "leadership",
"committees" and "key stakeholders". Some of these key people seem to
not even agree with it or know what it is really about and they cannot
or don't want to answer questions about the details.

In the last year we did some really nice work for the sourceware
infrastructure. We setup the shared buildbot, got various companies and
organisations to provide compute resources, workers for various
architectures. We now have CI, Try and Full builders for various
projects and doing 10.000+ builds a month. With a bunsen analysis
database with all those test-results. Did a resource analysis and wrote
up this public roadmap to make the email/git based workflow that
sourceware projects rely on more fun, secure and productive by
automating contribution tracking and testing. We now also have the
sourcehut mirror and the public-inbox instance to make the email
workflow nicer and support things like patch attestation. We are
working on better integration between patchwork and buildbot for pre-
commit checking. And we got the Software Freedom Conservancy to accept
sourceware as a member project to act as a fiscal sponsor. They are now
helping us with the future roadmap, setting up a organization,
budgeting, etc. And the FSF also is supportive of this.

All this was done in public, we even setup some public video chats
about how we wanted to do this in the future. And you were explicitly
invited to participate because we wanted to make sure it fit with any
other plans people might be having.

At the Cauldron, when we wanted to discuss with the community how to
use and set project policies around the sourceware infrastructure
services, one of the "leaders" ran around the room shouting down and
pushing people who wanted to discuss this. Telling people they didn't
got to decide what we would talk about. And finally yelling at me that
I lost all trust of the "gnu toolchain leadership". All just for
wanting to have a public discussion on some cool stuff we did and were
planning to do together. That isn't "leadership". That is just
intimidation and bullying. It made me really sad.

And now you again seem to not want to discuss any details on how to
work together. After Cauldron I thought we agreed we would discuss
goals on overseers and create sourceware infrastructure bugs. So we
could see what the community priorities were, write an updated
sourceware roadmap, setup a budget, etc.

I was really happy to see the discussions about setting up a video chat
system for projects, the FSF tech-team offering to setup mirrors,
backups and help coordinate secure release uploads. And I had hoped to
see some discussion on how the LF and potential sponsors could help,
working together with the sourceware community and the SFC.

We really would love for gdb, glibc, binutils and gcc to keep being
part of sourceware.

Cheers,

Mark

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-17 11:10, Mark Wielaard via Overseers wrote:
<snip>
> In the last year we did some really nice work for the sourceware
> infrastructure. We setup the shared buildbot, got various companies and

I feel like you're taking this personally as an overseer; the proposal 
to transition to LF IT is not a value judgment on your work.  It is a 
proposal to scale far beyond what our current resources can afford.  I 
personally am grateful for all that work and have participated in some 
of it too but I don't think that it's a scalable approach for gcc or to 
a lesser extent, glibc.

> organisations to provide compute resources, workers for various
> architectures. We now have CI, Try and Full builders for various
> projects and doing 10.000+ builds a month. With a bunsen analysis
> database with all those test-results. Did a resource analysis and wrote
> up this public roadmap to make the email/git based workflow that
> sourceware projects rely on more fun, secure and productive by
> automating contribution tracking and testing. We now also have the
> sourcehut mirror and the public-inbox instance to make the email
> workflow nicer and support things like patch attestation. We are
> working on better integration between patchwork and buildbot for pre-
> commit checking. And we got the Software Freedom Conservancy to accept
> sourceware as a member project to act as a fiscal sponsor. They are now

Fiscal sponsor for the *sourceware overseers*.  There's no way for SFC 
to be a fiscal sponsor for sourceware, which is infrastructure owned by 
Red Hat.

> helping us with the future roadmap, setting up a organization,
> budgeting, etc. And the FSF also is supportive of this.

The FSF has little stake in sourceware beyond the GNU toolchain, so FSF 
being supportive of sourceware overseers seeking funding has about the 
same relevance as the FSF being supportive of my decision to move to a 
different country.

They do have a significant stake in infrastructure for the GNU toolchain 
project and they've been part of discussions since the idea for GTI 
germinated.

> And now you again seem to not want to discuss any details on how to
> work together. After Cauldron I thought we agreed we would discuss
> goals on overseers and create sourceware infrastructure bugs. So we
> could see what the community priorities were, write an updated
> sourceware roadmap, setup a budget, etc.

On multiple occasions I've been told on this list that sourceware 
administration being transitioned to LF IT is out of question, so I 
don't see the point of having these discussions.  I don't see a 
retention plan that's viable for the next 5, 10 or 20 years.

I personally do not think the current sourceware infrastructure, even 
with the roadmap it promises is a viable alternative to what LF IT can 
provide.  There is a significant resource gap (e.g. service isolation to 
different machines and/or containers, full time administrators with 
global presence for FTS support, established security and administration 
practices, traffic acceleration, to name a few) that we seem to disagree 
about.

There seems to be little to discuss from the GNU toolchain perspective 
IMO; the overseers think that the LF IT proposal and the additional 
funding and infrastructure it brings in are not necessary and we think 
it is.

> I was really happy to see the discussions about setting up a video chat
> system for projects, the FSF tech-team offering to setup mirrors,
> backups and help coordinate secure release uploads. And I had hoped to
> see some discussion on how the LF and potential sponsors could help,
> working together with the sourceware community and the SFC.
> 
> We really would love for gdb, glibc, binutils and gcc to keep being
> part of sourceware.

IMO the best way to make this happen is for us to transition sourceware 
administration to LF IT.  However since all projects hosted on 
sourceware don't seem to agree on this and the overseers also do not 
desire to give up control over the infrastructure, I don't see another 
way out of this.

Of course, like I said to Chris, it's likely going to be a while before 
we fully transition away from sourceware (it'll likely happen one 
service at a time, or something like that, we need to figure that out at 
some point) and we'll need continued help from the overseers to help 
manage the infrastructure in the near future.

Sid

Re: Toolchain Infrastructure project statement of support

[Mark Wielaard]

Hi Siddhesh,

On Mon, Oct 17, 2022 at 12:11:53PM -0400, Siddhesh Poyarekar wrote:
> There seems to be little to discuss from the GNU toolchain perspective IMO;

Yes, it is clear you don't want any discussion or answer any questions
about the proposals, how funds can be used, what the budget is, what
the requirements are, how the governance structure works, what
alternatives we have, etc.

But personally I think it is healthy to have real discussions, doing
resource analysis, creating public roadmaps, collecting infrastructure
enahancement requests, discuss how to organize, argue about the needed
budget and how to use funding most efficiently, etc. To make sure that
sourceware keeps being a healthy and viable free software
infrastructure project for the next 24 years, hopefully including the
various GNU toolchain projects.

Cheers,

Mark

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-18 05:50, Mark Wielaard wrote:
> Hi Siddhesh,
> 
> On Mon, Oct 17, 2022 at 12:11:53PM -0400, Siddhesh Poyarekar wrote:
>> There seems to be little to discuss from the GNU toolchain perspective IMO;
> 
> Yes, it is clear you don't want any discussion or answer any questions
> about the proposals, 

That is not true, Mark.  Your objections and questions have been 
answered at every stage, privately as well as publicly.  What *is* clear 
is that we have been talking past each other because despite our common 
high level intentions, we appear to have little common ground for our 
goals.  You want to retain the current sourceware infrastructure and try 
and see what we can do within that framework and I want us to migrate 
services to infrastructure with better funding (that's not just limited 
to services), dedicated ops management and an actually scalable future.

> how funds can be used,
Let me turn that around: how *would* you like funds to be used beyond 
what is currently proposed in the LF IT proposal?

> what the budget is, 

Around $400,000.

> what
> the requirements are, 

Your lack of clarity about requirements IMO have more to do with you 
wanting to fulfill those requirements within sourceware and not with 
their existence.  I and others have repeated them here and the overseers 
have either questioned their validity or noted them in bugzilla as 
possible things to explore in the current sourceware context.  With 
sourceware migration to LF IT off the table, there's little incentive 
for me personally to explore them.

> how the governance structure works, 

I think you know how it works, maybe you meant to say that you don't 
like it?

The governance structure and their workings have been described in the 
GTI introduction.  There are two key bodies that govern the project: the 
Technical Advisory Council (comprised of project community members) 
manages the technical details of the infrastructure and the governing 
board (comprised of representatives from funding companies) manages the 
funding for those technical details.

The current TAC comprises of people from the initial community 
stakeholders who were contacted and subsequently accepted the invitation 
to be part of TAC.  You, along with other overseers, were invited too 
but most of you declined.

> what
> alternatives we have, etc.
For projects the alternatives they have are:

1. Migrate to LF IT infrastructure
2. Have a presence on sourceware as well as LF IT, contingent to Red 
Hat's decision on the hardware infrastructure
3. Stay fully on sourceware

For sourceware as infrastructure the alternatives are:

1. Migrate to LF IT infrastructure
2. Stay as it currently is

For sourceware overseers, the choices are contingent on what projects 
decide and what Red Hat decides w.r.t. sourceware.

All of the above has been clear all along.  Maybe the problem here is 
that you're not happy with the alternatives?

> But personally I think it is healthy to have real discussions, doing
> resource analysis, creating public roadmaps, collecting infrastructure
> enahancement requests, discuss how to organize, argue about the needed
> budget and how to use funding most efficiently, etc. To make sure that
> sourceware keeps being a healthy and viable free software
> infrastructure project for the next 24 years, hopefully including the
> various GNU toolchain projects.

You want to talk about sourceware without including the LF IT proposal 
whereas I'd love to talk about sourceware as an LF IT maintained 
infrastructure.  There's a real disconnect that precludes any real 
discussions.

Sid

Re: Toolchain Infrastructure project statement of support

[Christopher Faylor]

On Tue, Oct 18, 2022 at 11:17:15AM -0400, Siddhesh Poyarekar wrote:
>That is not true, Mark.  Your objections and questions have been answered at
>every stage, privately as well as publicly.

Actually, going back through this thread, I see outstanding
questions/issues raised by Mark, Frank, Alexandre Oliva, Jon Corbet, and
Andrew Pinski.

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-18 12:42, Christopher Faylor wrote:
> On Tue, Oct 18, 2022 at 11:17:15AM -0400, Siddhesh Poyarekar wrote:
>> That is not true, Mark.  Your objections and questions have been answered at
>> every stage, privately as well as publicly.
> 
> Actually, going back through this thread, I see outstanding
> questions/issues raised by Mark, Frank, Alexandre Oliva, Jon Corbet, and
> Andrew Pinski.

As far as actual questions regarding the proposal is concerned, I think 
only Job Corbet's questions to Carlos/David are pending an answer; I 
deferred them to Carlos or David because I think they're better placed 
to answer them in their entirety.  The rest, AFAICT, are either fear of 
some kind of corporate takeover, discussions about current sourceware 
infrastructure, or just rhetoric, none of which I'm interested in 
engaging with.

The corporate takeover fear especially is amusing to me given how much 
of GNU toolchain development and infrastructure is sponsored by 
corporations right now but that's just my personal opinion.

Maybe the FSF hosted call next week[1] would be a suitable forum to 
discuss fears of corporate control of the GNU toolchain project 
infrastructure due to the LF IT migration, or for that matter any other 
questions you or others think may have gone unanswered.  Since 
sourceware is neither a GNU nor an FSF project, it probably does not 
make sense to discuss current sourceware infrastructure there.

Sid

[1] https://sourceware.org/pipermail/overseers/2022q4/018997.html

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-18 14:13, Siddhesh Poyarekar wrote:
> only Job Corbet's questions to Carlos/David are pending an answer; I 

s/Job/Jon/ sorry about misspelling your name.

Sid

Re: Toolchain Infrastructure project statement of support

[Paul Smith]

On Tue, 2022-10-18 at 14:14 -0400, Siddhesh Poyarekar wrote:
> On 2022-10-18 14:13, Siddhesh Poyarekar wrote:
> > only Job Corbet's questions to Carlos/David are pending an answer;
> 
> s/Job/Jon/ sorry about misspelling your name.

I thought it was great!  We all have known for years that Jon has the
requisite patience for that role...

Re: Toolchain Infrastructure project statement of support

[Alexandre Oliva]

On Oct 18, 2022, Siddhesh Poyarekar <siddhesh@gotplt.org> wrote:

> The rest, AFAICT, are either fear of some kind of corporate takeover,
> discussions about current sourceware infrastructure, or just rhetoric,
> none of which I'm interested in engaging with.

So in which category do you place my question about the viability of
using the funds raised through the LF to pay for IT services provided by
third parties rather than by the LF?

-- 
Alexandre Oliva, happy hacker                https://FSFLA.org/blogs/lxo/
   Free Software Activist                       GNU Toolchain Engineer
Disinformation flourishes because many people care deeply about injustice
but very few check the facts.  Ask me about <https://stallmansupport.org>

Re: Toolchain Infrastructure project statement of support

[Ian Kelling]

Siddhesh Poyarekar via Overseers <overseers@sourceware.org> writes:

>> what
>> alternatives we have, etc.
> For projects the alternatives they have are:
>
> 1. Migrate to LF IT infrastructure
> 2. Have a presence on sourceware as well as LF IT, contingent to Red
> Hat's decision on the hardware infrastructure
> 3. Stay fully on sourceware
>
> For sourceware as infrastructure the alternatives are:
>
> 1. Migrate to LF IT infrastructure
> 2. Stay as it currently is
>
> For sourceware overseers, the choices are contingent on what projects
> decide and what Red Hat decides w.r.t. sourceware.
>
> All of the above has been clear all along.  Maybe the problem here is
> that you're not happy with the alternatives?

No, I don't think that was ever clear. I've just read this message, but
I've been keeping up with everything public since Cauldron.  All your
options assume that any specific service is 100% managed by LF IT, or
100% managed by sourceware. That is a bad assumption. They could do it
together, or another group could help.  So, you said you wanted
"dedicated ops management", and I assume sourceware is not currently
equipped for that. But there is no reason that an ops team from LF IT or
FSF could not provide dedicated ops management for existing sourceware
services in collaboration with sourceware. Another notable ops team is
OSU, https://osuosl.org/.

For example, at FSF tech team (where I work), we jointly maintain
services with many volunteers that volunteer for specific projects. They
are mainly: KDE, Linux Libre, Emacs, Savannah, Guix, GNU debbugs,
replicant, h-node.org, planet.gnu.org, and Trisquel. The FSF tech team
keeps a 24 hour on call rotation, so the services have a dedicated ops
team to fix issues and respond to alerts, but the day to day management
of the services those groups want, eg: upgrading them, configuring,
modifying, etc is mostly done by volunteers.

To give some very specific examples: a group of volunteer sysadmins for
emacs decided they wanted a new build service for Emacs, so they jumped
in a meeting with FSF tech team (I'm part of it) to discuss all the
technical details and requirements. We decided the volunteers would do
the primary installation and management of a gitlab that was only
configured to use the build server, and FSF tech team would setup
alerts, create the VM and the DNS. We agree on what kind of uptime is
expected and what kind of alerts the tech team will respond to in off
hours. The volunteer's ssh keys sit alongside the FSF tech team's keys
on that VM. Alternatively, for Trisquel, Trisquel orders hardware and we
go install it to the data center, and the Trisquel sysadmins spin up
their own virtual machines or whatever they want to run, we just go into
the data center with spare parts and fix things if the hardware breaks
down. For any service we are going to support, we learn enough about the
service to fix things things.

Anyways, basically, having a dedicated ops team does not imply removing
the sourceware's role, it could simply mean: adding a dedicated ops team
to sourceware.

To provide dedicated ops for the physical servers would require moving
the servers or into servers in a data center near the ops team, or
outsourcing the hardware management to one of many companies (usually by
renting a physical server), but that is all totally feasible and not a
big cost.


Siddhesh Poyarekar via Overseers <overseers@sourceware.org> writes:

> I want us to migrate
> services to infrastructure with better funding (that's not just limited
> to services),

What do you want to fund specifically? "Infrastructure" and "not limited
to services" is not specific enough to understand.

> and an actually scalable future.

What does "an actually scalable future" mean? That is very vague.


-- 
Ian Kelling | Senior Systems Administrator, Free Software Foundation
GPG Key: B125 F60B 7B28 7FF6 A2B7  DF8F 170A F0E2 9542 95DF
https://fsf.org | https://gnu.org

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-23 04:59, Ian Kelling wrote:
> No, I don't think that was ever clear. I've just read this message, but
> I've been keeping up with everything public since Cauldron.  All your
> options assume that any specific service is 100% managed by LF IT, or
> 100% managed by sourceware. That is a bad assumption. They could do it
> together, or another group could help.  So, you said you wanted
> "dedicated ops management", and I assume sourceware is not currently
> equipped for that. But there is no reason that an ops team from LF IT or
> FSF could not provide dedicated ops management for existing sourceware
> services in collaboration with sourceware. Another notable ops team is
> OSU, https://osuosl.org/.

Hybrid administration isn't part of the GTI proposal, why would that be 
considered an option?  Is this something you'd like to propose to the 
TAC, i.e. give named members from the community access to infrastructure 
administration?  We can bring that up in our discussion with the LF IT.

> Anyways, basically, having a dedicated ops team does not imply removing
> the sourceware's role, it could simply mean: adding a dedicated ops team
> to sourceware.

Given that the current sourceware admins have decided to block migration 
of all sourceware assets to the LF IT, I don't have a stake on how 
they'd like to handle this for sourceware.  I could however, as a member 
of TAC (and as member of projects that have agreed to migrate to LF IT, 
i.e. gcc and glibc), discuss with others the possibility of specific 
community volunteers being given some amount of access to manage 
infrastructure.

> To provide dedicated ops for the physical servers would require moving
> the servers or into servers in a data center near the ops team, or
> outsourcing the hardware management to one of many companies (usually by
> renting a physical server), but that is all totally feasible and not a
> big cost.
> 
> 
> Siddhesh Poyarekar via Overseers <overseers@sourceware.org> writes:
> 
>> I want us to migrate
>> services to infrastructure with better funding (that's not just limited
>> to services),
> 
> What do you want to fund specifically? "Infrastructure" and "not limited
> to services" is not specific enough to understand.
> 
>> and an actually scalable future.
> 
> What does "an actually scalable future" mean? That is very vague.

Both of those point to scaling hardware in addition to services, having 
things like physical isolation of services.  Scaling volunteers (which 
hasn't happened in the last 20-soemthing years; it has scaled *down*, if 
anything) and money to pay for dedicated ops isn't going to mostly 
pointless if we can't pay for hardware, which is owned by Red Hat. 
That, and FSF not being able to add resources for the GNU toolchain was 
in fact why we had to look elsewhere for funding.

I suggest you wait a day for the FSF hosted call so that the background 
is clearer.

Sid

Re: Toolchain Infrastructure project statement of support

[Frank Ch. Eigler]

Hi -

> [...]  Given that the current sourceware admins have decided to
> block migration of all sourceware assets to the LF IT [...]

If you're trying to say that projects have not unanimously shown
interest in moving infrastructure to LF IT, just say that.  Don't
blame overseers.

If you're trying to suggest that overseers, contrary to our repeated
public statements, wish to block all migration, that is untrue and you
will need to retract this.

- FChE

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-23 11:16, Frank Ch. Eigler wrote:
> Hi -
> 
>> [...]  Given that the current sourceware admins have decided to
>> block migration of all sourceware assets to the LF IT [...]
> 
> If you're trying to say that projects have not unanimously shown
> interest in moving infrastructure to LF IT, just say that.  Don't
> blame overseers.

I did not say that, although no project (barring maybe elfutils and 
systemtap, assuming that your and Mark's objection as overseers implies 
that you do not want to move to LF IT) has specifically *opposed* moving 
infrastructure to LF IT either.

To be specific, gcc steering committee and glibc FSF stewards have 
announced the decision for their projects, Nick announced for binutils 
that he supports moving to LF IT (with the caveat that he won't abandon 
sourceware, I assume that means he'd like to use sourceware as a mirror 
or something similar) but gdb folks have been silent so far.  Given how 
gdb and binutils are coupled, the gdb conversation really needs to 
happen at some point.  From private conversations with folks from the 
gdb community, it seems to me that they're primarily avoiding getting 
into this public spat.

I am not aware of any opposition from maintainers of libabigail or 
cygwin or any other active sourceware based project over moving either, 
but I haven't had any links to those projects, so I may be uninformed.

> If you're trying to suggest that overseers, contrary to our repeated
> public statements, wish to block all migration, that is untrue and you
> will need to retract this.

Here's a more precise statement: Two of the overseers are leaders of 
projects hosted on sourceware and three overseers (including those two) 
have stated clearly on multiple occasions that transitioning to LF IT is 
off the table, effectively announcing their decision on behalf of 
projects they lead.  It is hence clear that the overseers have 
effectively blocked full migration of sourceware to LF IT.

Sid

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-23 12:07, Siddhesh Poyarekar via Overseers wrote:
> sourceware, I assume that means he'd like to use sourceware as a mirror 
> or something similar) but gdb folks have been silent so far.  Given how 
> gdb and binutils are coupled, the gdb conversation really needs to 
> happen at some point.  From private conversations with folks from the 
> gdb community, it seems to me that they're primarily avoiding getting 
> into this public spat.

... and I've been corrected offline that a gdb GNU maintainer (Joel 
Brobecker) has indeed signed on to support the LF IT proposal and there 
have been no objections so far that I'm aware of, either publicly or 
privately.

Sid

Re: Toolchain Infrastructure project statement of support

[Jeff Law]

On 10/23/22 10:07, Siddhesh Poyarekar wrote:

>> If you're trying to suggest that overseers, contrary to our repeated
>> public statements, wish to block all migration, that is untrue and you
>> will need to retract this.
>
> Here's a more precise statement: Two of the overseers are leaders of 
> projects hosted on sourceware and three overseers (including those 
> two) have stated clearly on multiple occasions that transitioning to 
> LF IT is off the table, effectively announcing their decision on 
> behalf of projects they lead.  It is hence clear that the overseers 
> have effectively blocked full migration of sourceware to LF IT.

They can make those decisions for the projects they lead.  But making 
the decision or setting criteria for other projects is highly unreasonable.

Jeff

Re: Toolchain Infrastructure project statement of support

[Christopher Faylor]

On Sun, Oct 23, 2022 at 11:01:34AM -0600, Jeff Law wrote:
>On 10/23/22 10:07, Siddhesh Poyarekar wrote:
>>>If you're trying to suggest that overseers, contrary to our repeated
>>>public statements, wish to block all migration, that is untrue and you
>>>will need to retract this.
>>
>>Here's a more precise statement: Two of the overseers are leaders of
>>projects hosted on sourceware and three overseers (including those two)
>>have stated clearly on multiple occasions that transitioning to LF IT
>>is off the table, effectively announcing their decision on behalf of
>>projects they lead.  It is hence clear that the overseers have
>>effectively blocked full migration of sourceware to LF IT.
>
>They can make those decisions for the projects they lead.  But making
>the decision or setting criteria for other projects is highly
>unreasonable.

This is not, IMO, helping.

On Thu, Oct 13, 2022 at 02:25:29PM -0400, Christopher Faylor wrote:
>We'd like to assure the communities that, when and if any individual
>project formally expresses the decision of their developers to transfer
>their services, we'll endeavor to make the move as smooth as possible.
>Those projects that wish to stay will continue to receive the best
>services that the overseers can offer, with the ongoing assistance of
>Red Hat, the SFC, and, when relevant, the FSF tech team.

We can't help move anyone without first establishing some kind of
criteria.  The only reasonable criteria is a formal request from the
project being moved.

As an exercise in human psychology, these insinuations of anticipated
unhelpfulness *can* eventually be a self-fullfilling prophecy, though.

i.e., if you really do not *want* any help with any transitions of
projects then, just keep implying, despite evidence to the contrary,
that we might be unreasonable jerks.

Re: Toolchain Infrastructure project statement of support

[Frank Ch. Eigler]

Hi -

> [...]  To be specific, gcc steering committee and glibc FSF stewards
> have announced the decision for their projects [...]

I may be missing something.  All I've seen so far were some of the
leaders of some of the projects being joint signatories to a letter on
overseers@.  As far as I'm aware, that is not how any of these
projects make or announce **project decisions** with/to their
respective constituencies.


> I am not aware of any opposition from maintainers of libabigail or
> cygwin or any other active sourceware based project over moving
> either, but I haven't had any links to those projects, so I may be
> uninformed.

Indeed.  The onus is obviously on the shoulders of the proponents of
this proposal to convince each sourceware guest project to consent to
move.  If you wish to frame any dissent as "blocking full migration",
then I'd say the job of convincing everyone just has not been up to
par.  Perhaps it was the wrong goal all along.


- FChE

Re: Toolchain Infrastructure project statement of support

[Jeff Law]

On 10/23/22 11:09, Frank Ch. Eigler via Libc-alpha wrote:
> Hi -
>
>> [...]  To be specific, gcc steering committee and glibc FSF stewards
>> have announced the decision for their projects [...]
> I may be missing something.  All I've seen so far were some of the
> leaders of some of the projects being joint signatories to a letter on
> overseers@.  As far as I'm aware, that is not how any of these
> projects make or announce **project decisions** with/to their
> respective constituencies.

Right.  It's a general statement of support from a variety of leaders 
but I wouldn't consider it authoritative from any project.


For GCC the decision would be made by the overseers and relayed to the 
overseers as an official statement for the GCC project. That would 
happen after a vote by the steering committee members based on already 
established voting procedures.


>
>
>> I am not aware of any opposition from maintainers of libabigail or
>> cygwin or any other active sourceware based project over moving
>> either, but I haven't had any links to those projects, so I may be
>> uninformed.
> Indeed.  The onus is obviously on the shoulders of the proponents of
> this proposal to convince each sourceware guest project to consent to
> move.  If you wish to frame any dissent as "blocking full migration",
> then I'd say the job of convincing everyone just has not been up to
> par.  Perhaps it was the wrong goal all along.

Also agreed.  And I fully support needing a statement from project 
leadership for each project wishing to move.    That is reasonable and I 
wish we'd been clearer about that.

In simplest terms, overseers need  to be a neutral party here.

In cases where overseers have leadership roles on particular projects, 
then they will have to wear multiple hats, but it's not really 
complicated.  Each project makes a decision, by whatever means project 
leadership has in place to make decisions. overseers then honors those 
requests to stay or go.  It's a pretty simple separation of 
responsibilities.  It need not be contentious in any way shape or form.


Jeff

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-23 13:09, Frank Ch. Eigler wrote:
>> [...]  To be specific, gcc steering committee and glibc FSF stewards
>> have announced the decision for their projects [...]
> 
> I may be missing something.  All I've seen so far were some of the
> leaders of some of the projects being joint signatories to a letter on
> overseers@.  As far as I'm aware, that is not how any of these
> projects make or announce **project decisions** with/to their
> respective constituencies.

Fair enough.

>> I am not aware of any opposition from maintainers of libabigail or
>> cygwin or any other active sourceware based project over moving
>> either, but I haven't had any links to those projects, so I may be
>> uninformed.
> 
> Indeed.  The onus is obviously on the shoulders of the proponents of
> this proposal to convince each sourceware guest project to consent to
> move.  If you wish to frame any dissent as "blocking full migration",
> then I'd say the job of convincing everyone just has not been up to
> par.  Perhaps it was the wrong goal all along.

Are you saying then that your dissent so far is as maintainer for 
systemtap and not as an overseer?

Sid

Re: Toolchain Infrastructure project statement of support

[Corinna Vinschen]

On Oct 23 12:07, Siddhesh Poyarekar via Overseers wrote:
> I am not aware of any opposition from maintainers of libabigail or cygwin or
> any other active sourceware based project over moving either, but I haven't
> had any links to those projects, so I may be uninformed.

For the time being, as the project lead, I hearby declare that Cygwin
and the cygwin.com domain wants to stay on Sourceware.  We have exactly
what we need on sware, so I don't see any reason to move infrastructure
to another entity, which we, in fact, don't really know much about, and
which may just add additional red tape.

"For the time being", because I didn't take the prospect of moving sware
in its entirety to some 3-rd part service really serious.  I still have
to discuss this with other Cygwin maintainers, but I have a hard time
to see that there's any real advantage for Cygwin.


Corinna

Re: Toolchain Infrastructure project statement of support

[Christopher Faylor]

On Thu, Oct 13, 2022 at 02:25:29PM -0400, Christopher Faylor wrote:
>Re: https://sourceware.org/pipermail/overseers/2022q4/018981.html
>
>On Wed, Oct 12, 2022 at 12:43:09PM -0400, Carlos O'Donell wrote:
>>The GNU Toolchain project leadership supports the proposal[1] to move the
>>services for the GNU Toolchain to the Linux Foundation IT under the auspices of
>>the Toolchain Infrastructure project (GTI) with fiscal sponsorship from the
>>OpenSSF and other major donors.
>
>Noted, however, a list of signatories does not automatically confer
>authority over any particular project.  Any participation from
>overseers in moving projects to different infrastructure will require
>clear approval from the individual projects themselves.
>
>Also, the FSF, being the existing fiscal sponsor to these projects,
>surely needs to review the formal agreements before we sunset our
>infrastructural offerings to glibc, gcc, binutils, and gdb and hand
>control of the projects' infrastructure over to a different entity.
>
>We'd like to assure the communities that, when and if any individual
>project formally expresses the decision of their developers to transfer
>their services, we'll endeavor to make the move as smooth as possible.
>Those projects that wish to stay will continue to receive the best
>services that the overseers can offer, with the ongoing assistance of
>Red Hat, the SFC, and, when relevant, the FSF tech team.

On Sun, Oct 23, 2022 at 09:27:26AM -0400, Siddhesh Poyarekar wrote:
>Given that the current sourceware admins have decided to block migration of
>all sourceware assets to the LF IT, I don't have a stake on how they'd like
>to handle this for sourceware.  I could however, as a member of TAC (and as
>member of projects that have agreed to migrate to LF IT, i.e. gcc and glibc),
>discuss with others the possibility of specific community volunteers being
>given some amount of access to manage infrastructure.

Stop spreading FUD.  The "we" in my statement above, from October 13,
included fche, mjw, and myself.  You have no reason to be confused on
this subject.

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-23 16:57, Christopher Faylor wrote:
> On Thu, Oct 13, 2022 at 02:25:29PM -0400, Christopher Faylor wrote:
>> Re: https://sourceware.org/pipermail/overseers/2022q4/018981.html
>>
>> On Wed, Oct 12, 2022 at 12:43:09PM -0400, Carlos O'Donell wrote:
>>> The GNU Toolchain project leadership supports the proposal[1] to move the
>>> services for the GNU Toolchain to the Linux Foundation IT under the auspices of
>>> the Toolchain Infrastructure project (GTI) with fiscal sponsorship from the
>>> OpenSSF and other major donors.
>>
>> Noted, however, a list of signatories does not automatically confer
>> authority over any particular project.  Any participation from
>> overseers in moving projects to different infrastructure will require
>> clear approval from the individual projects themselves.
>>
>> Also, the FSF, being the existing fiscal sponsor to these projects,
>> surely needs to review the formal agreements before we sunset our
>> infrastructural offerings to glibc, gcc, binutils, and gdb and hand
>> control of the projects' infrastructure over to a different entity.
>>
>> We'd like to assure the communities that, when and if any individual
>> project formally expresses the decision of their developers to transfer
>> their services, we'll endeavor to make the move as smooth as possible.
>> Those projects that wish to stay will continue to receive the best
>> services that the overseers can offer, with the ongoing assistance of
>> Red Hat, the SFC, and, when relevant, the FSF tech team.
> 
> On Sun, Oct 23, 2022 at 09:27:26AM -0400, Siddhesh Poyarekar wrote:
>> Given that the current sourceware admins have decided to block migration of
>> all sourceware assets to the LF IT, I don't have a stake on how they'd like
>> to handle this for sourceware.  I could however, as a member of TAC (and as
>> member of projects that have agreed to migrate to LF IT, i.e. gcc and glibc),
>> discuss with others the possibility of specific community volunteers being
>> given some amount of access to manage infrastructure.
> 
> Stop spreading FUD.  The "we" in my statement above, from October 13,
> included fche, mjw, and myself.  You have no reason to be confused on
> this subject.
> 

Nope, I'm not spreading FUD, in fact that statement of yours is 
perfectly consistent with what I've said: the blocker at the moment is 
that the sourceware overseers have refused to hand over the server *in 
its entirety* to LF IT, not that any projects themselves have refused to 
move their services to LF IT.  I don't doubt that the overseers will 
help in smooth migration for projects that eventually state that they 
wish to move over.

Sid

Re: Toolchain Infrastructure project statement of support

[Christopher Faylor]

On Sun, Oct 23, 2022 at 05:17:40PM -0400, Siddhesh Poyarekar wrote:
>On 2022-10-23 16:57, Christopher Faylor wrote:
>> On Thu, Oct 13, 2022 at 02:25:29PM -0400, Christopher Faylor wrote:
>> > Re: https://sourceware.org/pipermail/overseers/2022q4/018981.html
>> > 
>> > On Wed, Oct 12, 2022 at 12:43:09PM -0400, Carlos O'Donell wrote:
>> > > The GNU Toolchain project leadership supports the proposal[1] to move the
>> > > services for the GNU Toolchain to the Linux Foundation IT under the auspices of
>> > > the Toolchain Infrastructure project (GTI) with fiscal sponsorship from the
>> > > OpenSSF and other major donors.
>> > 
>> > Noted, however, a list of signatories does not automatically confer
>> > authority over any particular project.  Any participation from
>> > overseers in moving projects to different infrastructure will require
>> > clear approval from the individual projects themselves.
>> > 
>> > Also, the FSF, being the existing fiscal sponsor to these projects,
>> > surely needs to review the formal agreements before we sunset our
>> > infrastructural offerings to glibc, gcc, binutils, and gdb and hand
>> > control of the projects' infrastructure over to a different entity.
>> > 
>> > We'd like to assure the communities that, when and if any individual
>> > project formally expresses the decision of their developers to transfer
>> > their services, we'll endeavor to make the move as smooth as possible.
>> > Those projects that wish to stay will continue to receive the best
>> > services that the overseers can offer, with the ongoing assistance of
>> > Red Hat, the SFC, and, when relevant, the FSF tech team.
>> 
>> On Sun, Oct 23, 2022 at 09:27:26AM -0400, Siddhesh Poyarekar wrote:
>> > Given that the current sourceware admins have decided to block migration of
>> > all sourceware assets to the LF IT, I don't have a stake on how they'd like
>> > to handle this for sourceware.  I could however, as a member of TAC (and as
>> > member of projects that have agreed to migrate to LF IT, i.e. gcc and glibc),
>> > discuss with others the possibility of specific community volunteers being
>> > given some amount of access to manage infrastructure.
>> 
>> Stop spreading FUD.  The "we" in my statement above, from October 13,
>> included fche, mjw, and myself.  You have no reason to be confused on
>> this subject.
>> 
>
>Nope, I'm not spreading FUD, in fact that statement of yours is perfectly
>consistent with what I've said: the blocker at the moment is that the
>sourceware overseers have refused to hand over the server *in its entirety*
>to LF IT, not that any projects themselves have refused to move their
>services to LF IT.  I don't doubt that the overseers will help in smooth
>migration for projects that eventually state that they wish to move over.

Your initial implication was that the unreasonable overseers would hold
all projects hostage on our current infrastructure.   Now you've "clarified"
that point by implying that we've been approached to transfer the server
"in its entirety" to the LF and have unreasonably refused.

Both of those are FUD.  You're either intentionally trying to muddy the
waters or you're just confused.  I'd submit that in either case you should
just think about shutting up.  You have no special authority to speak for
the GTI TAC and your increasingly hostile messages are not helping anyone.

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-23 17:59, Christopher Faylor wrote:
> On Sun, Oct 23, 2022 at 05:17:40PM -0400, Siddhesh Poyarekar wrote:
>> On 2022-10-23 16:57, Christopher Faylor wrote:
>>> On Thu, Oct 13, 2022 at 02:25:29PM -0400, Christopher Faylor wrote:
>>>> Re: https://sourceware.org/pipermail/overseers/2022q4/018981.html
>>>>
>>>> On Wed, Oct 12, 2022 at 12:43:09PM -0400, Carlos O'Donell wrote:
>>>>> The GNU Toolchain project leadership supports the proposal[1] to move the
>>>>> services for the GNU Toolchain to the Linux Foundation IT under the auspices of
>>>>> the Toolchain Infrastructure project (GTI) with fiscal sponsorship from the
>>>>> OpenSSF and other major donors.
>>>>
>>>> Noted, however, a list of signatories does not automatically confer
>>>> authority over any particular project.  Any participation from
>>>> overseers in moving projects to different infrastructure will require
>>>> clear approval from the individual projects themselves.
>>>>
>>>> Also, the FSF, being the existing fiscal sponsor to these projects,
>>>> surely needs to review the formal agreements before we sunset our
>>>> infrastructural offerings to glibc, gcc, binutils, and gdb and hand
>>>> control of the projects' infrastructure over to a different entity.
>>>>
>>>> We'd like to assure the communities that, when and if any individual
>>>> project formally expresses the decision of their developers to transfer
>>>> their services, we'll endeavor to make the move as smooth as possible.
>>>> Those projects that wish to stay will continue to receive the best
>>>> services that the overseers can offer, with the ongoing assistance of
>>>> Red Hat, the SFC, and, when relevant, the FSF tech team.
>>>
>>> On Sun, Oct 23, 2022 at 09:27:26AM -0400, Siddhesh Poyarekar wrote:
>>>> Given that the current sourceware admins have decided to block migration of
>>>> all sourceware assets to the LF IT, I don't have a stake on how they'd like
>>>> to handle this for sourceware.  I could however, as a member of TAC (and as
>>>> member of projects that have agreed to migrate to LF IT, i.e. gcc and glibc),
>>>> discuss with others the possibility of specific community volunteers being
>>>> given some amount of access to manage infrastructure.
>>>
>>> Stop spreading FUD.  The "we" in my statement above, from October 13,
>>> included fche, mjw, and myself.  You have no reason to be confused on
>>> this subject.
>>>
>>
>> Nope, I'm not spreading FUD, in fact that statement of yours is perfectly
>> consistent with what I've said: the blocker at the moment is that the
>> sourceware overseers have refused to hand over the server *in its entirety*
>> to LF IT, not that any projects themselves have refused to move their
>> services to LF IT.  I don't doubt that the overseers will help in smooth
>> migration for projects that eventually state that they wish to move over.
> 
> Your initial implication was that the unreasonable overseers would hold
> all projects hostage on our current infrastructure.   

Absolutely not, you and I have had multiple exchanges on this list so 
far and I'd have trusted you to take my statement above in the correct 
context.  I did not even negate your statement when you stated that the 
overseers would support seamless migration of services over to LF IT and 
in fact supplemented[1] by saying that the transition would likely take 
years.

> Now you've "clarified"
> that point by implying that we've been approached to transfer the server
> "in its entirety" to the LF and have unreasonably refused.

You literally have an email on the list with the subject like "Moving 
sourceware to the Linux Foundation? no thanks".

> Both of those are FUD.  You're either intentionally trying to muddy the
> waters or you're just confused.  I'd submit that in either case you should
> just think about shutting up.  You have no special authority to speak for
> the GTI TAC and your increasingly hostile messages are not helping anyone.

It's funny that you're asking me to shut up and also implying that my 
messages are hostile.

Sid

[1] https://sourceware.org/pipermail/overseers/2022q4/018987.html

Re: Toolchain Infrastructure project statement of support

[Ian Kelling]

Siddhesh Poyarekar via Overseers <overseers@sourceware.org> writes:

> I personally do not think the current sourceware infrastructure, even
> with the roadmap it promises is a viable alternative to what LF IT can
> provide.  There is a significant resource gap (e.g.
....
> established security and administration practices,
...
> that we seem to disagree about.


Let's consider some "established security and administration practices"

curl -v http://vger.kernel.org | head
...
< Server: Apache/2.0.52 (Red Hat)
< X-Powered-By: PHP/4.3.9

This is RHEL 3, released in 2003, according to
https://people.redhat.com/crunge/RHEL3-package-lists.pdf,

The final end of support for this distro was on 2014-01-30.

There are CVE's for that version of Apache. I assume their apache is not
running in a configuration that makes them actually exploitable, but it
is still better security practice upgrade.

The kernel is likely from RHEL 3 too. I'm reminded of Greg KH beating the
drum that old kernels need upgrades for security, especially because the
kernel devs don't always check if a bug is a security issue and
especially not for really old kernels (
https://thenewstack.io/design-system-can-update-greg-kroah-hartman-linux-security/
)

Notice that link is http because https is not supported by the apache
from 2003. Linux kernel development works through patches on mailing
lists, and how do you find the patches if you aren't already subscribed
to a list? You'd naturally go to the lists main webpage,
http://vger.kernel.org, and click "LIST INFO",
http://vger.kernel.org/vger-lists.html, and then click one of the list
archive links, or maybe the subscribe link. So, those vger.kerne.org
pages are an essential part of retrieving upstream kernel patches and
security information for some people. And being http only, my isp or
anyone in my network path could alter them to be malicious urls that
that appear to give the correct result, but actually give malicious
kernel patches, or hides away a security relevant patch. Obviously,
https for security sensitive pages like these is a basic 101 security
practice in 2022.

You might think when kernel.org had a major compromise in 2011, 11 years
later, they would have managed this basic upgrade. The fact is that the
Linux Foundation struggles with getting stuff to current versions and
following good security practices like everyone else does. This
narrative that there is a huge resource gap in security practices
between LF and sourceware is not true, and I don't think the kernel.org
IT team would claim that either. They certainly made no such claims in
their slide deck about the GTI proposal.

If LF IT were to get involved in services for GNU toolchain packages, it
should be more of a collaboration with sourceware instead of taking over
what sourceware is doing.

Competent sysadmin volunteers are rare and valuable to GNU. They help
build community, they help GNU stay independent, and they help GNU
practice what it preaches.

-- 
Ian Kelling | Senior Systems Administrator, Free Software Foundation
GPG Key: B125 F60B 7B28 7FF6 A2B7  DF8F 170A F0E2 9542 95DF
https://fsf.org | https://gnu.org

Re: Toolchain Infrastructure project statement of support

[Siddhesh Poyarekar]

On 2022-10-23 07:33, Ian Kelling wrote:
> Siddhesh Poyarekar via Overseers <overseers@sourceware.org> writes:
> 
>> I personally do not think the current sourceware infrastructure, even
>> with the roadmap it promises is a viable alternative to what LF IT can
>> provide.  There is a significant resource gap (e.g.
> ....
>> established security and administration practices,
> ...
>> that we seem to disagree about.
> 
> 
> Let's consider some "established security and administration practices"
> 
> curl -v http://vger.kernel.org | head
> ...
> < Server: Apache/2.0.52 (Red Hat)
> < X-Powered-By: PHP/4.3.9
> 
> This is RHEL 3, released in 2003, according to
> https://people.redhat.com/crunge/RHEL3-package-lists.pdf,
> 
> The final end of support for this distro was on 2014-01-30.
> 
> There are CVE's for that version of Apache. I assume their apache is not
> running in a configuration that makes them actually exploitable, but it
> is still better security practice upgrade.
> 
> The kernel is likely from RHEL 3 too. I'm reminded of Greg KH beating the
> drum that old kernels need upgrades for security, especially because the
> kernel devs don't always check if a bug is a security issue and
> especially not for really old kernels (
> https://thenewstack.io/design-system-can-update-greg-kroah-hartman-linux-security/
> )
> 
> Notice that link is http because https is not supported by the apache
> from 2003. Linux kernel development works through patches on mailing
> lists, and how do you find the patches if you aren't already subscribed
> to a list? You'd naturally go to the lists main webpage,
> http://vger.kernel.org, and click "LIST INFO",
> http://vger.kernel.org/vger-lists.html, and then click one of the list
> archive links, or maybe the subscribe link. So, those vger.kerne.org
> pages are an essential part of retrieving upstream kernel patches and
> security information for some people. And being http only, my isp or
> anyone in my network path could alter them to be malicious urls that
> that appear to give the correct result, but actually give malicious
> kernel patches, or hides away a security relevant patch. Obviously,
> https for security sensitive pages like these is a basic 101 security
> practice in 2022.

+Konstantin from LF IT since he's better equipped to speak to this, 
although ISTM that they started migrating off vger last year[1].

Sid

[1] https://www.kernel.org/lists-linux-dev.html

Sid

Re: Toolchain Infrastructure project statement of support

[Konstantin Ryabitsev]

On Sun, Oct 23, 2022 at 12:17:36PM -0400, Siddhesh Poyarekar wrote:
> > Let's consider some "established security and administration practices"
> > 
> > curl -v http://vger.kernel.org | head

These are all very fair observations, with one important caveat --
vger.kernel.org is the last remaining piece of kernel.org infrastructure that
is not managed by our team. It's been historically maintained by volunteers
that are not associated with the Linux Foundation (vger used to be hosted at
Red Hat, iirc).

> +Konstantin from LF IT since he's better equipped to speak to this, although
> ISTM that they started migrating off vger last year[1].

Correct, everything is ready for the migration on our end, but we cannot
unilaterally initiate the process. The mail server maintained by the Linux
Foundation is at subspace.kernel.org. I invite you to poke at it to see if it
fares any better compared to vger.

Best wishes,
Konstantin

Re: Toolchain Infrastructure project statement of support

[Alexandre Oliva]

On Oct 12, 2022, "Carlos O'Donell via Overseers" <overseers@sourceware.org> wrote:

> The GNU Toolchain project leadership

Is GNU Toolchain the name of a project?  This term has usually meant a
set of packages that are part of the GNU Project.  Each package has its
own set of maintainers appointed by GNU leadership, each one with a
commitment to advance the goals of the GNU Project through their
maintainership.  GNU Project leadership hasn't been consulted, and the
proposed move appears to be detrimental to the GNU Project, so it
follows that these people are not acting in their capacity of GNU
maintainers.  As such, any kind of leadership and decision-making
arising from these roles must be presumed void and null.

Speaking specifically of the GCC Steering Committee, I quote from the
very first sentence in https://gcc.gnu.org/steering.html

  The steering committee was founded in 1998 with the intent of
  *preventing* *any* particular individual, group or *organization* from
  *getting* *control* *over* *the* *project*.

(highlights are mine)

Turning over control over critical project infrastructure, in a way that
seemingly places the packages under the umbrella of that single
organization, is not in line with the existential reason of the GCC
Steering Committee.  As such, it must follow that the undersigners are
not acting in their capacity of GCC Steering Committee members, which
removes any decision-making weight that might have otherwise been
presumed from them, due to expectations associated with these roles.


That said, I acknowledge that several of them are prominent contributors
to GCC, and the GNU Project is happy to welcome contributions from
anyone willing to help advance its goals (please do not mistake this as
speaking on behalf of the GNU project; I merely state GNU policies and
positions to the extend that I'm familiar with and understand them).
Several others were relevant developers, as I and RMS have been for
quite some time in GNU libc and GCC, respectively.  It's interesting to
see that past contributors can have a voice, but it appears to me that
there has been some bias in the selection of which past voices to
actively seek out, and which to dismiss, that places the legitimacy of
the pronouncement under further doubt.


Anyhow, FWIW, I hereby raise my objection to the proposed outright move
of our infrastructure out of Sourceware and into LF.  I have a different
proposal that is more in line with GNU policies, that GNU maintainers
ought to uphold, and with the stated goals of the GCC SC quoted above.

I wish to be clear on GNU policy of welcoming contributions that help
advance its goals.  I, as contributor to the affected packages, and as
long-time GNU contributor, do not wish GNU to turn down the resources
offered as part of this proposal, any more than I wish GNU to turn down
the resources that have long been made available to us by Sourceware.
There are significant upsides to using both, to increase our autonomy,
rather than jumping from one single point of corporate dependence (AKA
autonomy failure :-) to another.

It seems fine to me, and also welcome, that groups of contributors pool
their individual efforts towards finding resources for the GNU packages
they wish to support.

I have no objection in principle to our relying on these resources, be
they financial or infrastructure, to advance our projects, even if the
governance model of each supporting group is independent from and
unrelated to the governance structure of GNU.  After all, we have no say
in Red Hat's internal governance structure and funding of Sourceware,
and that hasn't stopped and IMHO shouldn't stop us from accepting that
contribution either.  So it shouldn't stop us from accepting
contributions from the LF, even if we have no say in its governance
structure or in the composition of the governing board of the
organization or of the brands and subgroups that would affect directly
their offer to our packages.

As long as the offered resources advance our goals, they are welcome.  I
don't see that an outright move does, but I do see that increased
autonomy and robustness through replication would.

Should any such supporting group at any point in the future take actions
that conflict with our goal, the GNU Project may then turn them down,
and then, as long as critical project infrastructure doesn't make us
hostage, we will be no worse off than if the contributions had never
been offered in the first place, and presumably even a little ahead
because of already-accepted past contributions.

It seems very important, however, that we take steps to guard our own
autonomy, so as to avoid the risks of finding ourselves in such a
hostage situation or of being strongarmed into behaviors we'd not engage
in out of our own volition, and even of having our governance structures
confused, subverted or replaced by misperceptions about the nature of
these supporting structures.

To this end, it appears to me that the reliance on multiple supporting
groups, each with their own independent governance structures, is a
strength we should embrace, by keeping critical package resources
replicated across multiple such groups, instead of placing all eggs in a
single basket.

I urge responsible maintainers of all encompassed projects to take this
possibility into account, and also, while planning a transition
hopefully into a multi-party infrastructure arrangements, to see what
another organization that has long supported the GNU Project, namely the
FSF, has to offer, and rely on its resources as well.

I'd also encourage people involved in this initiative to look into
directing part of the raised funds towards paying *other* organizations
for infrastructure services, in addition to the LF itself.  I expect the
FSF could and would welcome this, and, just as there are contributors
and users who trust the LF better than the FSF, there are others who
trust the FSF better than the LF, and having trustworthy parties
involved in offering replicated/redundant infrastructure would likely
make everyone more comfortable.

I'd perceive this decentralization as yet another positive development
towards increasing our autonomy and robustness towards preventing any
particular individual, group or organization from getting control over
the project.

Accepting multiple such contributions, we also preemptively dispell any
potential assumptions that the GNU packages themselves, or their
governance structures, are in any way under the authority of the
governance structures of any of the infrastructure suppliers or funding
channels.


Please find a few other points below.

> We believe that utilizing the experience and infrastructure of the LF IT team
> that already is used by the Linux kernel community will provide the most
> effective solution and best experience for the GNU Toolchain developer
> community.

I'm curious as to any facts that support this statement.  Without
risk-cost-benefit analyses of the actual offer, and comparison with at
least a few alternatives, it feels more like an expression of
enchantment for promises made by a politician during an electoral run
than anything one could count on.  Gratis offers are often
bait-and-switch or other kinds of traps, so caution is advisable when
accepting them.  Considering that, in the end, the decisions wouldn't
even be made by the proponents themselves, it doesn't smell good to me.

But then, again, unless we're silly enough as to jump in without safety
jackets, or we are dragged in by force, I have no objections to
accepting contributions from this supporting group on a tentative basis.
That means, at least at first, replicating rather than moving; rolling
out new services (with viable plans to replicate them) rather than
replacing existing ones.

So far, there's no clarity to me as to the proposed plans of moving
services and how this would impact or benefit us, which makes it
impossible to have any rational grounds for any decision whatsoever.  I
look forward to tomorrow's presentation about expected benefits, so that
we can reason about them and see how to retain them in the multi-party
infrastructure scenario I propose.  It doesn't smell good, however, that
Sourceware has been prevented from presenting its own expansion plans
and proposals at the Cauldron.  I wish it too gets a chance to extend
their offer.  There's no basis for a rational decision in refusing to
listen to alternatives; it comes across to me as acknowledgment that a
weaker proposal wishes to prevail by denying others any consideration.


> As with kernel.org, GTI has requested from the Linux Foundation IT that all
> software implementing the infrastructure for the GNU Toolchain projects must
> be Free Software.

That is nice.  What has LF IT responded?

AFAICT, it doesn't look like they took it to heart.  So far, not even
the infrastructure for meetings has been based on Free Software, and
that's not a very hopeful sign.


Still, it's worth keeping in mind that, when it comes to infrastructure
offered by third parties, the use of Free Software benefits the
infrastructure providers, but as for the freedom of users of that
infrastructure, both as people and as packages or projects, the
arrangement is a Service as a Software Substitute, SaaSS, which is in
most freedom-related aspects even worse than nonfree software.
https://www.gnu.org/philosophy/who-does-that-server-really-serve.en.html


Outsourcing key infrastructure (as opposed to relying on community
members to maintain it) creates a detrimental power dynamic, that
requires working harder to maintain a group's autonomy.  Relying
exclusively on Free Software alleviates some of the extra difficulties,
but because of the SaaSS dynamics, it does not solve them entirely.
Thus my recommendation to pursue multiple redundant infrastructure
providers, for the sake of our autonomy.

> If Free Software versions of necessary security tools are missing, GTI
> will work with Free Software supporters to develop Free Software
> alternatives.

There are no "necessary security tools" for GNU that are nonfree, but an
implication of this phrasing is that, if any are found or framed as
such, they'd end up being used, because of "necessary", until an
alternative is developed and readied for use.

Still worrysome is that OpenSSF policies disfavor strong copyleft, so
even if such tools end up being developed and released as Free Software,
GNU might find itself relying on non-copyleft, push-over licenses.
While these are not unethical, associating ourselves with an
organization that has an anti-copyleft bias raises quite some red flags
for me.  Could your supporting infrastructure initiative negotiate some
arrangement with the OpenSSF so that any "necessary security tools"
developed in responses to our needs (even if addressing others' needs as
well) be released under strong copyleft, such as GNU GPLv3+?  Could they
be contributed to the GNU Project?


Thanks for listening,

-- 
Alexandre Oliva, happy hacker                https://FSFLA.org/blogs/lxo/
   Free Software Activist                       GNU Toolchain Engineer
Disinformation flourishes because many people care deeply about injustice
but very few check the facts.  Ask me about <https://stallmansupport.org>

Re: Toolchain Infrastructure project statement of support

[Christopher Faylor]

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv

On Sun, Oct 23, 2022 at 06:19:33PM -0300, Alexandre Oliva wrote:
>It doesn't smell good, however, that Sourceware has been prevented from presenting its own
>expansion plans and proposals at the Cauldron.  I wish it too gets a chance to extend their offer.
>There's no basis for a rational decision in refusing to listen to alternatives; it comes across to
>me as acknowledgment that a weaker proposal wishes to prevail by denying others any consideration.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^