Re: Role of sourceware for hosted projects

[Mark Wielaard <mark@klomp.org>]

Hi Paolo,

On Fri, Sep 23, 2022 at 04:04:12PM +0200, Paolo Bonzini via Overseers wrote:
> I have recently come across the discussions at Cauldron about Sourceware, by
> means of an article on LWN.net.

Which is 
https://lwn.net/SubscriberLink/908638/567de0001d86662c/

And I should really apologize for how I handled that BoF. I had
intended to show the infrastructure work the community had done this
last year and discuss the policies different projects could/had set to
use the new services, what had worked, what didn't work. Working up to
how the community could move these Sourceware infrastructure services
into the future with the Conservancy member proposal as a bridge to
the LF/GTI plan. For which we then also had the second BoF hour as
overflow if people wanted to discuss that even more. But clearly I
lost total control of the BoF. I thought I had prepared for that, to
make sure all discussion topics got enough time, but clearly hadn't
prepared enough.

> Even though I have not been a contributor
> to Sourceware projects for several years, they are still dear to my heart
> and I am familiar with several of them. Therefore, I would like to share my
> own observations about what sourceware.org can or should provide to the
> project it hosts.

Thanks.

> First of all, a few obligatory pieces of disclosure.  First, I am employed
> by Red Hat but not for anything related to the GNU Compiler Collection or
> any other Sourceware project.  Second, I was not at Cauldron and have only
> watched the online recording of the first hour of the BoFs; I trust the
> LWN.net editors and their article on the topic to provide a faithful
> account.  Third, even though I have discussed this with some of the people
> involved in the BoFs, that hasn't substantially changed my understanding of
> the situation as it is reflected below.

And you have relevant background as one of the Qemu Conservancy PLC
members. Also I would love to hear about patchew.org.

For those only having attended the Cauldron BoF or having just read
the lwn article maybe some of this background information helps a bit
with understanding where in the process we are:

- Sourceware roadmap discussions
  https://sourceware.org/pipermail/overseers/2022q2/018453.html
  https://sourceware.org/pipermail/overseers/2022q2/018529.html
  https://sourceware.org/pipermail/overseers/2022q3/018636.html
  https://sourceware.org/pipermail/overseers/2022q3/018716.html
- Sourceware as Conservancy member project proposal
  https://sourceware.org/pipermail/overseers/2022q3/018802.html
- Full Sourceware SFC application text
  https://sourceware.org/pipermail/overseers/2022q3/018804.html
- Public SFC video chat meeting notes
  https://sourceware.org/pipermail/overseers/2022q3/018837.html
- Statement from Zoë Kooyman, Executive director FSF
  https://sourceware.org/pipermail/overseers/2022q3/018843.html

> The obvious observation from the outside is that the two "opposing sides"
> (for lack of a better word)

Note that I am really unhappy this is seen as "opposing sides" or
different "visions". I really hope we can work this out as a community
with some shared values. Sourceware being a Conservancy member project
really shouldn't be incompatible with finding sponsors through the
Linux Foundation. And I hope we can also work out what might make
sense as a managed service (I really like the BBB idea as something we
can try out together).

> have different priorities on what Sourceware
> needs to provide for them.  There are several reasons for this: different
> projects that people work on, different emotional attachment, different
> jobs, whatnot.  However, both of them make the same mistake: they focus on
> the "next steps" without a full assessment of the *current* state of
> Sourceware.

I think we did do an assessment of the current state and the next step
really just is setting up a fiscal sponsor without any impact. Once we
have a PLC we can start thinking about the next steps. People have
ideas, and I think we really should take the security issues seriously
once we really understand them. First priority is making sure there is
no disruption to the projects and to make sure that we are "disaster
proof". That doesn't mean we shouldn't be a bit more ambitious, but
lets first make sure we have a solid fiscal sponsor and governance
structure.

> The first part of the assessment in my opinion should be that most projects
> on sourceware.org are dead.

Correct. We have 40 projects that have stopped being developed or
moved to other hosting services. We really should work with the
Software Heritage and maybe archive.org to get them properly archived.

> if I counted right, roughly ten projects that are alive: gcc, gdb,
> binutils and glibc ("the GNU toolchain"); elfutils, systemtap, cygwin/newlib
> and libabigail ("smaller projects"); and others that are alive but barely so
> (debugedit, dwz, etc.).  Also, most projects outside the GNU toolchain have
> only one main developer.

The projects page could use an update. There are 26 alive projects (we
just added one this week), plus a couple of "meta" projects (basically
for the services like builder.sourceware.org which is its own
community). Some are much more active than others, and some just use a
few services like having a mailinglist or git repo or bugzilla, but we
do find the "long tail" also important.

> If the bigger projects
> are worried about supply chain attacks, to begin with they can very easily
> migrate their source code control repositories elsewhere.  It could be
> operated by LF staff, or could be a "forge" like Gitlab/GitHub; several
> projects use the latter purely for hosting while keeping development on
> mailing lists.

I think a higher priority is to have good mirrors. We already have a
sourcehut mirror at https://sr.ht/~sourceware/ it would be great to
have some alternative mirrors, maybe LF/IT could set those up. Also
now that we have a public-inbox instance we could have the same for
the patches/mailinglists. With that and possibly commit signing and
patch attestation you can always verify the "supply chain" even when
one of the hosts is compromised since all verification is/can be done
against any mirror or your local copy.

> They don't even have to do the same thing between the four
> of them, though it would make sense for at least binutils, gcc and gdb to do
> so. I am rusty on how these three projects do release management, but
> perhaps they could even use a monorepo with per-project release branches,
> and tarballs that only include the relevant directories.

binutils, gdb and sim use a mono-repo, but do separate releases and
even have separate contribution policies. They do share libiberty but
not as a shared repo. And some use gnulib, but with different update
policies. It isn't completely clear if all of them have even heard of
the LF/GTI proposal yet, we have sent email to the various steering
committees to figure out where they all stand on this.

> That's all.  I admit I am not necessarily up to date on how Sourceware
> operates, so I apologize in advance for any incorrect assumptions I made.
> Apart from that, I hope that this writeup can provide useful ideas, or even
> a path forward for both Sourceware and the GNU toolchain.

Thanks, it was interesting. I am not sure it really fits with the
current roadmap and making sure projects don't need to migrate away
from sourceware. But if projects want to migrate some of their
services you do list some interesting options.

Cheers,

Mark