Core Toolchain Infrastructure - Services for glibc

Core Toolchain Infrastructure - Services for glibc

[Carlos O'Donell]

The Core Toolchain Infrastructure (renamed) project continues to
move forward the goal of creating a long-term sustainable set of
secure and state of the art services for the GNU Toolchain.

Some of the major goals include:

 * Isolating all services in VMs or containers to increase service
   security and reduce service resource interference. 

 * Allow volunteers to focus efforts outside of core infrastructure
   maintenance.

 * Prepare for additional software supply chain requirements from
   distributors.

As part of this effort the Technical Advisory Committee (TAC) has
evaluated the services used by glibc and how they could be migrated
and serviced by the CTI project.

The CTI TAC recommendation is to use Linux Foundation IT services
for core infrastructure. The LF IT team already supports many of
the same services for the Linux kernel and at scale. The migration
would involve moving services from Sourceware.org to LF IT servers.
We continue to be thankful and appreciative of the time spent by
Sourceware.org volunteers in support of the current services.

Appended to this message are the list of proposed services that
LF IT can provide and an enumeration of the current services.

The CTI TAC is looking for feedback from the community on the listed
services which would be migrated to LF IT services and overall
project service requirements. Please feel free to reply to this
thread or email me, or any CTI TAC member privately.

After the glibc release early in August the CTI project will be
taking the next step of asking the glibc stewards (GNU Maintainers)
to review the proposal and make a decision to move forward with the
migration.

Cheers,
Carlos O’Donell
CTI TAC Member

The following is the suggested list of services to be migrated (with notes):

* Mailing lists
 * Support public-inbox for mailing list archives.
 * Use of public-inbox means archives can be cloned and copied.
 * Use of LF IT Subspace mailing list services (mlmmj, postfix).

* bug database
 * Consider starting fresh in new Bugzilla 5.0.4+ instance and freeze old product.
 * glibc component in sourceware instance marked "Not open for new bugs."
 * No easy way to clone this but we can discuss options.
 * Isolate bugzilla from other services.

* git
 * Migrate to gitolite
 * Community manages access via gitolite keys.
 * Minimize all access to sources and isolate from other processes.
 * Minimal server side web hooks where required.
 * Isolate git service from other services.
 * Stop supporting svn/cvs and provide tarball dumps.

* wiki
 * Migrate to git-based documentation with existing content copied over.
 * Suggest rst/Sphinx or similar to existing discussions for GCC docs.
 * Sphinx with themes can provide a lot of flexibility for display.
 * Isolate wiki service from other services.

* patch management
 * Continue patchwork usage and maintenance of isolated instance
 * Required for community driven pre-commit CI
 * LF IT hosting patchwork instance with community hosting bots.
 * Isolate patchwork from other services.

* Website
 * Provide a simple static site.
 * Isolate web hosting from other services.

* Meeting
 * Already migrated away from proprietary solutions.
 * Continue to use LF IT BBB instance for glibc meetings including weekly patch review.
 * Isolate BBB from other services.

The current list of glibc services were put together as part of the
CTI TAC glibc service enumeration:

* mailing lists
  * Mailman 2 mailing lists: https://sourceware.org/mailman/listinfo/*
    * libc-announce
    * libc-alpha
    * libc-stable
    * libc-help
    * libc-locales
    * libc-testresults
    * glibc-cvs
    * glibc-bugs
    * glibc-bugs-regex (limited bugs just for regex).
    * Closed legacy mailing lists:
      * libc-ports: https://sourceware.org/mailman/listinfo/libc-ports
      * libc-hacker: https://sourceware.org/mailman/listinfo/libc-hacker
    * Mailing lists accept non-html email only.
    * Run through spamassasin
    * Run through clamav
  * Pipermail archives:
    * https://sourceware.org/pipermail/*
    * e.g. https://sourceware.org/pipermail/libc-alpha/
  * public-inbox archives:
    * https://inbox.sourceware.org/*
    * e.g. https://inbox.sourceware.org/libc-alpha/
  * MHonArc archives (/legacy-ml/) - no longer updated
    * The /legacy-ml/ URLs and /ml/ redirects to them need to keep working.
    * E.g. https://sourceware.org/legacy-ml/libc-alpha/2020-01/

* bugzilla 5.0.4+
  * Uses backend SQL database of MariaDB 10.3
  * Must be able to send email to glibc-bugs mailing list.
    * Don't know how email is routed to this list.
  * Must also send email glibc-bugs-regex mailing list.
    * Don't know how email is routed to this list.
  * Must be able to send email to all users on the bug.
  * Must be able to receive email when someone responds to a glibc-bugs
    email e.g. sourceware-bugzilla@sourceware.org.
  * Custom Administration->Groups settings for User RegExp.
    * canconfirm: Allow certain domains to always be able to confirm bugs.
    * editbugs: Likewise but for editbugs.
  * Must have REST API enabled to allow RM to generate release list
    of fixed bugs using the glibc/scripts/list-fixed-bugs.py script
    e.g. https://sourceware.org/bugzilla/rest.cgi/
    * Implies that non-logged-in users can list and view all bugs
      that were fixed for the release.
  * Must have account creation disabled due to spamming.
  * Must have someone with Bugzilla admin access to:
    * Add new users to bugzilla.
    * Add new Product components, versions, and milestones.
    * Add new Key Words
    * Remove users.

* git 2.31
  * Allows per-user access to commit to the glibc repo.
  * Allows per-user access to commit to the legacy glibc-ports repo.
  * Uses group access to control repository access.
  * Must be able to send email to glibc-cvs mailing list with one
    email for each commit made by a developer to any branch of the repository.
  * AdaCore hooks need more thorough audit for required services.
    * Must be able to send email to bugzilla to update bugs.
      * Done by AdaCore hook 'file-commit-cmd'
      * Configured to use email-to-bugzilla-filtered command.
        * Uses connection to SQL database to determine if bug exists.
  * Currently uses shared AdaCore hooks configured via origin/meta/config 
    * Active hooks:
      * post-receive
        * AdaCore post_receive
        * /git/glibc.git/hooks-bin/post-receive
          * Triggers irkerhook.py (see notes below).
          * Does not work today, likely due to requirement to register OFTC user.
      * post-update
	* Standard git-update-server-info.
      * pre-receive
	* AdaCore pre_receive
    * AdaCore config:
      * No max line lengths.
      * Allow UTF-8 in commit messages.
      * 5MiB max email size.
      * Max 500 commit messages for larger commit series sent to glibc-cvs.
      * Reject merge commits to master and release branches.
      * Allow rebasing only private branches (non master and non release).
      * Run minimal style checker, nominally for whitespace issue rejection.
        * Run extra commit checking to avoid source address for author being wrong.
          * /git/glibc.git/hooks-bin/commit_checker
            * From email format checker. No special requirements.
        * /git/glibc.git/hooks-bin/style_checker
          * Style chcker. No special requirements.
      * Send email to bugzilla if a commit mentions a bug.
        * /git/glibc.git/hooks-bin/email-to-bugzilla-filtered
          * Uses /sourceware/infra/bin/email-to-bugzilla
          * Must be able to connect to bugzilla SQL database.
          * Does not appear to work today. We don't get emails for commits with bugs.
      * Send IRC message to per-project configured IRC channel.
        * Involves irkerhook.py and git config information for project.
        * Hook must be able to connect to external IRC networks to post IRC notices.

* wiki
  * Uses MoinMoin 1.9.10
  * Must have account creation disabled due to spamming.
    * Uses EditorGroup permissions to allow any community member to add a new
      community member to the wiki e.g. human vetting another human.
  * Must be able to send notification emails.
  * Cron run to purge users not in EditorGroup to prevent wiki slowdown.

* patch management.
  * Uses patchwork v3.1.1.post18-g11cf1f3
  * Must be able to receive email (as part of collecting patch data)
  * Must be able to send emails as part of account verification.
  * Uses django for administration
  * Must allow authenticated REST API access for patchwork.
    * Currently rate limited.
    * Used by SLI tools (Carlos O'Donell)
      * Run manually on developer systems.
    * Auto-close on commit patchwork bot (Siddhesh Poyarekar)
      * Run on sourceware.org via cron.
  * Used for weekly patch management meetings.
  * git-pw integration used to access patchwork directly using REST API and API token.

* Red Hat Bluejeans remote meeting system.
  * Must allow remote video and audio for participants around the world.
  * Allows weekly glibc patch review meetings for patch review collaboration.
  * Meetings must operate without host needing to be present so community can host.
    * Delegating host is difficult in bluejeans.
  * Managed by Bluejeans/Verizon.

* pre-commit CI system.
  https://gitlab.com/djdelorie/glibc-cicd
  * Run inside a VM.
  * Uses networkless containers for further build isolation.
  * Highest risk system because it runs mailing list posted patches.
  * Event curation system (curator):
    * Must have network access to patchwork REST API.
    * Must have access to SQL database for storing state.
      * Currently using MariaDB.
    * Must allow runners to access curatore REST API URL.
    * One curator currently hosted by DJ Delorie.
  * Event running system (runner + trybots):
    * Must have network access to curator REST API.
    * Must have local network access to rabbitmq queue (job delegation)
    * trybots must have local network access to rabbitmq.
      * Must have network access to patchwork REST API to post results.
      * Must have network access to container registries to pull modern containers.w
      * Must have network git access to pull updated glibc git repo.
    * Generally the runner and trybots are on one site together.
      * Avoid passing rabbitmq traffic beyond the local network.
      * Eventual emailing of results to the mailing list will happen via another bot
        that is distinct from this system to avoid the runners needing anything but
        restricted network access.
    * One runner hosted by DJ Delorie	
    * One i686 trybot hosted by DJ Delorie
    * One "patch applies" trybot hosted by DJ Delorie

* Website (sourceware.org)
  * CVS hosted website.
  * Static redirect to gnu.org website.

* Website (gnu.org)
  https://www.gnu.org/software/libc/
  * CVS hosted website uploads along with manual.
    * Manuals are generated with scripts in the CVS repo and generated files committed.
  * All static content.
  * Website automatically updated after CVS commits.
  * Manged by the GNU Project/FSF.

* Release tarballs (ftp upload of gpg-signed release tarballs)
  https://ftp.gnu.org/gnu/libc/
  * Use gnupload script to gpg sign uploaded tarballs.
   * Uses ncftpput to place files into /incoming directories.
   * Network ftp access required.
  * Managed by the GNU Project/FSF.

* Translation project services
  https://translationproject.org/html/welcome.html
  * https network access to TP servers to fetch uploaded translation files.
  * Managed by the Translation Project.

Re: Core Toolchain Infrastructure - Services for glibc

[Florian Weimer]

* Carlos O'Donell via Libc-alpha:

> * Mailing lists
>  * Support public-inbox for mailing list archives.
>  * Use of public-inbox means archives can be cloned and copied.
>  * Use of LF IT Subspace mailing list services (mlmmj, postfix).

I assume LF IT is able to run mailing lists without “via” From:
rewriting?

> * bug database
>  * Consider starting fresh in new Bugzilla 5.0.4+ instance and freeze old product.
>  * glibc component in sourceware instance marked "Not open for new bugs."
>  * No easy way to clone this but we can discuss options.
>  * Isolate bugzilla from other services.

Does LF IT offer some Bugzilla anti-spam services?  To what extent do we
need to constrain new sign-ups?

We have one custom extension I know of: suppressing notifications when
setting security+ because it was deemed too noisy for old bugs.  This
should not be necessary anymore; most of the bugs missing security+
analysis are current.

> * git
>  * Migrate to gitolite
>  * Community manages access via gitolite keys.
>  * Minimize all access to sources and isolate from other processes.
>  * Minimal server side web hooks where required.
>  * Isolate git service from other services.
>  * Stop supporting svn/cvs and provide tarball dumps.

Can we keep using the AdaCore hooks?  Or would they have to run on the
side somehow?  Who is going to implement changes to the AdaCore scripts?

> * wiki
>  * Migrate to git-based documentation with existing content copied over.
>  * Suggest rst/Sphinx or similar to existing discussions for GCC docs.
>  * Sphinx with themes can provide a lot of flexibility for display.
>  * Isolate wiki service from other services.

Will this be a separate Git repository, with different commit rules?

I do not particularly like the current MoinMoin wiki, but at least it
has a preview button.  We'd lose that with a pure Git-based workflow.

> * Website
>  * Provide a simple static site.
>  * Isolate web hosting from other services.

How do we keep the online manual up-to-date?

> * Meeting
>  * Already migrated away from proprietary solutions.
>  * Continue to use LF IT BBB instance for glibc meetings including weekly patch review.
>  * Isolate BBB from other services.

Ironically, with that move we lost support for open standards like SIP
dial-in.  I assume there is no plan to bring that back, as the SIP
situation with BBB isn't great.

Thanks,
Florian

Re: Core Toolchain Infrastructure - Services for glibc

[Konstantin Ryabitsev]

On Fri, Jul 14, 2023 at 01:38:00PM +0200, Florian Weimer wrote:
> >  * Support public-inbox for mailing list archives.
> >  * Use of public-inbox means archives can be cloned and copied.
> >  * Use of LF IT Subspace mailing list services (mlmmj, postfix).
> 
> I assume LF IT is able to run mailing lists without “via” From:
> rewriting?

Yes, that practice is horrible and we do not support it in any way shape or
form for mailing lists.

> > * bug database
> >  * Consider starting fresh in new Bugzilla 5.0.4+ instance and freeze old product.
> >  * glibc component in sourceware instance marked "Not open for new bugs."
> >  * No easy way to clone this but we can discuss options.
> >  * Isolate bugzilla from other services.
> 
> Does LF IT offer some Bugzilla anti-spam services?  To what extent do we
> need to constrain new sign-ups?

We have various approaches here, depending on the project.

We do not constrain sign-ups for bugzilla.kernel.org. We have a script that
reports new comments containing links or potentially spammy attachments. These
comments are junked and accounts posting them are banned.

We do constrain sign-ups for bugzilla.yoctoproject.org -- users must request a
new account to be created before they can file any bugs.

> Can we keep using the AdaCore hooks?  Or would they have to run on the
> side somehow?  Who is going to implement changes to the AdaCore scripts?

This is the main point of contemplation -- we do not currently support custom
hooks on the server side:

- they tend to significantly slow down pushes
- they run extensive codebases with the same permissions as the owner of the
  repositories, significantly increasing security risks

Our recommendation was to move all CI tasks to a system that is better suited
for it. For example, CI can run on a patchwork system and the pre-commit hook
can then check that each commit matches a patchwork entry that passed CI.

> > * wiki
> >  * Migrate to git-based documentation with existing content copied over.
> >  * Suggest rst/Sphinx or similar to existing discussions for GCC docs.
> >  * Sphinx with themes can provide a lot of flexibility for display.
> >  * Isolate wiki service from other services.
> 
> Will this be a separate Git repository, with different commit rules?
> 
> I do not particularly like the current MoinMoin wiki, but at least it
> has a preview button.  We'd lose that with a pure Git-based workflow.

RTD documentation is written in ReST, for which there are many editors that
provide preview functionality.

-K

Re: Core Toolchain Infrastructure - Services for glibc

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1465 bytes --]

Hi Carlos,

On Thu, 2023-07-13 at 17:58 -0400, Carlos O'Donell via Libc-alpha
wrote:
> As part of this effort the Technical Advisory Committee (TAC) has
> evaluated the services used by glibc and how they could be migrated
> and serviced by the CTI project.
> 
> The CTI TAC recommendation is to use Linux Foundation IT services
> for core infrastructure. The LF IT team already supports many of
> the same services for the Linux kernel and at scale. The migration
> would involve moving services from Sourceware.org to LF IT servers.
> We continue to be thankful and appreciative of the time spent by
> Sourceware.org volunteers in support of the current services.

Your goals seem to align with those of Sourceware and the expanded
services we have been setting up over the last couple of years
together. So hopefully you don't have to leave the community.
https://sourceware.org/mission.html

A couple of months ago we finalized the incorporation of Sourceware as
a Software Freedom Conservancy member project with eight community
members serving as Project Leadership Committee:
https://sfconservancy.org/news/2023/may/15/sourceware-joins-sfc/

If the community needs any help with any of the Sourceware services
please join one of the Open Office talks (there is one today, at 18:00
UTC, in #overseers on irc.libera.chat) or contact us through the
overseers mailinglist, bugzilla, etc at any time. See attached.

Cheers,

Mark

[-- Attachment #2: Attached message - Sourceware Open Office, Friday July 14, 18:00 UTC --]
[-- Type: message/rfc822, Size: 6201 bytes --]

From: Mark Wielaard via Overseers <overseers@sourceware.org>
To: overseers@sourceware.org
Cc: Mark Wielaard <mark@klomp.org>
Subject: Sourceware Open Office, Friday July 14, 18:00 UTC
Date: Fri, 14 Jul 2023 02:11:34 +0200
Message-ID: <20230714001134.GA29669@gnu.wildebeest.org>

Every second Friday of the month is the Sourceware Overseers Open
Office hour in #overseers on irc.libera.chat from 18:00 till 19:00
UTC. That is this Friday July 14th.

Please feel free to drop by with any Sourceware hosting questions.
Help with email (archives, spam), secure supply chain, setting up new
buildbot CI, producing snapshots, helping write up some POCs,
discussing extending services, what the Software Freedom Sourceware
Project Leadership Committee should prioritize, or just to say things
are all working fine.

Of course you are welcome to drop into the #overseers channel at any
time and we can also be reached through email and bugzilla:
https://sourceware.org/mission.html#organization

If you aren't already and want to keep up to date on Sourceware
infrastructure services then please also subscribe to the overseers
mailinglist. https://sourceware.org/mailman/listinfo/overseers

We are also on the fediverse these days:
https://fosstodon.org/@sourceware

Re: Core Toolchain Infrastructure - Services for glibc

[Siddhesh Poyarekar]

On 2023-07-14 11:34, Konstantin Ryabitsev via Libc-alpha wrote:
>> Can we keep using the AdaCore hooks?  Or would they have to run on the
>> side somehow?  Who is going to implement changes to the AdaCore scripts?
> 
> This is the main point of contemplation -- we do not currently support custom
> hooks on the server side:
> 
> - they tend to significantly slow down pushes
> - they run extensive codebases with the same permissions as the owner of the
>    repositories, significantly increasing security risks
> 
> Our recommendation was to move all CI tasks to a system that is better suited
> for it. For example, CI can run on a patchwork system and the pre-commit hook
> can then check that each commit matches a patchwork entry that passed CI.

This would mean porting AdaCore hooks to a patchwork trybot.  This would 
be an acceptable solution for glibc, but I'm not sure how useful this 
would be on the whole since gcc doesn't use patchwork as extensively at 
the moment.  Also, we need to figure out who's going to do this.

Sid

Re: Core Toolchain Infrastructure - Services for glibc

[Jakub Jelinek]

On Tue, Jul 18, 2023 at 07:47:37AM -0400, Siddhesh Poyarekar wrote:
> On 2023-07-14 11:34, Konstantin Ryabitsev via Libc-alpha wrote:
> > > Can we keep using the AdaCore hooks?  Or would they have to run on the
> > > side somehow?  Who is going to implement changes to the AdaCore scripts?
> > 
> > This is the main point of contemplation -- we do not currently support custom
> > hooks on the server side:
> > 
> > - they tend to significantly slow down pushes
> > - they run extensive codebases with the same permissions as the owner of the
> >    repositories, significantly increasing security risks
> > 
> > Our recommendation was to move all CI tasks to a system that is better suited
> > for it. For example, CI can run on a patchwork system and the pre-commit hook
> > can then check that each commit matches a patchwork entry that passed CI.
> 
> This would mean porting AdaCore hooks to a patchwork trybot.  This would be
> an acceptable solution for glibc, but I'm not sure how useful this would be
> on the whole since gcc doesn't use patchwork as extensively at the moment.
> Also, we need to figure out who's going to do this.

It is definitely not acceptable for gcc, we strongly rely on server side
pre-commit hooks for various different tasks.

	Jakub

Re: Core Toolchain Infrastructure - Services for glibc

[Jeff Law]

On 7/18/23 06:26, Jakub Jelinek via Libc-alpha wrote:
> On Tue, Jul 18, 2023 at 07:47:37AM -0400, Siddhesh Poyarekar wrote:
>> On 2023-07-14 11:34, Konstantin Ryabitsev via Libc-alpha wrote:
>>>> Can we keep using the AdaCore hooks?  Or would they have to run on the
>>>> side somehow?  Who is going to implement changes to the AdaCore scripts?
>>>
>>> This is the main point of contemplation -- we do not currently support custom
>>> hooks on the server side:
>>>
>>> - they tend to significantly slow down pushes
>>> - they run extensive codebases with the same permissions as the owner of the
>>>     repositories, significantly increasing security risks
>>>
>>> Our recommendation was to move all CI tasks to a system that is better suited
>>> for it. For example, CI can run on a patchwork system and the pre-commit hook
>>> can then check that each commit matches a patchwork entry that passed CI.
>>
>> This would mean porting AdaCore hooks to a patchwork trybot.  This would be
>> an acceptable solution for glibc, but I'm not sure how useful this would be
>> on the whole since gcc doesn't use patchwork as extensively at the moment.
>> Also, we need to figure out who's going to do this.
> 
> It is definitely not acceptable for gcc, we strongly rely on server side
> pre-commit hooks for various different tasks.
BUt the key (I think) is they're not trybot/CI style hooks.  They're 
doing things like wiring up commits to the lists/bugzilla, verifying 
ChangeLog bits and the like.

I think the point is that any hooks need serious thought about what they 
do and the attack surface they represent.  So for example firing off 
pre-commit CI, not advisable.  Having a hook that queries another system 
for the state of pre-commit CI may be reasonable.

I would think that verifying ChangeLog format might fall into the 
reasonable space.  Not sure about lists/bz integration hooks.

jeff

Re: Core Toolchain Infrastructure - Services for glibc

[Siddhesh Poyarekar]

On 2023-07-18 09:19, Jeff Law wrote:
> 
> 
> On 7/18/23 06:26, Jakub Jelinek via Libc-alpha wrote:
>> On Tue, Jul 18, 2023 at 07:47:37AM -0400, Siddhesh Poyarekar wrote:
>>> On 2023-07-14 11:34, Konstantin Ryabitsev via Libc-alpha wrote:
>>>>> Can we keep using the AdaCore hooks?  Or would they have to run on the
>>>>> side somehow?  Who is going to implement changes to the AdaCore 
>>>>> scripts?
>>>>
>>>> This is the main point of contemplation -- we do not currently 
>>>> support custom
>>>> hooks on the server side:
>>>>
>>>> - they tend to significantly slow down pushes
>>>> - they run extensive codebases with the same permissions as the 
>>>> owner of the
>>>>     repositories, significantly increasing security risks
>>>>
>>>> Our recommendation was to move all CI tasks to a system that is 
>>>> better suited
>>>> for it. For example, CI can run on a patchwork system and the 
>>>> pre-commit hook
>>>> can then check that each commit matches a patchwork entry that 
>>>> passed CI.
>>>
>>> This would mean porting AdaCore hooks to a patchwork trybot.  This 
>>> would be
>>> an acceptable solution for glibc, but I'm not sure how useful this 
>>> would be
>>> on the whole since gcc doesn't use patchwork as extensively at the 
>>> moment.
>>> Also, we need to figure out who's going to do this.
>>
>> It is definitely not acceptable for gcc, we strongly rely on server side
>> pre-commit hooks for various different tasks.
> BUt the key (I think) is they're not trybot/CI style hooks.  They're 
> doing things like wiring up commits to the lists/bugzilla, verifying 
> ChangeLog bits and the like.
> 
> I think the point is that any hooks need serious thought about what they 
> do and the attack surface they represent.  So for example firing off 
> pre-commit CI, not advisable.  Having a hook that queries another system 
> for the state of pre-commit CI may be reasonable.
> 
> I would think that verifying ChangeLog format might fall into the 
> reasonable space.  Not sure about lists/bz integration hooks.

Yeah, putting ChangeLog format verification into a patchwork bot won't 
work anyway since patchwork ignores the git commit message when 
identifying patches.

Sid

Re: Core Toolchain Infrastructure - Services for glibc

[Konstantin Ryabitsev]

On Tue, Jul 18, 2023 at 02:26:44PM +0200, Jakub Jelinek wrote:
> > This would mean porting AdaCore hooks to a patchwork trybot.  This would be
> > an acceptable solution for glibc, but I'm not sure how useful this would be
> > on the whole since gcc doesn't use patchwork as extensively at the moment.
> > Also, we need to figure out who's going to do this.
> 
> It is definitely not acceptable for gcc, we strongly rely on server side
> pre-commit hooks for various different tasks.

I'm not saying "no hooks at all," but the general rule is:

- a hook can run checks on the contents of the commit message
- a hook can query external systems for validation results (with a reasonably
  short timeout)

The core principle is that hooks should be fast and shouldn't have a large
attack surface, because hooks are running with full access to the underlying
repository. Any CI builds or other validations should run prior to the push
and visible to the committer before the push is attempted. I mention patchwork
because it's a system we use, but other options exist (e.g. gerrit, though
I wouldn't use it just for CI purposes).

Regards,
-K

Re: Core Toolchain Infrastructure - Services for glibc

[Joseph Myers]

On Tue, 18 Jul 2023, Jeff Law via Libc-alpha wrote:

> BUt the key (I think) is they're not trybot/CI style hooks.  They're doing
> things like wiring up commits to the lists/bugzilla, verifying ChangeLog bits
> and the like.
> 
> I think the point is that any hooks need serious thought about what they do
> and the attack surface they represent.  So for example firing off pre-commit
> CI, not advisable.  Having a hook that queries another system for the state of
> pre-commit CI may be reasonable.
> 
> I would think that verifying ChangeLog format might fall into the reasonable
> space.  Not sure about lists/bz integration hooks.

Details of what the GCC hooks do / check are documented in the GCC service 
enumeration.

https://lore.kernel.org/cti-tac/7e59d8db-7b26-1249-ad5c-67b7f3411e1@codesourcery.com/T/#u

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Core Toolchain Infrastructure - Services for glibc

[Florian Weimer]

* Konstantin Ryabitsev:

> On Fri, Jul 14, 2023 at 01:38:00PM +0200, Florian Weimer wrote:
>> >  * Support public-inbox for mailing list archives.
>> >  * Use of public-inbox means archives can be cloned and copied.
>> >  * Use of LF IT Subspace mailing list services (mlmmj, postfix).
>> 
>> I assume LF IT is able to run mailing lists without “via” From:
>> rewriting?
>
> Yes, that practice is horrible and we do not support it in any way shape or
> form for mailing lists.

That's good to know.

>> > * bug database
>> >  * Consider starting fresh in new Bugzilla 5.0.4+ instance and freeze old product.
>> >  * glibc component in sourceware instance marked "Not open for new bugs."
>> >  * No easy way to clone this but we can discuss options.
>> >  * Isolate bugzilla from other services.
>> 
>> Does LF IT offer some Bugzilla anti-spam services?  To what extent do we
>> need to constrain new sign-ups?
>
> We have various approaches here, depending on the project.
>
> We do not constrain sign-ups for bugzilla.kernel.org. We have a script
> that reports new comments containing links or potentially spammy
> attachments. These comments are junked and accounts posting them are
> banned.
>
> We do constrain sign-ups for bugzilla.yoctoproject.org -- users must
> request a new account to be created before they can file any bugs.

Okay, so there are options, good.

>> Can we keep using the AdaCore hooks?  Or would they have to run on the
>> side somehow?  Who is going to implement changes to the AdaCore scripts?
>
> This is the main point of contemplation -- we do not currently support custom
> hooks on the server side:
>
> - they tend to significantly slow down pushes
> - they run extensive codebases with the same permissions as the owner of the
>   repositories, significantly increasing security risks
>
> Our recommendation was to move all CI tasks to a system that is better
> suited for it. For example, CI can run on a patchwork system and the
> pre-commit hook can then check that each commit matches a patchwork
> entry that passed CI.

One way to deal with this on our end would be a policy that allows
branch rebases to pull faulty commits for a time.  Lately we have pushed
some misformatted commit messages in the current setup, with its commit
hooks.

Thanks,
Florian

Re: Core Toolchain Infrastructure - Services for glibc

[Frank Ch. Eigler]

> [...]  Appended to this message are [...] an enumeration of the
> current services. [...]

Existing sourceware services for glibc also includes:

- the buildbot server, having run some 12,000 glibc builds since
  inception: https://builder.sourceware.org/buildbot/#/builders

- the bunsen instance accumulating glibc testsuite results, presently
  holding some 5,700 of them: 
  https://builder.sourceware.org/testruns/?has_keyvalue_k=testrun.git_describe&has_keyvalue_op=glob&has_keyvalue_v=*glibc*&has_keyvalue2_k=source.gitdescribe&has_keyvalue2_op=glob&has_keyvalue2_v=*&order_by=testrun.authored.time&order=desc&limit=10000&offset=0&offset=0

- the recently added 'snapshots' worker & server:
  https://snapshots.sourceware.org/glibc/trunk/


- FChE

Re: Sourceware Infrastructure Services for glibc

[Mark Wielaard]

Hi Frank,

On Tue, Aug 22, 2023 at 11:53:43AM -0400, Frank Ch. Eigler via Libc-alpha wrote:
> 
> > [...]  Appended to this message are [...] an enumeration of the
> > current services. [...]
> 
> Existing sourceware services for glibc also includes:
> 
> - the buildbot server, having run some 12,000 glibc builds since
>   inception: https://builder.sourceware.org/buildbot/#/builders
> 
> - the bunsen instance accumulating glibc testsuite results, presently
>   holding some 5,700 of them: 
>   https://builder.sourceware.org/testruns/?has_keyvalue_k=testrun.git_describe&has_keyvalue_op=glob&has_keyvalue_v=*glibc*&has_keyvalue2_k=source.gitdescribe&has_keyvalue2_op=glob&has_keyvalue2_v=*&order_by=testrun.authored.time&order=desc&limit=10000&offset=0&offset=0
> 
> - the recently added 'snapshots' worker & server:
>   https://snapshots.sourceware.org/glibc/trunk/

Thanks for the list. Note that Sourceware Infrastructure Services are
also listed per project at https://sourceware.org/projects.html

For glibc it is currently missing bunsen (would be nice to have short
URLs per project btw) and the fsf-tech team provided services for GNU
projects. Which should probably be added. There is also a generic
services list at https://sourceware.org/mission.html#services

Also note that over the last couple of years we setup a real
Sourceware organization to make sure the Sourceware community can be
sustained for another 25 years:
https://sourceware.org/sourceware-25-roadmap.html

So if there are any services missing or could be improved, please do
reach out to overseers@, file infrastructure bugs or come talk at one
of the open office hours on irc (or really at any time). See
https://sourceware.org/mission.html#organization

Cheers,

Mark

Re: Core Toolchain Infrastructure - Services for glibc

[Carlos O'Donell]

On 8/22/23 11:53, Frank Ch. Eigler wrote:
> 
>> [...]  Appended to this message are [...] an enumeration of the
>> current services. [...]
> 
> Existing sourceware services for glibc also includes:

Thanks, I've noted the additional services provided for sourceware projects.

> - the buildbot server, having run some 12,000 glibc builds since
>   inception: https://builder.sourceware.org/buildbot/#/builders

> - the bunsen instance accumulating glibc testsuite results, presently
>   holding some 5,700 of them: 
>   https://builder.sourceware.org/testruns/?has_keyvalue_k=testrun.git_describe&has_keyvalue_op=glob&has_keyvalue_v=*glibc*&has_keyvalue2_k=source.gitdescribe&has_keyvalue2_op=glob&has_keyvalue2_v=*&order_by=testrun.authored.time&order=desc&limit=10000&offset=0&offset=0

> - the recently added 'snapshots' worker & server:
>   https://snapshots.sourceware.org/glibc/trunk/

-- 
Cheers,
Carlos.