[Bug Infrastructure/29615] New: prototype & document SOP for signed-git-op repo

[Bug Infrastructure/29615] New: prototype & document SOP for signed-git-op repo

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29615

            Bug ID: 29615
           Summary: prototype & document SOP for signed-git-op repo
           Product: sourceware
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Infrastructure
          Assignee: overseers at sourceware dot org
          Reporter: fche at redhat dot com
  Target Milestone: ---

We should find a volunteer (bunsen?  builder?) project to experiment getting a
fully gpg-signed git workflow, and then publish the setup for other projects to
try.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29615] prototype & document SOP for signed-git-op repo

[serhei at serhei dot io]

https://sourceware.org/bugzilla/show_bug.cgi?id=29615

Serhei Makarov <serhei at serhei dot io> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |serhei at serhei dot io

--- Comment #1 from Serhei Makarov <serhei at serhei dot io> ---
I'm open to putting the Bunsen repo through all the latest and most stringent
in git supply chain security best practices. Got to keep out those Ken Thompson
omniscient code gremlins.

We'll need to coordinate with Martin Cermak, as he's also pushing commits
directly to the repo. (Keith Seitz has also contributed, but he's always sent
patches to the mailing list for review so I assume he'll be ok with added steps
to the contribution procedure.)

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29615] prototype & document SOP for signed-git-op repo

[serhei at serhei dot io]

https://sourceware.org/bugzilla/show_bug.cgi?id=29615

--- Comment #2 from Serhei Makarov <serhei at serhei dot io> ---
As a note to self, I assume we'll be trying a workflow as close as possible to

https://www.kernel.org/doc/html/latest/process/maintainer-pgp-guide.html

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29615] prototype & document SOP for signed-git-op repo

[ezannoni at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29615

Elena Zannoni <ezannoni at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ezannoni at gmail dot com

--- Comment #3 from Elena Zannoni <ezannoni at gmail dot com> ---
Butting in... 
Serhei, yes that document. I think this is also worth taking a look at:
https://www.kernel.org/doc/html/latest/process/submitting-patches.html
Has the use of the tags spelled out, and the roles of the reviewers and the
submitters of patches and what they "agree" to do/represent.

In addition, there is the SPDX license identifier thing, that the kernel has.
For further supply chain security/identification/SBOMs and whatnot. 

I am not saying to mimic everything exactly, but to get an idea of what the
kernel was trying to solve with these things.

Note that the kernel workflow is different, only the maintainers push code to
the repo (in the majority of the subsystems), for other projects it might not
be the same (such as the GNU toolchain write after approval case).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29615] prototype & document SOP for signed-git-op repo

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29615

Mark Wielaard <mark at klomp dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mark at klomp dot org

--- Comment #4 from Mark Wielaard <mark at klomp dot org> ---
It would be nice to go through the source integrity threats identified in
https://slsa.dev/spec/v0.1/threats

For a sourceware project that means checking section (A) "Submit unauthorized
change" of:
https://slsa.dev/spec/v0.1/threats#source-integrity-threats

Almost all of those are policy issues, but it would be good to note where our
setup doesn't support adopting a specific policy change (if wanted, I think
some of there policy changes are a bit heavy-handed, not everybody wants to be
SLSA4 compliant, but it would be nice to make sure that technically a project
can choose to adopt them).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29615] prototype & document SOP for signed-git-op repo

[serhei at serhei dot io]

https://sourceware.org/bugzilla/show_bug.cgi?id=29615

--- Comment #5 from Serhei Makarov <serhei at serhei dot io> ---
One issue with the SLSA4 checklist is that SLSA4 seems to be written in a way
that assumes developers are reviewing pull requests and there's a magic forge
that everyone trusts to securely decide when the merge criteria have been met.
Applying it to individual commits on a mailing-list workflow will be tricky and
AFAIK the kernel isn't currently doing that. The example on SLSA4 website of a
compliant project is SUSE Open Build System, which sounds magic-forge-y.

(Just trying to wrap my head around how it would hypothetically look. In
fully-paranoid SLSA4, we would need to verify

(a) the patch series submitted to the mailing list was reviewed & signed off by
<insert required number of reviewers>
(b) the patch series committed to the Git repo matches exactly what was signed
off in the mailing list (no last-minute good-Samaritan fixup by the maintainer)

and all of that would need to be recorded by GPG signatures on the mailing list
archives* and in the Git repo and verifiable by anyone after the fact, not 'oh,
we has a forge website, and the forge's security widget ok'ed the merge, so the
resulting Git repo must be good; hope no one hacked the forge'.

[*On an 8-patch series, 2 reviewers would need to attach a total of 16 GPG
signatures. Maybe with suitable tooling you could batch up the contents of the
8 patches into 1 email, and sign that, I dunno. {Forget doing that in your
corporate GMail web client.} You would then have a script anyone can run for
auditing purposes that slurps the Git repo and mailing list archives and flags
any commits that don't have two matching, GPG-signed Signed-Off-By: replies by
reviewers.]

[The Git repo would also need to consume mailing lists somehow and block any
patches that have not been signed off. But this is orthogonal to after-the-fact
audits, and can basically be thought of as a convenience measure to protect
maintainers from accidentally making un-approved changes that would immediately
be caught and reverted.])

Is there an existing example of an SLSA4 compliant (or at least
trying-in-good-faith) project that has grappled with this kind of workflow? I'm
honestly not sure if the tooling to enable it is 100% there, at LF or anywhere
else.

At the same time it's kind of intriguing to see if this level of paranoia can
be implemented in a somewhat convenient way.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29615] prototype & document SOP for signed-git-op repo

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29615

--- Comment #6 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Serhei Makarov from comment #5)
> One issue with the SLSA4 checklist is that SLSA4 seems to be written in a
> way that assumes developers are reviewing pull requests and there's a magic
> forge that everyone trusts to securely decide when the merge criteria have
> been met. Applying it to individual commits on a mailing-list workflow will
> be tricky and AFAIK the kernel isn't currently doing that. The example on
> SLSA4 website of a compliant project is SUSE Open Build System, which sounds
> magic-forge-y.

Yes, I think it is mostly used for "downstream" (distro like) projects where
the upstream (signed) release tarball is used as trusted input and then any
patches on top of that are treated at SLSA4 level.

But I don't think the point is to adopt all SLSA level 4, but to see how far a
project could get if it tried.

> (Just trying to wrap my head around how it would hypothetically look. In
> fully-paranoid SLSA4, we would need to verify
> 
> (a) the patch series submitted to the mailing list was reviewed & signed off
> by <insert required number of reviewers>
> (b) the patch series committed to the Git repo matches exactly what was
> signed off in the mailing list (no last-minute good-Samaritan fixup by the
> maintainer)
> 
> and all of that would need to be recorded by GPG signatures on the mailing
> list archives* and in the Git repo and verifiable by anyone after the fact,
> not 'oh, we has a forge website, and the forge's security widget ok'ed the
> merge, so the resulting Git repo must be good; hope no one hacked the forge'.
> 
> [*On an 8-patch series, 2 reviewers would need to attach a total of 16 GPG
> signatures. Maybe with suitable tooling you could batch up the contents of
> the 8 patches into 1 email, and sign that, I dunno. {Forget doing that in
> your corporate GMail web client.} You would then have a script anyone can
> run for auditing purposes that slurps the Git repo and mailing list archives
> and flags any commits that don't have two matching, GPG-signed
> Signed-Off-By: replies by reviewers.]

So there are different ways to attest a patch or email is valid.
Using a public-inbox instance like https://inbox.sourceware.org/ you can use b4
for patch attestation using dkim, gpg-signed emails or patatt.

See https://gnu.wildebeest.org/~mark/sourceware/presentation.html#slide14
https://people.kernel.org/monsieuricon/introducing-b4-and-patch-attestation
https://people.kernel.org/monsieuricon/end-to-end-patch-attestation-with-patatt-and-b4

Also tools like git-pw or b4 make it easy to collect all commit/patch emails
and create a new patch that has all Signed-Off-by/Reviewed-by/... signoffs.

The idea is not to carry-over all signatures on the individual email messages,
but to trust the final committer to have verified them all. And only added the
signoffs of those emails/reviews that could be verified. So the signature of
the final committer proofs they have verified all previous emails. You still
have the email archive to check that if you are really paranoid. 

> [The Git repo would also need to consume mailing lists somehow and block any
> patches that have not been signed off. But this is orthogonal to
> after-the-fact audits, and can basically be thought of as a convenience
> measure to protect maintainers from accidentally making un-approved changes
> that would immediately be caught and reverted.])

I don't believe it is that strict. At some point you just have to trust the
final committer to have done all these checks. SLSA doesn't really protect
against bad (but trusted insider) actors. See (A3) Code review bypasses that
are out of scope of SLSA.

> Is there an existing example of an SLSA4 compliant (or at least
> trying-in-good-faith) project that has grappled with this kind of workflow?
> I'm honestly not sure if the tooling to enable it is 100% there, at LF or
> anywhere else.
> 
> At the same time it's kind of intriguing to see if this level of paranoia
> can be implemented in a somewhat convenient way.

I don't know if anybody really does it full paranoid SLSA level 4 for a real
community, distributed, upstream development project. But I think tools like
gitpw and b4 plus inbox.sourceware.org get us pretty far if people want to get
close.

Also you can start with simply adding the patch signoffs to every commit to
show the intent of double review, etc. and don't require all the signing stuff
(which is basically what the kernel does). That isn't full SLSA compliant, but
shows the intent.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29615] prototype & document SOP for signed-git-op repo

[mark at klomp dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29615

--- Comment #7 from Mark Wielaard <mark at klomp dot org> ---
(In reply to Mark Wielaard from comment #6)
> (In reply to Serhei Makarov from comment #5)
> > One issue with the SLSA4 checklist is that SLSA4 seems to be written in a
> > way that assumes developers are reviewing pull requests and there's a magic
> > forge that everyone trusts to securely decide when the merge criteria have
> > been met. Applying it to individual commits on a mailing-list workflow will
> > be tricky and AFAIK the kernel isn't currently doing that. The example on
> > SLSA4 website of a compliant project is SUSE Open Build System, which sounds
> > magic-forge-y.
> 
> Yes, I think it is mostly used for "downstream" (distro like) projects where
> the upstream (signed) release tarball is used as trusted input and then any
> patches on top of that are treated at SLSA4 level.

SLSA v1.0 does not address source threats any more. It only deals with
dependency and build threats. So it just deals with "downstream" uses of
(trusted) releases.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug Infrastructure/29615] prototype & document SOP for signed-git-op repo

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=29615

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #8 from Frank Ch. Eigler <fche at redhat dot com> ---
gitsigur is available for enforcing signedness on local git repos on an opt-in
basis

Some work needs to be done but the basics are there.

https://inbox.sourceware.org/overseers/ZIz4NB%2FAqWpSNj5d@elastic.org/

-- 
You are receiving this mail because:
You are the assignee for the bug.