Machine maintainer veto.

Machine maintainer veto.

[Carlos O'Donell]

Community,

I have attempted to clarify what has always been in effect.
The machine maintainers have some level of veto for what goes
into their machine port. This allows some amount of control over
hardware support and ABI/API additions and removals.

The key issue is to balance the project goals and the needs of
the users of the particular machine. To do that effectively the
machine maintainers have to have some level of veto to add or
remove things to the machine they know and understand best.

https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers

Cheers,
Carlos.

Re: Machine maintainer veto.

[Rich Felker]

On Thu, Jul 02, 2015 at 11:51:55PM -0400, Carlos O'Donell wrote:
> Community,
> 
> I have attempted to clarify what has always been in effect.
> The machine maintainers have some level of veto for what goes
> into their machine port. This allows some amount of control over
> hardware support and ABI/API additions and removals.
> 
> The key issue is to balance the project goals and the needs of
> the users of the particular machine. To do that effectively the
> machine maintainers have to have some level of veto to add or
> remove things to the machine they know and understand best.
> 
> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers

I know I don't have any standing to change it, but I just want to
express a sentiment that I think this is bad policy. I can go into the
details of why if anyone is interested.

Rich

Re: Machine maintainer veto.

[Ondřej Bílka]

On Fri, Jul 03, 2015 at 02:20:20AM -0400, Rich Felker wrote:
> On Thu, Jul 02, 2015 at 11:51:55PM -0400, Carlos O'Donell wrote:
> > Community,
> > 
> > I have attempted to clarify what has always been in effect.
> > The machine maintainers have some level of veto for what goes
> > into their machine port. This allows some amount of control over
> > hardware support and ABI/API additions and removals.
> > 
> > The key issue is to balance the project goals and the needs of
> > the users of the particular machine. To do that effectively the
> > machine maintainers have to have some level of veto to add or
> > remove things to the machine they know and understand best.
> > 
> > https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers
> 
And Carlos could you elaborate what are maintainer responsibilities. We
are talking about prosperity of project as whole. If a maintainer
doesn't respond to any objections raised on list, ignores any consensus 
and just commits what he thinks best, closes bugs of his arch as invalid 
then it harms a project.

Also maintainer is a person. If whole community reaches consensus about some
feature and only he opposes how do you handle that?


> I know I don't have any standing to change it, but I just want to
> express a sentiment that I think this is bad policy. I can go into the
> details of why if anyone is interested.
>
Could you summarize that in three sentences to see if somebody shares
that sentiment? 

Re: Machine maintainer veto.

[Carlos O'Donell]

On 07/03/2015 02:20 AM, Rich Felker wrote:
> On Thu, Jul 02, 2015 at 11:51:55PM -0400, Carlos O'Donell wrote:
>> Community,
>>
>> I have attempted to clarify what has always been in effect.
>> The machine maintainers have some level of veto for what goes
>> into their machine port. This allows some amount of control over
>> hardware support and ABI/API additions and removals.
>>
>> The key issue is to balance the project goals and the needs of
>> the users of the particular machine. To do that effectively the
>> machine maintainers have to have some level of veto to add or
>> remove things to the machine they know and understand best.
>>
>> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers
> 
> I know I don't have any standing to change it, but I just want to
> express a sentiment that I think this is bad policy. I can go into the
> details of why if anyone is interested.

I am absolutely interested in hearing the details.

I am not documenting anything new, machine maintainers have altered
their machine files as they see fit and with respect to the two key
responsibilities they hold.

Where the machine maintenance touches generic code or other parts of
glibc, or where the community wishes to enact a cross-machine change
we get into obvious gray zones, but that's life.

To date I have not seen the machine maintainers abuse this veto, but
at times it has been done.

Cheers,
Carlos.

Re: Machine maintainer veto.

[Rich Felker]

On Fri, Jul 03, 2015 at 08:58:58AM -0400, Carlos O'Donell wrote:
> On 07/03/2015 02:20 AM, Rich Felker wrote:
> > On Thu, Jul 02, 2015 at 11:51:55PM -0400, Carlos O'Donell wrote:
> >> Community,
> >>
> >> I have attempted to clarify what has always been in effect.
> >> The machine maintainers have some level of veto for what goes
> >> into their machine port. This allows some amount of control over
> >> hardware support and ABI/API additions and removals.
> >>
> >> The key issue is to balance the project goals and the needs of
> >> the users of the particular machine. To do that effectively the
> >> machine maintainers have to have some level of veto to add or
> >> remove things to the machine they know and understand best.
> >>
> >> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers
> > 
> > I know I don't have any standing to change it, but I just want to
> > express a sentiment that I think this is bad policy. I can go into the
> > details of why if anyone is interested.
> 
> I am absolutely interested in hearing the details.

On a level of principles, I think it violates the principle of
consensus and preserves part of the toxic dictatorship impression that
glibc used to have. The project has come a long way in shedding this
impression, and in my opinion has become _the_ most open and healthy
GNU project in terms of development process. I'd like to see this
progress continue.

In terms of impact, I think there's a false notion that machine
maintainers' decisions don't impact anyone but their machine's users,
and therefore that as long as they act in the interest of their users,
nobody is harmed by their decisions. This is wrong for multiple
reasons. User-visible changes potentially affect anyone writing
portable software -- when their software behaves unexpectedly on the
machine maintainer's target, they (not the glibc machine maintainers)
are the ones who generally take the blame and deal with the costs of
working around whatever the issue is. And while non-user-visible
changes shouldn't affect applications, they do constrain general glibc
development. Every bit of machine-specific code/behavior increases the
cost of making changes to glibc in the affected area, and in a worst
case machine-specifics can even constrain the implementation in ways
that preclude serious improvements.

There's also the matter of bugs and conformance. Machine maintainers
are not necessarily experts in standards conformance, portability, and
related ABI issues, and unless they also happen to be experts in these
fields, they should not be treated as such. At least the following
bugs are results of machine-specific code/definitions being approved
for inclusion in glibc without regard for how they affect standards
conformance:

https://sourceware.org/bugzilla/show_bug.cgi?id=5945
https://sourceware.org/bugzilla/show_bug.cgi?id=16437
https://sourceware.org/bugzilla/show_bug.cgi?id=16919
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2f3a499e6b803802880aea1fb8d3b46f1959494f
(I don't think this one has a glibc bug tracker issue yet, but the
kernel-side "fix" was a regression and made the types wrong and
inconsistent -- st_mode is 32-bit but ipc_perm.mode is 16-bit -- and
it was made because the glibc userspace type was wrong.)

I don't know the history of how they were introduced, but the current
policy, at least as expressed by Carlos, seems to make it possible for
machine maintainers to dictate (by veto) the introduction of such bugs
even if they were detected and objected to by others. I think we
should be striving for the opposite: objective conformance checks that
machine-specific definitions for new targets have to pass before being
accepted into glibc.

Security also comes under the area of bugs. Being a machine maintainer
should not entitle someone to do things in a machine-specific way
that's less secure than the way the same thing is handled on other
machines. Even if it only affects users of their target, the resulting
CVE makes glibc look bad.

Rich

Re: Machine maintainer veto.

[Siddhesh Poyarekar]

On 3 July 2015 at 09:21, Carlos O'Donell <carlos@redhat.com> wrote:
> The key issue is to balance the project goals and the needs of
> the users of the particular machine. To do that effectively the
> machine maintainers have to have some level of veto to add or
> remove things to the machine they know and understand best.
>
> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers

I don't quite agree with this:

    "... This veto can be used to prevent changes that break ABI or
API, or it can be used to include new ABI support for the machine even
if there is no consensus on how best to proceed"

While I acknowledge that arch maintainers have a better understanding
of their architecture, it doesn't automatically translate to a deep
enough understanding of glibc in general and that can break things in
irreversible ways.  Also, arch maintainers are as accountable to the
glibc community as they are to their customers or their organization,
so ignoring consensus goes against that principle.

There is a middle ground though, where arch maintainers have enough
freedom to decide what goes into their port as long as it doesn't
conflict with the goals of the community.  So it should be OK if a
maintainer chooses to ignore objection for a patch from a community
member on minor differences.  However, if there is sustained
opposition from the community in general, it would be inappropriate
for the maintainer to ignore that feedback.

I admit this is still vague, but ISTM to be a better place to be than
a full veto described in the wiki page.

Siddhesh
-- 
http://siddhesh.in

Re: Machine maintainer veto.

[Carlos O'Donell]

On 07/03/2015 11:34 AM, Rich Felker wrote:
> On Fri, Jul 03, 2015 at 08:58:58AM -0400, Carlos O'Donell wrote:
>> On 07/03/2015 02:20 AM, Rich Felker wrote:
>>> On Thu, Jul 02, 2015 at 11:51:55PM -0400, Carlos O'Donell wrote:
>>>> Community,
>>>>
>>>> I have attempted to clarify what has always been in effect.
>>>> The machine maintainers have some level of veto for what goes
>>>> into their machine port. This allows some amount of control over
>>>> hardware support and ABI/API additions and removals.
>>>>
>>>> The key issue is to balance the project goals and the needs of
>>>> the users of the particular machine. To do that effectively the
>>>> machine maintainers have to have some level of veto to add or
>>>> remove things to the machine they know and understand best.
>>>>
>>>> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers
>>>
>>> I know I don't have any standing to change it, but I just want to
>>> express a sentiment that I think this is bad policy. I can go into the
>>> details of why if anyone is interested.
>>
>> I am absolutely interested in hearing the details.
> 
> On a level of principles, I think it violates the principle of
> consensus and preserves part of the toxic dictatorship impression that
> glibc used to have. The project has come a long way in shedding this
> impression, and in my opinion has become _the_ most open and healthy
> GNU project in terms of development process. I'd like to see this
> progress continue.
> 
> In terms of impact, I think there's a false notion that machine
> maintainers' decisions don't impact anyone but their machine's users,
> and therefore that as long as they act in the interest of their users,
> nobody is harmed by their decisions. This is wrong for multiple
> reasons. User-visible changes potentially affect anyone writing
> portable software -- when their software behaves unexpectedly on the
> machine maintainer's target, they (not the glibc machine maintainers)
> are the ones who generally take the blame and deal with the costs of
> working around whatever the issue is. And while non-user-visible
> changes shouldn't affect applications, they do constrain general glibc
> development. Every bit of machine-specific code/behavior increases the
> cost of making changes to glibc in the affected area, and in a worst
> case machine-specifics can even constrain the implementation in ways
> that preclude serious improvements.
> 
> There's also the matter of bugs and conformance. Machine maintainers
> are not necessarily experts in standards conformance, portability, and
> related ABI issues, and unless they also happen to be experts in these
> fields, they should not be treated as such. At least the following
> bugs are results of machine-specific code/definitions being approved
> for inclusion in glibc without regard for how they affect standards
> conformance:
> 
> https://sourceware.org/bugzilla/show_bug.cgi?id=5945
> https://sourceware.org/bugzilla/show_bug.cgi?id=16437
> https://sourceware.org/bugzilla/show_bug.cgi?id=16919
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2f3a499e6b803802880aea1fb8d3b46f1959494f
> (I don't think this one has a glibc bug tracker issue yet, but the
> kernel-side "fix" was a regression and made the types wrong and
> inconsistent -- st_mode is 32-bit but ipc_perm.mode is 16-bit -- and
> it was made because the glibc userspace type was wrong.)
> 
> I don't know the history of how they were introduced, but the current
> policy, at least as expressed by Carlos, seems to make it possible for
> machine maintainers to dictate (by veto) the introduction of such bugs
> even if they were detected and objected to by others. I think we
> should be striving for the opposite: objective conformance checks that
> machine-specific definitions for new targets have to pass before being
> accepted into glibc.
> 
> Security also comes under the area of bugs. Being a machine maintainer
> should not entitle someone to do things in a machine-specific way
> that's less secure than the way the same thing is handled on other
> machines. Even if it only affects users of their target, the resulting
> CVE makes glibc look bad.

Rich,

Thank you for that writeup. My initial text in the wiki is certainly a
starting point, and I think we need to refine it. The community does need
to decide how much power a machine maintainer should be given. Your comments
go a long way to adding to and broadening this discussion.

Would it be sufficient to soften the language to say that the machine
maintainer has blanket commit privileges for their machine, but that they
remain responsible to the distributions, users, the project as a whole and
that such blanket commit privileges should be used responsibly?

Therefore we couch the language in terms of responsibility to the community?

Cheers,
Carlos.

Re: Machine maintainer veto.

[Carlos O'Donell]

On 07/03/2015 11:44 AM, Siddhesh Poyarekar wrote:
> On 3 July 2015 at 09:21, Carlos O'Donell <carlos@redhat.com> wrote:
>> The key issue is to balance the project goals and the needs of
>> the users of the particular machine. To do that effectively the
>> machine maintainers have to have some level of veto to add or
>> remove things to the machine they know and understand best.
>>
>> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers
> 
> I don't quite agree with this:
> 
>     "... This veto can be used to prevent changes that break ABI or
> API, or it can be used to include new ABI support for the machine even
> if there is no consensus on how best to proceed"
> 
> While I acknowledge that arch maintainers have a better understanding
> of their architecture, it doesn't automatically translate to a deep
> enough understanding of glibc in general and that can break things in
> irreversible ways.  Also, arch maintainers are as accountable to the
> glibc community as they are to their customers or their organization,
> so ignoring consensus goes against that principle.

That is a very good point.

> There is a middle ground though, where arch maintainers have enough
> freedom to decide what goes into their port as long as it doesn't
> conflict with the goals of the community.  So it should be OK if a
> maintainer chooses to ignore objection for a patch from a community
> member on minor differences.  However, if there is sustained
> opposition from the community in general, it would be inappropriate
> for the maintainer to ignore that feedback.

This calls for consensus.

> I admit this is still vague, but ISTM to be a better place to be than
> a full veto described in the wiki page.

Agreed. I'll let others comment a bit further before I summarize the
positions with a new block of text to describe the machine maintainers
responsibilities.

Cheers,
Carlos.

Re: Machine maintainer veto.

[Torvald Riegel]

On Fri, 2015-07-03 at 11:34 -0400, Rich Felker wrote:
> In terms of impact, I think there's a false notion that machine
> maintainers' decisions don't impact anyone but their machine's users,
> and therefore that as long as they act in the interest of their users,
> nobody is harmed by their decisions. This is wrong for multiple
> reasons. User-visible changes potentially affect anyone writing
> portable software -- when their software behaves unexpectedly on the
> machine maintainer's target, they (not the glibc machine maintainers)
> are the ones who generally take the blame and deal with the costs of
> working around whatever the issue is. And while non-user-visible
> changes shouldn't affect applications, they do constrain general glibc
> development. Every bit of machine-specific code/behavior increases the
> cost of making changes to glibc in the affected area, and in a worst
> case machine-specifics can even constrain the implementation in ways
> that preclude serious improvements.

Agreed.  On example is that we seem to assume that if someone wants to
fix/change/... something that has both a generic implementation and an
arch-specific implementation, this person is responsible for both.  This
makes sense I think -- however, if the arch maintainer used her/his veto
rights for this particular piece of code, it seems the arch maintainer
should then have to deal with the fallout from the deviation from the
community's direction.  But enforcing that would mean that we track when
veto rights are exercised, in some way -- which I can't imagine being
fun for anyone.

Re: Machine maintainer veto.

[Rich Felker]

On Fri, Jul 03, 2015 at 11:46:14AM -0400, Carlos O'Donell wrote:
> Thank you for that writeup. My initial text in the wiki is certainly a
> starting point, and I think we need to refine it. The community does need
> to decide how much power a machine maintainer should be given. Your comments
> go a long way to adding to and broadening this discussion.
> 
> Would it be sufficient to soften the language to say that the machine
> maintainer has blanket commit privileges for their machine, but that they
> remain responsible to the distributions, users, the project as a whole and
> that such blanket commit privileges should be used responsibly?
> 
> Therefore we couch the language in terms of responsibility to the community?

Yes, I think that's a good approach. I would also suggest something
along the lines of what I said about sticking to their areas of
expertise, but perhaps written in a way that's less accusatory. The
idea is that maintainers should have some authority over (but still be
responsible to distros/users/project/etc.) things that are genuinely
machine issues, but that authority does not/should not extend to
things that affect API, especially API correctness. (Note that most of
the 'ABI' issues I cited are actually API issues.)

Rich

Re: Machine maintainer veto.

[Rich Felker]

On Fri, Jul 03, 2015 at 12:38:36PM -0400, Rich Felker wrote:
> On Fri, Jul 03, 2015 at 11:46:14AM -0400, Carlos O'Donell wrote:
> > Thank you for that writeup. My initial text in the wiki is certainly a
> > starting point, and I think we need to refine it. The community does need
> > to decide how much power a machine maintainer should be given. Your comments
> > go a long way to adding to and broadening this discussion.
> > 
> > Would it be sufficient to soften the language to say that the machine
> > maintainer has blanket commit privileges for their machine, but that they
> > remain responsible to the distributions, users, the project as a whole and
> > that such blanket commit privileges should be used responsibly?
> > 
> > Therefore we couch the language in terms of responsibility to the community?
> 
> Yes, I think that's a good approach. I would also suggest something
> along the lines of what I said about sticking to their areas of
> expertise, but perhaps written in a way that's less accusatory. The
> idea is that maintainers should have some authority over (but still be
> responsible to distros/users/project/etc.) things that are genuinely
> machine issues, but that authority does not/should not extend to
> things that affect API, especially API correctness. (Note that most of
> the 'ABI' issues I cited are actually API issues.)

BTW, another specific issue I expect to come up again and again is
performance, particularly situations where a machine maintainer wants
a "fast" implementation of some function that's fast in the cases they
care about but O(n²) whereas the generic implementation is O(n) or
O(n*log(n)). I would say this is a situation where allowing the
bad-big-O implementation should require consensus at the project level
rather than machine maintainer veto discretion. The right solution in
general is almost sure to be something like the introsort solution
(self-measurement to fall back to generic code for cases where
worst-case performance is imminent) which is more work for the
machine-specific code contributor or machine maintainer but better for
the project.

Rich

Re: Machine maintainer veto.

[David Miller]

From: Rich Felker <dalias@libc.org>
Date: Fri, 3 Jul 2015 12:54:38 -0400

> BTW, another specific issue I expect to come up again and again is
> performance, particularly situations where a machine maintainer
> wants a "fast" implementation of some function that's fast in the
> cases they care about but O(n²) whereas the generic implementation
> is O(n) or O(n*log(n)). I would say this is a situation where
> allowing the bad-big-O implementation should require consensus at
> the project level rather than machine maintainer veto
> discretion. The right solution in general is almost sure to be
> something like the introsort solution (self-measurement to fall back
> to generic code for cases where worst-case performance is imminent)
> which is more work for the machine-specific code contributor or
> machine maintainer but better for the project.

I don't think the project as a whole gets to dictate what is the best
performance selection or set of tradeoffs for CPU X in all possible
scenerios.

Port specific performance tradeoffs exist (f.e.: people who use CPU X
do _NOT_ care about case Y of string function Z so we don't have to
optimize it well), and you cannot enfore tree wide rules in this area.

Some of the viewpoints expressed here really concern me as a cpu port
maintainer, and makes me want to have stronger rather than weaker
veto powers.

Re: Machine maintainer veto.

[Steven Munroe]

On Fri, 2015-07-03 at 11:34 -0400, Rich Felker wrote:
> On Fri, Jul 03, 2015 at 08:58:58AM -0400, Carlos O'Donell wrote:
> > On 07/03/2015 02:20 AM, Rich Felker wrote:
> > > On Thu, Jul 02, 2015 at 11:51:55PM -0400, Carlos O'Donell wrote:
> > >> Community,
> > >>
> > >> I have attempted to clarify what has always been in effect.
> > >> The machine maintainers have some level of veto for what goes
> > >> into their machine port. This allows some amount of control over
> > >> hardware support and ABI/API additions and removals.
> > >>
> > >> The key issue is to balance the project goals and the needs of
> > >> the users of the particular machine. To do that effectively the
> > >> machine maintainers have to have some level of veto to add or
> > >> remove things to the machine they know and understand best.
> > >>
> > >> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers
> > > 
> > > I know I don't have any standing to change it, but I just want to
> > > express a sentiment that I think this is bad policy. I can go into the
> > > details of why if anyone is interested.
> > 
> > I am absolutely interested in hearing the details.
> 
> On a level of principles, I think it violates the principle of
> consensus and preserves part of the toxic dictatorship impression that
> glibc used to have. The project has come a long way in shedding this
> impression, and in my opinion has become _the_ most open and healthy
> GNU project in terms of development process. I'd like to see this
> progress continue.
> 
There has to be a balance between the consensus of the community and the
requirements of the platform (as represented by the maintainer).

Democratic consensus can quickly turn into the mob rule, where ignorance
or malice of community members with strong but unfounded opinions can
also do great harm. Read up on the Death of Socrates or later stages of
the French revolution.

I think we need a balance of Powers definition that protects both
interests.

> In terms of impact, I think there's a false notion that machine
> maintainers' decisions don't impact anyone but their machine's users,
> and therefore that as long as they act in the interest of their users,
> nobody is harmed by their decisions. This is wrong for multiple
> reasons. User-visible changes potentially affect anyone writing
> portable software -- when their software behaves unexpectedly on the
> machine maintainer's target, they (not the glibc machine maintainers)
> are the ones who generally take the blame and deal with the costs of
> working around whatever the issue is. And while non-user-visible
> changes shouldn't affect applications, they do constrain general glibc
> development. Every bit of machine-specific code/behavior increases the
> cost of making changes to glibc in the affected area, and in a worst
> case machine-specifics can even constrain the implementation in ways
> that preclude serious improvements.
> 
> There's also the matter of bugs and conformance. Machine maintainers
> are not necessarily experts in standards conformance, portability, and
> related ABI issues, and unless they also happen to be experts in these
> fields, they should not be treated as such. At least the following
> bugs are results of machine-specific code/definitions being approved
> for inclusion in glibc without regard for how they affect standards
> conformance:
> 
> https://sourceware.org/bugzilla/show_bug.cgi?id=5945
> https://sourceware.org/bugzilla/show_bug.cgi?id=16437
> https://sourceware.org/bugzilla/show_bug.cgi?id=16919
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2f3a499e6b803802880aea1fb8d3b46f1959494f
> (I don't think this one has a glibc bug tracker issue yet, but the
> kernel-side "fix" was a regression and made the types wrong and
> inconsistent -- st_mode is 32-bit but ipc_perm.mode is 16-bit -- and
> it was made because the glibc userspace type was wrong.)
> 
> I don't know the history of how they were introduced, but the current
> policy, at least as expressed by Carlos, seems to make it possible for
> machine maintainers to dictate (by veto) the introduction of such bugs
> even if they were detected and objected to by others. I think we
> should be striving for the opposite: objective conformance checks that
> machine-specific definitions for new targets have to pass before being
> accepted into glibc.
> 
> Security also comes under the area of bugs. Being a machine maintainer
> should not entitle someone to do things in a machine-specific way
> that's less secure than the way the same thing is handled on other
> machines. Even if it only affects users of their target, the resulting
> CVE makes glibc look bad.
> 
> Rich
> 

Re: Machine maintainer veto.

[Siddhesh Poyarekar]

On 6 July 2015 at 18:56, Steven Munroe <munroesj@linux.vnet.ibm.comcom> wrote:
> There has to be a balance between the consensus of the community and the
> requirements of the platform (as represented by the maintainer).
>
> Democratic consensus can quickly turn into the mob rule, where ignorance
> or malice of community members with strong but unfounded opinions can
> also do great harm. Read up on the Death of Socrates or later stages of
> the French revolution.
>
> I think we need a balance of Powers definition that protects both
> interests.

Agreed, I don't think I intended to counter this at all when I asked
for the veto wording to be altered.  As a maintainer if you think
there is an effort (either due to ignorance or malice) to stonewall a
patchset, please feel free to call it out.

Siddhesh
-- 
http://siddhesh.in

Re: Machine maintainer veto.

[Siddhesh Poyarekar]

On 3 July 2015 at 22:28, David Miller <davem@davemloft.net> wrote:
> I don't think the project as a whole gets to dictate what is the best
> performance selection or set of tradeoffs for CPU X in all possible
> scenerios.
>
> Port specific performance tradeoffs exist (f.e.: people who use CPU X
> do _NOT_ care about case Y of string function Z so we don't have to
> optimize it well), and you cannot enfore tree wide rules in this area.

Absolutely, but if the concerns or criticism have a sound base, the
arch maintainer should not simply ignore them.  Maybe another way to
look at it is that the arch maintainer is deemed correct about
decisions on his/her port until proven wrong.

Siddhesh
-- 
http://siddhesh.in

Re: Machine maintainer veto.

[Richard Earnshaw]

On 03/07/15 16:48, Carlos O'Donell wrote:
> On 07/03/2015 11:44 AM, Siddhesh Poyarekar wrote:
>> On 3 July 2015 at 09:21, Carlos O'Donell <carlos@redhat.com> wrote:
>>> The key issue is to balance the project goals and the needs of
>>> the users of the particular machine. To do that effectively the
>>> machine maintainers have to have some level of veto to add or
>>> remove things to the machine they know and understand best.
>>>
>>> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers
>>
>> I don't quite agree with this:
>>
>>     "... This veto can be used to prevent changes that break ABI or
>> API, or it can be used to include new ABI support for the machine even
>> if there is no consensus on how best to proceed"
>>
>> While I acknowledge that arch maintainers have a better understanding
>> of their architecture, it doesn't automatically translate to a deep
>> enough understanding of glibc in general and that can break things in
>> irreversible ways.  Also, arch maintainers are as accountable to the
>> glibc community as they are to their customers or their organization,
>> so ignoring consensus goes against that principle.
> 
> That is a very good point.
> 
>> There is a middle ground though, where arch maintainers have enough
>> freedom to decide what goes into their port as long as it doesn't
>> conflict with the goals of the community.  So it should be OK if a
>> maintainer chooses to ignore objection for a patch from a community
>> member on minor differences.  However, if there is sustained
>> opposition from the community in general, it would be inappropriate
>> for the maintainer to ignore that feedback.
> 
> This calls for consensus.

To me, consensus means 'a lack of objection'.  So if anyone is objecting
then you do not have consensus.  That means, in effect, that everyone
has a veto!

The problem then is that this could lead to deadlock and we'd need a way
to resolve that.  I'd hope that in general we'd want to either ensure
that the expressed concerns were not relevant (a misunderstanding) or
that an appropriate alternative that was generally acceptable could be
found.

I think also, the right to exercise (and maintain) a veto should come
with the responsibility to work *constructively* with the proposer to
find an acceptable resolution: it can't be a simple stone-wall.

What I think would be very dangerous would be if consensus came to mean
'simple majority'.

Just my 2 penn'orth.

R.

> 
>> I admit this is still vague, but ISTM to be a better place to be than
>> a full veto described in the wiki page.
> 
> Agreed. I'll let others comment a bit further before I summarize the
> positions with a new block of text to describe the machine maintainers
> responsibilities.
> 
> Cheers,
> Carlos.
> 

Re: Machine maintainer veto.

[Carlos O'Donell]

On 07/06/2015 12:08 PM, Richard Earnshaw wrote:
> On 03/07/15 16:48, Carlos O'Donell wrote:
>> On 07/03/2015 11:44 AM, Siddhesh Poyarekar wrote:
>>> On 3 July 2015 at 09:21, Carlos O'Donell <carlos@redhat.com> wrote:
>>>> The key issue is to balance the project goals and the needs of
>>>> the users of the particular machine. To do that effectively the
>>>> machine maintainers have to have some level of veto to add or
>>>> remove things to the machine they know and understand best.
>>>>
>>>> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers
>>>
>>> I don't quite agree with this:
>>>
>>>     "... This veto can be used to prevent changes that break ABI or
>>> API, or it can be used to include new ABI support for the machine even
>>> if there is no consensus on how best to proceed"
>>>
>>> While I acknowledge that arch maintainers have a better understanding
>>> of their architecture, it doesn't automatically translate to a deep
>>> enough understanding of glibc in general and that can break things in
>>> irreversible ways.  Also, arch maintainers are as accountable to the
>>> glibc community as they are to their customers or their organization,
>>> so ignoring consensus goes against that principle.
>>
>> That is a very good point.
>>
>>> There is a middle ground though, where arch maintainers have enough
>>> freedom to decide what goes into their port as long as it doesn't
>>> conflict with the goals of the community.  So it should be OK if a
>>> maintainer chooses to ignore objection for a patch from a community
>>> member on minor differences.  However, if there is sustained
>>> opposition from the community in general, it would be inappropriate
>>> for the maintainer to ignore that feedback.
>>
>> This calls for consensus.
> 
> To me, consensus means 'a lack of objection'.  So if anyone is objecting
> then you do not have consensus.  That means, in effect, that everyone
> has a veto!

Yes, and no.

Sustained opposition is different from a technical objection. You may
object, but realize that your objection is technically infeasible until
say a newer technology is available. However, your objection is recorded
for all to read at a future date when someone is reading the mailing
list with respect to the given change (and perhaps the newer technology
is now available).

See:
https://sourceware.org/glibc/wiki/Consensus

You could argue that an important part of the concerned interests have
been met if the distribution maintainers agree with your change, at which
point the machine maintainer might argue to checkin the patch given the
distro review.

Your response would have to make this crystal clear.

> The problem then is that this could lead to deadlock and we'd need a way
> to resolve that.  I'd hope that in general we'd want to either ensure
> that the expressed concerns were not relevant (a misunderstanding) or
> that an appropriate alternative that was generally acceptable could be
> found.

Exactly.

> I think also, the right to exercise (and maintain) a veto should come
> with the responsibility to work *constructively* with the proposer to
> find an acceptable resolution: it can't be a simple stone-wall.

Agreed, which is why I think veto is perhaps the wrong word to use in
the maintainer description.

> What I think would be very dangerous would be if consensus came to mean
> 'simple majority'.

Agreed.

Cheers,
Carlos.

Re: Machine maintainer veto.

[Steven Munroe]

On Mon, 2015-07-06 at 17:08 +0100, Richard Earnshaw wrote:
> On 03/07/15 16:48, Carlos O'Donell wrote:
> > On 07/03/2015 11:44 AM, Siddhesh Poyarekar wrote:
> >> On 3 July 2015 at 09:21, Carlos O'Donell <carlos@redhat.com> wrote:
> >>> The key issue is to balance the project goals and the needs of
> >>> the users of the particular machine. To do that effectively the
> >>> machine maintainers have to have some level of veto to add or
> >>> remove things to the machine they know and understand best.
> >>>
> >>> https://sourceware.org/glibc/wiki/MAINTAINERS#Machine_maintainers
> >>
> >> I don't quite agree with this:
> >>
> >>     "... This veto can be used to prevent changes that break ABI or
> >> API, or it can be used to include new ABI support for the machine even
> >> if there is no consensus on how best to proceed"
> >>
> >> While I acknowledge that arch maintainers have a better understanding
> >> of their architecture, it doesn't automatically translate to a deep
> >> enough understanding of glibc in general and that can break things in
> >> irreversible ways.  Also, arch maintainers are as accountable to the
> >> glibc community as they are to their customers or their organization,
> >> so ignoring consensus goes against that principle.
> > 
> > That is a very good point.
> > 
> >> There is a middle ground though, where arch maintainers have enough
> >> freedom to decide what goes into their port as long as it doesn't
> >> conflict with the goals of the community.  So it should be OK if a
> >> maintainer chooses to ignore objection for a patch from a community
> >> member on minor differences.  However, if there is sustained
> >> opposition from the community in general, it would be inappropriate
> >> for the maintainer to ignore that feedback.
> > 
> > This calls for consensus.
> 
> To me, consensus means 'a lack of objection'.  So if anyone is objecting
> then you do not have consensus.  That means, in effect, that everyone
> has a veto!
> 
> The problem then is that this could lead to deadlock and we'd need a way
> to resolve that.  I'd hope that in general we'd want to either ensure
> that the expressed concerns were not relevant (a misunderstanding) or
> that an appropriate alternative that was generally acceptable could be
> found.
> 
> I think also, the right to exercise (and maintain) a veto should come
> with the responsibility to work *constructively* with the proposer to
> find an acceptable resolution: it can't be a simple stone-wall.
> 
It takes at least two to be constructive. Some like argue perfection or
require convincing everyone (not just GLIBC community members) to change
what they are already doing to a more "correct" way. Like saying users
are stupid and they doing it wrong is not constructive. 

Straw-man, slipper slope, moral hazards argument should be excluded,
because there there no rational response to a an irrational argument. 

If we don't restrain this behavior, we allow individuals to block
platform specific patches indefinitely.

> What I think would be very dangerous would be if consensus came to mean
> 'simple majority'.
> 
> Just my 2 penn'orth.
> 
> R.
> 
> > 
> >> I admit this is still vague, but ISTM to be a better place to be than
> >> a full veto described in the wiki page.
> > 
> > Agreed. I'll let others comment a bit further before I summarize the
> > positions with a new block of text to describe the machine maintainers
> > responsibilities.
> > 
> > Cheers,
> > Carlos.
> > 
> 

Re: Machine maintainer veto.

[Torvald Riegel]

On Mon, 2015-07-06 at 13:09 -0500, Steven Munroe wrote:
> It takes at least two to be constructive.

Agreed.

> Some like argue perfection or
> require convincing everyone (not just GLIBC community members) to change
> what they are already doing to a more "correct" way.

I don't understand this sentence.

> Like saying users
> are stupid and they doing it wrong is not constructive. 

Agreed on the "are stupid", but saying that they are doing something
that is not something glibc wants to support is not something that's
necessarily bad.

> Straw-man, slipper slope, moral hazards argument should be excluded,
> because there there no rational response to a an irrational argument. 
> 
> If we don't restrain this behavior, we allow individuals to block
> platform specific patches indefinitely.

That goes both ways.  You can apply the same concerns to the other way
around.  We need to find rules that are acceptable for all sides.

Re: Machine maintainer veto.

[Torvald Riegel]

On Mon, 2015-07-06 at 08:26 -0500, Steven Munroe wrote:
> There has to be a balance between the consensus of the community and the
> requirements of the platform (as represented by the maintainer).

Note that the consensus of the community may be a requirement for the
community too (ie, there may be conflicting requirements).

> Democratic consensus can quickly turn into the mob rule, where ignorance
> or malice of community members with strong but unfounded opinions can
> also do great harm. Read up on the Death of Socrates or later stages of
> the French revolution.
> 
> I think we need a balance of Powers definition that protects both
> interests.

What would you suggest as mechanism to "protect" the community from
machine maintainers making decisions that conflict with the community's
interests?

For example, would it be okay for you if machine maintainers would have
to be responsible for their changes on their won if they exercised their
"veto rights"?  (That is, if they go their own machine-specific way
against a different community consensus, they can't necessarily expect
generic code to be compatible with their own way, nor authors of generic
code to put a lot of effort into making it compatible.)

Re: Machine maintainer veto.

[Steven Munroe]

On Tue, 2015-07-07 at 00:10 +0200, Torvald Riegel wrote:
> On Mon, 2015-07-06 at 08:26 -0500, Steven Munroe wrote:
> > There has to be a balance between the consensus of the community and the
> > requirements of the platform (as represented by the maintainer).
> 
> Note that the consensus of the community may be a requirement for the
> community too (ie, there may be conflicting requirements).
> 
> > Democratic consensus can quickly turn into the mob rule, where ignorance
> > or malice of community members with strong but unfounded opinions can
> > also do great harm. Read up on the Death of Socrates or later stages of
> > the French revolution.
> > 
> > I think we need a balance of Powers definition that protects both
> > interests.
> 
> What would you suggest as mechanism to "protect" the community from
> machine maintainers making decisions that conflict with the community's
> interests?
> 
> For example, would it be okay for you if machine maintainers would have
> to be responsible for their changes on their won if they exercised their
> "veto rights"?  (That is, if they go their own machine-specific way
> against a different community consensus, they can't necessarily expect
> generic code to be compatible with their own way, nor authors of generic
> code to put a lot of effort into making it compatible.)
> 

Yes I agree that I would not expect the community to support my platform
specific tricks that are not part of the GLIBC API. We (the community)
agreed to carved out the ./sys/platform/ extension for exactly this
purpose. So ./sys/platform/ppc.h is left for the PPC maintainers

But at the same time I would not expect the community (really just a few
individuals with strong opinions on the topic) to force a platform
maintainer to implement a GLIBC feature exactly like another platform.
Especially if the suggested design is impractical or highly inefficient
for that platform.

Over the years GLIBC has developed and supported mechanisms for platform
specific extension to allow all platforms to contribute to the common
implementation. I assume this will continue.

I would not expect and would actively oppose assertions that specific
feature like IFUNC and TLS storage are the only solution to specific
design problem. Because each platform has a different set of hardware
software trade-offs. 

Re: Machine maintainer veto.

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 739 bytes --]

On 06 Jul 2015 13:09, Steven Munroe wrote:
> It takes at least two to be constructive. Some like argue perfection or
> require convincing everyone (not just GLIBC community members) to change
> what they are already doing to a more "correct" way. Like saying users
> are stupid and they doing it wrong is not constructive. 

it doesn't logically follow that a bad solution should be deployed purely
because users can't be bothered to improve their practices.  so attempting
to justify a change based purely on "this is the only API users will use"
doesn't really fly in my mind.

(don't try and extrapolate this into specifics on the ppc/hwcaps issue as
that is a waste of time -- this thread is talking generalities.)
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Machine maintainer veto.

[Steven Munroe]

On Tue, 2015-07-07 at 00:03 +0200, Torvald Riegel wrote:
> On Mon, 2015-07-06 at 13:09 -0500, Steven Munroe wrote:
> > It takes at least two to be constructive.
> 
> Agreed.
> 
> > Some like argue perfection or
> > require convincing everyone (not just GLIBC community members) to change
> > what they are already doing to a more "correct" way.
> 
> I don't understand this sentence.
> 
Sorry lost in translation ... Said more plainly.

Have you looked at what is going on PHP, OpenSSL, Boost, FFmpeg. ... and
the thousands of other packages that make up a complete distro?

I suspect you will not be happy with what you find.

So if you think __builtin_cpu_supports() is a bad solution and they
should be using STD_GNU_IFUNC then you need to convince them.

And __builtin_cpu_supports exist and has existed for Intel in compilers
including GCC for a while. So you have to convince them.

Arguing with me, will not help you achieve your goal.

Just being pragmatic ...

> > Like saying users
> > are stupid and they doing it wrong is not constructive. 
> 
> Agreed on the "are stupid", but saying that they are doing something
> that is not something glibc wants to support is not something that's
> necessarily bad.
> 
But they you should be talking to them, explaining to them, convincing
them.

Holding a platform maintainer hostage does not help you.

> > Straw-man, slipper slope, moral hazards argument should be excluded,
> > because there there no rational response to a an irrational argument. 
> > 
> > If we don't restrain this behavior, we allow individuals to block
> > platform specific patches indefinitely.
> 
> That goes both ways.  You can apply the same concerns to the other way
> around.  We need to find rules that are acceptable for all sides.
> 

Re: Machine maintainer veto.

[Torvald Riegel]

On Wed, 2015-07-08 at 09:48 -0500, Steven Munroe wrote:
> On Tue, 2015-07-07 at 00:03 +0200, Torvald Riegel wrote:
> > > Like saying users
> > > are stupid and they doing it wrong is not constructive. 
> > 
> > Agreed on the "are stupid", but saying that they are doing something
> > that is not something glibc wants to support is not something that's
> > necessarily bad.
> > 
> But they you should be talking to them, explaining to them, convincing
> them.
> 
> Holding a platform maintainer hostage does not help you.

I don't think that the goal is to hold the platform maintainer hostage.
If the project, as a whole, should decide to not support something, then
it can do that.  Making sure that users at least understand the
background of the decision is worthwile, of course; likewise, serving
users makes sense too :)  However, the project must be able to make
decisions even if not all users agree with that decision.

One could also argue that users should not blame a platform maintainer
for a decision of the project as a whole -- IOW, the hostage situation
is not just due to the project's decisions.

As another example (with a somewhat GCC-ish background), consider that
likely lots of code out there assumes a memory model like x86' TSO
(e.g., some glibc code did).  It would probably be useful to many users
if we'd just promise those guarantees.  If such users would approach you
as platform maintainer and request you to give you those guarantees, I
guess you would decline this request :)  It would be even harder for
them to adapt their code for a weak memory model than in the specific
case that you want to see support for.

Re: Machine maintainer veto.

[Joseph Myers]

On Fri, 3 Jul 2015, Carlos O'Donell wrote:

> Would it be sufficient to soften the language to say that the machine
> maintainer has blanket commit privileges for their machine, but that they
> remain responsible to the distributions, users, the project as a whole and
> that such blanket commit privileges should be used responsibly?

I think machine maintainers are in exactly the same position as other 
subsystem maintainers.  That is, they have the discretion to assume 
consensus for a change of their own without waiting for other opinions, 
but if the discussion shows there is no consensus after all then the 
change still needs revising or reverting.  (This does not mean that all 
objections are relevant for establishing lack of consensus, e.g. if the 
reasons given are speculative, based on false analogies to other machines 
or a lack of understanding of the change and its context or themselves 
ignore other established consensus.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Machine maintainer veto.

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 2258 bytes --]

On Fri, 3 Jul 2015, David Miller wrote:

> From: Rich Felker <dalias@libc.org>
> Date: Fri, 3 Jul 2015 12:54:38 -0400
> 
> > BTW, another specific issue I expect to come up again and again is
> > performance, particularly situations where a machine maintainer
> > wants a "fast" implementation of some function that's fast in the
> > cases they care about but O(n²) whereas the generic implementation
> > is O(n) or O(n*log(n)). I would say this is a situation where
> > allowing the bad-big-O implementation should require consensus at
> > the project level rather than machine maintainer veto
> > discretion. The right solution in general is almost sure to be
> > something like the introsort solution (self-measurement to fall back
> > to generic code for cases where worst-case performance is imminent)
> > which is more work for the machine-specific code contributor or
> > machine maintainer but better for the project.
> 
> I don't think the project as a whole gets to dictate what is the best
> performance selection or set of tradeoffs for CPU X in all possible
> scenerios.
> 
> Port specific performance tradeoffs exist (f.e.: people who use CPU X
> do _NOT_ care about case Y of string function Z so we don't have to
> optimize it well), and you cannot enfore tree wide rules in this area.

I think that's only valid for constant factors, not for cases such as 
quadratic strstr implementations.  The exponent should be considered part 
of the architecture-independent, OS-independent GNU API for glibc, and so 
increased exponents should be unacceptable in any machine-specific 
implementation (and grounds for removal of such an implementation as a 
security regression - strstr can be used with untrusted inputs - if such 
an O(n^2) implementation gets in where the machine-independent 
implementation is O(n), and the machine maintainer doesn't fix it 
promptly).

The strstr case is also a case involving cooperation between GNU projects 
- gnulib has a test that strstr works in linear time, building a 
replacement implementation if libc's strstr is quadratic, and if gnulib 
considers a glibc function buggy that's a problem that needs to be worked 
out in collaboration between the projects.

-- 
Joseph S. Myers
joseph@codesourcery.com