Re: cmp missing from base

Re: cmp missing from base

[Warren Young]

On May 6, 2016, at 3:53 AM, Thomas Wolff <towo@towo.net> wrote:
> 
> after a recent fresh installation of cygwin, I was surprised that `cmp` was missing, which is part of the traditional Unix base commands.
> I think the diffutils package should be part of the base installation.

We’ve never really had a hard rule on what is in Base and what isn’t.  It’s always been a judgement call.

I wonder if the rule should just be “POSIX”?  That is, if it’s on this page, it should be in Base:

  http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html

That would exclude other things we’ve always excluded, such as Perl.

I’m not suggesting that we make this rule a strict one.  Most importantly, it cannot be an exclusion rule: Cygwin must contain things not in POSIX.  I’m just suggesting that it would be nice if Cygwin were as close to POSIX as practical out-of-the-box.

By that latter, I mean without extra effort other than adjusting some setup.hint files.  I mean, if there is a command on that list that doesn’t even have a Cygwin package, I don’t mean to propose with this rule that someone must go out and package it just to satisfy POSIX.

As a counterexample, that list contains pax(1), which is currently in Archive, not Base, so by that rule, pax(1) should also move to Base.

By that very example, though, I can argue against this proposed rule: as I understand it, pax(1) was added to POSIX at the same time they dropped cpio(1) and tar(1), thinking that by doing so, they’d change existing practice, moving everyone over to pax(1).  That just created a Standard in the XKCD sense:

  https://xkcd.com/927/
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cmp missing from base

[Thomas Wolff]

Am 07.05.2016 um 03:41 schrieb Warren Young:
> On May 6, 2016, at 3:53 AM, Thomas Wolff <towo@towo.net> wrote:
>> after a recent fresh installation of cygwin, I was surprised that `cmp` was missing, which is part of the traditional Unix base commands.
>> I think the diffutils package should be part of the base installation.
> We’ve never really had a hard rule on what is in Base and what isn’t.  It’s always been a judgement call.
>
> I wonder if the rule should just be “POSIX”?  That is, if it’s on this page, it should be in Base:
>
>    http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html
>
> That would exclude other things we’ve always excluded, such as Perl.
>
> I’m not suggesting that we make this rule a strict one.  Most importantly, it cannot be an exclusion rule: Cygwin must contain things not in POSIX.  I’m just suggesting that it would be nice if Cygwin were as close to POSIX as practical out-of-the-box.
>
> By that latter, I mean without extra effort other than adjusting some setup.hint files.  I mean, if there is a command on that list that doesn’t even have a Cygwin package, I don’t mean to propose with this rule that someone must go out and package it just to satisfy POSIX.
>
> As a counterexample, that list contains pax(1), which is currently in Archive, not Base, so by that rule, pax(1) should also move to Base.
>
> By that very example, though, I can argue against this proposed rule: as I understand it, pax(1) was added to POSIX at the same time they dropped cpio(1) and tar(1), thinking that by doing so, they’d change existing practice, moving everyone over to pax(1).  That just created a Standard in the XKCD sense:
>
>    https://xkcd.com/927/
Thanks, Warren, for some metaphysical insights :)
Right, there isn't and shouldn't be a strict rule. But maybe there can 
be a guideline, and more a guideline of common practice than of a 15th 
standard. By decade-long practice as a Unix user, I was simply expecting 
cmp to be basic, like I would expect tar. Maybe others would agree with 
that, or maybe not...
Cheers
Thomas

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cmp missing from base

[Warren Young]

On May 7, 2016, at 12:20 AM, Thomas Wolff <towo@towo.net> wrote:
> 
> Am 07.05.2016 um 03:41 schrieb Warren Young:
>> On May 6, 2016, at 3:53 AM, Thomas Wolff <towo@towo.net> wrote:
>>> after a recent fresh installation of cygwin, I was surprised that `cmp` was missing, which is part of the traditional Unix base commands.
>>> I think the diffutils package should be part of the base installation.
>> We’ve never really had a hard rule on what is in Base and what isn’t.  It’s always been a judgement call.
>> 
>> I wonder if the rule should just be “POSIX”?

Over the weekend, I realized that the rule can’t be that simple, because that would drag in GCC.

> maybe there can be a guideline, and more a guideline of common practice than of a 15th standard.

The problem, of course, is that there is no common practice.

I think the closest parallel to the Cygwin Base philosophy is the minimal or base installs of several other *ixes, particularly the BSDs.  It is also something like a throwback to the SysV Unixes, prior to about 1992, when you could count on things like X and the C compiler to be separate installs.

But even that guideline isn’t especially useful, since often such OSes *do* include the C compiler and all its support tooling.

Ultimately, I think Cygwin Base is whatever Corinna and Yaakov say it is. :)
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cmp missing from base

[Warren Young]

On May 6, 2016, at 7:41 PM, Warren Young <wyml@etr-usa.com> wrote:
> 
> On May 6, 2016, at 3:53 AM, Thomas Wolff <towo@towo.net> wrote:
>> 
>> after a recent fresh installation of cygwin, I was surprised that `cmp` was missing, which is part of the traditional Unix base commands.
>> I think the diffutils package should be part of the base installation.
> 
> I wonder if the rule should just be “POSIX”?

I’ve come up with a much better rule: mimic the Minimal package set for RHEL.[1]

Not only is this a nice nod toward the primary corporate sponsor for Cygwin, it turns out that the current Base set is already almost exactly the same thing after applying some fairly obvious exclusion rules:


1. We can exclude from consideration all of those packages that do something for RHEL that Cygwin leaves to Windows.  e.g. chrony, dmidecode, iptables, the Linux kernel, etc.

2. We can also exclude any user space program that is included in RHEL not for its own sake but instead primarily to satisfy a dependency for some other user space program that Cygwin doesn’t need.  Examples are curl, gpg, lua, make, and python.

3. We can also exclude all libraries.  Discussions of which libraries the Base packages should depend on are wholly separate from this one.  For example, RHEL’s coreutil depends on gmp, while Cygwin’s does not, but this does not tell us that Cygwin’s coreutils should be rebuilt to depend on gmp.

4. Perhaps controversially, I think we can also leave out anything where the use case set is far narrower on Cygwin due to the nature of the platform.  This is my justification for excluding openssh-server and postfix: you’re not likely to find a rack full of 1U Windows boxes running Cygwin in a datacenter, accessible only via SSH and sending important messages to the local mail spool.  Therefore, RHEL’s justification for including such packages in its Minimal set simply doesn’t apply to Cygwin.


After applying all of those rules, the only packages in RHEL 7 Minimal that aren’t in RHEL Base are diffutils and openssh-client, both of which I’d be happy to see in Cygwin Base.

I welcome anyone else to go through the RHEL/CentOS 7 Minimal package set[1] and see if they can provide a justification for one that currently isn’t in Cygwin Base which doesn’t run up against any of the exclusion rules above.

In terms of my original reply to this thread, adopting this rule removes one of the “standards” from the running, a good thing IMHO.



[1] https://gist.github.com/wyoung/42e7f6081e8d695f9656621be0f58eae
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cmp missing from base

[Warren Young]

On May 9, 2016, at 12:29 PM, Warren Young <wyml@etr-usa.com> wrote:
> 
> On May 6, 2016, at 7:41 PM, Warren Young <wyml@etr-usa.com> wrote:
>> 
>> On May 6, 2016, at 3:53 AM, Thomas Wolff <towo@towo.net> wrote:
>>> 
>>> after a recent fresh installation of cygwin, I was surprised that `cmp` was missing, which is part of the traditional Unix base commands.
>>> I think the diffutils package should be part of the base installation.
>> 
>> I wonder if the rule should just be “POSIX”?
> 
> I’ve come up with a much better rule: mimic the Minimal package set for RHEL.[1]

I decided to try a practical experiment: apply a simplified version of my rule set to a Cygwin Base installation.  My thought was that if the goal is to change Cygwin Base to be as close to RHEL Minimal as is practical given platform differences, that my exclusion rules 2 and 4 should be dropped.

A 64-bit Cygwin Base installation is currently 112 MiB as reckoned by du -sh on my system.  If you take that and add the packages in CentOS 7’s Minimal install that aren’t excluded by my exclusion rules 1 and 3, a “Cygwin Minimal” installation is 266 MiB.

That’s quite a jump in absolute terms, but still smallish by today’s standards.  The corresponding RHEL 7 Minimal installation is 1.2 GiB, by comparison.

This does drag in one major package that isn’t in RHEL 7’s Minimal set: perl, by way of postfix.  We’ve had repeated cases in the past where something in Base indirectly depended on Perl and dragged it in.  Maybe it’s time to give up on excluding Perl?
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cmp missing from base

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1185 bytes --]

On 05/09/2016 12:29 PM, Warren Young wrote:
>>> I think the diffutils package should be part of the base installation.

Based on just the start of the thread, I was already considering adding
diffutils to Base the next time I package it...

> 3. We can also exclude all libraries.  Discussions of which libraries the Base packages should depend on are wholly separate from this one.  For example, RHEL’s coreutil depends on gmp, while Cygwin’s does not, but this does not tell us that Cygwin’s coreutils should be rebuilt to depend on gmp.

Umm, the cygwin build of coreutils DOES depend on gmp, unless I made a
mistake packaging it.  [/me goes and checks...] Yep - libgmp10 is in the
coreutils 'requires:' setup.hint.

> 
> After applying all of those rules, the only packages in RHEL 7 Minimal that aren’t in RHEL Base are diffutils and openssh-client, both of which I’d be happy to see in Cygwin Base.

...But you've added even more good justification for me to do so.  All
right, sounds like something I should do when I next have the chance.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: cmp missing from base

[Warren Young]

On May 9, 2016, at 2:57 PM, Eric Blake <eblake@redhat.com> wrote:
> 
> On 05/09/2016 12:29 PM, Warren Young wrote:
>> RHEL’s coreutil depends on gmp, while Cygwin’s does not, but this does not tell us that Cygwin’s coreutils should be rebuilt to depend on gmp.
> 
> Umm, the cygwin build of coreutils DOES depend on gmp

I was just quickly vdiffing package lists, and didn’t fix up a mismatch in package naming: RHEL calls it gmp, Cygwin calls it libgmp10.  Oopsie.

The point still stands, though, because there are many other examples I could have chosen: glib, sqlite, expat...
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cmp missing from base

[DAGwyn]

I was unpleasantly surprised to find that the  "cmp" utility had been 
removed from the Cygwin base, and a search for "cmp" in  the Setup installer 
didn't find it.  (It used to exist.)  I eventually  found it hidden in the 
"diffutils" package.
Upon searching the mailing list I  found some discussion, most of it being 
clueless.  "cmp" is widely used in  Unix makefiles!  It's a basic utility 
that has been around for over 40  years.  Utilities like that are so essential 
that it disrupts users when  they are removed.  There is a reason that 
POSIX specifies them!  


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cmp missing from base

[Ken Brown]

On 10/30/2016 4:05 AM, DAGwyn@aol.com wrote:
> I was unpleasantly surprised to find that the  "cmp" utility had been
> removed from the Cygwin base, and a search for "cmp" in  the Setup installer
> didn't find it.  (It used to exist.)  I eventually  found it hidden in the
> "diffutils" package.
> Upon searching the mailing list I  found some discussion, most of it being
> clueless.  "cmp" is widely used in  Unix makefiles!  It's a basic utility
> that has been around for over 40  years.  Utilities like that are so essential
> that it disrupts users when  they are removed.  There is a reason that
> POSIX specifies them!

I think the diffutils maintainer intended to add it to Base 
(https://sourceware.org/ml/cygwin/2016-05/msg00099.html), but maybe he 
forgot.  Eric?

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: cmp missing from base

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 1098 bytes --]

On 10/30/2016 09:58 AM, Ken Brown wrote:
> On 10/30/2016 4:05 AM, DAGwyn@aol.com wrote:
>> I was unpleasantly surprised to find that the  "cmp" utility had been
>> removed from the Cygwin base, and a search for "cmp" in  the Setup
>> installer
>> didn't find it.  (It used to exist.)  I eventually  found it hidden in
>> the
>> "diffutils" package.
>> Upon searching the mailing list I  found some discussion, most of it
>> being
>> clueless.  "cmp" is widely used in  Unix makefiles!  It's a basic utility
>> that has been around for over 40  years.  Utilities like that are so
>> essential
>> that it disrupts users when  they are removed.  There is a reason that
>> POSIX specifies them!
> 
> I think the diffutils maintainer intended to add it to Base
> (https://sourceware.org/ml/cygwin/2016-05/msg00099.html), but maybe he
> forgot.  Eric?

Thanks for the reminder; I've finally found time to rectify it today.
Look for 3.5-2 to hit a mirror near you soon.


-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: cmp missing from base

[Steven Penny]

> There is a reason that POSIX specifies them!

Agreed. Both "cmp" and "diff" are required by POSIX:

http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple