Python extraneous dependencies

Python extraneous dependencies

[Steven Penny]

I noticed that "python36" requires "binutils". Further, I noticed this
dependency chain:

    python36 > libuuid-devel > pkg-config > libglib2.0_0

"binutils" is 5,863,216 bytes and "libglib" is 3,044,044 bytes. I am of the
opinion we should not be including large dependencies like these unless they are
actually needed. From my admittedly brief testing, they are not needed:

    $ setup-x86_64 -P python36
    $ setup-x86_64 -x binutils,libglib2.0_0,libuuid-devel,pkg-config
    $ python3 -c 'print("hello world")'
    hello world


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[LRN]

[-- Attachment #1.1: Type: text/plain, Size: 615 bytes --]

On 24.02.2019 3:29, Steven Penny wrote:
> I noticed that "python36" requires "binutils". Further, I noticed this
> dependency chain:
> 
>     python36 > libuuid-devel > pkg-config > libglib2.0_0
> 
> "binutils" is 5,863,216 bytes and "libglib" is 3,044,044 bytes. I am of the
> opinion we should not be including large dependencies like these

pkg-config has a built-in copy of glib partly for this reason.

P.S. Unrelated: Just noticed that cygwin ML auto-inserted signature is slightly
broken - it should have a space after "--", but doesn't. Whoever maintains this
mailing list, please fix that.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Python extraneous dependencies

[Yaakov Selkowitz]

On Sat, 2019-02-23 at 16:29 -0800, Steven Penny wrote:
> I noticed that "python36" requires "binutils".

This is needed for ctypes.util.find_library().  FWIW, on Linux, not
only binutils is used, but also gcc.

> Further, I noticed this dependency chain:
> 
>     python36 > libuuid-devel > pkg-config > libglib2.0_0

Until 3.7, the uuid stdlib module loads libuuid via ctypes, hence the
dependency.  In 3.7, there is a compiled binding, and so the -devel
dependency was dropped.  Also, pkgconf will soon be providing and
replacing pkg-config, which will cause the glib2.0 dependency to be
dropped.

--
Yaakov



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[Steven Penny]

On Sun, 24 Feb 2019 17:42:36, Yaakov Selkowitz wrote:
> On Sat, 2019-02-23 at 16:29 -0800, Steven Penny wrote:
>> I noticed that "python36" requires "binutils".
>
> This is needed for ctypes.util.find_library().  FWIW, on Linux, not
> only binutils is used, but also gcc.
>
>> Further, I noticed this dependency chain:
>> 
>>     python36 > libuuid-devel > pkg-config > libglib2.0_0
>
> Until 3.7, the uuid stdlib module loads libuuid via ctypes, hence the
> dependency.  In 3.7, there is a compiled binding, and so the -devel
> dependency was dropped.  Also, pkgconf will soon be providing and
> replacing pkg-config, which will cause the glib2.0 dependency to be
> dropped.

thanks. i was interested further in comparison so i took this file:

https://python.org/ftp/python/3.6.8/python-3.6.8-embed-amd64.zip

then use tar for fair comparison:

    tar -a -c -f python-3.6.8-embed-amd64.tar.xz python-3.6.8-embed-amd64

result:

    $ wc -c python-3.6.8-embed-amd64.tar.xz
    5630160 python-3.6.8-embed-amd64.tar.xz

compare this with cygwin version:

    $ wc -c python36-3.6.8-1.tar.xz
    5746648 python36-3.6.8-1.tar.xz

and that doesnt include the other packages i previously discussed. so it almost
makes more sense to not even use the cygwin version?


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[Yaakov Selkowitz]

On Mon, 2019-02-25 at 04:43 -0800, Steven Penny wrote:
> On Sun, 24 Feb 2019 17:42:36, Yaakov Selkowitz wrote:
> > On Sat, 2019-02-23 at 16:29 -0800, Steven Penny wrote:
> > > I noticed that "python36" requires "binutils".
> > 
> > This is needed for ctypes.util.find_library().  FWIW, on Linux, not
> > only binutils is used, but also gcc.
> > 
> > > Further, I noticed this dependency chain:
> > > 
> > >     python36 > libuuid-devel > pkg-config > libglib2.0_0
> > 
> > Until 3.7, the uuid stdlib module loads libuuid via ctypes, hence the
> > dependency.  In 3.7, there is a compiled binding, and so the -devel
> > dependency was dropped.  Also, pkgconf will soon be providing and
> > replacing pkg-config, which will cause the glib2.0 dependency to be
> > dropped.
> 
> thanks. i was interested further in comparison so i took this file:
> 
> https://python.org/ftp/python/3.6.8/python-3.6.8-embed-amd64.zip
> 
> then use tar for fair comparison:
> 
>     tar -a -c -f python-3.6.8-embed-amd64.tar.xz python-3.6.8-embed-amd64

Did you look at the contents of that zip file?  It's quite minimal, and
therefore not a fair comparison at all.

> and that doesnt include the other packages i previously discussed. so it almost
> makes more sense to not even use the cygwin version?

Hardly, but discussing how/why *not* to use Cygwin is OT IMO.

--
Yaakov



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[Ray Donnelly]

On Tue, Feb 26, 2019 at 5:12 AM Yaakov Selkowitz <yselkowitz@cygwin.com> wrote:
>
> On Mon, 2019-02-25 at 04:43 -0800, Steven Penny wrote:
> > On Sun, 24 Feb 2019 17:42:36, Yaakov Selkowitz wrote:
> > > On Sat, 2019-02-23 at 16:29 -0800, Steven Penny wrote:
> > > > I noticed that "python36" requires "binutils".
> > >
> > > This is needed for ctypes.util.find_library().  FWIW, on Linux, not
> > > only binutils is used, but also gcc.

I am very familiar with CPython's source code because I work on the
Anaconda Ditribution, maintaining our Python packages for Windows,
Linux and macOS and I can say that this is not the case, neither
regarding binutils nor gcc. In Python, objdump is used only in a
*fallback* case, and one that no one has run for many years [1]. If it
is used in Cygwin's port of Python then the reason for needing the
fallback needs to be investigated and removed. I might submit a patch
to CPython removing this fallback one day and it is just code-rot at
this point (and leads people to draw incorrect conclusions).

gcc is not used at all in ctypes. It (or some compiler) is used by
disutils when compiling python extension modules, obviously, but
that's not something all (or even most) users of Python probably need
to do.

--

Cheers,

Ray.

[1] https://github.com/python/cpython/blob/e42b705188271da108de42b55d9344642170aa2b/Lib/ctypes/util.py#L193-L212

> > >
> > > > Further, I noticed this dependency chain:
> > > >
> > > >     python36 > libuuid-devel > pkg-config > libglib2.0_0
> > >
> > > Until 3.7, the uuid stdlib module loads libuuid via ctypes, hence the
> > > dependency.  In 3.7, there is a compiled binding, and so the -devel
> > > dependency was dropped.  Also, pkgconf will soon be providing and
> > > replacing pkg-config, which will cause the glib2.0 dependency to be
> > > dropped.
> >
> > thanks. i was interested further in comparison so i took this file:
> >
> > https://python.org/ftp/python/3.6.8/python-3.6.8-embed-amd64.zip
> >
> > then use tar for fair comparison:
> >
> >     tar -a -c -f python-3.6.8-embed-amd64.tar.xz python-3.6.8-embed-amd64
>
> Did you look at the contents of that zip file?  It's quite minimal, and
> therefore not a fair comparison at all.
>
> > and that doesnt include the other packages i previously discussed. so it almost
> > makes more sense to not even use the cygwin version?
>
> Hardly, but discussing how/why *not* to use Cygwin is OT IMO.
>
> --
> Yaakov
>
>
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[Steven Penny]

On Sun, 24 Feb 2019 17:42:36, Yaakov Selkowitz wrote:
> On Sat, 2019-02-23 at 16:29 -0800, Steven Penny wrote:
>> I noticed that "python36" requires "binutils".
> 
> This is needed for ctypes.util.find_library().  FWIW, on Linux, not
> only binutils is used, but also gcc.

Are you sure about that? It seems that function doesnt even work:

$ python3.6 -q
>>> from ctypes.util import find_library
>>> print(find_library('z'))
None
>>> print(find_library('cygz'))
None
>>> print(find_library('z.dll'))
None
>>> print(find_library('cygz.dll'))
None

$ python3.7 -q
>>> from ctypes.util import find_library
>>> print(find_library('z'))
None
>>> print(find_library('cygz'))
None
>>> print(find_library('z.dll'))
None
>>> print(find_library('cygz.dll'))
None

$ python3.8 -q
>>> from ctypes.util import find_library
>>> print(find_library('z'))
None
>>> print(find_library('cygz'))
None
>>> print(find_library('z.dll'))
None
>>> print(find_library('cygz.dll'))
None

Also see this post:

https://cygwin.com/ml/cygwin/2019-02/msg00432.html


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[Yaakov Selkowitz]

On Mon, 2019-03-25 at 19:39 -0700, Steven Penny wrote:
> On Sun, 24 Feb 2019 17:42:36, Yaakov Selkowitz wrote:
> > On Sat, 2019-02-23 at 16:29 -0800, Steven Penny wrote:
> > > I noticed that "python36" requires "binutils".
> > 
> > This is needed for ctypes.util.find_library().  FWIW, on Linux, not
> > only binutils is used, but also gcc.
> 
> Are you sure about that? 

Of course I am.

> It seems that function doesnt even work:
> 
> $ python3.6 -q
> > > > from ctypes.util import find_library
> > > > print(find_library('z'))
> None

WFM (returns cygz.dll), do you have the respective -devel packages
installed?  Those are also required for find_library.

> > > > print(find_library('cygz'))
> None
> > > > print(find_library('z.dll'))
> None
> > > > print(find_library('cygz.dll'))
> None

The 'cygz' forms won't work, but that's to be expected.

> Also see this post:
> 
> https://cygwin.com/ml/cygwin/2019-02/msg00432.html

This is incorrect, look at ctypes/util.py and you will see that.

--
Yaakov



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[Steven Penny]

On Tue, 26 Mar 2019 10:08:27, Yaakov Selkowitz wrote:
>> It seems that function doesnt even work:
>> 
>> $ python3.6 -q
>> > > > from ctypes.util import find_library
>> > > > print(find_library('z'))
>> None
> 
> WFM (returns cygz.dll), do you have the respective -devel packages
> installed?  Those are also required for find_library.

still failing for me after installing "python36-devel". however even if it
works, it seems we have an issue. if you need "python36-devel" to use
"find_library", then it seems you only need "binutils" if you are installing
"python36-devel", not "python36".

so the "binutils" dep should be moved from "python36" to "python36-devel". this
way it only hurts people installing the development python, not also people
wanting to run "print('hello world')"

if you have another reason why "python36" should require "binutils" i would be
interested to read it - thanks.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[Yaakov Selkowitz]

On Tue, 2019-03-26 at 11:25 -0700, Steven Penny wrote:
> On Tue, 26 Mar 2019 10:08:27, Yaakov Selkowitz wrote:
> > > It seems that function doesnt even work:
> > > 
> > > $ python3.6 -q
> > > > > > from ctypes.util import find_library
> > > > > > print(find_library('z'))
> > > None
> > 
> > WFM (returns cygz.dll), do you have the respective -devel packages
> > installed?  Those are also required for find_library.
> 
> still failing for me after installing "python36-devel".

I mean the respective library's -devel package, e.g. zlib-devel for
find_library('z'), etc.

> so the "binutils" dep should be moved from "python36" to "python36-devel". this
> way it only hurts people installing the development python, not also people
> wanting to run "print('hello world')"

Nak.  ctypes is part of the Python standard library and therefore the
dependency needs to be on the main package(s).

BTW, binutils is usually included by default in all but the most
minimal base installs, so I'm not quite sure why you object so much to
this requirement.

--
Yaakov



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[Steven Penny]

On Tue, 26 Mar 2019 14:58:12, Yaakov Selkowitz wrote:
> Nak.  ctypes is part of the Python standard library and therefore the
> dependency needs to be on the main package(s).
> 
> BTW, binutils is usually included by default in all but the most
> minimal base installs, so I'm not quite sure why you object so much to
> this requirement.

Using Python 3.6 I noticed that a:

    print('hello world')

will actually work with only the Base packages and the "python36" package, and
none of the current "python36" dependencies installed. Obviously this comes with
some caveats as you have pointed out. But my counter is this:

- Current "python36" package is 5,746,648 bytes
- "binutils" 2.29 package is 5,863,216 bytes

So by choosing to have "binutils" as a dependency we are literally doubling the
size of Cygwin Python install. To me this is not trivial. I know with Git it
used to "require" Python but not really - it was only some ancillary scripts
that were eventually broken out into their own package so that the Python
requirement could be removed from Cygwin Git.

To contrast however the Perl dependency remained with Cygwin Git as currently
its too ingrained to be removed. What Im saying is that while you are
technically correct that "binutils" is a requirement for current Cygwin Python,
it doesnt have to be that way. We could move the Python elements that require
binutils into their own package, or we could create a "python_base" similar to
"perl_base".

I think if we just say "something somewhere in the bowels of Python needs it, so
it must be a hard dependency", even if that thing is rarely used by anyone, we
are doing a disservice to the Cygwin community. However that may not be the case
and perhaps "ctypes" is used much more than I am aware of.

Thanks


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Python extraneous dependencies

[Steven Penny]

On Tue, 26 Mar 2019 14:58:12, Yaakov Selkowitz wrote:
> Nak.  ctypes is part of the Python standard library and therefore the
> dependency needs to be on the main package(s).
> 
> BTW, binutils is usually included by default in all but the most
> minimal base installs, so I'm not quite sure why you object so much to
> this requirement.

https://cygwin.com/ml/cygwin/2019-03/msg00589.html

Is it though? With both Debian and Ubuntu, binutils is a suggested package, not
a required one:

- https://packages.debian.org/stretch/python3.5
- https://packages.ubuntu.com/bionic/python3.6


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple