* rebuilding python pip from source @ 2019-03-04 11:57 Doug Henderson 2019-03-04 16:39 ` Yaakov Selkowitz 0 siblings, 1 reply; 6+ messages in thread From: Doug Henderson @ 2019-03-04 11:57 UTC (permalink / raw) To: Cygwin Apps I am trying to rebuild the python37-pip package from source. Checking source in setup gives me /usr/src/python-pip-19.0.2-1.src/ When I try to build the package I get an error: $ cygport python-pip.cygport all *** ERROR: python27-pip is required to build this package Initially I got errors saying python36-pip, etc. were required, so I installed the binary and source packages for them, but there is no python27-pip or python2-pip package available to install. Ultimately I intend to completely rebuild from source both Python 2.7 and 3.7 and all the other packages named "^python.*". Doug -- Doug Henderson, Calgary, Alberta, Canada - from gmail.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rebuilding python pip from source 2019-03-04 11:57 rebuilding python pip from source Doug Henderson @ 2019-03-04 16:39 ` Yaakov Selkowitz 2019-03-04 18:09 ` Doug Henderson 0 siblings, 1 reply; 6+ messages in thread From: Yaakov Selkowitz @ 2019-03-04 16:39 UTC (permalink / raw) To: Cygwin Apps On Mon, 2019-03-04 at 04:57 -0700, Doug Henderson wrote: > I am trying to rebuild the python37-pip package from source. > Checking source in setup gives me /usr/src/python-pip-19.0.2-1.src/ > When I try to build the package I get an error: > > $ cygport python-pip.cygport all > *** ERROR: python27-pip is required to build this package > > Initially I got errors saying python36-pip, etc. were required, so I > installed the binary and source packages for them, but there is no > python27-pip or python2-pip package available to install. There is: https://cygwin.com/packages/x86_64/python27-pip/ > Ultimately I intend to completely rebuild from source both Python 2.7 and 3.7 > and all the other packages named "^python.*". To what end? -- Yaakov ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rebuilding python pip from source 2019-03-04 16:39 ` Yaakov Selkowitz @ 2019-03-04 18:09 ` Doug Henderson 2019-03-04 19:00 ` Yaakov Selkowitz 0 siblings, 1 reply; 6+ messages in thread From: Doug Henderson @ 2019-03-04 18:09 UTC (permalink / raw) To: Cygwin Apps On Mon, 4 Mar 2019 at 09:39, Yaakov Selkowitz <> wrote: > > On Mon, 2019-03-04 at 04:57 -0700, Doug Henderson wrote: > > I am trying to rebuild the python37-pip package from source. > > Checking source in setup gives me /usr/src/python-pip-19.0.2-1.src/ > > When I try to build the package I get an error: > > > > $ cygport python-pip.cygport all > > *** ERROR: python27-pip is required to build this package > > > > Initially I got errors saying python36-pip, etc. were required, so I > > installed the binary and source packages for them, but there is no > > python27-pip or python2-pip package available to install. > > There is: https://cygwin.com/packages/x86_64/python27-pip/ I have that package installed. I want to be able to compile it from source, as a small step towards my ultimate goal. > > Ultimately I intend to completely rebuild from source both Python 2.7 and 3.7 > > and all the other packages named "^python.*". > > To what end? Because that is a right granted to me by the open source license under which some source code is released. Perhaps not explicitly by the Python and cygwin licences, but as an implied ability. What is the point of having a source package but to be able to rebuild the binary package. For me, it is more a learning process than anything else. I understand that you are the maintainer of the main Python packages, as well as many other related packages. Thank you for your contributions. I must keep you very busy, so automating the process using cygport makes a lot of sense. There are other python related packages maintained by others, or orphaned. Starting with pip seemed a good idea. Previously, before python3 was available on cygwin, I rebuilt python2 from source, and also attempted to port python3. I don't think I used cygport back then. Back in the early 80's I received gcc, emacs, X11, sun-rpc, etc. on 9 track tape, and built them on the wide variety of Unix workstations and servers in our office. And over the years I built Python every where I had a C compiler but no Python. Building software from a source distribution is not a problem for me. Currently I maintain the cygwin expat package. I suspect that the Python build process requires some manual steps in addition to perhaps running cygport in small step, or even editing the.cygport file between those steps, that are not obvious from examining the source package. I suspect there is a recipe that you use every time you build a new Python release. I hope you can share that with me. I am thinking about porting some other packages to cygwin, and Python represents a complex system of packages that appears to make good use of cygport. Along the way, I need a better understanding of cygport, and how to wrap a provided build system within it. Thanks, Doug -- Doug Henderson, Calgary, Alberta, Canada - from gmail.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rebuilding python pip from source 2019-03-04 18:09 ` Doug Henderson @ 2019-03-04 19:00 ` Yaakov Selkowitz 2019-03-04 23:07 ` Doug Henderson 0 siblings, 1 reply; 6+ messages in thread From: Yaakov Selkowitz @ 2019-03-04 19:00 UTC (permalink / raw) To: Cygwin Apps On Mon, 2019-03-04 at 11:09 -0700, Doug Henderson wrote: > On Mon, 4 Mar 2019 at 09:39, Yaakov Selkowitz <> wrote: > > On Mon, 2019-03-04 at 04:57 -0700, Doug Henderson wrote: > > > I am trying to rebuild the python37-pip package from source. > > > Checking source in setup gives me /usr/src/python-pip-19.0.2-1.src/ > > > When I try to build the package I get an error: > > > > > > $ cygport python-pip.cygport all > > > *** ERROR: python27-pip is required to build this package > > > > > > Initially I got errors saying python36-pip, etc. were required, so I > > > installed the binary and source packages for them, but there is no > > > python27-pip or python2-pip package available to install. > > > > There is: https://cygwin.com/packages/x86_64/python27-pip/ > > I have that package installed. I want to be able to compile it from > source, as a small step towards my ultimate goal. To be clear, to build the python-pip package from source with the included .cygport file, you still need to have pythonXY-pip installed. > > > Ultimately I intend to completely rebuild from source both Python 2.7 and 3.7 > > > and all the other packages named "^python.*". > > > > To what end? > > Because that is a right granted to me by the open source license under > which some source code is released. Perhaps not explicitly by the > Python and cygwin licences, but as an implied ability. > > What is the point of having a source package but to be able to rebuild > the binary package. For me, it is more a learning process than > anything else. Fair enough, but keep in mind that Cygwin is a binary distribution, not a BLFS manual. > I suspect that the Python build process requires some manual steps in > addition to perhaps running cygport in small step, or even editing > the.cygport file between those steps, that are not obvious from > examining the source package. I suspect there is a recipe that you use > every time you build a new Python release. I hope you can share that > with me. The .cygport file together with the cygport program in whatever state it is at the time of building (keep in mind, as the primary author thereof, I always use git master) is exactly how the package is built, and how it can be rebuilt. Now, if you're asking about how I bootstrap python-pip's build-time requirement on pip for a new X.Y version of Python (wrt the brand new scheme where each is supported concurrently), that would be: 1. build pythonXY with cygport and install locally 2. pythonX.Y -m ensurepip --altinstall 3. rebuild/update python-wheel with cygport and install pythonXY-wheel locally 4. rebuild/update python-setuptools with cygport and install pythonXY- setuptools locally 5. rebuild/update python-pip with cygport and install pythonXY-pip locally 6. remove any cruft from step 2. 7. rebuild/update python-virtualenv with cygport This isn't reflected in the .cygport files because exactly how pip became present on the system it is irrelevant to the building of the packages themselves. > I am thinking about porting some other packages to cygwin, and Python > represents a complex system of packages that appears to make good use > of cygport. Along the way, I need a better understanding of cygport, > and how to wrap a provided build system within it. My Ports repos cover a wide variety of packages from which you may be able to learn more: https://github.com/cygwinports/ https://github.com/cygwinports-extras/ Also, the manual is included with the cygport package and is also available online: https://cygwinports.github.io/cygport/ -- Yaakov ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rebuilding python pip from source 2019-03-04 19:00 ` Yaakov Selkowitz @ 2019-03-04 23:07 ` Doug Henderson 2019-03-05 2:48 ` Brian Inglis 0 siblings, 1 reply; 6+ messages in thread From: Doug Henderson @ 2019-03-04 23:07 UTC (permalink / raw) To: Cygwin Apps tOn Mon, 4 Mar 2019 at 12:00, Yaakov Selkowitz <> wrote: > > On Mon, 2019-03-04 at 11:09 -0700, Doug Henderson wrote: > > On Mon, 4 Mar 2019 at 09:39, Yaakov Selkowitz <> wrote: > > > On Mon, 2019-03-04 at 04:57 -0700, Doug Henderson wrote: > The .cygport file together with the cygport program in whatever state > it is at the time of building (keep in mind, as the primary author > thereof, I always use git master) is exactly how the package is built, > and how it can be rebuilt. Hopefully I can get by with your releases, rather than tracking the HEAD. Do you have tags that correspond to the releases? I don't know how to tell Github to show tags, if it can. > Now, if you're asking about how I bootstrap python-pip's build-time > requirement on pip for a new X.Y version of Python (wrt the brand new > scheme where each is supported concurrently), that would be: Yes, this is what I want to understand. > 1. build pythonXY with cygport and install locally > 2. pythonX.Y -m ensurepip --altinstall > 3. rebuild/update python-wheel with cygport and install pythonXY-wheel > locally > 4. rebuild/update python-setuptools with cygport and install pythonXY- > setuptools locally > 5. rebuild/update python-pip with cygport and install pythonXY-pip > locally > 6. remove any cruft from step 2. > 7. rebuild/update python-virtualenv with cygport > > This isn't reflected in the .cygport files because exactly how pip > became present on the system it is irrelevant to the building of the > packages themselves. Thanks. This will help a lot, if I understand it correctly. It looks like you do not clear your build directories using 'cygport finish' after uploading new/changed packages, so a lot of inter-package state is retained in your /usr/src tree. Is this correct? Or am I misunderstanding the recipe you show above. In step 2, should I be executing the just built executable: ../inst/usr/bin/python3.7m.exe ? That seems to work. In step 3, I get: $ pwd /usr/src/python-wheel-0.32.3-1.src $ cygport python-wheel.cygport prep *** ERROR: python27-pip is required to build this package I don't understand how to get past this point. Am I calling cygport correctly at this point? Do I need any environment variables? What does this mean, besides the obvious? What kind of thing is `python27-pip' and where should it be? Is it looking in a wheel file somewhere? In the /usr/src/python37-3.7.2-1.src tree? In the /usr/src/python2-2.7.15-1.src tree? Do I need to cygport pe & compile & install and then install pip2 in that local install tree? And likewise for each of Python 3.4, 3.5, 3.6? Or do these versions of pip need to be installed into my system via setup.exe? Do I need to study Gentoo's portage to get a better understanding of how cygport works? Sorry about all the questions. Perhaps I'm trying to move ahead too quickly, without pouring through the source for cygport. I've found the document at https://cygwinports.github.io/cygport/cygport_in.html to be a bit sparse in some important places (and I know exactly how this kind of documentation gets created, as I've done it myself a time or two). Thanks Doug -- Doug Henderson, Calgary, Alberta, Canada - from gmail.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: rebuilding python pip from source 2019-03-04 23:07 ` Doug Henderson @ 2019-03-05 2:48 ` Brian Inglis 0 siblings, 0 replies; 6+ messages in thread From: Brian Inglis @ 2019-03-05 2:48 UTC (permalink / raw) To: cygwin-apps On 2019-03-04 16:07, Doug Henderson wrote: > On Mon, 4 Mar 2019 at 12:00, Yaakov Selkowitz wrote: >> On Mon, 2019-03-04 at 11:09 -0700, Doug Henderson wrote: >>> On Mon, 4 Mar 2019 at 09:39, Yaakov Selkowitz wrote: >>>> On Mon, 2019-03-04 at 04:57 -0700, Doug Henderson wrote: >> The .cygport file together with the cygport program in whatever state >> it is at the time of building (keep in mind, as the primary author >> thereof, I always use git master) is exactly how the package is built, >> and how it can be rebuilt. > Hopefully I can get by with your releases, rather than tracking the > HEAD. Do you have tags that correspond to the releases? I don't know > how to tell Github to show tags, if it can. Left button dropdowns for branches or tags or git tag --list command. >> Now, if you're asking about how I bootstrap python-pip's build-time >> requirement on pip for a new X.Y version of Python (wrt the brand new >> scheme where each is supported concurrently), that would be: > Yes, this is what I want to understand. >> 1. build pythonXY with cygport and install locally >> 2. pythonX.Y -m ensurepip --altinstall >> 3. rebuild/update python-wheel with cygport and install pythonXY-wheel >> locally >> 4. rebuild/update python-setuptools with cygport and install pythonXY- >> setuptools locally >> 5. rebuild/update python-pip with cygport and install pythonXY-pip >> locally >> 6. remove any cruft from step 2. >> 7. rebuild/update python-virtualenv with cygport >> This isn't reflected in the .cygport files because exactly how pip >> became present on the system it is irrelevant to the building of the >> packages themselves. > Thanks. This will help a lot, if I understand it correctly. > It looks like you do not clear your build directories using 'cygport > finish' after uploading new/changed packages, so a lot of > inter-package state is retained in your /usr/src tree. Is this > correct? Or am I misunderstanding the recipe you show above. > In step 2, should I be executing the just built executable: > ../inst/usr/bin/python3.7m.exe ? That seems to work. > In step 3, I get: > $ pwd > /usr/src/python-wheel-0.32.3-1.src > $ cygport python-wheel.cygport prep > *** ERROR: python27-pip is required to build this package > I don't understand how to get past this point. > Am I calling cygport correctly at this point? > Do I need any environment variables? > What does this mean, besides the obvious? What kind of thing is > `python27-pip' and where should it be? Is it looking in a wheel file > somewhere? In the /usr/src/python37-3.7.2-1.src tree? In the > /usr/src/python2-2.7.15-1.src tree? Do I need to cygport pe & compile > & install and then install pip2 in that local install tree? And > likewise for each of Python 3.4, 3.5, 3.6? Or do these versions of pip > need to be installed into my system via setup.exe? > Do I need to study Gentoo's portage to get a better understanding of > how cygport works? > Sorry about all the questions. Perhaps I'm trying to move ahead too > quickly, without poring through the source for cygport. I've found > the document at https://cygwinports.github.io/cygport/cygport_in.html > to be a bit sparse in some important places (and I know exactly how > this kind of documentation gets created, as I've done it myself a time > or two). Symlink /usr/share/doc/cygport/html/manual/index.html -> cyport-doc, also alias opening with cygstart, also bookmark in your browser, and start reading; browse around in /usr/share/cygport/cygclass and that neighbourhood. When third party builds have problems under Cygwin, try using cygport to solve some of the problems, often more easily than hacking the build directly, although that can't always be avoided, in which case those patches are candidates for submitting upstream. -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-03-05 2:48 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-03-04 11:57 rebuilding python pip from source Doug Henderson 2019-03-04 16:39 ` Yaakov Selkowitz 2019-03-04 18:09 ` Doug Henderson 2019-03-04 19:00 ` Yaakov Selkowitz 2019-03-04 23:07 ` Doug Henderson 2019-03-05 2:48 ` Brian Inglis
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).