Building from source packages

Building from source packages

[Jon Turney]

Recently I've done a little prototyping of a system to build from 
uploaded source packages in a VM, to verify that they recreate packages 
with the same contents as the uploaded packages.

Issues I noted were:

* build time dependencies aren't recorded

Out of the 3135 cygported packages for x86, only 175 set DEPEND.

I don't know how we can ensure DEPEND contains accurate information 
without some sort of build service which tests that it does.

Also, there's probably room here for some policy on what is and isn't 
required in DEPEND i.e. packages essential for building, which are 
already dependencies of cygport, probably don't need to be listed. 
Additionally, some cygclasses probably imply additional build 
dependencies, i.e. 'inherit gnome' implies gnome-common is a build-dep, etc.

Further, turning DEPEND into a list of cygwin packages to install can be 
complex (e.g., if it contains some pkgconfig atoms, it requires 
knowledge of the files contained in all packages to find the one which 
contains a particular .pc file).

* a few packages mistakenly use DEPENDS

A few packages mistakenly use DEPENDS rather than DEPEND, which is 
silently ignored

pkg           maintainer
---           ----------
cloog-isl     Achim Gratz
isl           Achim Gratz
mpclib        Achim Gratz
svn_load_dirs David Rothenberger

* a handful of packages are oddities

These often contain source and patches, but no build script, so the 
configuration used by the maintainer to build the package isn't recorded 
(at least in a way accessible to automation)

pkg       ver         arch
---       ---         ----
bzr       2.6+b2-2    x86    contains source for 2.6b2-1 and 2.6+b2-2
exim      4.86-1      both
fetchmail 6.3.21-1    x86
flex      2.5.39-1    both   orphaned
ipc-utils 1.0-1       x86
mined     2015.25-0   x86_64 a note from maintainer pointing to x86 src
nmh       1.6-2       both   uniquely, g-b-s script name doesn't end .sh
popt      1.16-1      both   orphaned
suite3270 3.3.15ga9-1 both   uniquely, multiple g-b-s and one tarfile

Re: Building from source packages

[Andrew Schulman]

> * a few packages mistakenly use DEPENDS
> 
> A few packages mistakenly use DEPENDS rather than DEPEND, which is 
> silently ignored

I'm surprised none of them are mine.  It confuses me every time that I have to
use REQUIRES, but DEPEND instead of DEPENDS. I have to go look it up again once
or twice a month.

To minimize the cognitive load on maintainers' brains, I humbly propose that
cygport honor both REQUIRE and REQUIRES, and DEPEND and DEPENDS.

Or it that's too ugly, make them both of the same form, i.e. REQUIRE and DEPEND.
A one-time backwards-incompatible change that would make things easier in the
future.

Andrew

Re: Building from source packages

[Achim Gratz]

Am 13.05.2016 um 16:33 schrieb Jon Turney:
> Out of the 3135 cygported packages for x86, only 175 set DEPEND.
>
> I don't know how we can ensure DEPEND contains accurate information
> without some sort of build service which tests that it does.

That's the trouble exactly.  There is no way to find out if your build 
dependencies are complete short of starting with a base installation and 
then adding them one-by-one.  So far I've typically only added 
dependencies when the build would otherwise succeed anyway (due to 
alternatives picked up in configury).

> Also, there's probably room here for some policy on what is and isn't
> required in DEPEND i.e. packages essential for building, which are
> already dependencies of cygport, probably don't need to be listed.
> Additionally, some cygclasses probably imply additional build
> dependencies, i.e. 'inherit gnome' implies gnome-common is a build-dep,
> etc.

That will occasionally fall flat on its face when the dependencies of 
those tools themselves change, but then it is hard to figure out what 
the actual dependencies are unless you try to build manually, see above.

> Further, turning DEPEND into a list of cygwin packages to install can be
> complex (e.g., if it contains some pkgconfig atoms, it requires
> knowledge of the files contained in all packages to find the one which
> contains a particular .pc file).

Isn't that what cygcheck does?  What's missing is a record of all files 
read during the build from the installation.

> A few packages mistakenly use DEPENDS rather than DEPEND, which is
> silently ignored
>
> pkg           maintainer
> ---           ----------
> cloog-isl     Achim Gratz
> isl           Achim Gratz
> mpclib        Achim Gratz

Heh.  I'm pretty sure I've inherited the packages that way...  I'll try 
to remember that these need cleanup.

-- 
Achim.

(on the road :-)

Re: Building from source packages

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]

On May 13 15:33, Jon Turney wrote:
> 
> Recently I've done a little prototyping of a system to build from uploaded
> source packages in a VM, to verify that they recreate packages with the same
> contents as the uploaded packages.
> 
> Issues I noted were:
> [...]
> * a handful of packages are oddities
> 
> These often contain source and patches, but no build script, so the
> configuration used by the maintainer to build the package isn't recorded (at
> least in a way accessible to automation)
> 
> pkg       ver         arch
> ---       ---         ----
> bzr       2.6+b2-2    x86    contains source for 2.6b2-1 and 2.6+b2-2
> exim      4.86-1      both
> fetchmail 6.3.21-1    x86
> flex      2.5.39-1    both   orphaned
> ipc-utils 1.0-1       x86

The orignal x86 package predates me using cygport.  The x86_64 package
has then probably been built by Yaakov, who's much less a slacker than
I am.  Just rebuilding the 32 bit packafge with Yaakov's 64 bit
cygport file will probably do.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Building from source packages

[Marco Atzeri]

On 13/05/2016 16:33, Jon Turney wrote:
>
> Recently I've done a little prototyping of a system to build from
> uploaded source packages in a VM, to verify that they recreate packages
> with the same contents as the uploaded packages.
>
> Issues I noted were:
>
>
> * a handful of packages are oddities
>
> These often contain source and patches, but no build script, so the
> configuration used by the maintainer to build the package isn't recorded
> (at least in a way accessible to automation)
>
> pkg       ver         arch
> ---       ---         ----
> bzr       2.6+b2-2    x86    contains source for 2.6b2-1 and 2.6+b2-2
> exim      4.86-1      both
> fetchmail 6.3.21-1    x86
> flex      2.5.39-1    both   orphaned
> ipc-utils 1.0-1       x86
> mined     2015.25-0   x86_64 a note from maintainer pointing to x86 src
> nmh       1.6-2       both   uniquely, g-b-s script name doesn't end .sh
> popt      1.16-1      both   orphaned
> suite3270 3.3.15ga9-1 both   uniquely, multiple g-b-s and one tarfile


flex and popt were probably built with a g-b-s like script by CGF.
I had the same issue when I adopted make.

popt version is last upstream, while
flex had a new release last November after long time

Regards
Marco

Re: Building from source packages

[Jon Turney]

On 13/05/2016 20:44, Andrew Schulman wrote:
>> * a few packages mistakenly use DEPENDS
>>
>> A few packages mistakenly use DEPENDS rather than DEPEND, which is
>> silently ignored
>
> I'm surprised none of them are mine.  It confuses me every time that I have to
> use REQUIRES, but DEPEND instead of DEPENDS. I have to go look it up again once
> or twice a month.
>
> To minimize the cognitive load on maintainers' brains, I humbly propose that
> cygport honor both REQUIRE and REQUIRES, and DEPEND and DEPENDS.
>
> Or it that's too ugly, make them both of the same form, i.e. REQUIRE and DEPEND.
> A one-time backwards-incompatible change that would make things easier in the
> future.

Yes, I get confused by it too, which is how I noticed this problem.  But 
I'm not sure how to fix this.

I wonder if it's possible to check the set of variables which exist 
after evaluating the cygport file, and warn if there are unexpected ones.

(Any existing uses of temporary variables inside a cygport file would 
have to be unset before it exits to avoid warning about them, but that 
seems a fair price to pay...)