cygport-0.9.3 in release-2

cygport-0.9.3 in release-2

[Yaakov (Cygwin Ports)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I've just made cygport-0.9.3 available in release-2 with the following
changes:

* PV is now an array; members 1-* replace PVP[].
* foo_CONTENTS can now be used in place of PKG_CONTENTS[].
* cygtest(): Doesn't exit when tests fail.
* autotools.cygclass: Detect ac-2.63+; detect missing LT_OUTPUT.
* fossil.cygclass: NEW for Fossil RCS checkouts.
* ruby.cygclass: Added rubyinto, doruby. Accept RDOC_MODULE.
* ruby-gnome2.cygclass: Added ruby-goocanvas.
* xorg.cygclass: Added GIT_URI; renamed font- packages.
* zope.cygclass: REMOVED.  Use distutils instead.

If there are any changes you want to see in cygport, please let me know
ASAP; I would like to have the major changes out of the way soon,
so that cygport isn't a moving target once everything is ready for a 1.7
rebuild.


Yaakov
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREIAAYFAkkGsvgACgkQpiWmPGlmQSMfVwCg9OQzknDJv2bQPWVfrCO2Zkif
+7EAoP4eQbdkvhbYSJFv3Yyvmj7ilSZb
=sDSo
-----END PGP SIGNATURE-----

Re: cygport-0.9.3 in release-2

[Charles Wilson]

[-- Attachment #1: Type: text/plain, Size: 855 bytes --]

Yaakov (Cygwin Ports) wrote:

> If there are any changes you want to see in cygport, please let me know
> ASAP; I would like to have the major changes out of the way soon,
> so that cygport isn't a moving target once everything is ready for a 1.7
> rebuild.

Forward port of my three remaining patches (against SVN as of about a
week ago, but I expect no merge issues).

patch-cygport-0.9.3/cygport-custom-cmds.patch
patch-cygport-0.9.3/cygport-cvs-topdir.patch
patch-cygport-0.9.3/cygport-postinst-hook.patch

Also attached the same forward port against the 0.4x branch SVN as of
about a week ago:

patch-cygport-0.4.2/cygport-custom-cmds.patch
patch-cygport-0.4.2/cygport-cvs-topdir.patch
patch-cygport-0.4.2/cygport-postinst-hook.patch

The "relocatable" stuff is still...pending or abandoned; I haven't
decided whether it is worth pursuing.

--
Chuck

[-- Attachment #2: patch-cygport-0.9.3.tar.bz2 --]
[-- Type: application/octet-stream, Size: 2554 bytes --]

[-- Attachment #3: patch-cygport-0.4.2.tar.bz2 --]
[-- Type: application/octet-stream, Size: 2542 bytes --]

Re: cygport-0.9.3 in release-2

[Andrew Schulman]

> If there are any changes you want to see in cygport, please let me know
> ASAP;

As I requested before at
http://www.cygwin.com/ml/cygwin-apps/2008-09/msg00070.html, I routinely use
Charles Wilson's patch for src_prep_fini_hook()
(http://www.cygwin.com/ml/cygwin/2007-01/msg00110.html), and would like to see
it included in cygport.

As I explained in the thread referenced above, src_patch_hook() and
src_unpack_hook() are not adequate substitutes for src_prep_fini_hook(), because
both of the former act *before* origsrc is mirrored to src.  So, using them it
is not possible to make changes to src independently from origsrc.
src_prep_fini_hook(), as Charles proposed it, is called after origsrc is
mirrored to src, so that I can patch src independent of origsrc, and the changes
are captured in cygwin.patch-- which is essential for creating a working source
package.

If you don't like src_prep_fini_hook(), then maybe src_unpack_hook() could be
moved down below the call to rsync, which would make it the same as
src_prep_fini_hook().

Thanks,
Andrew.

Re: cygport-0.9.3 in release-2

[Charles Wilson]

Andrew Schulman wrote:
> As I explained in the thread referenced above, src_patch_hook() and
> src_unpack_hook() are not adequate substitutes for src_prep_fini_hook(), because
> both of the former act *before* origsrc is mirrored to src.  So, using them it
> is not possible to make changes to src independently from origsrc.

This is true.  In general, changing src independently of origsrc is what
 .src.patch and cygwin.patch are for.  The only thing you really can't
do via a patch at that point, is to chmod any files created by those
patches (pre-existing files can be chmoded in src_unpack_hook before
patching them). And as Yaakov has pointed out, there's also no reason
why you can't do those chmods in src_compile(), except for aesthetic
reasons ("setting up your src tree is part of prep, not build").

> src_prep_fini_hook(), as Charles proposed it, is called after origsrc is
> mirrored to src, so that I can patch src independent of origsrc, and the changes
> are captured in cygwin.patch-- which is essential for creating a working source
> package.

This I don't get.  Once you've captured the desired differences between
origsrc and src in .src.patch and .cygwin.patch...you are done. Those
patches can be used to regenerate your entire build environment: "mirror
 (the possibly src_unpack_hook() and PATCH_URI-modified) origsrc and
then apply the standard two patches". But if you try to rebuild using a
.cygport that specifies BOTH src_prep_fini_hook() AND a .*.patch file
that "captures" those same changes...BOOM: "target already contains
changes; reverse?"

> If you don't like src_prep_fini_hook(), then maybe src_unpack_hook() could be
> moved down below the call to rsync, which would make it the same as
> src_prep_fini_hook().

Uhm, please no.  src_unpack_hook() is Yaakov's replacement for the first
half of my original hooks patch, src_prep_init_hook().  THAT one is
needed for LOTS of reasons -- mostly, when you have big changes you want
to make to the origsrc tree precisely because it occurs before mirroring
(such as rearranging files) -- that would otherwise require really
gigantic .src.patch files.  In the past, src_prep_init_hook ne'
src_unpack_hook was also used to apply "upstream" standard patches, when
the cygwin maintainer wants to keep them "unbundled" from
cygwin-specific changes to the src (e.g. smaller .src.patch).  However,
with the advent of PATCH_URI, "apply standard upstream patches" is not
as large a use ccase for src_unpack_hook().

Andrew, please post one of your .cygports that actually requires the
src_prep_fini_hook(). I'd like to reach a resolution on this (these)
issues, so I can either drop the patches entirely or get some
replacement/reimplementation in cygport.

FYI, the patch set I posted yesterday does NOT include
src_prep_fini_hook(). It only includes src_postinst_hook().

Yaakov is strongly opposed to src_prep_fini_hook(), and I'm more-or-less
convinced that it is not necessary, as well. I'd like to see an actual
example where it is the only solution -- or demonstrating that other
workarounds are significantly inelegant.

With regards to src_postinst_hook() ... well, Yaakov and I can argue
about that in another subthread.

--
Chuck

Re: cygport-0.9.3 in release-2

[Andrew Schulman]

> Andrew, please post one of your .cygports that actually requires the
> src_prep_fini_hook(). I'd like to reach a resolution on this (these)
> issues, so I can either drop the patches entirely or get some
> replacement/reimplementation in cygport.

OK, here's how I use it.

Several of my packages require multiple patches to compile and run properly in
Cygwin.  Instead of maintaining them all together as One Big Patch, I find it
easier to manage them as individual, discrete patch files, and apply them all at
package build time.

I also have extra files, such as README and setup.hint, that I like to just copy
in before building, instead of maintaining them as patches.  Again, this is
easier for me in the long run.

So, in general I include the following code in my .cygport files:

src_prep_fini_hook ()
{
    cd "${top}"

    # copy in extra files
    if [ -d extras -a "$(ls extras)" ] ; then
        __step "Copying in extra files"
        cp -a extras/* "${S}"
    fi

    # apply patches
    if [ -d patches -a "$(ls patches | grep '\.patch$')" ] ; then
        __step "Applying patches"
        cd "${S}"
        cygpatch ${top}/patches/*.patch
    fi
}

The above is from screen-4.0.3-1.cygport.

If there's a better way to do this, then I'm all ears.  I hadn't thought of
doing it during compile() but that seems fine.

Note that the problem you mentioned of the patches being applied twice doesn't
occur.  If I'm building the packages, then the extras/ and patches/ directories
are present.  Someone else who's building the binary package from source doesn't
have those directories, so there's no conflict.

Thanks,
Andrew.

Re: cygport-0.9.3 in release-2

[Charles Wilson]

Andrew Schulman wrote:
> Several of my packages require multiple patches to compile and run properly in
> Cygwin.  Instead of maintaining them all together as One Big Patch, I find it
> easier to manage them as individual, discrete patch files, and apply them all at
> package build time.

Okay, so these are (mostly) your own custom patches needed to port the
code to cygwin, and not "official" patches from somewhere else, like

  1) bugfixes taken wholesale from another distro
(http://sources.gentoo.org/viewcvs.py/gentoo-x86/sys-libs/ncurses/files/)
  2) intra-release patches (see ftp://invisible-island.net/ncurses/5.6/)

If it were 1) or 2), I'd suggest using

PATCH_URI="http://sources.gentoo.org/viewcvs.py/gentoo-x86/sys-libs/ncurses/files/ncurses-5.6-build.patch

ftp://invisible-island.net/ncurses/5.6/ncurses-5.6-coverity.patch.gz"

etc.  Of course, there are a few issues there:

a) my example, ncurses, has a LOT (50 or so) "official" patches
b) they are all gz-compressed; cygport might not support compressed
patches in PATCH_URI

So, in fact, for ncurses (where some of the official upstream "patches"
are actually shell scripts with shar-compressed patches!) I actually
DON'T specify these files in PATCH_URI. Instead, I specify them in
SRC_URI, and then use src_unpack_hook to apply (that is, execute!) them.


However, that's not the case for your screen port, so...moving on.

> I also have extra files, such as README and setup.hint, that I like to just copy
> in before building, instead of maintaining them as patches.  Again, this is
> easier for me in the long run.

Ah. So, what you're really trying to do is automate the procedure of
"forward porting" your current changeset to the next release -- whether
it's the -NEXT release of the same upstream version, or the -1 release
an entirely new upstream version.  Effectively, you're using
src_prep_fini_hook() as a "maintainer mode", and "maintainer mode" is
activated by the presence of ${topdir}/extra/ and ${topdir}/patch/.

> So, in general I include the following code in my .cygport files:
> 
> src_prep_fini_hook ()
> {
>     cd "${top}"
> 
>     # copy in extra files
>     if [ -d extras -a "$(ls extras)" ] ; then
>         __step "Copying in extra files"
>         cp -a extras/* "${S}"
>     fi
> 
>     # apply patches
>     if [ -d patches -a "$(ls patches | grep '\.patch$')" ] ; then
>         __step "Applying patches"
>         cd "${S}"
>         cygpatch ${top}/patches/*.patch
>     fi
> }
> 
> The above is from screen-4.0.3-1.cygport.

The way *I* would do that, is the following:

Assuming Yaakov adopts the custom-cmds patch, I'd include a function in
my cygport like so:

import_changeset() {
   # contents of your src_prep_fini_hook
}

Then, my workflow for a new release would be:

$ cp pkg-VER-1.cygport pkg-VER-2.cygport
## notice, there are no existing -2.cygwin.patch or -2.src.patch files

$ cygport pkg-VER-2.cygport prep
## so, everything gets unpacked, and mirrored -- but src is identical
## origsrc, because there are no -2.*.patch files

$ cygport pkg-VER-2.cygport custom0-import_changeset
## FYI, what I do here is 'cygport pkg-VER-2.cygport oldpatch VER-1'
## but that's because I don't mind "maintaining" any changes that
## don't fall into categories 1) and 2) above as patch files:
## .src.patch and .cygwin.patch

$ cygport pkg-VER-2.cygport build install pkg ## etc

One advantage of this method is that some other user won't get a
surprising error if they *happen* to have an extra/ directory present
when they try to build your package. They'd have to explicitly invoke
custom0-import_changeset -- and hopefully if they do THAT, then they
know what they are doing. Your way, random files from their extra/
directory will automatically get copied in, which could be...unexpected.

Another advantage -- to my thinking -- is that I *never* import the
original files from extra/ and patch/ unless I explicitly choose to do
so; it won't happen by accident.  I think that's good; however, you may
value the "automatic" behavior of embedding this action as part of 'prep'.

> Note that the problem you mentioned of the patches being applied twice doesn't
> occur.  If I'm building the packages, then the extras/ and patches/ directories
> are present.  Someone else who's building the binary package from source doesn't
> have those directories

You hope.

> so there's no conflict.

My typical workflow at present involves doing the following after I'm
done with development for a new release:

<1>
$ cygport foo-VER-REL.cygport pkg
$ tar xvjf foo-VER-REL-src.tar.bz2
$ cygport foo-VER-REL.cygport finish almostall

So that I am assured that the distributed src package works. With your
system, you'd have to do

<2>
$ cygport foo-VER-REL.cygport pkg
$ mkdir temp
$ cd temp
$ tar xvjf ../foo-VER-REL-src.tar.bz2
$ cygport foo-VER-REL.cygport all
$ mv *.tar.bz2 ..
$ cd ..
$ rmdir temp

to have the same effect. If you did it as in <1> with your
src_prep_fini_hook, you'd have BOTH the .src.patch and .cygwin.patch
present, AND the extra/ and patch/ directories. That's not an issue with
the extra/ files, but the changes represented by patch/* are duplicated
by the .src.patch changes, and that would definitely bomb out.

However, <1> would work fine, if you used the custom0-import_changeset
method.

But, having said all that, I realize I'm now just arguing for using a
DIFFERENT non-standard cygport patch.  'Course, that makes sense: my
original intent for the custom- stuff was for maintainance -- like
running individual components of the cvs testsuite via cygport, etc.

Yaakov? Pretty please?

--
Chuck

Re: cygport-0.9.3 in release-2

[Yaakov (Cygwin Ports)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Charles Wilson wrote:
> Okay, so these are (mostly) your own custom patches needed to port the
> code to cygwin, and not "official" patches from somewhere else, like
> 
>   1) bugfixes taken wholesale from another distro
> (http://sources.gentoo.org/viewcvs.py/gentoo-x86/sys-libs/ncurses/files/)
>   2) intra-release patches (see ftp://invisible-island.net/ncurses/5.6/)
> 
> If it were 1) or 2), I'd suggest using
> 
> PATCH_URI="http://sources.gentoo.org/viewcvs.py/gentoo-x86/sys-libs/ncurses/files/ncurses-5.6-build.patch
> 
> ftp://invisible-island.net/ncurses/5.6/ncurses-5.6-coverity.patch.gz"

BTW, that's mirror://portage/sys-libs/ncurses/... for short, but you've
got the idea.

> a) my example, ncurses, has a LOT (50 or so) "official" patches
> b) they are all gz-compressed; cygport might not support compressed
> patches in PATCH_URI

Compressed single patches are supported in PATCH_URI since 0.3.5.

> So, in fact, for ncurses (where some of the official upstream "patches"
> are actually shell scripts with shar-compressed patches!) I actually
> DON'T specify these files in PATCH_URI. Instead, I specify them in
> SRC_URI, and then use src_unpack_hook to apply (that is, execute!) them.

Interesting, I'd like to see those "patches"; with the recent release of
ncurses-5.7, all the 5.6 patches are gone.

> Yaakov? Pretty please?

I think you know that I haven't been a big fan of this idea.
Nevertheless, cygport 0.9.x allows you to call src_compile(),
src_install(), or any self-defined function on the command line, but it
doesn't allow for any arguments to the function.  Adding that limitation
made the implementation much easier than what you were proposing, and I
hope that it will be sufficient.


Yaakov

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREIAAYFAkkWgHAACgkQpiWmPGlmQSMO6wCZAYYFbbjEz7ufFJ5fkdJHXslc
3c4AoOCOv5aK+iO/Q2+3Z0g0p+f+A5vV
=/NPh
-----END PGP SIGNATURE-----

Re: cygport-0.9.3 in release-2

[Charles Wilson]

Yaakov (Cygwin Ports) wrote:
> Charles Wilson wrote:
>> a) my example, ncurses, has a LOT (50 or so) "official" patches
>> b) they are all gz-compressed; cygport might not support compressed
>> patches in PATCH_URI
> 
> Compressed single patches are supported in PATCH_URI since 0.3.5.

Good to know. I /thought/ that was the case, but was too lazy to check
while composing the email.

>> So, in fact, for ncurses (where some of the official upstream "patches"
>> are actually shell scripts with shar-compressed patches!) I actually
>> DON'T specify these files in PATCH_URI. Instead, I specify them in
>> SRC_URI, and then use src_unpack_hook to apply (that is, execute!) them.
> 
> Interesting, I'd like to see those "patches"; with the recent release of
> ncurses-5.7, all the 5.6 patches are gone.

They were called "rollup" patches. T.E.D. releases patches roughly every
week, and then every month (or two, or three), he combines all patches
dating back to an official release into a "rollup" patch -- and these
"rollup" patches are in the shar-archive-auto-apply format.  You can see
an example in the current cygwin ncurses-5.5 package (yes, two years
old. I know...)  T.E.D. also provides the patches to update a 5.x tree
to 5.x+1 in that form:

ftp://invisible-island.net/ncurses/patches/

> I think you know that I haven't been a big fan of this idea.
> Nevertheless, cygport 0.9.x allows you to call src_compile(),
> src_install(), or any self-defined function on the command line, but it
> doesn't allow for any arguments to the function.  Adding that limitation
> made the implementation much easier than what you were proposing, and I
> hope that it will be sufficient.

Yes, that will do.

If one really needs to pass arguments -- which I (usually) don't --
there are a number of workarounds:
  1. export MY_FUNC_ARGS=("a b" c "d e"); cygport foo.cygport my_func
     where my_func() processes MY_FUNC_ARGS[@] (which in this
     case has three elements, "a b", c, and "d e").
  2. You could do something similar with response files
  3. Reduce granularity and then have a few specific functions.
     For instance, rather than:
       cygport cvs.cygport custom1-run_test N
     where N is any number from 1 to 300, instead do
       cygport cvs.cygport run_test_group1
       cygport cvs.cygport run_test_group2
       cygport cvs.cygport run_test_group3
       cygport cvs.cygport run_test_group4
     where each group comprises 75 or so tests.
I'd probably lean to #1. If it ever came up.

In any event, all of those workarounds can be implemented solely inside
a specific .cygport file, without any additional (or intrusive, or
ugly-as-sin) changes to /usr/bin/cygport.

The custom- stuff is a marvelous (ab)use of bash, though, ain't it?

That leaves only:
  cvs topdir support
  postinst hook support
  relocatable [*]

=====

[*] relocatable. I've ported those changes to 0.4.2, but...they are
really, just, wow.  Awesomely ugly. I've decided to drop them entirely,
for the following reasons:

1)
According to Bruno, "official" packages aren't supposed to be
relocatable. "I hope that people will learn when to use
--enable-relocatable by themselves. For example, I don't think a Linux
distributor should use --enable-relocatable for anything installed in
/usr - it would only be bloat that makes the system slower."

Since the relocatable magic applies only to libintl, gettext, (and my
personal patched versions of proj and libgeotiff) -- all of which I
maintain and which go into cygwin's /usr -- Bruno's argument against
--enable-relocatable applies.

2)
cygport is intended for use in packaging "official" cygwin stuff. Not
random compiled-with-wierd-options stuff. So, it doesn't really need to
support the rather esoteric --enable-relocatable, as official packages
aren't supposed to use that option (see 1).

3)
--enable-relocatable is a bad fit for cygport. As a consequence of
cygport's focus on "official packages only", it always installs into
/usr. It's sorta hard to force it into installing to
/tmp/cygport-reloc-au723g/ -- the whole --enable-relocatable scheme
relies on using such odd, random, never-repeated, --prefixes.

4)
The only reason I was even bothering was to help iron out the bugs in
Bruno's reloc support.  Well, as of 0.17, it needed almost ZERO
additional effort from me to make it work (a single two-line patch --
and THAT was actually gnulib's fault, not Bruno's reloc stuff). I think
that qualifies as "good enough" and I can, in the future, test reloc
support "normally" -- that is, configure/make/install sans-cygport.

5)
Ye GODS, the patch is ugly. I'll post it for historical interest,
but...yuck!

=====

FWIW, here's the current ncurses prep function (eventually to become
src_unpack_hook when I release 5.7). ${PF}.extra.patch is a separate
patch (of mine) that I wanted to keep separate from .src.patch.

SRC_URI="mirror://gnu/ncurses/${PN}-${PV}.tar.gz \
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20060909-patch.sh.bz2 \
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20060916.patch.gz \
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20060923.patch.gz \
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20060930.patch.gz \
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20061007.patch.gz \
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20061014.patch.gz \
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20061021.patch.gz \
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20061028.patch.gz \
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20061104.patch.gz \
${PF}.extra.patch"

src_prep_init_hook() {
  local -i n=1
  local -a mySrcURI=($SRC_URI)
  local breakout=""
  local pnz
  local pn
  local msg

  cd ${origsrcdir}/${SRC_DIR}

  # This "loop" is only executed once (at most). We
  # use 'while' instead of 'if' because it is a more
  # flexible control flow structure.
  while [ ${n} -lt ${#mySrcURI[*]} -a -z "${breakout}" ]
  do
    breakout="yes"
    pnz=${mySrcURI[${n}]##*/}
    case "${pnz}" in
    *.gz  ) pn=${pnz%.gz} ;;
    *.bz2 ) pn=${pnz%.bz2} ;;
    *     ) pn=${pnz} ;;
    esac

    case "${pn}" in
    *.sh ) n+=1 ;;
    *    ) continue ;;
    esac

    if [ -f ${origsrcdir}/${pn} ]
    then
      inform "APPLYING OFFICIAL ROLLUP PATCH: ${pn}"
      ${origsrcdir}/${pn}
    else
      if [ -f ${origsrcdir}/${SRC_DIR}/${pn} ]
      then
        inform "APPLYING OFFICIAL ROLLUP PATCH: ${pn}"
        ${origsrcdir}/${SRC_DIR}/${pn}
      else
        warning "COULD NOT FIND OFFICIAL ROLLUP PATCH: ${pn}"
      fi
    fi
  done

  while [ ${n} -lt ${#mySrcURI[*]} ]
  do
    pnz=${mySrcURI[${n}]##*/}
    msg="OFFICIAL INCREMENTAL PATCH"
    case "${pnz}" in
    *.gz  ) pn=${pnz%.gz} ;;
    *.bz2 ) pn=${pnz%.bz2} ;;
    *.patch ) pn=${pnz} ; msg="CYGWIN SPECIAL PATCH" ;;
    * )
      warning "Skipping unknown file: ${pnz}"
      n+=1
      continue
      ;;
    esac

    if [ -f ${origsrcdir}/${pn} ]
    then
      inform "APPLYING ${msg}: ${pn}"
      apply_patch ${origsrcdir}/${pn}
    else
      if [ -f ${origsrcdir}/${SRC_DIR}/${pn} ]
      then
        inform "APPLYING ${msg}: ${pn}"
        apply_patch ${origsrcdir}/${SRC_DIR}/${pn}
      else
        warning "COULD NOT FIND ${msg}: ${pn}"
      fi
    fi
    n+=1
  done
}

--
Chuck

Re: cygport-0.9.3 in release-2

[Yaakov (Cygwin Ports)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Charles Wilson wrote:
> They were called "rollup" patches. T.E.D. releases patches roughly every
> week, and then every month (or two, or three), he combines all patches
> dating back to an official release into a "rollup" patch -- and these
> "rollup" patches are in the shar-archive-auto-apply format.  You can see
> an example in the current cygwin ncurses-5.5 package (yes, two years
> old. I know...)  T.E.D. also provides the patches to update a 5.x tree
> to 5.x+1 in that form:
> 
> ftp://invisible-island.net/ncurses/patches/

Looking at 5.5-3, all those patches can be in PATCH_URI instead.

> Yes, that will do.
> 
> If one really needs to pass arguments -- which I (usually) don't --
> there are a number of workarounds:
>   1. export MY_FUNC_ARGS=("a b" c "d e"); cygport foo.cygport my_func
>      where my_func() processes MY_FUNC_ARGS[@] (which in this
>      case has three elements, "a b", c, and "d e").
>   2. You could do something similar with response files
>   3. Reduce granularity and then have a few specific functions.
>      For instance, rather than:
>        cygport cvs.cygport custom1-run_test N
>      where N is any number from 1 to 300, instead do
>        cygport cvs.cygport run_test_group1
>        cygport cvs.cygport run_test_group2
>        cygport cvs.cygport run_test_group3
>        cygport cvs.cygport run_test_group4
>      where each group comprises 75 or so tests.
> I'd probably lean to #1. If it ever came up.

I was thinking #3, but the other methods would work as well.

> In any event, all of those workarounds can be implemented solely inside
> a specific .cygport file, without any additional (or intrusive, or
> ugly-as-sin) changes to /usr/bin/cygport.

Exactly.

> The custom- stuff is a marvelous (ab)use of bash, though, ain't it?

Definitely abuse. :-)

> That leaves only:
>   cvs topdir support
>   postinst hook support

cvs topdir: this has little benefit and would break ABI.

postinst hook: RESTRICT="postinst-doc" was supposed to be a workaround.
 How was this not sufficient?

> [*] relocatable. I've ported those changes to 0.4.2, but...they are
> really, just, wow.  Awesomely ugly. I've decided to drop them entirely,
> for the following reasons: [snip]

Couldn't have said it better.


Yaakov
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREIAAYFAkkXeQoACgkQpiWmPGlmQSOm+wCg7nPLuDby9RnkAZl5PNTXtO7v
P5cAn3jnPlzigHZncZO6DF6R4lHx0unf
=0HSh
-----END PGP SIGNATURE-----

Re: cygport-0.9.3 in release-2

[Charles Wilson]

Yaakov (Cygwin Ports) wrote:

>> They were called "rollup" patches. T.E.D. releases patches roughly every
>> week, and then every month (or two, or three), he combines all patches
>> dating back to an official release into a "rollup" patch -- and these
>> "rollup" patches are in the shar-archive-auto-apply format.  You can see
>> an example in the current cygwin ncurses-5.5 package (yes, two years
>> old. I know...)  T.E.D. also provides the patches to update a 5.x tree
>> to 5.x+1 in that form:
> 
>> ftp://invisible-island.net/ncurses/patches/
> 
> Looking at 5.5-3, all those patches can be in PATCH_URI instead.

Even the "rollup" patch?
ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20060909-patch.sh.bz2

>> That leaves only:
>>   cvs topdir support
>>   postinst hook support
> 
> cvs topdir: this has little benefit and would break ABI.

It only breaks the ABI (B?) if you do it the way you suggested:
http://cygwin.com/ml/cygwin/2008-05/msg00038.html
> I suppose a cleaner solution would have been to add a '-d
> ${CVS_MODULE##*/}' to the checkout command, but doing that now would
> break existing -src packages.

Anyway, that wouldn't actually work either. Modules aren't necessarily
directory names.  For instance, take the binutils module:

binutils        -a naked-binutils naked-opcodes naked-bfd naked-libiberty \
                naked-include naked-gas naked-gprof naked-ld naked-gold \
                naked-elfcpp naked-intl src-support naked-texinfo naked-cpu

If I wanted to checkout the 'naked-binutils' module, in /THAT/ case I
would really expect to get

src/binutils/<STUFF>

and would be annoyed to get

naked-binutils/<STUFF>

There's really no "rule" like '-d ${CVS_MODULE##*/}' that is universally
applicable: the module name and the -d option (if present) are
orthogonal controls. You don't want to tie them together.


The way my patch does it, there is no API breakage -- but you have a new
API entry point [the new CVS_DIR variable].  The price of API
preservation is a proliferation of new ones; just ask Bill Gates.


The benefit of allowing some mechanism to pass a -d option to the cvs
checkout is that I can ensure that my cvs.cygclass-generated origsrc
tarball has the same directory layout as a make-dist-generated one.

But whatever. I'll live with it. Worst case, I'll manually repack the
tarball and comment-out the inherit cvs.cygclass. Besides, the only
package I know of that has this issue is libgeotiff -- which moved to
subversion a few months ago, anyway.

So, it's probably moot for all current packages.

> postinst hook: RESTRICT="postinst-doc" was supposed to be a workaround.
>  How was this not sufficient?

It is sufficient to override the default behavior of the post-install
phase with respect to documentation; that's what the urxvt packages care
about. But other packages (unknown at this time, but I'm not possessed
of sufficient hubris to rule them out) might need to override/customize
some other phase:

__src_postinst() {
        __prep_symlinks;
        __prepdoc;
        __prepetc;
        __prepman;
        __prepinfo;
        __prep_empty_dirs;
        __prepstrip;
        __prep_libtool_modules;
}

But I guess you can continue adding new RESTRICT options on an ad-hoc
basis. A draw-back to that method is all I can do is STOP the automated
system from doing something at the end of the install-phase. The only
chance I have to actually DO anything during the install phase is in
src_install, which precedes all that.

Hopefully there are no, and never will be any, ordering dependencies
between the post-install phases; For example: suppose, for some reason,
that cygport develops an assumption that __prepinfo MUST always come
after __prepman.  If I want to do something funky with the info files,
all I can do is RESTRICT_postinst-info -- and do my funky thing MUCH
earlier, in src_install.

So, now, MY "prepinfo" happens before __prepman.

But, I suppose there's no sense borrowing trouble; "Sufficient unto the
day is the evil thereof".

--
Chuck

Re: cygport-0.9.3 in release-2

[Yaakov (Cygwin Ports)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Charles Wilson wrote:
> Even the "rollup" patch?
> ftp://invisible-island.net/${PN}/${PV}/${PN}-${PV}-20060909-patch.sh.bz2

Yes.

> It only breaks the ABI (B?) if you do it the way you suggested:
> http://cygwin.com/ml/cygwin/2008-05/msg00038.html

Yes, the concept is the same:  ABI breakage affects previously-built
source packages but doesn't necessarily break API (IOW the .cygport
wouldn't need to be changed, but you'd need to fetch() from scratch).
API breakage affects the programmer, i.e. the .cygport itself would
require a change.

> There's really no "rule" like '-d ${CVS_MODULE##*/}' that is universally
> applicable: the module name and the -d option (if present) are
> orthogonal controls. You don't want to tie them together.
> 
> The way my patch does it, there is no API breakage -- but you have a new
> API entry point [the new CVS_DIR variable].  The price of API
> preservation is a proliferation of new ones; just ask Bill Gates.

I should learn about API preservation from Micro$oft?!?  Are you joking?
 http://en.wikipedia.org/wiki/Criticism_of_Windows_Vista#Software_compatibility

OTOH one can definitely learn from GNOME, whose developer platform has
maintained API compatiblity for 6 years while steadily adding new
features every six months.

I have been trying to maintain ABI/API stability, although I wouldn't be
surprised if I broke something along the way.

> The benefit of allowing some mechanism to pass a -d option to the cvs
> checkout is that I can ensure that my cvs.cygclass-generated origsrc
> tarball has the same directory layout as a make-dist-generated one.
> 
> But whatever. I'll live with it. Worst case, I'll manually repack the
> tarball and comment-out the inherit cvs.cygclass. Besides, the only
> package I know of that has this issue is libgeotiff -- which moved to
> subversion a few months ago, anyway.

Out of 2018 packages in Ports SVN, a quick grep showed that 13 currently
inherit cvs.cygclass (vs. 32 using SVN), and out of those, only one
(ocaml-xml-light) has a deep CVS_MODULE.  The fact that the sources are
deeper than they need be is IMO not an issue because it is handled by
cvs.cygclass without intervention.

> So, it's probably moot for all current packages.

Agreed.

> It is sufficient to override the default behavior of the post-install
> phase with respect to documentation; that's what the urxvt packages care
> about. But other packages (unknown at this time, but I'm not possessed
> of sufficient hubris to rule them out) might need to override/customize
> some other phase:

IOW it's purely hypothetical.  I haven't found a case where this would
be necessary either.  If you do find something, I'll be happy to look at
it, but cygport development has always been driven by practical usage.


Yaakov

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREIAAYFAkkXtmQACgkQpiWmPGlmQSND/QCg2ZEvZZiR8mRJ5deX+o8QqBhi
M5AAniHfGhbNbtPK9QRmbbAXOR8dj8Lr
=Xat9
-----END PGP SIGNATURE-----

Re: cygport-0.9.3 in release-2

[Charles Wilson]

Yaakov (Cygwin Ports) wrote:
>> It only breaks the ABI (B?) if you do it the way you suggested:
>> http://cygwin.com/ml/cygwin/2008-05/msg00038.html
> 
> Yes, the concept is the same:  ABI breakage affects previously-built
> source packages but doesn't necessarily break API (IOW the .cygport
> wouldn't need to be changed, but you'd need to fetch() from scratch).
> API breakage affects the programmer, i.e. the .cygport itself would
> require a change.

Ack.

>> There's really no "rule" like '-d ${CVS_MODULE##*/}' that is universally
>> applicable: the module name and the -d option (if present) are
>> orthogonal controls. You don't want to tie them together.
> 
>> The way my patch does it, there is no API breakage -- but you have a new
>> API entry point [the new CVS_DIR variable].  The price of API
>> preservation is a proliferation of new ones; just ask Bill Gates.
> 
> I should learn about API preservation from Micro$oft?!?  Are you joking?
>  http://en.wikipedia.org/wiki/Criticism_of_Windows_Vista#Software_compatibility

Um, yeah. That WAS the joke. Vista (and the old DOS-vs-Lotus123 wars)
notwithstanding, the long-term joke about w32api is that Microsoft went
to extreme lengths to not break existing apps; rather than change an
API, they'd add 57 new ones (all those ...Ex() functions).  With Vista
(and arguably, when they started pushing .NET), API compatibility went
out the window. But there were 13 years between Win95 and EOL-XP, with a
major kernel change underneath -- and the w32api kept plugging away
(getting bigger, and bigger, and bigger...)

See:

Strategy Letter II: Chicken and Egg Problems
http://www.joelonsoftware.com/articles/fog0000000054.html
"Windows 95? No problem. Nice new 32 bit API, but it still ran old 16
bit software perfectly. Microsoft obsessed about this, spending a big
chunk of change testing every old program they could find with Windows
95. Jon Ross, who wrote the original version of SimCity for Windows 3.x,
told me that he accidentally left a bug in SimCity where he read memory
that he had just freed. Yep. It worked fine on Windows 3.x, because the
memory never went anywhere. Here's the amazing part: On beta versions of
Windows 95, SimCity wasn't working in testing. Microsoft tracked down
the bug and added specific code to Windows 95 that looks for SimCity. If
it finds SimCity running, it runs the memory allocator in a special mode
that doesn't free memory right away. That's the kind of obsession with
backward compatibility that made people willing to upgrade to Windows 95."


Of course, this article (written four years later, same author)
How Microsoft Lost the API War
http://www.joelonsoftware.com/articles/APIWar.html
is about how that USED to be true of Microsoft, but isn't true any
longer. They'd lost "the backwards-compatibility religion," as of 2004.
('Course, one of the pieces of evidence -- in 2004 -- was...wait for
it...Longhorn. That's right, our friend Vista!  Hence your wiki page
Criticism_of_Windows_Vista#Software_compatibility)

> OTOH one can definitely learn from GNOME, whose developer platform has
> maintained API compatiblity for 6 years while steadily adding new
> features every six months.

Right. Adding a new feature -- an API entry point, such as CVS_DIR --
doesn't break anything that doesn't use it. Ditto
gnome_fancy_new_file_chooser() so long as
gnome_boring_old_file_chooser() is still there.

> I have been trying to maintain ABI/API stability, although I wouldn't be
> surprised if I broke something along the way.

/stability/ is different than /compatibility/.  The former implies a
resistance to ANY change, backwards-compatible or not.  A corpse is
"stable". I agree you have succeeded admirably in maintaining cygport's
/stability/.

>> The benefit of allowing some mechanism to pass a -d option to the cvs
>> checkout is that I can ensure that my cvs.cygclass-generated origsrc
>> tarball has the same directory layout as a make-dist-generated one.
> 
>> So, it's probably moot for all current packages.
> 
> Agreed.
> 
> IOW it's purely hypothetical.  I haven't found a case where this would
> be necessary either.  If you do find something, I'll be happy to look at
> it, but cygport development has always been driven by practical usage.

YOUR practical usage.

MY practical usage has, well, taken a back seat, to your theoretical
ideas about software design.

It's taken over two years of regular and repeated complaints, to get
grudging acceptance of changes that /almost/-but-not-quite support
features I had working in Nov 2006. Features I implemented because I had
an actual -- practical -- need for them: I couldn't use cygport
effectively to support certain packages without them.

Forgive me if I view platitudes about how cygport development is "driven
by practical usage" with a somewhat jaundiced eye.

Don't get me wrong; I appreciate that you created cygport in the first
place -- it was a brilliant idea, and put the execrable gbs in its
well-earned grave.  And I definitely appreciate all your work with
cygwin-ports and now cygwin-X.  But this cygport "patch acceptance
cycle" has, well, not exactly been a shining example of open source
development; very much the cathedral, and nothing resembling a bazaar.

--
Chuck

Re: cygport-0.9.3 in release-2

[Andrew Schulman]

> Okay, so these are (mostly) your own custom patches needed to port the
> code to cygwin, and not "official" patches from somewhere else

Correct.  Or anyway, they're patches that aren't included in the source tarball,
and that I maintain separately as discrete patch files.

> Effectively, you're using
> src_prep_fini_hook() as a "maintainer mode", and "maintainer mode" is
> activated by the presence of ${topdir}/extra/ and ${topdir}/patch/.

Exactly.  In fact, it's seemed to me for a long time that there was a need to
separate the set of packager/maintainer commands in cygport, from those a user
wants if they just want to build the package from source.

> $ cygport pkg-VER-2.cygport custom0-import_changeset

Sure, this seems fine, if I understand correctly that it runs my custom
import_changeset() function.

> One advantage of this method is that some other user won't get a
> surprising error if they *happen* to have an extra/ directory present
> when they try to build your package. They'd have to explicitly invoke
> custom0-import_changeset -- and hopefully if they do THAT, then they
> know what they are doing. Your way, random files from their extra/
> directory will automatically get copied in, which could be...unexpected.

Agreed.

> Another advantage -- to my thinking -- is that I *never* import the
> original files from extra/ and patch/ unless I explicitly choose to do
> so; it won't happen by accident.  I think that's good; however, you may
> value the "automatic" behavior of embedding this action as part of 'prep'.

I always import patches/ and extra/.  It's part of my standard build procedure.
But, it's no big deal to have a separate command for that, if it fits better in
the design of cygport.  Although... something with a little shorter name than
custom0-import_changeset would be nice :|

> > Note that the problem you mentioned of the patches being applied twice doesn't
> > occur.  If I'm building the packages, then the extras/ and patches/ directories
> > are present.  Someone else who's building the binary package from source doesn't
> > have those directories
> 
> You hope.

No one's complained yet :)

> My typical workflow at present involves doing the following after I'm
> done with development for a new release:
> 
> <1>
> $ cygport foo-VER-REL.cygport pkg
> $ tar xvjf foo-VER-REL-src.tar.bz2
> $ cygport foo-VER-REL.cygport finish almostall
> 
> So that I am assured that the distributed src package works. With your
> system, you'd have to do
> 
> <2>
> $ cygport foo-VER-REL.cygport pkg
> $ mkdir temp
> $ cd temp
> $ tar xvjf ../foo-VER-REL-src.tar.bz2
> $ cygport foo-VER-REL.cygport all
> $ mv *.tar.bz2 ..
> $ cd ..
> $ rmdir temp
> 
> to have the same effect. If you did it as in <1> with your
> src_prep_fini_hook, you'd have BOTH the .src.patch and .cygwin.patch
> present, AND the extra/ and patch/ directories. That's not an issue with
> the extra/ files, but the changes represented by patch/* are duplicated
> by the .src.patch changes, and that would definitely bomb out.

Well, that is my problem it seems.  I generally don't go back and rebuild the
source package, but when I have I've just moved to a different directory to do
it.

> But, having said all that, I realize I'm now just arguing for using a
> DIFFERENT non-standard cygport patch.

As long as there's some reasonable way to do it, I don't care too much which way
it is.

Thanks to you and Yaakov for looking at this.
Andrew.

Re: cygport-0.9.3 in release-2

[Yaakov (Cygwin Ports)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Andrew Schulman wrote:
> Several of my packages require multiple patches to compile and run properly in
> Cygwin.  Instead of maintaining them all together as One Big Patch, I find it
> easier to manage them as individual, discrete patch files, and apply them all at
> package build time.

I do that all the time.  Just keep the patches in the same directory as
the .cygport, and add their file names (just the basename, no full URI
or path) to PATCH_URI.

> I also have extra files, such as README and setup.hint, that I like to just copy
> in before building, instead of maintaining them as patches.  Again, this is
> easier for me in the long run.

I don't think anybody maintains these as a patch; just copy them into
CYGWIN-PATCHES sometime before the install step.  I prefer to do this
manually so that I'm sure to check/update them before packaging.

> If there's a better way to do this, then I'm all ears.  I hadn't thought of
> doing it during compile() but that seems fine.

src_compile() isn't meant for what you're trying to do here.


Yaakov

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREIAAYFAkkWfBcACgkQpiWmPGlmQSMk7QCffEpzYd8urS3rG9ycC/eSw9+z
PEgAnRYjlF7FDhijGftnGj2/uV59+RzS
=NcW0
-----END PGP SIGNATURE-----

Re: cygport-0.9.3 in release-2

[Andrew Schulman]

> > Several of my packages require multiple patches to compile and run properly in
> > Cygwin.  Instead of maintaining them all together as One Big Patch, I find it
> > easier to manage them as individual, discrete patch files, and apply them all at
> > package build time.
> 
> I do that all the time.  Just keep the patches in the same directory as
> the .cygport, and add their file names (just the basename, no full URI
> or path) to PATCH_URI.

OK, but what if instead of having my patch files clutter up the top-level
directory, I want to keep them tucked neatly away in patches/?  (Which I do.)
Will cygport accept PATCH_URI=patches/* ?

> > I also have extra files, such as README and setup.hint, that I like to just copy
> > in before building, instead of maintaining them as patches.  Again, this is
> > easier for me in the long run.
> 
> I don't think anybody maintains these as a patch; just copy them into
> CYGWIN-PATCHES sometime before the install step.  I prefer to do this
> manually so that I'm sure to check/update them before packaging.

I prefer to do it automatically, so I don't forget or have to do it manually
every time I rebuild (which I sometimes repeat several times from scratch, for
the same pkg-ver-rel).  So should I do that at the start of src_compile(), or is
there a better place?

Thanks,
Andrew.

Re: cygport-0.9.3 in release-2

[Reini Urban]

2008/10/28 Yaakov (Cygwin Ports):
> I've just made cygport-0.9.3 available in release-2 with the following
> changes:
>
> * PV is now an array; members 1-* replace PVP[].
> * foo_CONTENTS can now be used in place of PKG_CONTENTS[].
> * cygtest(): Doesn't exit when tests fail.
> * autotools.cygclass: Detect ac-2.63+; detect missing LT_OUTPUT.
> * fossil.cygclass: NEW for Fossil RCS checkouts.
> * ruby.cygclass: Added rubyinto, doruby. Accept RDOC_MODULE.
> * ruby-gnome2.cygclass: Added ruby-goocanvas.
> * xorg.cygclass: Added GIT_URI; renamed font- packages.
> * zope.cygclass: REMOVED.  Use distutils instead.
>
> If there are any changes you want to see in cygport, please let me know
> ASAP; I would like to have the major changes out of the way soon,
> so that cygport isn't a moving target once everything is ready for a 1.7
> rebuild.

I've put
  CC="gcc-4"
  CXX="g++-4"
into /etc/cygport.conf (in cygwin-2)

I noticed in trying out some new release-2 packages, that
$  cygport icu-3.8-5.cygport prep
fails with
 * cygport 0.9 is required for Cygwin 1.7.
but
$ bash cygport icu-3.8-5.cygport prep
works okay


$ which cygport
/usr/bin/cygport
$ cygport --version
 * cygport 0.9 is required for Cygwin 1.7.
$ bash cygport --version
cygport 0.9.3
$ /usr/bin/cygport --version
cygport 0.9.3

There's no symlink - mount clash for the cygport files in
/usr/lib and /usr/share.
Strange.

cygport shebang => -x
$ cygport --version
+ readonly -f defined
+ export -f defined
+ alias 'ifdef=if defined'
+ alias 'ifndef=if ! defined'
+ set -e
+ case $(uname -r) in
++ uname -r
+ echo -e ' \e[1;31m*\e[0;0m cygport 0.9 is required for Cygwin 1.7.'
 * cygport 0.9 is required for Cygwin 1.7.
+ exit 127

even more strange:
$ joe /usr/bin/cygport
#!/bin/bash -x
################################################################################
...

$ head /usr/bin/cygport
#!/bin/bash
################################################################################
...

There's something fishy, which I haven't figures out yet.
There's no symlink /usr/bin/cygport, just the file.
-- 
Reini Urban
http://phpwiki.org/              http://murbreak.at/