per-version hints proposal

per-version hints proposal

[Jon Turney]

Currently, the setup.hint file is shared between all versions.

This means that manual intervention (by the package maintainer, or on 
sourceware) is needed when versions have different dependencies.

To automate this problem out of existence, I suggest replacing the 
setup.hint file in an upload with a package-version-release.hint file.

This will be basically identical to the existing setup.hint, with the 
advantage that it can't be trampled on by a future version, with the 
following changes:

* 'skip' doesn't seem meaningful on a per-version basis.  Since it seems 
we can automatically detect packages which should have this applied 
anyhow (see the discussion in [1]), I'd suggest ignoring this hint, to 
be retired at some future date.

* 'curr', 'prev' and 'test' don't make sense on a per-version basis.  So 
I suggest a separate file for these version overrides (versions.hint?)


cygport will be updated to create a pvr.hint rather than setup.hint


calm will be changed so that:

* The requires: line written in setup.ini will contain the union of the 
requires: from each pvr.hint

* The sdesc:, ldesc:, category: and message: lines in setup.ini will be 
taken from the pvr.hint for the curr version

* While it's already effectively mandatory that a package has a curr 
version, this requirement will be documented and enforced.

* The source: line in setup.ini for a package version will consider any 
external-source: from the corresponding pvr.hint

* Uploads with a setup.hint will continue to work as before, for a 
transitional period.


No setup changes are required.


Immediately, this avoids the need for manual intervention when versions 
have different dependencies, and going forward, this is lays some 
groundwork for supporting per-version dependencies.


[1] https://cygwin.com/ml/cygwin-apps/2016-02/msg00017.html

Re: per-version hints proposal

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2807 bytes --]

On Jun 20 16:28, Jon Turney wrote:
> 
> Currently, the setup.hint file is shared between all versions.
> 
> This means that manual intervention (by the package maintainer, or on
> sourceware) is needed when versions have different dependencies.
> 
> To automate this problem out of existence, I suggest replacing the
> setup.hint file in an upload with a package-version-release.hint file.
> 
> This will be basically identical to the existing setup.hint, with the
> advantage that it can't be trampled on by a future version, with the
> following changes:
> 
> * 'skip' doesn't seem meaningful on a per-version basis.  Since it seems we
> can automatically detect packages which should have this applied anyhow (see
> the discussion in [1]), I'd suggest ignoring this hint, to be retired at
> some future date.
> 
> * 'curr', 'prev' and 'test' don't make sense on a per-version basis.  So I
> suggest a separate file for these version overrides (versions.hint?)

Ideally we wouldn't need something like "prev" at all since the version
number itself is sufficient to specify what's curr and what's old.

As for test, IMHO it would make sense to specify "this is a test
release" right in the cygport file.  This in turn could create a
per-version hint with a test marker which is evaluated by calm
accordingly.  For instance, the name of the file could take over this
role.  Or even better, the package version number itself.

This would have an additional benefit:  We couldn't just move a package
from test to curr, it would have to be explicitely rebuilt as non-test
release.

I think this is the cleanest solution.

> cygport will be updated to create a pvr.hint rather than setup.hint
> 
> 
> calm will be changed so that:
> 
> * The requires: line written in setup.ini will contain the union of the
> requires: from each pvr.hint
> 
> * The sdesc:, ldesc:, category: and message: lines in setup.ini will be
> taken from the pvr.hint for the curr version
> 
> * While it's already effectively mandatory that a package has a curr
> version, this requirement will be documented and enforced.
> 
> * The source: line in setup.ini for a package version will consider any
> external-source: from the corresponding pvr.hint
> 
> * Uploads with a setup.hint will continue to work as before, for a
> transitional period.
> 
> 
> No setup changes are required.
> 
> 
> Immediately, this avoids the need for manual intervention when versions have
> different dependencies, and going forward, this is lays some groundwork for
> supporting per-version dependencies.

Sounds good to me.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: per-version hints proposal

[Marco Atzeri]

On 21/06/2016 14:03, Corinna Vinschen wrote:
> On Jun 20 16:28, Jon Turney wrote:
>>
>> Currently, the setup.hint file is shared between all versions.
>>
>> This means that manual intervention (by the package maintainer, or on
>> sourceware) is needed when versions have different dependencies.
>>
>> To automate this problem out of existence, I suggest replacing the
>> setup.hint file in an upload with a package-version-release.hint file.
>>
>> This will be basically identical to the existing setup.hint, with the
>> advantage that it can't be trampled on by a future version, with the
>> following changes:
>>

fine for me.


> Ideally we wouldn't need something like "prev" at all since the version
> number itself is sufficient to specify what's curr and what's old.
>
> As for test, IMHO it would make sense to specify "this is a test
> release" right in the cygport file.  This in turn could create a
> per-version hint with a test marker which is evaluated by calm
> accordingly.  For instance, the name of the file could take over this
> role.  Or even better, the package version number itself.
>
> This would have an additional benefit:  We couldn't just move a package
> from test to curr, it would have to be explicitely rebuilt as non-test
> release.

not a huge fan of this.
The last time we made the perl transition we put a lot of package in
test as temporary solution. Rebuild all just to change a label
seems a waste of time.

>>
>> No setup changes are required.
>>
>>
>> Immediately, this avoids the need for manual intervention when versions have
>> different dependencies, and going forward, this is lays some groundwork for
>> supporting per-version dependencies.
>
> Sounds good to me.

+1


> Corinna

Marco

Re: per-version hints proposal

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1977 bytes --]

On Jun 21 15:49, Marco Atzeri wrote:
> 
> On 21/06/2016 14:03, Corinna Vinschen wrote:
> > On Jun 20 16:28, Jon Turney wrote:
> > > 
> > > Currently, the setup.hint file is shared between all versions.
> > > 
> > > This means that manual intervention (by the package maintainer, or on
> > > sourceware) is needed when versions have different dependencies.
> > > 
> > > To automate this problem out of existence, I suggest replacing the
> > > setup.hint file in an upload with a package-version-release.hint file.
> > > 
> > > This will be basically identical to the existing setup.hint, with the
> > > advantage that it can't be trampled on by a future version, with the
> > > following changes:
> > > 
> 
> fine for me.
> 
> 
> > Ideally we wouldn't need something like "prev" at all since the version
> > number itself is sufficient to specify what's curr and what's old.
> > 
> > As for test, IMHO it would make sense to specify "this is a test
> > release" right in the cygport file.  This in turn could create a
> > per-version hint with a test marker which is evaluated by calm
> > accordingly.  For instance, the name of the file could take over this
> > role.  Or even better, the package version number itself.
> > 
> > This would have an additional benefit:  We couldn't just move a package
> > from test to curr, it would have to be explicitely rebuilt as non-test
> > release.
> 
> not a huge fan of this.
> The last time we made the perl transition we put a lot of package in
> test as temporary solution. Rebuild all just to change a label
> seems a waste of time.

Not a huge fan of what part?  I think in general it makes sense to
keep the "test" info in the ${version}.hint file.  If a simple
change to this file moves ${version} to non-test, ok with me.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: per-version hints proposal

[Marco Atzeri]

On 21/06/2016 16:28, Corinna Vinschen wrote:
> On Jun 21 15:49, Marco Atzeri wrote:
>>
>> On 21/06/2016 14:03, Corinna Vinschen wrote:
>>> On Jun 20 16:28, Jon Turney wrote:
>>>>
>
>>
>>> Ideally we wouldn't need something like "prev" at all since the version
>>> number itself is sufficient to specify what's curr and what's old.
>>>
>>> As for test, IMHO it would make sense to specify "this is a test
>>> release" right in the cygport file.  This in turn could create a
>>> per-version hint with a test marker which is evaluated by calm
>>> accordingly.  For instance, the name of the file could take over this
>>> role.  Or even better, the package version number itself.
>>>
>>> This would have an additional benefit:  We couldn't just move a package
>>> from test to curr, it would have to be explicitely rebuilt as non-test
>>> release.
>>
>> not a huge fan of this.
>> The last time we made the perl transition we put a lot of package in
>> test as temporary solution. Rebuild all just to change a label
>> seems a waste of time.
>
> Not a huge fan of what part?  I think in general it makes sense to
> keep the "test" info in the ${version}.hint file.  If a simple
> change to this file moves ${version} to non-test, ok with me.

changing ${version}.hint is fine also for me.

Rebuilding packages not ;-)

>
>
> Corinna
>

Re: per-version hints proposal

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 701 bytes --]

On 06/21/2016 06:03 AM, Corinna Vinschen wrote:

>> * 'curr', 'prev' and 'test' don't make sense on a per-version basis.  So I
>> suggest a separate file for these version overrides (versions.hint?)
> 
> Ideally we wouldn't need something like "prev" at all since the version
> number itself is sufficient to specify what's curr and what's old.

Except when upstream version numbers go backwards.  We'd have to adopt
something like Fedora's "epoch" numbering if we want our version numbers
to always be increasing (by bumping the epoch any time upstream versions
go backwards).

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: per-version hints proposal

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 881 bytes --]

On Jun 21 08:08, Eric Blake wrote:
> On 06/21/2016 06:03 AM, Corinna Vinschen wrote:
> 
> >> * 'curr', 'prev' and 'test' don't make sense on a per-version basis.  So I
> >> suggest a separate file for these version overrides (versions.hint?)
> > 
> > Ideally we wouldn't need something like "prev" at all since the version
> > number itself is sufficient to specify what's curr and what's old.
> 
> Except when upstream version numbers go backwards.  We'd have to adopt
> something like Fedora's "epoch" numbering if we want our version numbers
> to always be increasing (by bumping the epoch any time upstream versions
> go backwards).

I think Yaakov is desperately asking for this for a long time :)


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: per-version hints proposal

[Achim Gratz]

Eric Blake writes:
> Except when upstream version numbers go backwards.  We'd have to adopt
> something like Fedora's "epoch" numbering if we want our version numbers
> to always be increasing (by bumping the epoch any time upstream versions
> go backwards).

Oh please not.  I've looked into this as I thought it would be helping a
few things.  The bummer is that once you've introduced that epoch part
to any package, there is no way to drop it again.  I'd rather keep a
list of exceptions someplace for those rare cases where the versioning
algorithm doesn't do the right thing.  Actually, I'm already doing it
for the Perl distributions to override dodgy information in metadata
about dependencies.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: per-version hints proposal

[Jon Turney]

On 21/06/2016 13:03, Corinna Vinschen wrote:
> On Jun 20 16:28, Jon Turney wrote:
[...]
>> * 'curr', 'prev' and 'test' don't make sense on a per-version basis.  So I
>> suggest a separate file for these version overrides (versions.hint?)
>
> Ideally we wouldn't need something like "prev" at all since the version
> number itself is sufficient to specify what's curr and what's old.
>
> As for test, IMHO it would make sense to specify "this is a test
> release" right in the cygport file.  This in turn could create a
> per-version hint with a test marker which is evaluated by calm
> accordingly.  For instance, the name of the file could take over this
> role.  Or even better, the package version number itself.
>
> This would have an additional benefit:  We couldn't just move a package
> from test to curr, it would have to be explicitely rebuilt as non-test
> release.
>
> I think this is the cleanest solution.

I'm not sure I agree with this reasoning.

Without control over the other elements which determine the build 
product (e.g. compiler version, headers, static libraries, etc.), there 
is the risk that any testing you have done on the test package is 
invalidated when it is rebuilt.

Otoh, if you did have perfect build reproducibility, you are rebuilding 
the package just to change a label applied to it, which seems a bit 
inefficient.

I take the point that 'test' could be more useful, but I think that's 
out of scope of what I want to achieve, for the moment.

Re: per-version hints proposal

[Jon Turney]

On 20/06/2016 16:28, Jon Turney wrote:
> Currently, the setup.hint file is shared between all versions.
>
> This means that manual intervention (by the package maintainer, or on
> sourceware) is needed when versions have different dependencies.
>
> To automate this problem out of existence, I suggest replacing the
> setup.hint file in an upload with a package-version-release.hint file.

I deployed an update to calm today which adds support for this proposal, 
with some minor adjustments noted below.

Nobody can actually use this until a cygport with the corresponding 
changes is released, which still needs some testing, after which I will 
update the website documentation.

> This will be basically identical to the existing setup.hint, with the
> advantage that it can't be trampled on by a future version, with the
> following changes:
>
> * 'skip' doesn't seem meaningful on a per-version basis.  Since it seems
> we can automatically detect packages which should have this applied
> anyhow (see the discussion in [1]), I'd suggest ignoring this hint, to
> be retired at some future date.

skip is honoured, if present, but not required.

> * 'curr', 'prev' and 'test' don't make sense on a per-version basis.  So
> I suggest a separate file for these version overrides (versions.hint?)

This file is called override.hint.

> cygport will be updated to create a pvr.hint rather than setup.hint

I'll send a patch to update cygport separately.

> calm will be changed so that:
>
> * The requires: line written in setup.ini will contain the union of the
> requires: from each pvr.hint
>
> * The sdesc:, ldesc:, category: and message: lines in setup.ini will be
> taken from the pvr.hint for the curr version
>
> * While it's already effectively mandatory that a package has a curr
> version, this requirement will be documented and enforced.

calm will issue a warning, not an error, when a package doesn't have a 
current version.  For the purpose above, information from the highest 
version will be used in place of information from the curr version.

per the discussion [1], it's possibly useful in mksetupini, so the 
warning can be suppressed there.

[1] https://cygwin.com/ml/cygwin-apps/2016-07/msg00052.html

> * The source: line in setup.ini for a package version will consider any
> external-source: from the corresponding pvr.hint
>
> * Uploads with a setup.hint will continue to work as before, for a
> transitional period.
>
>
> No setup changes are required.
>
>
> Immediately, this avoids the need for manual intervention when versions
> have different dependencies, and going forward, this is lays some
> groundwork for supporting per-version dependencies.
>
>
> [1] https://cygwin.com/ml/cygwin-apps/2016-02/msg00017.html

Re: per-version hints proposal

[Achim Gratz]

Jon Turney writes:
>> calm will be changed so that:
>>
>> * The requires: line written in setup.ini will contain the union of the
>> requires: from each pvr.hint
>>
>> * The sdesc:, ldesc:, category: and message: lines in setup.ini will be
>> taken from the pvr.hint for the curr version

I've not given this much more thought, but I think we should change the
setup.ini syntax to allow different values for _all_ keys in a package
for each version.  The ones that are not in a specific section
(i.e. where they are now, right after the package line) would just
provide a default that gets used when there is no version-specific value
provided in the corresponding section.  Calm could/should move such
key/value pairs to the default when they are identical among the
majority of versions, otherwise use those from the current version.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: per-version hints proposal

[Jon Turney]

On 31/08/2016 20:13, Achim Gratz wrote:
> Jon Turney writes:
>>> calm will be changed so that:
>>>
>>> * The requires: line written in setup.ini will contain the union of the
>>> requires: from each pvr.hint
>>>
>>> * The sdesc:, ldesc:, category: and message: lines in setup.ini will be
>>> taken from the pvr.hint for the curr version
>
> I've not given this much more thought, but I think we should change the
> setup.ini syntax to allow different values for _all_ keys in a package
> for each version.

Yes, ultimately.

There's not much value until we have some functionality or UI which uses 
that, though, and changing setup.ini syntax like that is probably 
incompatible with older setup.

 > The ones that are not in a specific section
> (i.e. where they are now, right after the package line) would just
> provide a default that gets used when there is no version-specific value
> provided in the corresponding section.  Calm could/should move such
> key/value pairs to the default when they are identical among the
> majority of versions, otherwise use those from the current version.

This seems perhaps a bit of a premature optimization.

The setup.ini will be compressed, so any redundancy between versions 
won't affect the file size much.

Re: per-version hints proposal

[Jon Turney]

On 30/08/2016 13:24, Jon Turney wrote:
> On 20/06/2016 16:28, Jon Turney wrote:
>> Currently, the setup.hint file is shared between all versions.
>>
>> This means that manual intervention (by the package maintainer, or on
>> sourceware) is needed when versions have different dependencies.
>>
>> To automate this problem out of existence, I suggest replacing the
>> setup.hint file in an upload with a package-version-release.hint file.
[...]
>> * 'curr', 'prev' and 'test' don't make sense on a per-version basis.  So
>> I suggest a separate file for these version overrides (versions.hint?)
>
> This file is called override.hint.

While not a great deal of use is made of test versions, this mechanism 
doesn't seem to be working well for that, and there have been several 
requests to improve it.

There's no automation to generate override.hint, and writing it 
correctly requires too much knowledge about how calm is going to process it.

(e.g. if the curr: version is N, we can upload a version N+1 as test: 
with an override.hint that says 'test: N+1', but if we then want to 
upload version N+2 as a replacement test, you need to know that you 
should write an override.hint that says 'curr: N test: N+2' otherwise 
calm will automatically promote N+1 to curr (and vault any N-1, which 
makes this a pain to revert))

The proposal to address this is:

Add support to calm for a 'test:' line in PVR.hint, marking a version as 
a test version.

If multiple versions are marked test, the highest version will be used 
as the test version in the generated setup.ini (and thus offered for 
installation using the 'exp' control in setup.)

(Note to self: why isn't this control labelled 'test', which is an 
actual english word???)

Versions marked as test cannot be used as curr: (so test versions are 
never automatically promoted to curr)

override.hint will continue to work, and, if one exists it takes 
precedence over these rules.

cygport will be updated to (details TBC) accept a --test flag which is 
significant to the cygport package stage, and adds this 'test:' line to 
all the generated PVR.hint files.

To promote a package from test to curr, a script will be run on 
sourceware to remove the test: line from the existing PVR.hints in a 
given package subtree, for a given VR.

Since this requires shell access on sourceware, if you don't have that, 
you can ask here or on #cygwin-developers.

If all the people with shell access are unavailable, if you still have 
the cygport build directory, you can manually amend all the PVR.hints 
under PVR.arch/dist/ to remove the 'test:' line, and re-upload them.

Finally, the package can be rebuilt with a bumped release number and 
without --test, although opinion is mixed as to if this is a good idea 
or not.

I would like to provide an automatic mechanism to allow package 
maintainers to promote their own test packages, but there are a few 
stumbling blocks in the way of that, currently.

Re: per-version hints proposal

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3192 bytes --]

On Dec  8 19:30, Jon Turney wrote:
> On 30/08/2016 13:24, Jon Turney wrote:
> > This file is called override.hint.
> 
> While not a great deal of use is made of test versions, this mechanism
> doesn't seem to be working well for that, and there have been several
> requests to improve it.
> 
> There's no automation to generate override.hint, and writing it correctly
> requires too much knowledge about how calm is going to process it.
> [...]

ACK.  It's kind of backwards compared to the new layout with per-version
hint files.

> The proposal to address this is:
> 
> Add support to calm for a 'test:' line in PVR.hint, marking a version as a
> test version.

ACK

> If multiple versions are marked test, the highest version will be used as
> the test version in the generated setup.ini (and thus offered for
> installation using the 'exp' control in setup.)
> 
> (Note to self: why isn't this control labelled 'test', which is an actual
> english word???)

Note to jturney, change it, don't be shy.

> Versions marked as test cannot be used as curr: (so test versions are never
> automatically promoted to curr)
> 
> override.hint will continue to work, and, if one exists it takes precedence
> over these rules.

I would rather drop it as an evolutionary dead end.

> cygport will be updated to (details TBC) accept a --test flag which is
> significant to the cygport package stage, and adds this 'test:' line to all
> the generated PVR.hint files.

--test would be fine.

> To promote a package from test to curr, a script will be run on sourceware
> to remove the test: line from the existing PVR.hints in a given package
> subtree, for a given VR.
> 
> Since this requires shell access on sourceware, if you don't have that, you
> can ask here or on #cygwin-developers.

I don't think this is feasible.  The maintainer should have control
over the promotion from test to curr.  I'm not affected by this since
I generate new versions as soon as I promote, so this is more maintainers
like JonY, for whom a rebuild and reupload of the gcc packages just to
promote test to curr is quite a burden.

First, not well thought out proposal:

- cygport gets a new command, e. g.

    cygport foo.cygport {promote|untest|currify}

  This command has only one purpose.  It uploads a file !untest
  to the maintainers upload area, with a single line containing
  the version number from the foo.cygport file, i. e.

  - Fetch $PVR from foo.cygport, e. g.  2.24-1.
  - echo "$PVR" > !untest
  - lftp !untest to cygwin.com:maintainer-area

- While creating the ini file, calm looks for !untest files.  If one
  is available, check the version number.  If there's a matching PVR.hints
  file, drop the test marker.  Continue with creating the ini file.

> I would like to provide an automatic mechanism to allow package maintainers
> to promote their own test packages, but there are a few stumbling blocks in
> the way of that, currently.

-v?


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: per-version hints proposal

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

On Dec  9 11:46, Corinna Vinschen wrote:
> I don't think this is feasible.  The maintainer should have control
> over the promotion from test to curr.  I'm not affected by this since
> I generate new versions as soon as I promote, so this is more maintainers
> like JonY, for whom a rebuild and reupload of the gcc packages just to
> promote test to curr is quite a burden.
> 
> First, not well thought out proposal:
> 
> - cygport gets a new command, e. g.
> 
>     cygport foo.cygport {promote|untest|currify}
> 
>   This command has only one purpose.  It uploads a file !untest
>   to the maintainers upload area, with a single line containing
>   the version number from the foo.cygport file, i. e.
> 
>   - Fetch $PVR from foo.cygport, e. g.  2.24-1.
>   - echo "$PVR" > !untest
>   - lftp !untest to cygwin.com:maintainer-area
> 
> - While creating the ini file, calm looks for !untest files.  If one
>   is available, check the version number.  If there's a matching PVR.hints
>   file, drop the test marker.  Continue with creating the ini file.

Even simpler:

- cygport gets a new command, e. g.

    cygport foo.cygport {promote|untest|currify}

  This command has only one purpose.  It creates and uploades new
  $PVR.hint files without the test: marker to the maintainers upload
  area.

  Shouldn't these files be picked up by calm and overwrite the existing
  PVR.hint files and the test marker is gone?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: per-version hints proposal

[Jon Turney]

On 09/12/2016 11:10, Corinna Vinschen wrote:
> On Dec  9 11:46, Corinna Vinschen wrote:
>> I don't think this is feasible.  The maintainer should have control
>> over the promotion from test to curr.  I'm not affected by this since
>> I generate new versions as soon as I promote, so this is more maintainers
>> like JonY, for whom a rebuild and reupload of the gcc packages just to
>> promote test to curr is quite a burden.

I mentioned 'ask for the package to be promoted' first, assuming that 
would be the first method tried.

[...]
>
> - cygport gets a new command, e. g.
>
>     cygport foo.cygport {promote|untest|currify}
>
>   This command has only one purpose.  It creates and uploades new
>   $PVR.hint files without the test: marker to the maintainers upload
>   area.
>
>   Shouldn't these files be picked up by calm and overwrite the existing
>   PVR.hint files and the test marker is gone?

Embarrassingly, this was supposed to work, but didn't.  Thanks to eblake 
for being the guinea pig for discovering that.  I deployed a fix for that.

Re: per-version hints proposal

[Jon Turney]

On 09/12/2016 10:46, Corinna Vinschen wrote:
> On Dec  8 19:30, Jon Turney wrote:
>> On 30/08/2016 13:24, Jon Turney wrote:
>>
>> (Note to self: why isn't this control labelled 'test', which is an actual
>> english word???)
>
> Note to jturney, change it, don't be shy.

Patch sent :)

>> Versions marked as test cannot be used as curr: (so test versions are never
>> automatically promoted to curr)
>>
>> override.hint will continue to work, and, if one exists it takes precedence
>> over these rules.
>
> I would rather drop it as an evolutionary dead end.

It still has some potential uses, such as overriding curr:

[...]
>> I would like to provide an automatic mechanism to allow package maintainers
>> to promote their own test packages, but there are a few stumbling blocks in
>> the way of that, currently.
>
> -v?

So, this requires:

Making this change in the release area in a transactional way (so we can 
rollback if the upload fails.

Some way to request the removal of the test label.  This would probably 
be best if things were restructured as some kind of 'commands' from a 
'script', rather than certain files with funny names implying special 
actions, and moving files by default.

These are both new functionality, and I have no good ideas yet about how 
to do it, or when I might be able to work on it.

There are only a handful of packages using test versions at the moment 
(base-files, cygwin, gdb, readline, socat, xorg-server), so some sort of 
manual process to promote things until this gets a proper solution 
doesn't seem too demanding of manpower...

Re: per-version hints proposal

[Achim Gratz]

Jon Turney writes:
> cygport will be updated to (details TBC) accept a --test flag which is
> significant to the cygport package stage, and adds this 'test:' line
> to all the generated PVR.hint files.

I've opted to make it a command (package-test or pkg-test) instead, as
that seems to fall more in line with my usage pattern and types easier.

http://repo.or.cz/cygport/rpm-style.git/commitdiff/95282c614f0991e0cada4f65ac0c6888803f71a3


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: per-version hints proposal

[Ken Brown]

[-- Attachment #1: Type: text/plain, Size: 608 bytes --]

On 4/8/2017 1:00 PM, Achim Gratz wrote:
> Jon Turney writes:
>> cygport will be updated to (details TBC) accept a --test flag which is
>> significant to the cygport package stage, and adds this 'test:' line
>> to all the generated PVR.hint files.
>
> I've opted to make it a command (package-test or pkg-test) instead, as
> that seems to fall more in line with my usage pattern and types easier.

What about just using a variable (say PKG_TEST) in the cygport file. 
For example, the line

   PKG_TEST=1

would cause the 'test:' line to be added to the hint files.

The attached patch implements this.

Ken


[-- Attachment #2: 0001-pkg_pkg-implement-PKG_TEST.patch --]
[-- Type: text/plain, Size: 702 bytes --]

From a9a63675dd83416a4ec510c3a2e2c2ca9eda9ad4 Mon Sep 17 00:00:00 2001
From: Ken Brown <kbrown@cornell.edu>
Date: Wed, 12 Apr 2017 15:39:26 -0400
Subject: [PATCH] pkg_pkg: implement PKG_TEST

---
 lib/pkg_pkg.cygpart | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart
index 5552040..69910fc 100644
--- a/lib/pkg_pkg.cygpart
+++ b/lib/pkg_pkg.cygpart
@@ -717,6 +717,12 @@ _EOF
 message: ${pkg_name[${n}]} "${!pkg_message_var}"
 _EOF
 			fi
+			if defined PKG_TEST
+			then
+				cat >> ${distdir}/${PN}/${distsubdir}/${pkg_name[${n}]}-${PVR}.hint <<-_EOF
+test:
+_EOF
+			fi
 		else
 			warning "${pkg_hint[${n}]%.hint}.hint is missing";
 		fi
-- 
2.8.3

Re: per-version hints proposal

[Achim Gratz]

Ken Brown writes:
> What about just using a variable (say PKG_TEST) in the cygport
> file. For example, the line
>
>   PKG_TEST=1
>
> would cause the 'test:' line to be added to the hint files.

That was booed down by Jon (and Yaakov earlier).  It's become clear that
everyone does their packaging in their own peculiar way and I don't see
a point in discussing which is the right one.

> The attached patch implements this.

You don't need another patch.  Just use mine and set TESTPACKAGE
non-null in the cygport file and it still works (oops, now everyone
knows… :-).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: per-version hints proposal

[Achim Gratz]

Jon Turney writes:
> Currently, the setup.hint file is shared between all versions.
>
> This means that manual intervention (by the package maintainer, or on
> sourceware) is needed when versions have different dependencies.
>
> To automate this problem out of existence, I suggest replacing the
> setup.hint file in an upload with a package-version-release.hint file.

Since this is now moving into reality, the documentation in

https://cygwin.com/setup.html

has become out-of-date.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: per-version hints proposal

[Marco Atzeri]

On 17/09/2016 08:14, Achim Gratz wrote:
> Jon Turney writes:
>> Currently, the setup.hint file is shared between all versions.
>>
>> This means that manual intervention (by the package maintainer, or on
>> sourceware) is needed when versions have different dependencies.
>>
>> To automate this problem out of existence, I suggest replacing the
>> setup.hint file in an upload with a package-version-release.hint file.
>
> Since this is now moving into reality, the documentation in
>
> https://cygwin.com/setup.html
>
> has become out-of-date.
>
>
> Regards,
> Achim.
>

are we not yet missing an updated cygport release ?
Or a test one


Regards
Marco

Re: per-version hints proposal

[Jon Turney]

On 18/09/2016 06:17, Marco Atzeri wrote:
> On 17/09/2016 08:14, Achim Gratz wrote:
>> Jon Turney writes:
>>> Currently, the setup.hint file is shared between all versions.
>>>
>>> This means that manual intervention (by the package maintainer, or on
>>> sourceware) is needed when versions have different dependencies.
>>>
>>> To automate this problem out of existence, I suggest replacing the
>>> setup.hint file in an upload with a package-version-release.hint file.
>>
>> Since this is now moving into reality, the documentation in
>>
>> https://cygwin.com/setup.html
>>
>> has become out-of-date.
>>
>>
>> Regards,
>> Achim.
>>
>
> are we not yet missing an updated cygport release ?
> Or a test one

Yes, this is only usable by people running cygport from git, currently.

After I wrote the documentation update, I found it needs to describe
the backwards-compatible use of setup.hint.

Given that, there didn't seem to be any harm in pushing the 
documentation update now.


Achim,

I notice the section about postinstall scripts doesn't mention permanent 
postinsall scripts at all.

Could you possibly provide a couple of paragraphs?

Re: per-version hints proposal

[Achim Gratz]

Jon Turney writes:
> I notice the section about postinstall scripts doesn't mention
> permanent postinsall scripts at all.
>
> Could you possibly provide a couple of paragraphs?

I had sent them to this very list at the time.  I need to dig them out
of the archive unless you're faster.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: per-version hints proposal

[Achim Gratz]

Achim Gratz writes:
> Jon Turney writes:
>> I notice the section about postinstall scripts doesn't mention
>> permanent postinsall scripts at all.
>>
>> Could you possibly provide a couple of paragraphs?
>
> I had sent them to this very list at the time.  I need to dig them out
> of the archive unless you're faster.

There it is:
https://cygwin.com/ml/cygwin-apps/2014-12/msg00148.html


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: per-version hints proposal

[Ken Brown]

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

On 9/18/2016 12:28 PM, Achim Gratz wrote:
> Achim Gratz writes:
>> Jon Turney writes:
>>> I notice the section about postinstall scripts doesn't mention
>>> permanent postinsall scripts at all.
>>>
>>> Could you possibly provide a couple of paragraphs?
>>
>> I had sent them to this very list at the time.  I need to dig them out
>> of the archive unless you're faster.
>
> There it is:
> https://cygwin.com/ml/cygwin-apps/2014-12/msg00148.html

The attached patch attempts to extract two paragraphs from this for the 
web page.

Ken


[-- Attachment #2: 0001-Add-information-about-perpetual-postinstall-scripts.patch --]
[-- Type: text/plain, Size: 3093 bytes --]

From 5950633d23c69cf84200409f2f5354901ecb7933 Mon Sep 17 00:00:00 2001
From: Ken Brown <kbrown@cornell.edu>
Date: Mon, 19 Sep 2016 11:30:34 -0400
Subject: [PATCH] Add information about perpetual postinstall scripts

Also mention ".dash" and ".cmd" suffixes.
---
 packaging-package-files.html | 39 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 34 insertions(+), 5 deletions(-)

diff --git a/packaging-package-files.html b/packaging-package-files.html
index fdf7fdc..e5db8c5 100755
--- a/packaging-package-files.html
+++ b/packaging-package-files.html
@@ -160,13 +160,17 @@ etc...
 <p>
   If your package requires certain commands to be executed after the files in
   the package are installed, include them in a file in the package called
-  /etc/postinstall/<var>package</var>.sh or
-  /etc/postinstall/<var>package</var>.bat.
+  /etc/postinstall/<var>package</var>.sh,
+  /etc/postinstall/<var>package</var>.dash,
+  /etc/postinstall/<var>package</var>.bat, or
+  /etc/postinstall/<var>package</var>.cmd.
 </p>
 <p>
-  If the file's name ends in ".sh", it is executed with the Cygwin shell; if it
-  ends in ".bat", it is executed with the DOS command interpreter. If it doesn't
-  end with either of these suffixes, it is ignored.
+  If the file's name ends in ".sh", it is executed with the Cygwin
+  bash shell; if it ends in ".dash", it is executed with the Cygwin
+  dash shell; if it ends in ".bat" or ".cmd", it is executed with the
+  Windows command interpreter. If it doesn't end with any of these
+  suffixes, it is ignored.
 </p>
 <p>
   After the script has been successfully run it is renamed by appending the
@@ -183,6 +187,31 @@ etc...
   scripts have already been run.
 </p>
 
+<h2>Perpetual postinstall scripts</h2>
+<p>
+  In addition to the ordinary ("run-once") postinstall scripts
+  described above, the setup program supports "perpetual" postinstall
+  scripts.  These are run on every invocation of setup and do not have
+  ".done" appended to their names after they are run.  Perpetual
+  postinstall scripts are distinguished from run-once scripts by
+  having names that start with "0p_" or "zp_".  Those that start with
+  "0p_" are run before the run-once scripts, and those that start with
+  "zp_" are run after the run-once scripts.  Examples include
+  0p_000_autorebase.dash (provided by the _autorebase package) and
+  0p_update-info-dir.dash (provided by the info package).
+</p>
+<p>
+  For those package maintainers wanting to employ perpetual scripts,
+  the first thing to keep in mind is to only use this feature for
+  things that really can't be done with run-once scripting.  Any
+  perpetual script should minimize the resources used (use dash
+  instead of bash for instance) and exit at the earliest possible
+  moment if no action is required.  Scripts of type "0p_" must be able
+  to run with the Base packages installed but the postinstall scripts
+  not yet executed; in practical terms that rules out using bash
+  scripts.  This limitation does not apply to scripts of type "zp_".
+</p>
+
 </div>
 </body>
 </html>
-- 
2.8.3

Re: per-version hints proposal

[Achim Gratz]

> The attached patch attempts to extract two paragraphs from this for
> the web page.

Thank you, Ken.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: per-version hints proposal

[Jon Turney]

On 19/09/2016 19:24, Achim Gratz wrote:
>> The attached patch attempts to extract two paragraphs from this for
>> the web page.
>
> Thank you, Ken.

Great, thanks.  I applied this.

Re: per-version hints proposal

[Ken Brown]

On 9/18/2016 11:14 AM, Jon Turney wrote:
> On 18/09/2016 06:17, Marco Atzeri wrote:
>> On 17/09/2016 08:14, Achim Gratz wrote:
>>> Jon Turney writes:
>>>> Currently, the setup.hint file is shared between all versions.
>>>>
>>>> This means that manual intervention (by the package maintainer, or on
>>>> sourceware) is needed when versions have different dependencies.
>>>>
>>>> To automate this problem out of existence, I suggest replacing the
>>>> setup.hint file in an upload with a package-version-release.hint file.
>>>
>>> Since this is now moving into reality, the documentation in
>>>
>>> https://cygwin.com/setup.html
>>>
>>> has become out-of-date.
>>>
>>>
>>> Regards,
>>> Achim.
>>>
>>
>> are we not yet missing an updated cygport release ?
>> Or a test one
>
> Yes, this is only usable by people running cygport from git, currently.
>
> After I wrote the documentation update, I found it needs to describe
> the backwards-compatible use of setup.hint.
>
> Given that, there didn't seem to be any harm in pushing the
> documentation update now.
>
>
> Achim,
>
> I notice the section about postinstall scripts doesn't mention permanent
> postinsall scripts at all.
>
> Could you possibly provide a couple of paragraphs?

Another thing: The page still says that all release announcements should 
contain information about unsubscribing from the cygwin-announce mailing 
list.  But it seems that many (most?) maintainers have stopped doing 
this.  And 'cygport announce' doesn't do it.  Should that instruction be 
removed?

Ken

Re: per-version hints proposal

[Marco Atzeri]

On 18/09/2016 18:41, Ken Brown wrote:

>
> Another thing: The page still says that all release announcements should
> contain information about unsubscribing from the cygwin-announce mailing
> list.  But it seems that many (most?) maintainers have stopped doing
> this.  And 'cygport announce' doesn't do it.  Should that instruction be
> removed?
>
> Ken

I suggest to remove the requirements.

The main list have in automatic:

Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

that is it more than enough.

Regards
Marco

Re: per-version hints proposal

[Achim Gratz]

Marco Atzeri writes:
> I suggest to remove the requirements.
>
> The main list have in automatic:
>
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>
> that is it more than enough.

That doesn't help you much when you're subscribed to the announcement
list instead of the main list, which I think was the point of the
requirement.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: per-version hints proposal

[Marco Atzeri]

On 18/09/2016 19:16, Achim Gratz wrote:
> Marco Atzeri writes:
>> I suggest to remove the requirements.
>>
>> The main list have in automatic:
>>
>> Problem reports:       http://cygwin.com/problems.html
>> FAQ:                   http://cygwin.com/faq/
>> Documentation:         http://cygwin.com/docs.html
>> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>>
>> that is it more than enough.
>
> That doesn't help you much when you're subscribed to the announcement
> list instead of the main list, which I think was the point of the
> requirement.
>
>
> Regards,
> Achim.
>

May be, but then it should be added in automatic, like main list.
I see no added value to the current instructions.

Regards
Marco

Re: per-version hints proposal

[Eric Blake]

[-- Attachment #1.1: Type: text/plain, Size: 1709 bytes --]

On 06/20/2016 10:28 AM, Jon Turney wrote:
> 
> Currently, the setup.hint file is shared between all versions.
> 
> This means that manual intervention (by the package maintainer, or on
> sourceware) is needed when versions have different dependencies.
> 
> To automate this problem out of existence, I suggest replacing the
> setup.hint file in an upload with a package-version-release.hint file.
> 
> This will be basically identical to the existing setup.hint, with the
> advantage that it can't be trampled on by a future version, with the
> following changes:
> 
> * 'skip' doesn't seem meaningful on a per-version basis.  Since it seems
> we can automatically detect packages which should have this applied
> anyhow (see the discussion in [1]), I'd suggest ignoring this hint, to
> be retired at some future date.
> 
> * 'curr', 'prev' and 'test' don't make sense on a per-version basis.  So
> I suggest a separate file for these version overrides (versions.hint?)
> 
> 
> cygport will be updated to create a pvr.hint rather than setup.hint
> 

It seems like the normal 'cygport foo.cygport upload' would be
responsible for just pushing a single pvr.hint.  I wonder if it would
make sense to add a new cygport command, 'cygport foo.cygport override
prev=pvr1 curr=pvr2 test=pvr3' that is responsible for pushing a new
override.hint (and nothing else), where the prev, curr, and test
parameters are optional according to what is desired (if all three are
omitted, push a directive to delete any existing override.hint and fall
back to normal version picking).

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

How to override previous version?

[David Rothenberger]

When I package Subversion, I usually leave the last major release as the 
previous version in setup because there are compatibility issues when 
updating major releases.

I'm now a little confused about how to accomplish this with the new 
pvr.hint scheme. I think I just need to upload an override.hint file, 
but I'm not sure what its contents should be nor whether I need to make 
one for each package or just one for the entire directory.

Subversion has many child packages, so I get a dist tree with the 
top-level subversion package and then subdirectories for each child 
package. Now each of these have a separate pvr.hint file.

I'm trying to package 1.9.5-1 and 1.8.17-1. Would I do this by creating 
an override.hint file in the top-level dist directory with the contents:

prev: 1.8.17-1

Is that sufficient? Any advice would be appreciated.

Thanks,
David


-- 
David Rothenberger  ----  daveroth@acm.org

... Had this been an actual emergency, we would have fled in terror,
and you would not have been informed.

Re: How to override previous version?

[Jon Turney]

On 10/12/2016 22:42, David Rothenberger wrote:
> When I package Subversion, I usually leave the last major release as the
> previous version in setup because there are compatibility issues when
> updating major releases.
>
> I'm now a little confused about how to accomplish this with the new
> pvr.hint scheme. I think I just need to upload an override.hint file,
> but I'm not sure what its contents should be nor whether I need to make
> one for each package or just one for the entire directory.
>
> Subversion has many child packages, so I get a dist tree with the
> top-level subversion package and then subdirectories for each child
> package. Now each of these have a separate pvr.hint file.
>
> I'm trying to package 1.9.5-1 and 1.8.17-1. Would I do this by creating
> an override.hint file in the top-level dist directory with the contents:
>
> prev: 1.8.17-1
>
> Is that sufficient? Any advice would be appreciated.

That looks like the right contents.

But currently, an override.hint is required for each (sub)package.