Possible "stage" token for SCALLYWAG builds

Possible "stage" token for SCALLYWAG builds

[Ken Brown]

Jon,

When building TeX Live (once a year), I have to build texlive, 
asymptote, and all the texlive-collection-* packages.  I currently can't 
use SCALLYWAG to do the builds because these packages need to all be 
deployed at once.  So I have to build them all locally, upload them to 
my staging area, and then upload !ready files, one for x86_64 and one 
for noarch.

It would be convenient for me if SCALLYWAG would accept a "stage" token 
that would upload the files to my staging area without deploying them. 
Then I could let SCALLYWAG do the builds, and I could upload the !ready 
files when everything is staged.

If I'm the only maintainer who would find this useful, then I can 
continue doing it the way I always have.  But maybe others would find it 
useful too.

Thanks.

Ken

Re: Possible "stage" token for SCALLYWAG builds

[Jon Turney]

On 15/02/2023 21:48, Ken Brown via Cygwin-apps wrote:
> Jon,
> 
> When building TeX Live (once a year), I have to build texlive, 
> asymptote, and all the texlive-collection-* packages.  I currently can't 
> use SCALLYWAG to do the builds because these packages need to all be 
> deployed at once.  So I have to build them all locally, upload them to 
> my staging area, and then upload !ready files, one for x86_64 and one 
> for noarch.
> 
> It would be convenient for me if SCALLYWAG would accept a "stage" token 
> that would upload the files to my staging area without deploying them. 
> Then I could let SCALLYWAG do the builds, and I could upload the !ready 
> files when everything is staged.
> 
> If I'm the only maintainer who would find this useful, then I can 
> continue doing it the way I always have.  But maybe others would find it 
> useful too.

I'm not opposed to adding a some feature to support this if needed, but 
from the way you are describing it, this sounds more like a constraint 
that the dependency solver should be aware of.

(just because texlive-2023 and texlive-collection-foo-2023 are made 
available to download at the same time, doesn't mean that they always 
end up installed together, as the user might alter the version of one or 
the other)

If they really must be kept in lockstep to work correctly, then there's 
at least a couple of ways of doing that:

* give texlive an additional provide, such as texlive_2023, and make 
everything that requires it, require that (something similar is done 
with perl and perl modules)

* make things which require texlive do so with a version constraint like 
'requires: texlive (>= 20230000), texlive (<20240000)' (in theory this 
works, but I have no doubt that a bug will emerge when someone tries to 
use it, and ofc, it relies on the range of future versions which are 
compatible being correctly known in advance)

Assuming I've understood correctly, you might want to survey how other 
distros approach this problem.

Re: Possible "stage" token for SCALLYWAG builds

[Ken Brown]

On 2/16/2023 3:31 PM, Jon Turney wrote:
> On 15/02/2023 21:48, Ken Brown via Cygwin-apps wrote:
>> Jon,
>>
>> When building TeX Live (once a year), I have to build texlive, 
>> asymptote, and all the texlive-collection-* packages.  I currently 
>> can't use SCALLYWAG to do the builds because these packages need to 
>> all be deployed at once.  So I have to build them all locally, upload 
>> them to my staging area, and then upload !ready files, one for x86_64 
>> and one for noarch.
>>
>> It would be convenient for me if SCALLYWAG would accept a "stage" 
>> token that would upload the files to my staging area without deploying 
>> them. Then I could let SCALLYWAG do the builds, and I could upload the 
>> !ready files when everything is staged.
>>
>> If I'm the only maintainer who would find this useful, then I can 
>> continue doing it the way I always have.  But maybe others would find 
>> it useful too.
> 
> I'm not opposed to adding a some feature to support this if needed, but 
> from the way you are describing it, this sounds more like a constraint 
> that the dependency solver should be aware of.
> 
> (just because texlive-2023 and texlive-collection-foo-2023 are made 
> available to download at the same time, doesn't mean that they always 
> end up installed together, as the user might alter the version of one or 
> the other)
> 
> If they really must be kept in lockstep to work correctly, then there's 
> at least a couple of ways of doing that:
> 
> * give texlive an additional provide, such as texlive_2023, and make 
> everything that requires it, require that (something similar is done 
> with perl and perl modules)
> 
> * make things which require texlive do so with a version constraint like 
> 'requires: texlive (>= 20230000), texlive (<20240000)' (in theory this 
> works, but I have no doubt that a bug will emerge when someone tries to 
> use it, and ofc, it relies on the range of future versions which are 
> compatible being correctly known in advance)

Thanks for the suggestions.  I think I should be able to make things 
work with one or both of your ideas.

Ken

Re: Possible "stage" token for SCALLYWAG builds

[Brian Inglis]

On 2023-02-16 17:13, Ken Brown via Cygwin-apps wrote:
> On 2/16/2023 3:31 PM, Jon Turney wrote:
>> On 15/02/2023 21:48, Ken Brown via Cygwin-apps wrote:
>>> When building TeX Live (once a year), I have to build texlive, asymptote, and 
>>> all the texlive-collection-* packages.  I currently can't use SCALLYWAG to do 
>>> the builds because these packages need to all be deployed at once.  So I have 
>>> to build them all locally, upload them to my staging area, and then upload 
>>> !ready files, one for x86_64 and one for noarch.
>>>
>>> It would be convenient for me if SCALLYWAG would accept a "stage" token that 
>>> would upload the files to my staging area without deploying them. Then I 
>>> could let SCALLYWAG do the builds, and I could upload the !ready files when 
>>> everything is staged.
>>>
>>> If I'm the only maintainer who would find this useful, then I can continue 
>>> doing it the way I always have.  But maybe others would find it useful too.
>>
>> I'm not opposed to adding a some feature to support this if needed, but from 
>> the way you are describing it, this sounds more like a constraint that the 
>> dependency solver should be aware of.
>>
>> (just because texlive-2023 and texlive-collection-foo-2023 are made available 
>> to download at the same time, doesn't mean that they always end up installed 
>> together, as the user might alter the version of one or the other)
>>
>> If they really must be kept in lockstep to work correctly, then there's at 
>> least a couple of ways of doing that:
>>
>> * give texlive an additional provide, such as texlive_2023, and make 
>> everything that requires it, require that (something similar is done with perl 
>> and perl modules)
>>
>> * make things which require texlive do so with a version constraint like 
>> 'requires: texlive (>= 20230000), texlive (<20240000)' (in theory this works, 
>> but I have no doubt that a bug will emerge when someone tries to use it, and 
>> ofc, it relies on the range of future versions which are compatible being 
>> correctly known in advance)
> 
> Thanks for the suggestions.  I think I should be able to make things work with 
> one or both of your ideas.

Would make sense if scallywag could stage as well as deploy, perhaps using a 
cygport all-stage command similar to all-test, perhaps when time permits?

Could also deploy in test SCALLYWAG="deploy", then untest each when you want to 
deploy.

-- 
Take care. Thanks, Brian Inglis			Calgary, Alberta, Canada

La perfection est atteinte			Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter	not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer	but when there is no more to cut
			-- Antoine de Saint-Exupéry

Re: Possible "stage" token for SCALLYWAG builds

[Ken Brown]

On 2/16/2023 7:13 PM, Ken Brown via Cygwin-apps wrote:
> On 2/16/2023 3:31 PM, Jon Turney wrote:
>> On 15/02/2023 21:48, Ken Brown via Cygwin-apps wrote:
>>> Jon,
>>>
>>> When building TeX Live (once a year), I have to build texlive, 
>>> asymptote, and all the texlive-collection-* packages.  I currently 
>>> can't use SCALLYWAG to do the builds because these packages need to 
>>> all be deployed at once.  So I have to build them all locally, upload 
>>> them to my staging area, and then upload !ready files, one for x86_64 
>>> and one for noarch.
>>>
>>> It would be convenient for me if SCALLYWAG would accept a "stage" 
>>> token that would upload the files to my staging area without 
>>> deploying them. Then I could let SCALLYWAG do the builds, and I could 
>>> upload the !ready files when everything is staged.
>>>
>>> If I'm the only maintainer who would find this useful, then I can 
>>> continue doing it the way I always have.  But maybe others would find 
>>> it useful too.
>>
>> I'm not opposed to adding a some feature to support this if needed, 
>> but from the way you are describing it, this sounds more like a 
>> constraint that the dependency solver should be aware of.
>>
>> (just because texlive-2023 and texlive-collection-foo-2023 are made 
>> available to download at the same time, doesn't mean that they always 
>> end up installed together, as the user might alter the version of one 
>> or the other)
>>
>> If they really must be kept in lockstep to work correctly, then 
>> there's at least a couple of ways of doing that:
>>
>> * give texlive an additional provide, such as texlive_2023, and make 
>> everything that requires it, require that (something similar is done 
>> with perl and perl modules)
>>
>> * make things which require texlive do so with a version constraint 
>> like 'requires: texlive (>= 20230000), texlive (<20240000)' (in theory 
>> this works, but I have no doubt that a bug will emerge when someone 
>> tries to use it, and ofc, it relies on the range of future versions 
>> which are compatible being correctly known in advance)
> 
> Thanks for the suggestions.  I think I should be able to make things 
> work with one or both of your ideas.

Just to finish this off for the sake of the archives, here's the 
solution I've decided on.  First, the only two packages that absolutely 
have to be in lockstep are texlive and texlive-collection-basic.  I 
achieve this as follows:

  - Make texlive provide tl_2023 and require tl_basic_2023.
  - Make texlive-collection-basic provide tl_basic_2023 and require tl_2023.

In addition, I prefer (but don't require) that the other 
texlive-collection-* packages stay in lockstep with texlive and 
texlive-collection-basic.  To this end,

  - Make the other texlive-collection-* packages require tl_basic_2023.

As long as I deploy texlive after all the collections, I get what I 
want.  Alternatively, I could deploy texlive-collection-basic after all 
the other collections.  If I forget to do one of those things, it won't 
be a disaster.

I've done some local testing, and this appears to work.

Ken