Cygwin command line download issue

Cygwin command line download issue

[Jim Rather]

I use a script to download cygwin so I can install the utilities on disconnected systems. I have an issue. The package crypto-policies does not download even though I have chosen the Base category. I noticed when I manually download this package that it shows up in the noarch directory so I am not sure if this is why? I do choose --arch x86_64 in my script. I am assuming that I do not enter noarch there since I do get a lot of noarch packages just not sure why crypto-policies is not one of them. I noticed this issue when trying to update openssh to version 8 and the message I was receiving was that crypto-policies was missing. I can share my script, just not sure how much information you want me to post. Please let me know what else you need.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin command line download issue

[Ken Brown]

On 10/21/2019 1:12 PM, Jim Rather wrote:
> I use a script to download cygwin so I can install the utilities on disconnected systems. I have an issue. The package crypto-policies does not download even though I have chosen the Base category. I noticed when I manually download this package that it shows up in the noarch directory so I am not sure if this is why? I do choose --arch x86_64 in my script. I am assuming that I do not enter noarch there since I do get a lot of noarch packages just not sure why crypto-policies is not one of them. I noticed this issue when trying to update openssh to version 8 and the message I was receiving was that crypto-policies was missing. I can share my script, just not sure how much information you want me to post. Please let me know what else you need.

I think you need to say exactly what you did, with all details.  Everything in 
the Base category should be downloaded/installed by default.

You might also look at /var/log/setup.log and /var/log/setup.log.full for clues 
as to what went wrong.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin command line download issue

[Jim Rather]

I only get two lines in setup.log.full with regards to crypto-policies

Found category Base in package crypto-policies
Added manual package crypto-policies

I am confused, this script has worked faithfully for at least a year. I just emptied by %cyg_dir% and re-ran it. Afterwards, I am missing many basic packages like alternatives, bash, etc. so I am not sure what has changed and welcome any suggestions.

Here is my .bat script, I use the powershell command to grab the latest setup-x86_64.exe

@echo off
setlocal

set url=http://cygwin.com/setup-x86_64.exe
set cyg_dir=t:\cygwin64
set cyg_setup=%cyg_dir%\setup-x86_64.exe

powershell -command "invoke-webrequest -uri %url% -outfile %cyg_setup%"

%cyg_setup% ^
 --arch x86_64 ^
 --categories Admin,Archive,Base,Database,Editors,GNOME,Graphics,Interpreters,KDE,Libs,^
Lua,LXDE,Mail,MATE,Math,Net,Ocaml,Office,Perl,PHP,Publishing,Python,Ruby,Scheme,Science,^
Security,Shells,Sugar,System,Tcl,Text,Utils,Web,X11,Xfce ^
 --delete-orphans ^
 --disable-buggy-antivirus ^
 --download ^
 --force-current ^
 --local-package-dir %cyg_dir% ^
 --no-admin ^
 --no-desktop ^
 --no-shortcuts ^
 --no-startmenu ^
 --no-version-check ^
 --only-site ^
 --prune-install ^
 --quiet-mode ^
 --remove-categories Accessibility,Audio,Debug,Devel,Doc,Games,Video ^
 --site http://mirrors.xmission.com/cygwin/ ^
 --verbose

:end


________________________________
From: cygwin-owner@cygwin.com <cygwin-owner@cygwin.com> on behalf of Ken Brown <kbrown@cornell.edu>
Sent: Tuesday, October 22, 2019 2:52 PM
To: cygwin@cygwin.com <cygwin@cygwin.com>
Subject: Re: Cygwin command line download issue

On 10/21/2019 1:12 PM, Jim Rather wrote:
> I use a script to download cygwin so I can install the utilities on disconnected systems. I have an issue. The package crypto-policies does not download even though I have chosen the Base category. I noticed when I manually download this package that it shows up in the noarch directory so I am not sure if this is why? I do choose --arch x86_64 in my script. I am assuming that I do not enter noarch there since I do get a lot of noarch packages just not sure why crypto-policies is not one of them. I noticed this issue when trying to update openssh to version 8 and the message I was receiving was that crypto-policies was missing. I can share my script, just not sure how much information you want me to post. Please let me know what else you need.

I think you need to say exactly what you did, with all details.  Everything in
the Base category should be downloaded/installed by default.

You might also look at /var/log/setup.log and /var/log/setup.log.full for clues
as to what went wrong.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin command line download issue

[Ken Brown]

[Please don't top-post on this list.]

On 10/22/2019 3:42 PM, Jim Rather wrote:
> I only get two lines in setup.log.full with regards to crypto-policies
> 
> Found category Base in package crypto-policies
> Added manual package crypto-policies

setup.log.full contains information only about the most recent run of setup. 
That's why I suggested looking in both setup.log and setup.log.full.

> I am confused, this script has worked faithfully for at least a year. I just emptied by %cyg_dir% and re-ran it. Afterwards, I am missing many basic packages like alternatives, bash, etc. so I am not sure what has changed and welcome any suggestions.

Once again, setup.log and setup.log.full should provide some clues.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin command line download issue

[Ken Brown]

On 10/22/2019 8:33 PM, Ken Brown wrote:
> [Please don't top-post on this list.]
> 
> On 10/22/2019 3:42 PM, Jim Rather wrote:
>> I only get two lines in setup.log.full with regards to crypto-policies
>>
>> Found category Base in package crypto-policies
>> Added manual package crypto-policies
> 
> setup.log.full contains information only about the most recent run of setup.
> That's why I suggested looking in both setup.log and setup.log.full.
> 
>> I am confused, this script has worked faithfully for at least a year. I just emptied by %cyg_dir% and re-ran it. Afterwards, I am missing many basic packages like alternatives, bash, etc. so I am not sure what has changed and welcome any suggestions.
> 
> Once again, setup.log and setup.log.full should provide some clues.

One other thing: I notice that your setup command line doesn't set the 
installation root directory.  By default, setup will use whatever root it used 
on its last run.  So if you previously used setup to install Cygwin, setup will 
look at /etc/setup/installed.db to see what you have installed.  It won't 
re-download the installed packages.

If you want to force download, I think you need to use --root to specify a 
directory in which Cygwin is not already installed.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin command line download issue

[Jim Rather]

My script is not trying to "install" cygwin. I am trying to download packages so that I can install them on systems without internet connectivity. I added --root in my script but that didn't really help. I got a new etc directory but not much in it, just timestamp and setup.rc . I removed the --prune option in case that was affecting something but it still did not work as I was expecting.

I created a new directory, c:\Users\jrather\Documents\cygwin64. Edited my .bat file to only get Base and that seemed to work. My feeling is that I might be running into some sort of limitation on the length of my update command. So I broke the categories list out into a for loop.

This would be better if  there was a 'test' option in setup-x86_64.exe? Downloading 8.5 GB of files just to test this also seems to be overkill. In any case here is my new updated script which appears to work even though I am unsure of how to solve the initial problem.

@echo off
setlocal

set url=http://cygwin.com/setup-x86_64.exe
set cyg_dir=C:\Users\jrather\Documents\cygwin64
set cyg_setup=%cyg_dir%\setup-x86_64.exe

powershell -command "invoke-webrequest -uri %url% -outfile %cyg_setup%"

for %%i in (Admin,Archive,Base,Database,Editors,^
GNOME,Graphics,Interpreters,KDE,Libs,Lua,LXDE,^
Mail,MATE,Math,Net,Ocaml,Office,Perl,PHP,Publishing,^
Python,Ruby,Scheme,Science,Security,Shells,Sugar,^
System,Tcl,Text,Utils,Web,X11,Xfce) do (call :cyg "%%i")
goto :eof

:cyg
%cyg_setup% ^
 --arch x86_64 ^
 --categories %1 ^
 --delete-orphans ^
 --disable-buggy-antivirus ^
 --download ^
 --force-current ^
 --local-package-dir %cyg_dir% ^
 --no-admin ^
 --no-desktop ^
 --no-shortcuts ^
 --no-startmenu ^
 --no-version-check ^
 --only-site ^
 --prune-install ^
 --quiet-mode ^
 --remove-categories Accessibility,Audio,Debug,Devel,Doc,Games,Video ^
 --root %cyg_dir% ^
 --site http://mirrors.xmission.com/cygwin/ ^
 --verbose )

:eof
:end






________________________________
From: cygwin-owner@cygwin.com <cygwin-owner@cygwin.com> on behalf of Ken Brown <kbrown@cornell.edu>
Sent: Wednesday, October 23, 2019 8:34 AM
To: cygwin@cygwin.com <cygwin@cygwin.com>
Subject: Re: Cygwin command line download issue

On 10/22/2019 8:33 PM, Ken Brown wrote:
> [Please don't top-post on this list.]
>
> On 10/22/2019 3:42 PM, Jim Rather wrote:
>> I only get two lines in setup.log.full with regards to crypto-policies
>>
>> Found category Base in package crypto-policies
>> Added manual package crypto-policies
>
> setup.log.full contains information only about the most recent run of setup.
> That's why I suggested looking in both setup.log and setup.log.full.
>
>> I am confused, this script has worked faithfully for at least a year. I just emptied by %cyg_dir% and re-ran it. Afterwards, I am missing many basic packages like alternatives, bash, etc. so I am not sure what has changed and welcome any suggestions.
>
> Once again, setup.log and setup.log.full should provide some clues.

One other thing: I notice that your setup command line doesn't set the
installation root directory.  By default, setup will use whatever root it used
on its last run.  So if you previously used setup to install Cygwin, setup will
look at /etc/setup/installed.db to see what you have installed.  It won't
re-download the installed packages.

If you want to force download, I think you need to use --root to specify a
directory in which Cygwin is not already installed.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin command line download issue

[Ken Brown]

On 10/23/2019 11:33 AM, Jim Rather wrote:
> My script is not trying to "install" cygwin. I am trying to download packages so 
> that I can install them on systems without internet connectivity.

You're misunderstanding the meaning of the "download" option.  It tells setup to 
download packages for later install.  It doesn't simply download all packages 
that you've specified on the command line.  In particular, it won't download 
packages that it thinks don't need to be installed.

I suggested using the --root option in order to trick setup into thinking that 
you're doing a new install.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin command line download issue

[Jim Rather]

If I am installing cygwin on a system without internet connectivity that means I need to "downloading packages for later install", since I cant download them without connectivity. Also, I do not define the packages to install but the categories, so I am not sure how the cygwin installer knows what packages it thinks I need installed since there are none downloaded in the first place. This script worked faithfully for over a year but now doesn't and I was not sure why. I agree the --root command might help trick it into thinking that it is a new install, especially when running my script from a system wtih cygwin installed so adding that might help, but at first glance it didn't seem to change anything since there was nothing of note populated there.

I am pretty sure my issue is that there is some sort of conflict or strange condition when I define --catagories as

Admin,Archive,Base,Database,Editors,GNOME,Graphics,Interpreters,KDE,Libs,Lua,LXDE,Mail,MATE,Math,Net,Ocaml,Office,Perl,PHP,Publishing,Python,Ruby,Scheme,Science,Security,Shells,Sugar,System,Tcl,Text,Utils,Web,X11,Xfce as opposed to choosing a category individually. I do not really have the time to go through all the permutations of those categories to figure out the problem, so I am fine with my for loop. My problem is probably some crazy, one-off edge case but I am not sure how to diagnose it.

If I am doing this incorrectly, I welcome any suggestion on how to download cygwin and install it for offline systems. I really do not want to rsync the whole cygwin repository so my script works for me even if the for loop adds additional time cycling through each category. I have resolved my issue for now.

________________________________
From: cygwin-owner@cygwin.com <cygwin-owner@cygwin.com> on behalf of Ken Brown <kbrown@cornell.edu>
Sent: Wednesday, October 23, 2019 12:48 PM
To: cygwin@cygwin.com <cygwin@cygwin.com>
Subject: Re: Cygwin command line download issue

On 10/23/2019 11:33 AM, Jim Rather wrote:
> My script is not trying to "install" cygwin. I am trying to download packages so
> that I can install them on systems without internet connectivity.

You're misunderstanding the meaning of the "download" option.  It tells setup to
download packages for later install.  It doesn't simply download all packages
that you've specified on the command line.  In particular, it won't download
packages that it thinks don't need to be installed.

I suggested using the --root option in order to trick setup into thinking that
you're doing a new install.

Ken

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin command line download issue

[Achim Gratz]

Jim Rather writes:
[…]

>  --delete-orphans ^

I don't think this does anything in download-only mode.  A mirror by
definition doesn't have any orphan packages as far as setup is
concerned.

>  --disable-buggy-antivirus ^
>  --no-desktop ^
>  --no-shortcuts ^
>  --no-startmenu ^

These are not needed.

>  --prune-install ^

This doesn't do what you apparently think it does.  It's effective on
installation only and will remove all existing packages that were not
explicitly or transitively selected for installation.

>  --remove-categories Accessibility,Audio,Debug,Devel,Doc,Games,Video ^

Ditto.  This option is for removing packages from an installation,
selected by category.  If you also select them to be installed at the
same time (via package name or the same or another category), then the
package(s) in question will be re-installed (i.e. removed first, then
installed again).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple