RE: setup 2.194.2.24: Bug (?) in downloading from internet

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

> -----Original Message-----
> From: Sam Edge [mailto:sam_edgeZZZ@hotmail.com] 
> Sent: Sunday, April 28, 2002 8:17 PM

> Every version of setup.exe I've used exhibits this annoying 
> behaviour but them I'm new here.
> 
> I agree with Daniele. I install different subsets of the 
> packages on several machines. I also like to keep a copy of 
> the most up-to-date versions of packages that I don't 
> currently use on any of them, in case I need them where I've 
> no (or poor) access to the 'Net. 

So use a mirroring tool! Setup.exe is -not- designed for this. If you
want a setup.ini parsing mirroring tool, take me up on my offer to help
someone leverage the setup.ini codebase to build a mirroring tool. 

Rob

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Sam Edge]

Robert Collins wrote in
<FC169E059D1A0442A04C40F86D9BA7600C5F42@itdomain003.itdomain.net.au>
in gmane.os.cygwin on Sun, 28 Apr 2002 20:22:30 +1000:

> > I agree with Daniele. I install different subsets of the 
> > packages on several machines. I also like to keep a copy of 
> > the most up-to-date versions of packages that I don't 
> > currently use on any of them, in case I need them where I've 
> > no (or poor) access to the 'Net. 
> 
> So use a mirroring tool! Setup.exe is -not- designed for this. 

Surely if setup.exe isn't designed with at least a nod to this way of
working why does it have "download from Internet" and "install from
local directory" options, instead of just instisting on connecting to
a server every time?

> If you
> want a setup.ini parsing mirroring tool, take me up on my offer to help
> someone leverage the setup.ini codebase to build a mirroring tool. 

Although I've not looked at it, setup.exe must already have most of
the code required to operate in the way we'd like, surely?

If you tell me the module name(s) for setup.exe and any documentation
about setup.ini and the local and FTP/HTTP server directory structures
I'd certainly be interested to download them from the CVS server and
take a look.

-- 
Sam Edge

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Gary R. Van Sickle]

> > -----Original Message-----
> > From: Sam Edge [mailto:sam_edgeZZZ@hotmail.com]
> > Sent: Sunday, April 28, 2002 8:17 PM
>
> > Every version of setup.exe I've used exhibits this annoying
> > behaviour but them I'm new here.
> >
> > I agree with Daniele. I install different subsets of the
> > packages on several machines. I also like to keep a copy of
> > the most up-to-date versions of packages that I don't
> > currently use on any of them, in case I need them where I've
> > no (or poor) access to the 'Net.
>
> So use a mirroring tool! Setup.exe is -not- designed for this. If you
> want a setup.ini parsing mirroring tool, take me up on my offer to help
> someone leverage the setup.ini codebase to build a mirroring tool.
>
> Rob
>

What am I missing in all these posts?  Didn't somebody post a while ago that if
you set your "Local Package Directory" to a network share and do "Install From
Internet", you get all the functionality people are wanting here?  Now I admit
I've not tried it myself (though I have plenty of reason to do so and should,
maybe next week), but what more do people want here?

Maybe we should dump the "Download" option entirely.  Seems that all it serves
to do is confuse people that should really be looking at rsync.

--
Gary R. Van Sickle
Brewer.  Patriot.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Sam Edge]

Gary R. Van Sickle wrote in
<NCBBIHCHBLCMLBLOBONKKEOLCNAA.g.r.vansickle@worldnet.att.net>
in gmane.os.cygwin on Sun, 28 Apr 2002 14:12:36 -0500:

> What am I missing in all these posts?  Didn't somebody post a while ago that if
> you set your "Local Package Directory" to a network share and do "Install From
> Internet", you get all the functionality people are wanting here?

No. You can use the same local package directory to install on
multiple machines very nicely. That aspect works fine.

The problem is that when you do "download from Internet" the list
tells you what you have installed in c:\cygwin on the machine where
you're running setup.exe not what you've previously downloaded to the
package directory. Add to this the lack of an option to skip
previously downloaded packages when they're manually selected (or
auto-selected by the dependency management) and it's a problem if
you're using a single local package directory for several different
machines that have different package sets installed.

On a single machine where you download and immediately install only
what'll be used locally it can still be a problem for those on slow,
unreliable or time-limited connections.

For example, many people in the UK use a 56kbps (or more usually
around 45kbps) connection. Many unmetered ISP packages disconnect
every two hours to prevent users hogging dial-in capacity they're not
using.

Downloading even a basic package with all its dependencies can run
over this time limit so you can't use the "install from Internet"
option.

But when you try to use "download from Internet" and then re-start
setup.exe after re-connecting it doesn't show you the packages you
already have. Worse, when you select your package again it
auto-selects all the dependencies and tries to download them all again
and so will never complete because you'll keep hitting the disconnect
timeout.

What you have to do is have an Explorer window open looking at the
package directory and manually go back and de-select all the ones
you've already got after auto-select has added them. This is prone to
error, especially if you've not used clean_setup.pl to flatten the
structure between each download session. It's also annoying because it
should be easy to automate into setup.exe itself.

> Maybe we should dump the "Download" option entirely.  Seems that all it serves
> to do is confuse people that should really be looking at rsync.

Please, no!

Apart from the reason I give above, on my main machine here I always
download first in my normal NT user account so I can continue to work
while it's going on. I then log in as ADMINISTRATOR to install because
I have the Cygwin directory hierarchy protected against write by
normal users. That way I'm running with potentially dangerous
privileges for as short a time as possible.

-- 
Sam Edge

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Charles Wilson]

<begin rant>
Robert, you are a saint.

I am sick and tired of the whining about setup and downloads.

SETUP IS NOT A MIRRORING TOOL.

In order to make this clear, I am in favor of completely removing the 
download-only option, leaving just install-from-internet and 
install-from-localdir.  This will force the whiners to either shut up 
and use a fscking mirroring tool, or help out:

Robert has already said what the correct solution to all this is: create 
a separate program that leverages the setup.exe codebase, to do 
"multi-source merged mirroring".

This will require:

1) Add command-line options for setup.exe.  Robert has added a helpful 
set of examples.  He even created a whole new project, GetOpt++, to 
support this.  What more can Robert, all by himself, do?

2) Create a new cygmirror.cc as part of the setup module, that uses some 
(but not all) of the other classes in the setup module.  According to 
Robert:

 > The differences between setup and a mirroring tool include:
 > * All the install code isn't needed.
 > * Attention to local packages isn't needed.
 > * Persistent selection of packages irrespective of install status
 >   should be present.
 > * command line driven may be useful?

I'll add one more: reading options (or package lists) from a file.

Now, unless somebody who WANTS this functionality actually steps up to 
the plate and digs in, can we PLEASE stop discussing this issue?

Removing the download-only option will also have the salutory effect of 
forcing the whiners to either:

1) use a REAL mirroring tool, then use setup.exe to
    install-from-local-dir
   1a) heck, you could even use wget; I've posted a script
       to this mailing list that does exactly that.
2) help out with the coding.

Either way, it kills "setup download only mode doesn't do what I want" 
"What's up with the wacky %%%'ified directory names in my local setup 
directory" "Setup sucks" threads that NEVER DIE!.

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Sam Edge]

Charles Wilson wrote in <3CCC7290.706@ece.gatech.edu>
in gmane.os.cygwin on Sun, 28 Apr 2002 18:07:12 -0400:

> <begin rant>
> Robert, you are a saint.
> I am sick and tired of the whining about setup and downloads.
> SETUP IS NOT A MIRRORING TOOL.

LOL. No need to get your knickers in a twist! I was merely explaining
the limitations of setup.exe to Gary. ;-)

The problem with time limited slow connections is valid even when
using it purely for local setup on a single machine i.e. when NOT
using it as a mirroring tool. 

A simple checkbox option on the download-only page to skip
pre-downloaded packages even if they're selected in the list would
deal with that and I really don't see how this would compromise its
function in any way, as long as it were off by default.

Unless someone can give a good reason why it /would/ be a bad idea,
I'll look at the code and see if I can write a version that has this.

> In order to make this clear, I am in favor of completely removing the 
> download-only option, leaving just install-from-internet and 
> install-from-localdir.  This will force the whiners to either shut up 
> and use a fscking mirroring tool, or help out:

It will also stop users on slow time-limited connections from being
able to use setup.exe at all unless they first use a conventional
FTP/HTTP mirroring client and set up their own local FTP server to
which to connect setup.exe. They then have to work out which parts of
one of the current mirrors are required for their particular needs and
download those packages. They'll also end up with two copies of
everything on their hard discs.

This is hardly a good way of increasing the Cygwin user/developer
base!

-- 
Sam Edge

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Christopher Faylor]

On Sun, Apr 28, 2002 at 06:07:12PM -0400, Charles Wilson wrote:
><begin rant>
>Robert, you are a saint.
>
>I am sick and tired of the whining about setup and downloads.
>
>SETUP IS NOT A MIRRORING TOOL.
>
>In order to make this clear, I am in favor of completely removing the 
>download-only option, leaving just install-from-internet and 
>install-from-localdir.  This will force the whiners to either shut up 
>and use a fscking mirroring tool, or help out:

I am really torn on this one.  I fully agree that setup.exe is not a
mirroring tool.  I've vigorously made that assertion on many occasions.

However, I think I may have been responsible for setup's current
behavior of basing what is downloaded on what is installed.  Even if
I was not, though, I think that the explanations about why some people
need this are rather convincing.

If you're on the other end of a slow connection and are apt to be
disconnected every couple of hours, you will *never* be able to do a
complete install since the setup.exe will start from scratch every time
you reconnect.  It doesn't matter if you chose "download from internet" or
not since setup.exe will only consider what's installed when considering
what to download.  That just seems like a bug to me.

If I have this right then even removing download from internet does not
eliminate this problem.

So, I think the right answer to this is the standard one "patches
gratefully accepted".

>Robert has already said what the correct solution to all this is: create 
>a separate program that leverages the setup.exe codebase, to do 
>"multi-source merged mirroring".

I do agree with this philosophy for anything beyond fixing the above, IMO,
glitch.  setup.exe should be a fairly simple tool.  If you find that it
isn't meeting your needs then you should be looking for a better tool.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Sam Edge]

Christopher Faylor wrote in <20020428223000.GB13849@redhat.com>
in gmane.os.cygwin on Sun, 28 Apr 2002 18:30:00 -0400:

> If you're on the other end of a slow connection and are apt to be
> disconnected every couple of hours, you will *never* be able to do a
> complete install since the setup.exe will start from scratch every time
> you reconnect.  It doesn't matter if you chose "download from internet" or
> not since setup.exe will only consider what's installed when considering
> what to download.  That just seems like a bug to me.

Thanks. Even something like the XFree86 "basic" install can take a
long time, let alone trying to install everything in one hit.
Sometimes people forget that there are many still out there on creaky
old dial-up connections.

> If I have this right then even removing download from internet does not
> eliminate this problem.

No, it makes it worse.

> So, I think the right answer to this is the standard one "patches
> gratefully accepted".

I'll look but I'm making no promises! ;-D

-- 
Sam Edge

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Michael A Chase]

On Sun, 28 Apr 2002 18:30:00 -0400 Christopher Faylor <cgf@redhat.com> wrote:

> On Sun, Apr 28, 2002 at 06:07:12PM -0400, Charles Wilson wrote:

> If you're on the other end of a slow connection and are apt to be
> disconnected every couple of hours, you will *never* be able to do a
> complete install since the setup.exe will start from scratch every time
> you reconnect.  It doesn't matter if you chose "download from internet"
> or
> not since setup.exe will only consider what's installed when considering
> what to download.  That just seems like a bug to me.

That's what the discussion has been about for the last week or so.  After
lots of requests for this functionality, the position eventually hardened
to "it's not a download tool".

> If I have this right then even removing download from internet does not
> eliminate this problem.

That depends on how capable wget or rsync is.  I have not looked into it
but if we do remove the download only function from setup.exe, we (me?)
should probably add something under cygwin.com describing how to mirror
selected packages.

> So, I think the right answer to this is the standard one "patches
> gratefully accepted".

If download handling is going to change, at least Robert will have to buy
into it.  A request for patches can follow that.

-- 
Mac :})
** I normally forward private questions to the appropriate mail list. **
Ask Smarter: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Give a hobbit a fish and he eats fish for a day.
Give a hobbit a ring and he eats fish for an age.



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Christopher Faylor]

On Sun, Apr 28, 2002 at 04:04:36PM -0700, Michael A Chase wrote:
>On Sun, 28 Apr 2002 18:30:00 -0400 Christopher Faylor <cgf@redhat.com> wrote:
>
>> On Sun, Apr 28, 2002 at 06:07:12PM -0400, Charles Wilson wrote:
>
>>If you're on the other end of a slow connection and are apt to be
>>disconnected every couple of hours, you will *never* be able to do a
>>complete install since the setup.exe will start from scratch every time
>>you reconnect.  It doesn't matter if you chose "download from internet"
>>or not since setup.exe will only consider what's installed when
>>considering what to download.  That just seems like a bug to me.
>
>That's what the discussion has been about for the last week or so.
>After lots of requests for this functionality, the position eventually
>hardened to "it's not a download tool".

Right.  But it started with my suggesting this behavior was a bug and (I
thought) Robert agreeing.  So, I thought it was time to chime in again.

>> If I have this right then even removing download from internet does not
>> eliminate this problem.
>
>That depends on how capable wget or rsync is.  I have not looked into it
>but if we do remove the download only function from setup.exe, we (me?)
>should probably add something under cygwin.com describing how to mirror
>selected packages.

It's not a question of mirroring.  It's a question of setup.exe being
completely unusable for some people.  We can't say that setup.exe is the
only method we'll support but then tell people to use some other method
when it fails for them as they try to use it in a perfectly acceptable
way.

The bottom line is that if we have to tell someone to use wget or rsync
just to do a normal install, something is wrong.

It seems like this problem would be trivially fixed by not attempting to
redownload something that has already been downloaded.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Cliff Hones]

Christopher Faylor <cgf@redhat.com> wrote:
> The bottom line is that if we have to tell someone to use wget or rsync
> just to do a normal install, something is wrong.
> 
> It seems like this problem would be trivially fixed by not attempting to
> redownload something that has already been downloaded.

Hooray.  Seems lots of people have been trying to say this.  A week
ago when it was, for a moment, acknowledged as a bug, I downloaded
the source with ideas of trying to fix it.  But then the mood shifted
back to it being a feature, so I lost interest.

Now Sam Edge is taking the same path.  Good luck.

What would happen if someone did produce a version of setup identical
to the current, except for the removal of redownload?  Would it
be adopted?  One gets an unfortunate undercurrent from this list that
if it wasn't approved by the right people it might be suppressed
(and left to circulate on the black market - possibly the worst
scenario).

-- Cliff



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Max Bowsher]

Robert Collins <robert.collins@itdomain.com.au> wrote:
>> -----Original Message-----
>> From: Sam Edge [mailto:sam_edgeZZZ@hotmail.com]
>> Sent: Sunday, April 28, 2002 8:17 PM
>> Every version of setup.exe I've used exhibits this annoying
>> behaviour but them I'm new here.
>>
>> I agree with Daniele. I install different subsets of the
>> packages on several machines. I also like to keep a copy of
>> the most up-to-date versions of packages that I don't
>> currently use on any of them, in case I need them where I've
>> no (or poor) access to the 'Net.
>
> So use a mirroring tool! Setup.exe is -not- designed for this. If you
> want a setup.ini parsing mirroring tool, take me up on my offer to help
> someone leverage the setup.ini codebase to build a mirroring tool.
>
> Rob

But setup.exe has operated in the correct way in the past! Why write a new tool,
when fixing a bug in the current one would do all that needs to be done? NB: I'm
not saying 'go fix that now'. When (if? ;-) I find time to understand the setup
code, I will do this, if no one has beaten me to it.

Max.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

===
----- Original Message -----
From: "Max Bowsher" <maxb@ukf.net>
To: <cygwin@cygwin.com>

> But setup.exe has operated in the correct way in the past! Why write a
new tool,
> when fixing a bug in the current one would do all that needs to be
done?

Because the current tool is more complex than it should be, and a new
tool can be accurately target. The current tool can then be simplified
somewhat (i.e. download only could go away if we had a (non cygwin
linked) mirroring tool. Or a variation of this.

> NB: I'm
> not saying 'go fix that now'. When (if? ;-) I find time to understand
the setup
> code, I will do this, if no one has beaten me to it.

Thanks.

Rob


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

===
----- Original Message -----
From: "Cliff Hones" <cliff@aonix.co.uk>
To: <cygwin@cygwin.com>

>
> Now Sam Edge is taking the same path.  Good luck.
>
> What would happen if someone did produce a version of setup identical
> to the current, except for the removal of redownload?  Would it
> be adopted?

As it stands, I don't know. I've asked everyone to write in and say
whether they use the redownload capability.

> One gets an unfortunate undercurrent from this list that
> if it wasn't approved by the right people it might be suppressed
> (and left to circulate on the black market - possibly the worst
> scenario).

Heh, this is open source, there is no black market :}. Realistically,
anyone can use anything they want to install, but 'you' the users have a
realistic expectation that 'we' the developers will provide a tool, and
support that tool. Conversely we expect that anyone wanting assistance
with install will use our provided tool.

Rob


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Using a real mirroring tool...

[Charles Wilson]

Poor archive.progeny.com ...

First, rsync places a heavy load on the server, so please do not 
EVERYBODY run out and try this immediately.  I suggest, for initial 
rsyncs (when you need to download EVERYTHING), that folks follow this 
rotation:

Surname Begins With       Initial rysnc
ABC                          May 1
DEF                          May 2
GHI                          May 3
JKL                          May 4
MNO                          May 5
PQR                          May 6
STU                          May 7
VWX                          May 8
YZ                           May 9

Incremental rsync's don't place as much of a load on the server, so it 
won't matter as much after that...

Here is a script that will mirror the cygwin/release tree and 
setup.exe/setup.ini (you'll need to change LOCALDIR).  Also, you'll need 
to remove --dry-run from EXTRAOPTS.  If anybody can identify other 
mirrors that provide rsync: support, please post the appropriate MIRROR= 
line as a reply to this message.

---------------------
#!/bin/sh
MIRROR=rsync://archive.progeny.com/cygwin
LOCALDIR=/annex1/private/software/cygwin/
OPTS="-rlptgoDzu"
EXTRAOPTS="--delete -vv --dry-run"
RSYNC=/usr/bin/rsync

${RSYNC} ${OPTS} ${EXTRAOPTS} ${MIRROR}/release ${LOCALDIR}
${RSYNC} ${OPTS} ${EXTRAOPTS} ${MIRROR}/setup.* ${LOCALDIR}
---------------------

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Using a real mirroring tool...

[Charles Wilson]

Actually, this is a better set of options: we don't need to preserve 
owner, groups, or devices, and the modify-window helps with the limited 
timestamp granularity of the FAT filesystem.  Remember to fix LOCALDIR 
and remove --dry-run.

---------------------
#!/bin/sh
MIRROR=rsync://archive.progeny.com/cygwin
LOCALDIR=/annex1/private/software/cygwin/
OPTS="-rlptzu --modify-window=2"
EXTRAOPTS="--delete -vv --dry-run"
RSYNC=/usr/bin/rsync

${RSYNC} ${OPTS} ${EXTRAOPTS} ${MIRROR}/release ${LOCALDIR}
${RSYNC} ${OPTS} ${EXTRAOPTS} ${MIRROR}/setup.* ${LOCALDIR}
---------------------

--Chuck

Charles Wilson wrote:

> Poor archive.progeny.com ...
> 
> First, rsync places a heavy load on the server, so please do not 
> EVERYBODY run out and try this immediately.  I suggest, for initial 
> rsyncs (when you need to download EVERYTHING), that folks follow this 
> rotation:
> 
> Surname Begins With       Initial rysnc
> ABC                          May 1
> DEF                          May 2
> GHI                          May 3
> JKL                          May 4
> MNO                          May 5
> PQR                          May 6
> STU                          May 7
> VWX                          May 8
> YZ                           May 9
> 
> Incremental rsync's don't place as much of a load on the server, so it 
> won't matter as much after that...
> 
> Here is a script that will mirror the cygwin/release tree and 
> setup.exe/setup.ini (you'll need to change LOCALDIR).  Also, you'll need 
> to remove --dry-run from EXTRAOPTS.  If anybody can identify other 
> mirrors that provide rsync: support, please post the appropriate MIRROR= 
> line as a reply to this message.
> 
> ---------------------
> #!/bin/sh
> MIRROR=rsync://archive.progeny.com/cygwin
> LOCALDIR=/annex1/private/software/cygwin/
> OPTS="-rlptgoDzu"
> EXTRAOPTS="--delete -vv --dry-run"
> RSYNC=/usr/bin/rsync
> 
> ${RSYNC} ${OPTS} ${EXTRAOPTS} ${MIRROR}/release ${LOCALDIR}
> ${RSYNC} ${OPTS} ${EXTRAOPTS} ${MIRROR}/setup.* ${LOCALDIR}
> ---------------------
> 
> --Chuck
> 



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Using a real mirroring tool...

[Charles Wilson]

A few notes:

Using rsync does not help the following problem:  I have four favorite 
mirrors, plus a "private" setup-compatible website from my friend Bob. 
setup.exe will show me a merged view of the contents of all of those 
sites, insuring that I get the latest version of a package wherever it 
is found.  By merging in the "private" site, setup enables me to get my 
friend Bob's latest version of "King of Frobozz, Tyrant" game using the 
same setup tool/view, along with the latest apache package or binutils 
package from the "official" cygwin mirrors.

rsync can't merge.

I could rsync each of the four mirrors down to my machine into separate 
directories (assuming the mirrors all had anonymous rsync servers), and 
rsync Bob's private website (assuming his webhost provided rsync -- 
hah!).  But even then, setup can't merge multiple local directories, at 
least not right now.

So, we still need some variant of setup.exe that can merge multiple 
setup-compatible sites, and "mirror" only the latest stuff into a single 
  local "release" tree / setup.ini.

Also, rsync does some stuff we don't really care about: since most of 
the files are compressed, rsync's sophisticated "download changed 
chunks" algorithm is pretty useless.  In compressed files, the first 
byte that is different in the uncompressed file will cause a cascade of 
changes throughout the remainder of the compressed file.  So: 1 byte 
diff in uncompressed file == entire compressed file is different.

OTOH, since we typically (never?) upload a modified version of a given 
file USING THE SAME NAME, we don't really care too much about this. 
"Our" mirroring tool can just use the md5 hashes on the mirror and 
locally to determine the rare instance when a given file has had its 
contents changed -- but then we'd download the ENTIRE file, not "changed 
chunks".  (However, if this ever happens, isn't it a sign that something 
went wrong somewhere?)

Also, there's really no need to use rsync's compression engine, since 
most of the data to be transferred has already been compressed.

Finally, the local "setup.ini" will never(?) match the setup.ini at any 
one of the mirrors or private sites; it will be a merger of them.  So, 
rsync can't do that.

So, if someone cares enough to do the work, there is still a need for 
the "additional tool" that uses the setup.exe codebase to do 
"cygwin-specific distributed mirroring".  See my earlier rant.

But, the rsync method should enable folks with slow connections to limp 
along for a while...without trying to turn setup.exe ITSELF into a 
mirror tool.

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Using a real mirroring tool...

[Charles Wilson]

Charles Wilson wrote:

> So, if someone cares enough to do the work, there is still a need for 
> the "additional tool" that uses the setup.exe codebase to do 
> "cygwin-specific distributed mirroring".  See my earlier rant.


Oh, I left out the most obvious reason why rsync won't solve all of our 
problems: we have lots of ftp and http mirrors, but very few (ONE?) 
rsync mirrors.  That really kills the "distributed" nature of the mirror 
system; that poor guy in Thailand with the slow connection would be 
rsync'ing across the Pacific from Indiana...that can't be good.

A tool that leverages the setup.exe codebase would inherit the ability 
to use the existing ftp and http mirror sites.

--Chuck
</stating the obvious>

P.S. you might want to add --exclude *-src.tar.* to the option string, 
if you don't want the -src tarballs.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Using a real mirroring tool...

[Sam Edge]

Charles Wilson wrote in <3CCDA73E.5000503@ece.gatech.edu>
in gmane.os.cygwin on Mon, 29 Apr 2002 16:04:14 -0400:

> P.S. you might want to add --exclude *-src.tar.* to the option string, 
> if you don't want the -src tarballs.

As I understand it you can also exclude the setup.hint and md5.sum
files if you're just populating a local package directory. They're
very small but every little helps and it reduces the clutter and can
save a few sectors on the local disc.

Or am I missing something? ("Again" do I hear from the chorus? Kindly
leave the auditorium, sir!) ;-)

-- 
Sam Edge

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Using a real mirroring tool...

[Volker Quetschke]

[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

Hi!

Charles Wilson wrote:

> A few notes:
> 
> ...
> 
> rsync can't merge.


Yes, but Michael A. Chases clean_setup.pl can! I allowed myself to add a 
new option to clean_setup.pl, it exports the list of missing files to a 
file. Then you can use wget to get all the missing files.

I attached the modified clean_setup.pl plus a shell scrip to get the 
missing files from a mirror. You have to modify the getmissing.sh to 
choose your mirror and your target download directory. The directory and 
both files must exist the same directory as your setup.exe. Just a quick 
hack, I guarantee nothing!

Bye
    Volker

[-- Attachment #2: getmissing.sh --]
[-- Type: text/plain, Size: 757 bytes --]

#!/bin/bash

# Mirror script for cygwin
# Uses Michael A. Chase "cleanup_setup.pl"
# in a modified version (V1.0303)
# and wget

# getmissing.sh and cleanup_setup.pl must reside in the same directory
# as setup.exe

# Files to download
filelist="missing.lst"
# Choose your mirror with directory of setup.ini
mirror="ftp://ftp.inf.tu-dresden.de/software/windows/cygwin32/"
# Cut directories                   1        2       3
cutdir=3
# Name your download directory
mirrordir="_mymirror"

wget --mirror --no-host-directories --cut-dirs=$cutdir --passive-ftp -P $mirrordir ${mirror}setup.ini
./clean_setup.pl -Arch -writelist
wget --mirror --no-host-directories --cut-dirs=$cutdir --passive-ftp -P $mirrordir -i $filelist -B $mirror
./clean_setup.pl -Setup

[-- Attachment #3: clean_setup.pl --]
[-- Type: text/plain, Size: 13836 bytes --]

#! /usr/bin/perl -w

require 5.005;

# Delete old files from Cygwin setup.exe archive subdirectories

# Name of file:  cleanup_setup.pl
#
# Type of file:  Perl script
#
# Author:        Michael A. Chase
#
# Purpose:       Clean out setup.exe archive subdirectories.

# Syntax:        See $HelpText Below
#--------------------------  MODIFICATION HISTORY -----------------------------
$VERSION = '1.0303';
# 020430 V. Quetschke   Add -writelist
#                       Writes list of missing files to missing.lst.
# 020423 M. Chase       Add -source, -install, -H options.
#                       Removed directory names from Missing Files list.
# $VERSION = '1.0301';
# 020417 M. Chase       Stop lc()ing file names during setup.ini parsing.
# $VERSION = '1.0300';
# 020416 M. Chase       Ignore setup.ini while collecting archive file names.
#                       Remove directories in reverse length order.
# $VERSION = '1.0202';
# 020416 M. Chase       Properly ignore directories in obsolete file search.
# $VERSION = '1.0201';
# 020404 M. Chase       Ignore files found more than once in move loop.
#                       Protect against attempts to move files from different
#                          trees over each other in the base tree.
# $VERSION = '1.0200';
# 020326 M. Chase       Properly handle directory case.
#                       Optionally move archives to base directory tree.
# $VERSION = '1.0100';
# 020212 M. Chase       Handle multiple setup.ini files and file sources.
# $VERSION = '1.0000';
# 010422 M. Chase       First draft.

use FindBin qw( $RealBin $RealScript );
use File::Basename qw( &basename &dirname &fileparse );
use File::Find;
use File::Spec;
use File::Path qw( &mkpath );
use File::Copy qw( &copy &move );
use Getopt::Long;

use strict;
use integer;
use vars qw( $VERSION );

# Initialize options
my $bArch    = 0;
my $bMove    = 0;
my $bSetup   = 0;
my $bCurr    = 1;
my $bPrev    = 1;
my $bTest    = 1;
my $bExp     = 1;
my $bInstall = 1;
my $bSource  = 0;
my $bWrite   = 0;
my @sHide    = ();
my @sIgnore  = ();
my $sDir0    = File::Spec -> rel2abs( "." );

# Syntax description
sub usage {
   my ( $sOpt, $sVal, @sMsg ) = @_;

   my $sHelpText = <<END_HELP_TEXT;
Cleanup Cygwin setup.exe package cache directories
syntax: $RealScript [opt]
Opt: ($VERSION)
   -[no]Move    = [Don't] Move archive files to base directory tree ($bMove)
   -[no]Arch    = [Don't] Delete obsolete archives and directories ($bArch)
   -[no]Setup   = [Don't] Delete obsolete setup.ini files ($bSetup), forces -Move
   -[no]curr    = [Don't] Report missing [curr] files ($bCurr)
   -[no]prev    = [Don't] Report missing [prev] files ($bPrev)
   -[no]test    = [Don't] Report missing [test] files ($bTest)
   -[no]exp     = [Don't] Report missing [exp]  files ($bExp)
   -[no]install = [Don't] Report missing install archives ($bInstall)
   -[no]source  = [Don't] Report missing source archives ($bSource)
   -writelist   = Write missing files report (missing.lst) to disk ($bWrite)
   -base dir    = Archive cache base directory ($sDir0)
   -I mask      = Ignore files and directories that match mask, multiple allowed
   -H mask      = Ignore packages that match mask, multiple allowed
      mask is a filename wildcard mask, not a regular expression
Arg:
END_HELP_TEXT
# Balance quotes in here document # ' # "

   my $nRet = 'help' eq $sOpt ? 0 : 0 + $sVal;
   select STDERR if $nRet;
   foreach ( @sMsg, $sHelpText ) { s/\s+$//; print "$_\n"; }
   exit $nRet;
}

# Parse command line
Getopt::Long::config( qw( no_ignore_case no_auto_abbrev require_order ) );
GetOptions(
   'Arch!'       => \$bArch,
   'Move!'       => \$bMove,
   'Setup!'      => \$bSetup,
   'curr!'       => \$bCurr,
   'prev!'       => \$bPrev,
   'test!'       => \$bTest,
   'exp!'        => \$bExp,
   'install!'    => \$bInstall,
   'source!'     => \$bSource,
   'writelist!'  => \$bWrite,
   'base=s'      => \$sDir0,
   'Hide|H=s@'   => \@sHide,
   'Ignore|I=s@' => \@sIgnore,
   'help|h' => \&usage ) or usage( 'die', 1 );
$bMove ||= $bSetup; # Don't remove setup files unless also moving archves
chdir $sDir0 or usage( 'die', 1, "Can't change directory to $sDir0, $!\n" );

# Provide default values for unset options and parameters

# Report arguments
$sDir0 = File::Spec -> rel2abs( "." );
$sDir0 =~ s,\\,/,g;
my $sOpt = '';
$sOpt .= "\nMoving archives to base directory tree" if $bMove;
$sOpt .= "\nDeleting unused files and empty directores" if $bArch;
$sOpt .= "\nDeleting obsolete setup.ini files" if $bSetup;
$sOpt .= "\nIgnoring files and directories: " . join ' ',  map { "'$_'" }
   @sIgnore if @sIgnore;
$sOpt .= "\nIgnoring packages: "   . join ' ',  map { "'$_'" } @sHide if @sHide;
print <<HERE;
Base Directory: $sDir0$sOpt
HERE

# Build file or directory name matcher
#    Adapted from Recipe 6.10 in Perl Cookbook
sub rfMatch {
   my ( $bDefault, $op, @sMask ) = @_; # @sMask must be a lexical array
   return sub { return $bDefault; } if 3 > @_;
   my $sExpr = join " $op\n",
      map {
         # Convert file expansion mask to regular expression
         $sMask[$_] =~ s/\./\\./g;
         $sMask[$_] =~ s/\?/.?/g;
         $sMask[$_] =~ s/\*/.*/g;
         "m:^\$sMask[$_]\$:oi";
         } 0 .. $#sMask;
   my $rfMatch = eval "sub { local \$_ = shift;\nreturn $sExpr; };";
   die $@ if $@;
   return $rfMatch;
}
local *bHide   = rfMatch( 0, '||', @sHide );
local *bIgnore = rfMatch( 0, '||', @sIgnore );

# Find and parse setup.ini files, collect other filenames at the same time
my ( %aSetup, $sRel, $sName, %sTarBall, %bDir );
my $wanted = sub {
   # Skip ., .., and files and directories in ignore list
   if ( '.' eq $_ || '..' eq $_ ) {
      $File::Find::prune = 1 if '..' eq $_;
      return;
   }
   if ( bIgnore( $_ ) ) {
      $File::Find::prune = 1 if -d $_; # Prune if a directory
      return;
   }

   # Handle directory or file
   $sRel = sRel( $File::Find::dir, $sDir0 );
   if ( -d $_ ) { $bDir{sRel( $File::Find::name, $sDir0 )} = 1; }
      # Remember directory name for possible removal
   elsif ( "setup.ini" eq $_ ) {
      # Parse setup.ini
      my @aSetup;

      # Get list of files to leave alone, includes subdirectory path
      # setup-timestamp: 1012849221
      # install: latest/bash/bash-2.05-1.tar.gz 576828
      # source:  latest/bash/bash-2.05-1-src.tar.gz 1792319
      my ( $bHide, $sGroup, $sType, $sFile, $sSize, $sVol, $sSubDir, $sName );
      my $sSetup = $File::Find::name;
      open( SETUP, $sSetup ) or usage( 'die', 1, "Can't open $sSetup, $!" );
      while ( <SETUP> ) {
         ( $sType, $sFile, $sSize )  = split /\s+/, $_;
         if    ( 'setup-timestamp:' eq $sType ) {
            $aSetup[0]         = $sFile; # Actually timestamp
         }
         elsif ( 'install:'         eq $sType ) {
            next if $bHide;
            ( $sVol, $sSubDir, $sName ) = File::Spec -> splitpath( $sFile );
            $sSubDir                    = File::Spec -> canonpath( $sSubDir );
            $aSetup[1]{$sName}          = [ $sGroup, $sSize, $sSubDir ];
         }
         elsif ( 'source:'          eq $sType ) {
            next if $bHide;
            ( $sVol, $sSubDir, $sName ) = File::Spec -> splitpath( $sFile );
            $sSubDir                    = File::Spec -> canonpath( $sSubDir );
            $aSetup[2]{$sName}          = [ $sGroup, $sSize, $sSubDir ];
         }
         elsif ( '[' eq substr( $_, 0, 1 ) )    { s/\s+$//; $sGroup = $_; } # ]
         elsif ( s/^@\s+// ) {
            s/\s+$//;
            $bHide  = bHide( $_ );
            $sGroup = '[curr]';
         }
      }
      close SETUP;
      usage( 'die', 1, "Nothing found in $sSetup" )
         if 3 != @aSetup || ! $aSetup[0];
      $aSetup{$sRel} = \@aSetup;
   }
   elsif ( ".tar.bz2" eq substr( $_, -8 ) || ".tar.gz" eq substr( $_, -7 ) ) {
      # Save name of archive file
      $sTarBall{$_}{File::Spec -> canonpath( $sRel )} = -s $_;
   }
   # Currently ignoring other types of files
};
find( $wanted, $sDir0 );
usage( "die", 1, "No setup.ini files found" ) if ! %aSetup;

# Pick newest setup.ini
my ( $tNewest, $sNewest, @sOldSetup ) = ( 0, "" );
foreach ( sort keys %aSetup ) {
   if ( $tNewest < $aSetup{$_}[0] ) {
      $tNewest = $aSetup{$_}[0];
      $sNewest = $_;
   }
}
print "Newest setup.ini files: ", scalar gmtime( $tNewest ), " GMT\n";
foreach ( sort keys %aSetup ) {
   print "   ", sUnPercent( sRel( $_ ) ), "\n" if $tNewest == $aSetup{$_}[0];
}

# Report or remove obsolete setup.ini files
foreach ( sort keys %aSetup ) {
   push @sOldSetup, $_ if $tNewest > $aSetup{$_}[0];
}
print "Obsolete setup.ini files:\n" if @sOldSetup;
foreach ( @sOldSetup ) {
   print "   ", scalar gmtime( $aSetup{$_}[0] ), " GMT ",
     sUnPercent( sRel( $_ ) ), "\n";
   unlink "$_/setup.ini" or print "      *** Can't remove, $!" if $bSetup;
}

# Check found files against those listed in latest setup.ini
my ( @sDir, $sDir, $sFile, @sDup, %sMove, @sRemove, @sWrongSize );
my %aInstall = ( %{$aSetup{$sNewest}[1]} );
my %aSource  = ( %{$aSetup{$sNewest}[2]} );
foreach $sName ( sort keys %sTarBall ) {
   @sDir  = sort keys %{$sTarBall{$sName}};
   $sDir  = $sDir[0];
   $sFile = File::Spec -> catfile( $sDir, $sName );
   if    ( 1 < @sDir ) {
      # More than one copy of a file was found
      push @sDup, "$sName: " . join( ", ", map { sUnPercent( $_ ) } @sDir );
   }
   elsif ( exists $aInstall{$sName} ) {
      if    ( $sDir eq $aInstall{$sName}[2] ) {} # Already in right place
      elsif ( $sTarBall{$sName}{$sDir} != $aInstall{$sName}[1] ) {
         # Wrong size
         push @sWrongSize,
            "$sFile: $sTarBall{$sName}{$sDir} != $aInstall{$sName}[1] in " .
            sUnPercent( $sDir );
      }
      else {
         $sMove{$sFile} = File::Spec -> catfile( $aInstall{$sName}[2], $sName );
      }
   }
   elsif ( exists $aSource{$sName} ) {
      if    ( $sDir eq $aSource{$sName}[2] ) {} # Already in right place
      elsif ( $sTarBall{$sName}{$sDir} != $aSource{$sName}[1] ) {
         # Wrong size
         push @sWrongSize,
            "$sFile: $sTarBall{$sName}{$sDir} != $aSource{$sName}[1] in " .
            sUnPercent( $sDir );
      }
      else {
         $sMove{$sFile} = File::Spec -> catfile( $aSource{$sName}[2], $sName );
      }
   }
   else {
      # File not in setup.ini
      push @sRemove, $sFile;
   }
}

# Check for missing files
my @sMissing = ();
my @sMissingList = ();

if ( $bInstall ) {
  foreach ( keys %aInstall ) {
	if ( ! exists $sTarBall{$_} &&
	  ( $bCurr || '[curr]' ne $aInstall{$_}[0] ) &&
	  ( $bPrev || '[prev]' ne $aInstall{$_}[0] ) &&
      ( $bTest || '[test]' ne $aInstall{$_}[0] ) &&
	  ( $bExp  || '[exp]'  ne $aInstall{$_}[0] ) ) {
		push @sMissing, "$aInstall{$_}[0] $aInstall{$_}[2]/$_" ;
		  # Alternative: $aInstall{$_}[0] . " " .
		  #              File::Spec -> catfile( $aInstall{$_}[2],  $_ ) }
		push @sMissingList, "$aInstall{$_}[2]/$_" ;
	  }
  }
}
if ( $bSource ) {
  foreach ( keys %aSource ) {
	if ( ! exists $sTarBall{$_} &&
	  ( $bCurr || '[curr]' ne $aSource{$_}[0] ) &&
      ( $bPrev || '[prev]' ne $aSource{$_}[0] ) &&
      ( $bTest || '[test]' ne $aSource{$_}[0] ) &&
      ( $bExp  || '[exp]'  ne $aSource{$_}[0] ) ) {
		push @sMissing, "$aSource{$_}[0] $_" ;
		  # Alternative: $aSource{$_}[0] . " " .
		  #              File::Spec -> catfile( $aSource{$_}[2],  $_ ) }
		push @sMissingList, "$aSource{$_}[2]/$_" ;
	  }
  }
}
@sMissing = sort @sMissing;

# Complain about errors
print join( "\n   ", "\nDuplicate Files",  @sDup ),       "\n" if @sDup;
print join( "\n   ", "\nWrong Size Files", @sWrongSize ), "\n" if @sWrongSize;
print join( "\n   ", "\nMissing Files:",   @sMissing ),   "\n" if @sMissing;

# Report list of missing files to disk
if ( $bWrite ) {
  @sMissingList = sort @sMissingList;
  open(REPORT, '>', 'missing.lst');
  print( REPORT join( "\n", @sMissingList ),"\n" ) if @sMissingList;
  close(REPORT);
}

# Move queued files to base directory tree
if ( %sMove ) {
   print $bMove ? "\n" : "\nNot ", "Moving files to base directory tree\n";
   my ( $sFrom, $sTo );
   foreach $sFrom ( sort keys %sMove ) {
      $sTo  = $sMove{$sFrom};
      $sDir = dirname( $sTo );
      print sUnPercent( "   $sFrom -> $sDir\n" );
      if ( -e $sTo ) { print "      *** Target already exists\n"; }
      elsif ( $bMove ) {
         mkpath( $sDir ) if ! -d $sDir;
         move( $sFrom, $sTo ) or print "      *** Can't move, $!\n";
      }
   }
}

# Remove files not listed in setup.ini
if ( @sRemove ) {
   print $bArch ? "\n" : "\nNot ", "Removing files not in setup.ini\n";
   foreach $sFile ( @sRemove ) {
      print sUnPercent( "   $sFile\n" );
      if ( $bArch ) {
         unlink( $sFile ) or print "      *** Can't remove, $!\n";
      }
   }
}

# Remove empty directories
my ( @g, $bFound );
foreach $sRel ( sort { length $b <=> length $a || $a cmp $b } keys %bDir ) {
   @g = glob( File::Spec -> catfile( $sRel, "*" ) );
   if ( ! @g ) {
      print $bArch ? "\n" : "\nNot ", "Removing Empty Directories\n"
         if ! $bFound++;
      print sUnPercent( "   $sRel\n" );
      if ( $bArch ) {
         rmdir $sRel or print "      *** Can't rmdir, $!\n";
      }
   }
}

# Copy latest setup.ini to base directory
if ( "." ne $sNewest ) {
   print sUnPercent( "\nCopying setup.ini from $sNewest to $sDir0\n" );
   copy( "$sNewest/setup.ini", "$sDir0/setup.ini" )
      or print "   *** Can't copy setup.ini, $!\n";
}

exit 0;

# Produce relative file or directory
sub sRel {
   my ( $sAbs, $sBase ) = @_;
   $sBase = "." if ! defined $sBase || ! length $sBase;
   my $sRel = File::Spec -> abs2rel( $sAbs, $sBase );
   $sRel =~ s,\\,/,g;
   $sRel =~ s/^\w://;
   $sRel =  "." if ! length $sRel;
   return $sRel;
}

# Convert %xx to characters
sub sUnPercent {
   local ( $_ ) = @_;

   s/\%([0-9a-f]{2})/chr(hex($1))/gie;
   return $_;
}


[-- Attachment #4: Type: text/plain, Size: 214 bytes --]

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Using a real mirroring tool...

[Volker Quetschke]

[-- Attachment #1: Type: text/plain, Size: 753 bytes --]

Hi!

> Charles Wilson wrote:
> 
>> A few notes:
>> ...
>> rsync can't merge.
> 
> Yes, but Michael A. Chases clean_setup.pl can! I allowed myself to add a 
> new option to clean_setup.pl, it exports the list of missing files to a 
> file. Then you can use wget to get all the missing files.
> 
> I attached the modified clean_setup.pl plus a shell scrip to get the 
> missing files from a mirror. You have to modify the getmissing.sh to 
> choose your mirror and your target download directory. The directory and 
> both files must exist the same directory as your setup.exe. Just a quick 
> hack, I guarantee nothing!


Use the attached getmissing.sh instead, some options for wget were not 
correct, they left directory listing files.

Bye
    Volker

[-- Attachment #2: getmissing.sh --]
[-- Type: text/plain, Size: 878 bytes --]

#!/bin/bash

# Mirror script for cygwin
# Uses Michael A. Chase "cleanup_setup.pl"
# in a modified version (V1.0303)
# and wget

# getmissing.sh and cleanup_setup.pl must reside in the same directory
# as setup.exe

# Files to download
filelist="missing.lst"
# Choose your mirror with directory of setup.ini
#mirror="ftp://ftp.inf.tu-dresden.de/software/windows/cygwin32/"
# Cut directories                   1        2       3
mirror="ftp://mirrors.rcn.net/mirrors/sources.redhat.com/cygwin/"
# Cut directories             1       2                  3
cutdir=3
# Name your download directory
mirrordir="_mymirror"

wget -r -N --no-host-directories --cut-dirs=$cutdir --passive-ftp -P $mirrordir ${mirror}setup.ini
./clean_setup.pl -writelist
wget -r -N --no-host-directories --cut-dirs=$cutdir --passive-ftp -P $mirrordir -i $filelist -B $mirror
./clean_setup.pl -Arch -Setup


[-- Attachment #3: Type: text/plain, Size: 214 bytes --]

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Using a real mirroring tool...

[Michael A Chase]

On Tue, 30 Apr 2002 16:00:52 +0200 Volker Quetschke <quetschke@scytek.de> wrote:

> I attached the modified clean_setup.pl plus a shell scrip to get the 
> missing files from a mirror. You have to modify the getmissing.sh to 
> choose your mirror and your target download directory. The directory and 
> both files must exist the same directory as your setup.exe. Just a quick 
> hack, I guarantee nothing!

You started with a little bit old version of the script.  I have uploaded a
new version 1.0401 which uses a '-misslist file' option to write the list
to the specified file.

The new version also allows you to exclude packages that you are not
interested in with the '-H mask' option.  This will allow your wget with
the list more selective.

Some of the other options have been changed as well, so be sure to run with
'-h' before using the script.

-- 
Mac :})
** I normally forward private questions to the appropriate mail list. **
Ask Smarter: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Give a hobbit a fish and he eats fish for a day.
Give a hobbit a ring and he eats fish for an age.



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Gary R. Van Sickle]

> Robert Collins wrote in <016301c1ef7b$4d769f00$0200a8c0@lifelesswks>
> in gmane.os.cygwin on Mon, 29 Apr 2002 22:42:18 +1000:
> 
> > Actually, the current functionality wil do the trick, as long as you
> > install what you have downloaded. 
> 
> Hmmm. I've just tried using "install from Internet" with bzip2,
> sharutils, unzip and zip, on a machine that doesn't have them
> installed and killing the connection after bzip2 has downloaded.
> 
> When it says "download incomplete, try again" I say "No" and it
> attempts to do the install.
> 
> I'm then getting "Can't open (null) for reading" errors during the
> attempted installation of sharutils (the sharutils sub-directory has
> been created but is empty) and a "installation incomplete" message box
> at the end but it /has/ installed bzip2.
> 
> It's pretty ugly but it does seem to be working and if I reselect
> bzip2 when I try again it isn't attempting to download it again so
> yes, you could get there even on a slow link. (Not exactly good UI
> design though!) ;-)
> 

My ears are burning... ;-)

-- 
Gary R. Van Sickle
Brewer.  Patriot. 

 

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Christopher Faylor]

On Mon, Apr 29, 2002 at 10:42:18PM +1000, Robert Collins wrote:
>Checkbox's, command line options, and 'special case code' and the
>redownload itself are all kludges around fixing the key problem.

That's for sure.  The design that I proposed for setup's UI eliminated
most of these.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Sam Edge]

Robert Collins wrote in <016301c1ef7b$4d769f00$0200a8c0@lifelesswks>
in gmane.os.cygwin on Mon, 29 Apr 2002 22:42:18 +1000:

> Actually, the current functionality wil do the trick, as long as you
> install what you have downloaded. 

Hmmm. I've just tried using "install from Internet" with bzip2,
sharutils, unzip and zip, on a machine that doesn't have them
installed and killing the connection after bzip2 has downloaded.

When it says "download incomplete, try again" I say "No" and it
attempts to do the install.

I'm then getting "Can't open (null) for reading" errors during the
attempted installation of sharutils (the sharutils sub-directory has
been created but is empty) and a "installation incomplete" message box
at the end but it /has/ installed bzip2.

It's pretty ugly but it does seem to be working and if I reselect
bzip2 when I try again it isn't attempting to download it again so
yes, you could get there even on a slow link. (Not exactly good UI
design though!) ;-)

(While I'm here; BUG REPORT: When I get the "Download incomplete.
Retry? Yes/No" message box in "install from Internet" mode and choose
"No" without it then going on to install any packages I then get a
final message box with "Download incomplete. Retry? Yes/No" again but
only an "OK" button.)

> http://www106.pair.com/rhp/free-software-ui.html.

LOL. Yes. I think I can agree with most of this, especially the lack
of UI consistency and the problems with debugging over-featured
design.

> Checkbox's, command line options, and 'special case code' and the
> redownload itself are all kludges around fixing the key problem.

I agree. The best solution is to make a bulletproof method of
detecting whether the local copies are uncorrupted and to keep the UI
uncluttered. But from the general tenor of the discussion at the
beginning I was under the impression that there was some philosophical
or technical reason why this wasn't appropriate. A manual option would
therefore be the next best thing. 

Sorry if I got the wrong end of the stick.

> The real solution is to positively identify corrupt archives and
> transparently remove them (perhaps asking the user whether we should
> delete, backup, or skip over the package).

Yup. Agreed. Thesis, antithesis, synthesis! :-)

Can I also suggest adding reget behaviour at some point, if it can be
done reliably. Getting nine tenths of the way into a 15MB package when
your line goes down is /very/ annoying! ;-(

Finally, I stand by my wish to retain a separate download and
installation invocation option to allow them to occur while logged on
to NT/2k/XP with different permissions, again out of paranoia.

Thanks.

-- 
Sam Edge

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

----- Original Message -----
From: "Sam Edge" <sam_edgeZZZ@hotmail.com>
To: <cygwin@cygwin.com>

> > A 'normal' install - download and install - works fine, no problems.
>
> Read my and Christopher Faylor's posts especially
> <20020429050712.GC27298@redhat.com> where we explain where for some
> people download and install with the current version of setup.exe does
> not work fine.

Actually, the current functionality wil do the trick, as long as you
install what you have downloaded. Combined with multi-mirror failure,
unless you have a *really* bad phone line connection, you should be able
to successfully install bit by bit. Notwithstanding that I do understand
and (mostly) agree with whats been said though.


> > A reference for this is
> > http://sources.redhat.com/ml/cygwin-apps/2001-11/msg00271.html.
>
> Ah. Exactly. :-)
>
> So is it worth my while looking at the code with an eye to providing a
> patch for this (specifically the checkbox option) or is it work in
> progress elsewhere?

Work is in progress. You (and everyone who's jumped in on this) might
like to read Havoc Penningtons recent paper
http://www106.pair.com/rhp/free-software-ui.html.

What follows is not a critique of your suggestion, just an explanation
as to why the checkbox is not appropriate IMO. Before reading the
following, be sure to have read Havoc's paper above.

We have a problem with setup.exe.
* Corrupt package files are not always detected and automatically
removed. *

That's it. "Redownloading" is a workaround for that. (If in fact there
are -other- reasons for "redownloading" then I will revise that
statement.)

Checkbox's, command line options, and 'special case code' and the
redownload itself are all kludges around fixing the key problem.

And thats why this has become a sudden issue. I *unknowningly* removed a
bit of special case code that interfaced between good logic and the
redownload capability to make it seem 'correct' to the user. (BTW:
Chris, it was NOT your fault. Really.)

The real solution is to positively identify corrupt archives and
transparently remove them (perhaps asking the user whether we should
delete, backup, or skip over the package).

Rob


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Sam Edge]

Robert Collins wrote in
<FC169E059D1A0442A04C40F86D9BA7600C5F52@itdomain003.itdomain.net.au>
in gmane.os.cygwin on Mon, 29 Apr 2002 17:47:16 +1000:

> The *redownloading* is a bug, and will get fixed. 

Okay. Good.

> The 'I want to mirror
> > non-installed packages by using download' is not.

Actually, the re-download fix would go 90% of the way to doing this
anyway, since you could just reselect everything you want again and
again and it'd eventually have the lot even on a slow link.

> A 'normal' install - download and install - works fine, no problems.

Read my and Christopher Faylor's posts especially
<20020429050712.GC27298@redhat.com> where we explain where for some
people download and install with the current version of setup.exe does
not work fine.

Even a single package such as the XFree86-fnts one can be a trial on
dial-up connections, taking the best part of an hour. A clean minimum
install of the Cygwin environment takes about the same, I think. 

A model that didn't allow multiple download sessions before a single
install one would effectively bar many dial-up users from installing
Cygwin and/or XFree86 at all. The selective mirroring and local FTP
server route would simply be too much hassle for many just to evaluate
something.

> A reference for this is
> http://sources.redhat.com/ml/cygwin-apps/2001-11/msg00271.html.

Ah. Exactly. :-)

So is it worth my while looking at the code with an eye to providing a
patch for this (specifically the checkbox option) or is it work in
progress elsewhere?

-- 
Sam Edge

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

> -----Original Message-----
> From: Christopher Faylor [mailto:cgf@redhat.com] 
> Sent: Monday, April 29, 2002 8:30 AM

> However, I think I may have been responsible for setup's 
> current behavior of basing what is downloaded on what is 
> installed.  Even if I was not, though, I think that the 
> explanations about why some people need this are rather convincing.

Yup.
 
> If I have this right then even removing download from 
> internet does not eliminate this problem.

I don't actually like the idea of removing download from the net. It's
such a useful get-this-for-me capacity that I feel it can stay. It's the
redownload aspect that s***s me.
 
Rob

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

> -----Original Message-----
> From: Christopher Faylor [mailto:cgf@redhat.com] 
> Sent: Monday, April 29, 2002 3:07 PM

> >That's what the discussion has been about for the last week or so. 
> >After lots of requests for this functionality, the position 
> eventually 
> >hardened to "it's not a download tool".
> 
> Right.  But it started with my suggesting this behavior was a 
> bug and (I
> thought) Robert agreeing.  So, I thought it was time to chime 
> in again.

The *redownloading* is a bug, and will get fixed. The 'I want to mirror
non-installed packages by using download' is not.

> The bottom line is that if we have to tell someone to use 
> wget or rsync just to do a normal install, something is wrong.

These folk aren't doing a normal install. They want to use download
mode, and then *not* install what was downloaded. 
A 'normal' install - download and install - works fine, no problems.
 
> It seems like this problem would be trivially fixed by not 
> attempting to redownload something that has already been downloaded.

I've also expressed this opinion. But... redownloading is a current
'feature'. It's one I don't think we need, and that we can get rid of -
which will correct the problem.

A reference for this is
http://sources.redhat.com/ml/cygwin-apps/2001-11/msg00271.html.

Rob

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Charles Wilson]

Alec Mihailovs wrote:

> I have a perfect installation program for MiKTeX made 
> by Christian Schenk, see http://www.miktex.org/ 
> 
> You people seem to be not able to make anything even close to it. 
> Why don't you just copy it and use for cygwin instead of inventing 
> a wheel yourself? 


I'll ignore your rudeness, and pretend that you actually are trying to 
help.  I don't believe it for a minute, but I'm a decent hobbyist actor...

1) We are providing an entire development environment, that includes tex 
as just one small part.  MikTeX, which is a fine program that I use for 
all my document creation, is MUCH more narrowly focused -- includes less 
than 50 executables -- and thus places far fewer demands on its 
installer.  To be sure, there are some similarities: you might be 
tempted to draw a parallel between MikTeX's packages, and Cygwin's 
package system; but that parallel is false.  The caption2 package and 
the fancyhdr package are both just modification scripts for the same 
engine; they both help make pretty documents.  The cvs package and the 
tetex-beta package are quite different; one is a source revision control 
program and the other includes all of MikTeX's functionality.

It's a bit more complicated to install cygwin stuff.

2) We predate the MikTeX installer by quite some time.

3) Christian provides CVS access to his source code -- but the installer 
is restricted in use, and is not licensed under the GPL.  We cannot take 
it and adapt it without his permission.  Call us old-fashioned, but we 
prefer truly free (as in speech) software.

4) To claim that MikTeX's installation program is "perfect" -- e.g. bug 
free -- is utter lunacy.  Check the MikTeX forums...

5) Christian's installer, I believe, is built using MSVC and seems to 
actually leverage the MS installer code from Microsoft's installer SDK. 
  That's a fine idea -- but is probably off limits in a truly free 
(speech) project.  We require that our tools -- including the installer 
-- are built using free (speech) tools.   Now, perhaps you could make 
the argument that this proves that free tools aren't as good as the MS 
stuff; I disagree and believe it's just another example of MS leveraging 
an existing monopoly (OS) to take over another market 
(SoftwareInstallers).  In any case, the "free software vs. microsoft" 
debate belongs somewhere other than the main cygwin mailing list.

 > Best wishes,
 > Alec Mihailovs

Somehow, I doubt that you really have the "best wishes" of the cygwin 
project at heart, Dr. Mihailovs, since you began your message accusing 
us of incompetence.

--Charles Wilson




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Alec Mihailovs]

I have a perfect installation program for MiKTeX made 
by Christian Schenk, see http://www.miktex.org/ 

You people seem to be not able to make anything even close to it. 
Why don't you just copy it and use for cygwin instead of inventing 
a wheel yourself? 

Best wishes, 
Alec Mihailovs 
http://webpages.shepherd.edu/amihailo/


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

Kurt,
	you've jumped in half way through a discussion. I've addressed
every point you made, as have other people, in various emails in this
list over the last few months.

Rob

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Kurt Roeckx]

On Sun, Apr 28, 2002 at 09:47:14PM +1000, Robert Collins wrote:
> > -----Original Message-----
> > From: Sam Edge [mailto:sam_edgeZZZ@hotmail.com] 
> > Sent: Sunday, April 28, 2002 9:32 PM
> 
> > > So use a mirroring tool! Setup.exe is -not- designed for this.
> > 
> > Surely if setup.exe isn't designed with at least a nod to 
> > this way of working why does it have "download from Internet" 
> > and "install from local directory" options, instead of just 
> > instisting on connecting to a server every time?
> 
> It's designed to allow an off-line mode, as opposed to having spare
> copies of potentially desired pacakges around. The redownloading
> behaviour is a result of a kludge to allow deliberate redownloading of
> packages in 'download only' mode.

Why is the redownloading a "good thing" in case of download only
mode?  If I want to redownload something, I could just remove it
and then say to download it again.

> If we remove that, then packages won't
> get downloaded again and again and again - but setup will be no more
> suited to being a mirroring tool. 

The "no more suited to being a mirroring tool" really confuses
me!  How is redownloading good for a mirror tool?  The only case
I can see to redownload a certain file is when the time of the
file has changed (or the size).  If it *looks* the same, don't
download it again.

The only file that really should need that, afaik, is setup.ini,
and I think you download that anyway?

> > > If you
> > > want a setup.ini parsing mirroring tool, take me up on my offer to 
> > > help someone leverage the setup.ini codebase to build a mirroring 
> > > tool.

Why do you even need to parse setup.ini to be able to mirror
something?

Using something as wget should work, I just didn't try it yet.


Kurt


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

> -----Original Message-----
> From: Sam Edge [mailto:sam_edgeZZZ@hotmail.com] 
> Sent: Sunday, April 28, 2002 9:32 PM

> > So use a mirroring tool! Setup.exe is -not- designed for this.
> 
> Surely if setup.exe isn't designed with at least a nod to 
> this way of working why does it have "download from Internet" 
> and "install from local directory" options, instead of just 
> instisting on connecting to a server every time?

It's designed to allow an off-line mode, as opposed to having spare
copies of potentially desired pacakges around. The redownloading
behaviour is a result of a kludge to allow deliberate redownloading of
packages in 'download only' mode. If we remove that, then packages won't
get downloaded again and again and again - but setup will be no more
suited to being a mirroring tool. The cache directory is shareable with
multiple cygwin installs (say over a SMB share). So yes, there is a nod
towards what you do, but that's abou tit.
 
> > If you
> > want a setup.ini parsing mirroring tool, take me up on my offer to 
> > help someone leverage the setup.ini codebase to build a mirroring 
> > tool.
> 
> Although I've not looked at it, setup.exe must already have 
> most of the code required to operate in the way we'd like, surely?

It has a significant amount of code, that is getting more flexible and
paramterised all the time. If you look at
http://sources.redhat.com/cygwin-apps/ you can access the setup.exe
developers page. It's a but rough at the moment, but it does have the
CVS location amongst other things.
 
> If you tell me the module name(s) for setup.exe and any 
> documentation about setup.ini and the local and FTP/HTTP 
> server directory structures I'd certainly be interested to 
> download them from the CVS server and take a look.

Setup.ini is documented at http://www.cygwin.com/setup.html. There is no
ftp/http data structure - setup.ini determines everything. Setup's
multiple setup.ini capability allows for advanced merging and
per-organisation tweaking.

The differences between setup and a mirroring tool include:
* All the install code isn't needed.
* Attention to local packages isn't needed.
* Persistent selection of packages irrespective of install status should
be present.
* command line driven may be useful? 

Anyway, there's some notes, have a look, and we can talk further.

Rob

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Sam Edge]

Daniele wrote in
<Pine.A41.3.96.1020428003038.145792A-100000@egeo.unipg.it>
in gmane.os.cygwin on Sun, 28 Apr 2002 00:44:25 +0200 (MDT):

> After dutifully reading the mailing list discussion on the subject,
> I would like to point out that this behavior - redownloading uninstalled
> packages - was absent 
> in previous versions of setup. In my humble opinion, the downloaded
> packages and the installed packages should be treated independently by
> setup for the simple reason that you may download a set of packages for
> installing on multiple machines, and install a subset of it on the
> downloading machine. In other words should be:
> download from internet: compares repository with packages
> install from internet:  compares repository with installation
> install from local:     comapres packages with installation  
> that is, previous behaviour.

Every version of setup.exe I've used exhibits this annoying behaviour
but them I'm new here.

I agree with Daniele. I install different subsets of the packages on
several machines. I also like to keep a copy of the most up-to-date
versions of packages that I don't currently use on any of them, in
case I need them where I've no (or poor) access to the 'Net. 

Doing this is something of a headache and requires manual checking in
the local package directory (now even more difficult because of its
fragmented nature even when mitigated by clean_setup.pl - thanks
Michael) and in the setup.exe list.

-- 
Sam Edge

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

setup 2.194.2.24: Bug (?) in downloading from internet

[Daniele Mezzetti]

(I hope this will show up in the correct thread.. excuse me if not)

refers to msg01259.html msg01481.html

After dutifully reading the mailing list discussion on the subject,
I would like to point out that this behavior - redownloading uninstalled
packages - was absent 
in previous versions of setup. In my humble opinion, the downloaded
packages and the installed packages should be treated independently by
setup for the simple reason that you may download a set of packages for
installing on multiple machines, and install a subset of it on the
downloading machine. In other words should be:

download from internet: compares repository with packages
install from internet:  compares repository with installation
install from local:     comapres packages with installation  

that is, previous behaviour.




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

> -----Original Message-----
> From: Alexei Lioubimov [mailto:e-complex@mtu-net.ru] 
> Sent: Saturday, April 27, 2002 12:32 AM
> To: Robert Collins; cygwin@cygwin.com; Larry Hall (RFK Partners, Inc)
> Subject: Re: setup 2.194.2.24: Bug (?) in downloading from internet
> 
> 
> Hello Robert,
> Excuse me, but my English is not so good, and i didn't 
> understand one sentence from your answer: "...which is 
> something I'm seriously considering nuking." What does the 
> word "nuking" mean?

Removing/destroying/eliminating. With a sense of anger/frustration.
 
> What is the re-download functionality you are speaking about?

When you select 'download from internet', if you select a package that
is already cached, it deletes the cached copy and downloads the file
again. As it doesn't keep corrupt files, I see no value in this - and it
is the reason you see setup downloading stuff again and again.

Rob

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Alexei Lioubimov]

Hello Robert,
Excuse me, but my English is not so good, and i didn't understand one
sentence from your answer:
"...which is something I'm seriously considering nuking."
What does the word "nuking" mean?

What is the re-download functionality you are speaking about?

Alexei Lioubimov


----- Original Message -----
From: "Robert Collins" <robert.collins@itdomain.com.au>
To: "Alexei Lioubimov" <e-complex@mtu-net.ru>; <cygwin@cygwin.com>; "Larry
Hall (RFK Partners, Inc)" <lhall@rfk.com>
Sent: Friday, April 26, 2002 5:57 PM
Subject: RE: setup 2.194.2.24: Bug (?) in downloading from internet




> -----Original Message-----
> From: Alexei Lioubimov [mailto:e-complex@mtu-net.ru]
> Sent: Friday, April 26, 2002 11:44 PM
> To: cygwin@cygwin.com; Larry Hall (RFK Partners, Inc)
> Subject: Re: setup 2.194.2.24: Bug (?) in downloading from internet
>
>
> Hello Larry,
> I know, that this problem has already been reported
> (msg01259.html). And i've read your answer to use some
> mirroring tool (wget) instead before i sent my message to the
> forum. It seemes to me that you are wrong proposing such
> solution and it is not the answer to the question.

The solution you propose is also less than satisfactory. It means that
setup won't be looking at what you need, but rather at what you have
'available'. The real cause of the problem is setup's re-download
functionality, which is something I'm seriously considering nuking.

Rob



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

RE: setup 2.194.2.24: Bug (?) in downloading from internet

[Robert Collins]

> -----Original Message-----
> From: Alexei Lioubimov [mailto:e-complex@mtu-net.ru] 
> Sent: Friday, April 26, 2002 11:44 PM
> To: cygwin@cygwin.com; Larry Hall (RFK Partners, Inc)
> Subject: Re: setup 2.194.2.24: Bug (?) in downloading from internet
> 
> 
> Hello Larry,
> I know, that this problem has already been reported 
> (msg01259.html). And i've read your answer to use some 
> mirroring tool (wget) instead before i sent my message to the 
> forum. It seemes to me that you are wrong proposing such 
> solution and it is not the answer to the question.

The solution you propose is also less than satisfactory. It means that
setup won't be looking at what you need, but rather at what you have
'available'. The real cause of the problem is setup's re-download
functionality, which is something I'm seriously considering nuking.

Rob

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Alexei Lioubimov]

Hello Larry,
I know, that this problem has already been reported (msg01259.html). And
i've read your answer to use some mirroring tool (wget) instead before i
sent my message to the forum. It seemes to me that you are wrong proposing
such solution and it is not the answer to the question. Of course i can
download everything "by hand" - without the SETUP program, but why should i?
SETUP "knows" more about the mirror sites, directory structure, package
versions etc.
I'm only asking about one small bug in SETUP -- since it doesn't show
already downloaded (but NOT installed)files, it's a bit confusing.

Alexei Lioubimov

----- Original Message -----
From: "Larry Hall (RFK Partners, Inc)" <lhall@rfk.com>
To: "Alexei Lioubimov" <e-complex@mtu-net.ru>; <cygwin@cygwin.com>
Sent: Thursday, April 25, 2002 7:24 PM
Subject: Re: setup 2.194.2.24: Bug (?) in downloading from internet


> At 11:08 AM 4/25/2002, Alexei Lioubimov wrote:
> >Hello,
> >I have a problem using setup.exe. Here is the description
> >
> >setup.exe: 2.194.2.24;
> >mode: Download from Inernet;
> >keyword: "skip" flag;
> >
> >The problem: if i've downloaded some tarballs and didn't install them -
then
> >during the following updates the SETUP program doesn't report me about
that.
> >It simply marks those files as "skip" by default and allows me to change
> >this to the current version. But i may have already downloaded the
stuff...
> >So i caught myself downloading the same tarballs twice or triple times.
If
> >you take into account the slow speed of dial-up connection, that i have,
> >then you can imagine what i'm feeling.
> >
> >The suggestion: may be it's possible force SETUP consider "current" be
> >"downloaded" but not "installed" stuff during "Download from Internet"
> >process?
>
>
>
> This has already been reported.  If you're interested in the discussion on
> this subject, please see the email list archives.
>
>
>
> Larry Hall                              lhall@rfk.com
> RFK Partners, Inc.                      http://www.rfk.com
> 838 Washington Street                   (508) 893-9779 - RFK Office
> Holliston, MA 01746                     (508) 893-9889 - FAX
>
>


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: setup 2.194.2.24: Bug (?) in downloading from internet

[Larry Hall (RFK Partners, Inc)]

At 11:08 AM 4/25/2002, Alexei Lioubimov wrote:
>Hello,
>I have a problem using setup.exe. Here is the description
>
>setup.exe: 2.194.2.24;
>mode: Download from Inernet;
>keyword: "skip" flag;
>
>The problem: if i've downloaded some tarballs and didn't install them - then
>during the following updates the SETUP program doesn't report me about that.
>It simply marks those files as "skip" by default and allows me to change
>this to the current version. But i may have already downloaded the stuff...
>So i caught myself downloading the same tarballs twice or triple times. If
>you take into account the slow speed of dial-up connection, that i have,
>then you can imagine what i'm feeling.
>
>The suggestion: may be it's possible force SETUP consider "current" be
>"downloaded" but not "installed" stuff during "Download from Internet"
>process?



This has already been reported.  If you're interested in the discussion on
this subject, please see the email list archives.



Larry Hall                              lhall@rfk.com
RFK Partners, Inc.                      http://www.rfk.com
838 Washington Street                   (508) 893-9779 - RFK Office
Holliston, MA 01746                     (508) 893-9889 - FAX


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

setup 2.194.2.24: Bug (?) in downloading from internet

[Alexei Lioubimov]

Hello,
I have a problem using setup.exe. Here is the description

setup.exe: 2.194.2.24;
mode: Download from Inernet;
keyword: "skip" flag;

The problem: if i've downloaded some tarballs and didn't install them - then
during the following updates the SETUP program doesn't report me about that.
It simply marks those files as "skip" by default and allows me to change
this to the current version. But i may have already downloaded the stuff...
So i caught myself downloading the same tarballs twice or triple times. If
you take into account the slow speed of dial-up connection, that i have,
then you can imagine what i'm feeling.

The suggestion: may be it's possible force SETUP consider "current" be
"downloaded" but not "installed" stuff during "Download from Internet"
process?

Thank you,
Alexei Lioubimov


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/