Package file lists archive

Package file lists archive

[Steven Penny]

The MSYS2 project keep an up to date archive with files lists for all packages

http://downloads.sourceforge.net/project/msys2/REPOS/MINGW/x86_64/mingw64.files.tar.gz

This file is not very large considering

    $ gzip -l mingw64.files.tar.gz
             compressed        uncompressed  ratio uncompressed_name
                 982852             9585152  89.7% mingw64.files.tar

I was wondering if something like this is possible with Cygwin. Currently if you
want to search a file that is not installed you must use

    cygcheck --package-query

which I have noticed can be very slow. I would be happy to help with maintaining
a Cygwin archive with file lists for all packages.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Christopher Faylor]

On Sun, May 25, 2014 at 05:37:45PM -0500, Steven Penny wrote:
>The MSYS2 project keep an up to date archive with files lists for all packages
>
>http://downloads.sourceforge.net/project/msys2/REPOS/MINGW/x86_64/mingw64.files.tar.gz
>
>This file is not very large considering
>
>    $ gzip -l mingw64.files.tar.gz
>             compressed        uncompressed  ratio uncompressed_name
>                 982852             9585152  89.7% mingw64.files.tar
>
>I was wondering if something like this is possible with Cygwin. Currently if you
>want to search a file that is not installed you must use
>
>    cygcheck --package-query
>
>which I have noticed can be very slow. I would be happy to help with maintaining
>a Cygwin archive with file lists for all packages.

Not sure how you'd do that.  I suspect that MSYS is a lot more static
than Cygwin.  I don't think package maintainers wants to notify you
every time they upload a package.

And, frankly, I think it's a bad idea to keep a monolithic list of
packages available for people to download anyway.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Mon, May 26, 2014 at 3:12 PM, Christopher Faylor wrote:
> And, frankly, I think it's a bad idea to keep a monolithic list of
> packages available for people to download anyway.

Could you not just require each package includes a file list? This way you could
get the information from any mirror.

This would ease the burden of the cygwin.com site, as it is the only site with
that information currently

http://google.com/search?filter=0&q="etc/postinstall/cygcheck-dep.sh"

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Christopher Faylor]

On Mon, May 26, 2014 at 10:18:20PM -0500, Steven Penny wrote:
>On Mon, May 26, 2014 at 3:12 PM, Christopher Faylor wrote:
>> And, frankly, I think it's a bad idea to keep a monolithic list of
>> packages available for people to download anyway.
>
>Could you not just require each package includes a file list? This way you could
>get the information from any mirror.

Not interested.

>This would ease the burden of the cygwin.com site, as it is the only site with
>that information currently

It is surviving just fine.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Tue, May 27, 2014 at 5:11 PM, Christopher Faylor wrote:
>>This would ease the burden of the cygwin.com site, as it is the only site with
>>that information currently
>
> It is surviving just fine.

In response to this apathy, I have found my own way. Using Google Search API is
in many cases faster than "cygcheck -p". Code is here

http://github.com/transcode-open/apt-cyg/blob/29cd844/apt-cyg#L308-L360

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Warren Young]

On 5/27/2014 16:11, Christopher Faylor wrote:
>
>> This would ease the burden of the cygwin.com site, as it is the only site with
>> that information currently
>
> It is surviving just fine.

Speaking of, is the source for package-grep.cgi available somewhere?  I 
could quarter the page weight, and probably improve the render time in 
browsers considerably, too.  It currently takes about 16 seconds to 
completely load in Firefox on my system, most of which is just page 
rendering time, due to the huge table used for layout.

(I'd also need either a static copy of its input, or a way to easily 
generate it locally.)

The /packages page load time is a considerable fraction of this, at 
roughly 2-4 seconds.  It's only returning about 120 kB of text 
(compressed) so that comes to roughly half a megabit/sec.  I assume that 
means the script is running every time, re-generating the whole page 
text anew instead of caching the output, despite the fact that the input 
probably changes less often than once a day on average.  Caching should 
give a page load time of a few hundred ms, and save some CPU time to boot.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Mon, Jun 9, 2014 at 10:17 AM, Warren Young wrote:
> Speaking of, is the source for package-grep.cgi available somewhere?  I

http://cygwin.com/cgi-bin/cvsweb.cgi/htdocs/cgi-bin2?cvsroot=cygwin

Of course because of Cygwin flawless website, this is not indexed by Google and
not even linked on Cygwin.com, so you are a left with "Isla de Muerta".

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Mon, Jun 9, 2014 at 5:45 AM, Steven Penny wrote:
> Using Google Search API is in many cases faster than "cygcheck -p". Code is
> here

Adding to this, here is the sometimes frustrating experience of "cygcheck -p"

    $ time cygcheck -p bin/aclocal
    cygcheck: unable to contact cygwin.com site, InternetOpenUrl() failed: The
              operation timed out
     (win32 error 12002)

    real    0m31.132s
    user    0m0.016s
    sys     0m0.031s

And here is current iteration of my tool

    $ time apt-cyg searchall bin/aclocal
    getting pages . . .
    getting start 0 . . .
    getting start 8 . . .
    gcc-tools-epoch1-automake
    gcc-tools-epoch2-automake
    automake1.4
    automake1.13
    automake1.12
    automake1.11
    automake1.7
    automake1.14
    automake1.10
    automake1.8
    automake1.9

    real    0m1.295s
    user    0m0.165s
    sys     0m0.182s

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[PolarStorm]

Steven Penny wrote
> Adding to this, here is the sometimes frustrating experience of "cygcheck
> -p"
> ...
> And here is current iteration of my tool
> ...

Thank you very much for that apt-cyg patch! 

I don't use cygcheck very often, but when I do it's been mostly a waste 
of time. Ever since I started using apt-cyg, I've been a much more happy 
Cygwin user. Now, with your patch that is even better.  I'm sure there are 
loads of Cygwin users who greatly appreciate your effort.

BR,



--
View this message in context: http://cygwin.1069669.n5.nabble.com/Package-file-lists-archive-tp108857p109216.html
Sent from the Cygwin list mailing list archive at Nabble.com.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Mon, May 26, 2014 at 3:12 PM, Christopher Faylor wrote:
> And, frankly, I think it's a bad idea to keep a monolithic list of
> packages available for people to download anyway.

I would counter that Debian and others who use "apt-get" already do this,
Example

http://http.debian.net/debian/dists/wheezy/main/Contents-amd64.gz

> This file maps each file available in the Debian GNU/Linux system to
> the package from which it originates.  It includes packages from the
> DIST distribution for the ARCH architecture.
> You can use this list to determine which package contains a specific
> file, or whether or not a specific file is available.  The list is
> updated weekly, each architecture on a different day.

so maybe you could voice your opinion to them as well.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Christopher Faylor]

On Thu, Jun 12, 2014 at 04:58:25PM -0500, Steven Penny wrote:
>On Mon, May 26, 2014 at 3:12 PM, Christopher Faylor wrote:
>> And, frankly, I think it's a bad idea to keep a monolithic list of
>> packages available for people to download anyway.
>
>I would counter that Debian and others who use "apt-get" already do this,
>Example
>
>http://http.debian.net/debian/dists/wheezy/main/Contents-amd64.gz
>
>> This file maps each file available in the Debian GNU/Linux system to
>> the package from which it originates.  It includes packages from the
>> DIST distribution for the ARCH architecture.
>> You can use this list to determine which package contains a specific
>> file, or whether or not a specific file is available.  The list is
>> updated weekly, each architecture on a different day.
>
>so maybe you could voice your opinion to them as well.

FYI, trolling is obviously off-topic here.

It's nice that you found a solution to your problem, however.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Christopher Faylor]

On Mon, Jun 09, 2014 at 11:17:11AM -0500, Steven Penny wrote:
>On Mon, Jun 9, 2014 at 10:17 AM, Warren Young wrote:
>> Speaking of, is the source for package-grep.cgi available somewhere?  I
>
>http://cygwin.com/cgi-bin/cvsweb.cgi/htdocs/cgi-bin2?cvsroot=cygwin
>
>Of course because of Cygwin flawless website, this is not indexed by Google and
>not even linked on Cygwin.com, so you are a left with "Isla de Muerta".

Ok Mr Penny.  Here's warning 1 for you:

Can the attitude.  I'm sick of it.

If you want to experimentally figure out how many warnings you get,
please go right ahead and continue with this style of communication.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Tue, Jun 17, 2014 at 9:29 AM, Christopher Faylor wrote:
> If you want to experimentally figure out how many warnings you get,
> please go right ahead and continue with this style of communication.

Really, I can do without veiled threats. If you are going to ban me or whatever,
go ahead and do it.

I have brought some constructive criticisms of this project with the hope of
starting a discussion around fixing/improving certain issues. Instead I am met
with brick wall attitude and threats of being pushed off the mailing list.

My suggestion has merit: Debian is no niche operating system and "apt-get" is no
niche package manager. To look to them for "how it should be done" or
"one way to do it" is not a mistake in my opinion. However I am met with this

> And, frankly, I think it's a bad idea to keep a monolithic list of
> packages available for people to download anyway.

Which is little more than your opinion with no explanation or citations as to
why it is a bad idea. If it is such a bad idea, why are these major Linux
distros using it?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Christopher Faylor]

On Tue, Jun 17, 2014 at 11:29:53PM -0500, Steven Penny wrote:
>On Tue, Jun 17, 2014 at 9:29 AM, Christopher Faylor wrote:
>>On Mon, Jun 09, 2014 at 11:17:11AM -0500, Steven Penny wrote:
>>>Of course because of Cygwin flawless website...
>>>
>> If you want to experimentally figure out how many warnings you get,
>> please go right ahead and continue with this style of communication.
>
>Really, I can do without veiled threats. If you are going to ban me or whatever,
>go ahead and do it.
>
>I have brought some constructive criticisms of this project with the hope of
>starting a discussion around fixing/improving certain issues. Instead I am met
>with brick wall attitude and threats of being pushed off the mailing list.

Your idea of "constructive" is apparently flawed.  The comment that you
cut (and I just put back) illustrates what I was talking about, as does
your suggestion that I should contact Debian maintainers because I
disagreed with the idea of maintaining a package file list.

Your truculent insistence that I should knight you as the coreutils/bash
maintainer is another example.

>My suggestion has merit: Debian is no niche operating system and
>"apt-get" is no niche package manager.

You can make any suggestions you want.  Just avoid the snide asides.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Wed, Jun 18, 2014 at 1:18 AM, Christopher Faylor wrote:
> Your idea of "constructive" is apparently flawed.

This is an interesting comment, taking in that your response ignores the
original question and focuses on off topic quibbling. I will pose your viewpoint
and my question again

> And, frankly, I think it's a bad idea to keep a monolithic list of
> packages available for people to download anyway.

Do you have an explanation for this opinion, or citations as to why this is a
bad idea, taking into account that major Linux distros are already using this
very method?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Larry Hall (Cygwin)]

On 06/18/2014 02:53 AM, Steven Penny wrote:
> On Wed, Jun 18, 2014 at 1:18 AM, Christopher Faylor wrote:
>> Your idea of "constructive" is apparently flawed.
>
> This is an interesting comment, taking in that your response ignores the
> original question and focuses on off topic quibbling. I will pose your viewpoint
> and my question again

I also find it interesting that you dismiss your "off topic quibbling"
since you were the one that was feeding the fire while Chris was
on vacation last week.

> Do you have an explanation for this opinion, or citations as to why this is a
> bad idea, taking into account that major Linux distros are already using this
> very method?

If this was the question you wanted to ask originally, this is all you
needed to add to the thread.

-- 
Larry

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Wed, Jun 18, 2014 at 7:28 AM, Larry Hall (Cygwin) wrote:
> If this was the question you wanted to ask originally, this is all you
> needed to add to the thread.

Hey Larry, thanks for your comments. However, did you have anything constructive
to add, like an answer to the question?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Christopher Faylor]

On Wed, Jun 18, 2014 at 01:53:14AM -0500, Steven Penny wrote:
>On Wed, Jun 18, 2014 at 1:18 AM, Christopher Faylor wrote:
>> Your idea of "constructive" is apparently flawed.
>
>This is an interesting comment, taking in that your response ignores the
>original question and focuses on off topic quibbling. I will pose your viewpoint
>and my question again

I was actually staying on-topic with what I wanted to address: namely
the thin ice that you are walking with your posting style.

>>And, frankly, I think it's a bad idea to keep a monolithic list of
>>packages available for people to download anyway.
>
>Do you have an explanation for this opinion, or citations as to why
>this is a bad idea, taking into account that major Linux distros are
>already using this very method?

Frankly, I don't have to justify my opinion.  I understand how the
package generation would have to work and don't see a major benefit to
introducing a package list file given the amount of work and the fact
that I would have to do the work.  I also don't want to spend my time
debating with you about why it is or isn't a good idea.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Christopher Faylor]

On Wed, Jun 18, 2014 at 09:24:13AM -0500, Steven Penny wrote:
>On Wed, Jun 18, 2014 at 7:28 AM, Larry Hall (Cygwin) wrote:
>>If this was the question you wanted to ask originally, this is all you
>>needed to add to the thread.
>
>Hey Larry, thanks for your comments.  However, did you have anything
>constructive to add, like an answer to the question?

Although we are friends, there is no reason to expect Larry to be
able to explain my opinions.

Please move on.  This isn't going to happen.

Or take it to cygwin-talk.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

In June I presented the idea of a "Package file lists archive" and was met with
this

> I understand how the package generation would have to work and don't see a
> major benefit to introducing a package list file given the amount of work and
> the fact that I would have to do the work.

http://cygwin.com/ml/cygwin/2014-06/msg00276.html

However Christopher has since left the project

> In the last several months I've told various key players in the Cygwin project
> that I was thinking about withdrawing from the project. Today is the day that
> this becomes official.

http://cygwin.com/ml/cygwin/2014-07/msg00392.html

I would like to revisit this issue. I would like to see if I can help in any way
to make this happen. My work on "apt-cyg" has been hindered by the issue with
"cygcheck --package-query"

http://github.com/transcode-open/apt-cyg/issues/21

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Yaakov Selkowitz]

On Sun, 2014-08-03 at 16:07 -0500, Steven Penny wrote:
> In June I presented the idea of a "Package file lists archive" and was met with
> this
> 
> > I understand how the package generation would have to work and don't see a
> > major benefit to introducing a package list file given the amount of work and
> > the fact that I would have to do the work.
> 
> http://cygwin.com/ml/cygwin/2014-06/msg00276.html
> 
> However Christopher has since left the project

Please don't assume that this means that we're ready to automatically
rehash every decision he made over the last however-many years.

> I would like to revisit this issue. I would like to see if I can help in any way
> to make this happen. My work on "apt-cyg" has been hindered by the issue with
> "cygcheck --package-query"

package_grep.cgi could probably use some optimization; PTC.


Yaakov



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Sun, Aug 3, 2014 at 10:00 PM, Yaakov Selkowitz wrote:
> package_grep.cgi could probably use some optimization; PTC.

No, the whole point of this thread is to kill "package_grep.cgi" in favor of a
master file list, like is used with

- Debian
- Ubuntu
- Arch Linux
- MSYS2
- apt-get
- pacman

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Sun, Aug 3, 2014 at 10:00 PM, Yaakov Selkowitz wrote:
> Please don't assume that this means that we're ready to automatically
> rehash every decision he made over the last however-many years.

I will add that the problem with "cygcheck" still exists, it has been an issue
since at least May when I started tracking it

    $ date; time cygcheck -p bin/aclocal
    Thu Aug  7 12:07:27 CDT 2014
    cygcheck: unable to contact cygwin.com site, InternetOpenUrl() failed:
     The operation timed out
     (win32 error 12002)

    real    0m29.172s
    user    0m0.000s
    sys     0m0.015s

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Larry Hall (Cygwin)]

On 08/07/2014 01:38 PM, Steven Penny wrote:
> On Sun, Aug 3, 2014 at 10:00 PM, Yaakov Selkowitz wrote:
>> Please don't assume that this means that we're ready to automatically
>> rehash every decision he made over the last however-many years.
>
> I will add that the problem with "cygcheck" still exists, it has been an issue
> since at least May when I started tracking it
>
>      $ date; time cygcheck -p bin/aclocal
>      Thu Aug  7 12:07:27 CDT 2014
>      cygcheck: unable to contact cygwin.com site, InternetOpenUrl() failed:
>       The operation timed out
>       (win32 error 12002)

The above command works fine for me.  Have you tried this on a different
machine or though a different network provider?


-- 
Larry

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Thu, Aug 7, 2014 at 12:49 PM, Larry Hall (Cygwin) wrote:
> The above command works fine for me.  Have you tried this on a different
> machine or though a different network provider?

It is an intermittent problem. I will be having timeouts with Cygwin when other
websites will be working perfectly fine. This is the typical situation

- try "cygcheck -p", timeout
- try again, will work after 15 seconds
- try again, will work after 7 seconds
- try again, will work after 1 second, as expected

After that all will be normal, until I try it again several hours later and the
process just repeats.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Larry Hall (Cygwin)]

On 08/07/2014 05:04 PM, Steven Penny wrote:
> On Thu, Aug 7, 2014 at 12:49 PM, Larry Hall (Cygwin) wrote:
>> The above command works fine for me.  Have you tried this on a different
>> machine or though a different network provider?
>
> It is an intermittent problem. I will be having timeouts with Cygwin when other
> websites will be working perfectly fine. This is the typical situation
>
> - try "cygcheck -p", timeout
> - try again, will work after 15 seconds
> - try again, will work after 7 seconds
> - try again, will work after 1 second, as expected
>
> After that all will be normal, until I try it again several hours later and the
> process just repeats.

Interesting.  OK, I've been running it in a loop for the last half-hour and
haven't seen any timeouts.  I suppose there could be a difference in local
configuration if you're still seeing the issue.  If you think that could be
coming into play, there may be some benefit to posting your cygcheck output
to see if that suggests something.

-- 
Larry

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Andrey Repin]

Greetings, Larry Hall (Cygwin)!

>>> The above command works fine for me.  Have you tried this on a different
>>> machine or though a different network provider?
>>
>> It is an intermittent problem. I will be having timeouts with Cygwin when other
>> websites will be working perfectly fine. This is the typical situation
>>
>> - try "cygcheck -p", timeout
>> - try again, will work after 15 seconds
>> - try again, will work after 7 seconds
>> - try again, will work after 1 second, as expected
>>
>> After that all will be normal, until I try it again several hours later and the
>> process just repeats.

> Interesting.  OK, I've been running it in a loop for the last half-hour and
> haven't seen any timeouts.  I suppose there could be a difference in local
> configuration if you're still seeing the issue.  If you think that could be
> coming into play, there may be some benefit to posting your cygcheck output
> to see if that suggests something.

There's little to see in Cygcheck, when you have flaky DNS servers, that
unable to respond timely.


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 08.08.2014, <02:43>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Thu, Aug 7, 2014 at 5:44 PM, Andrey Repin wrote:
> There's little to see in Cygcheck, when you have flaky DNS servers, that
> unable to respond timely.

As I have already said, when I am having problems with "cygcheck" other websites
will be working perfectly fine.

I have noticed a pattern of you posting content with little or no value. Please
going forward do not post any replys to my topics.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Andrey Repin]

Greetings, Steven Penny!

>> There's little to see in Cygcheck, when you have flaky DNS servers, that
>> unable to respond timely.

> As I have already said, when I am having problems with "cygcheck" other websites
> will be working perfectly fine.

"Other websites" probably requested by alot of other users of your DNS server,
and never leave cache.
Either way, without seeing in first person, what's going on, it's all just a
guess.

> I have noticed a pattern of you posting content with little or no value. Please
> going forward do not post any replys to my topics.

Check the image in your mirror, please.


--
WBR,
Andrey Repin (anrdaemon@yandex.ru) 08.08.2014, <05:50>

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Thu, Aug 7, 2014 at 9:01 PM, Andrey Repin wrote:
> Check the image in your mirror, please.

I see your listening skills are on par with your posting skills.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

So, to respond to the DNS allegations, I have this script

    url='http://209.132.180.131/cgi-bin2/package-grep.cgi'
    qs='text=1&arch=x86_64&grep=aclocal'
    while :
    do
      time wget --spider --timeout 30 "$url?$qs"
    done

Here are results of first 10 iterations

    real    0m16.957s
    real    0m17.425s
    real    0m17.160s
    real    0m17.004s
    real    0m16.895s
    real    0m17.207s
    real    0m16.910s
    real    0m16.770s
    real    0m17.316s
    real    0m16.770s

and here is detail on 11th iteration

    Spider mode enabled. Check if remote file exists.
    --2014-08-07 22:06:49--
    http://209.132.180.131/cgi-bin2/package-grep.cgi?
    text=1&arch=x86_64&grep=aclocal
    Connecting to 209.132.180.131:80... connected.
    HTTP request sent, awaiting response...
    Read error (Connection timed out) in headers. Retrying.

    Spider mode enabled. Check if remote file exists.
    --2014-08-07 22:07:20-- (try: 2)
    http://209.132.180.131/cgi-bin2/package-grep.cgi?
    text=1&arch=x86_64&grep=aclocal
    Connecting to 209.132.180.131:80... connected.
    HTTP request sent, awaiting response...
    Read error (Connection timed out) in headers. Retrying.

    Spider mode enabled. Check if remote file exists.
    --2014-08-07 22:07:52-- (try: 3)
    http://209.132.180.131/cgi-bin2/package-grep.cgi?
    text=1&arch=x86_64&grep=aclocal
    Connecting to 209.132.180.131:80... connected.
    HTTP request sent, awaiting response...
    Read error (Connection timed out) in headers. Retrying.

    Spider mode enabled. Check if remote file exists.
    --2014-08-07 22:08:25-- (try: 4)
    http://209.132.180.131/cgi-bin2/package-grep.cgi?
    text=1&arch=x86_64&grep=aclocal
    Connecting to 209.132.180.131:80... connected.
    HTTP request sent, awaiting response...
    Read error (Connection timed out) in headers. Retrying.

    Spider mode enabled. Check if remote file exists.
    --2014-08-07 22:08:59-- (try: 5)
    http://209.132.180.131/cgi-bin2/package-grep.cgi?
    text=1&arch=x86_64&grep=aclocal
    Connecting to 209.132.180.131:80... connected.
    HTTP request sent, awaiting response...
    Read error (Connection timed out) in headers. Retrying.

    Spider mode enabled. Check if remote file exists.
    --2014-08-07 22:09:34--  (try: 6)
    http://209.132.180.131/cgi-bin2/package-grep.cgi?
    text=1&arch=x86_64&grep=aclocal
    Connecting to 209.132.180.131:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: unspecified [text/plain]
    Remote file exists.

    real    3m3.128s
    user    0m0.000s
    sys     0m0.045s

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Yaakov Selkowitz]

On Thu, 2014-08-07 at 20:30 -0500, Steven Penny wrote:
> I have noticed a pattern of you posting content with little or no value. Please
> going forward do not post any replys to my topics.

This is out of line.  This is a community forum, and you are in no
position to dictate who says what here.  We will not tolerate any
further attacks of this kind.

If you wish to continue participating in this forum, then cease these
attacks immediately.


Yaakov



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Steven Penny]

On Thu, Aug 7, 2014 at 10:31 PM, Yaakov Selkowitz wrote:
> If you wish to continue participating in this forum, then cease these
> attacks immediately.

Thank you for the reply, I will heed this going forward. However, do you have
any input regarding the topic?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Package file lists archive

[Yaakov Selkowitz]

On Thu, 2014-08-07 at 22:59 -0500, Steven Penny wrote:
> On Thu, Aug 7, 2014 at 10:31 PM, Yaakov Selkowitz wrote:
> > If you wish to continue participating in this forum, then cease these
> > attacks immediately.
> 
> Thank you for the reply, I will heed this going forward. 

Thank you.

> However, do you have any input regarding the topic?

It is somewhat of a known issue; cygcheck -p sometimes times out for me
too, and AFAIK the cause is package-grep.cgi being too slow.  Some of
that might depend on how busy sourceware is at that moment; the fact
that repeat attempts are faster makes me think that is a result of
caching.

Given that we're still working on getting all the infrastructure tasks
under control, optimizing package-grep.cgi would be the only short term
solution; once again, PTC.

In the long term, if were going to drastically change the package
database, IMO it should be in a way that would be used by the website,
cygcheck, and setup*.exe.  A sqlite database comes to mind (as yum uses
one); there are already mingw*-sqlite packages in Ports, and the website
could use DBI and DBD::sqlite.  This is *really* in the realm of PTC
though.


Yaakov


 


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple