* setup @ 2015-05-29 19:37 Achim Gratz 2015-05-31 10:24 ` setup Corinna Vinschen 2015-06-06 20:58 ` setup Achim Gratz 0 siblings, 2 replies; 125+ messages in thread From: Achim Gratz @ 2015-05-29 19:37 UTC (permalink / raw) To: cygwin-apps The new SHA512 checksums are rather lengthy. Could we switch them to Base64 (perhaps the URL and file safe variant) instead of the current hex encoding instead (maybe with an SHA512: prefix if we want to support both)? Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-05-29 19:37 setup Achim Gratz @ 2015-05-31 10:24 ` Corinna Vinschen 2015-06-01 18:11 ` setup Achim Gratz 2015-06-06 20:58 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-05-31 10:24 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 541 bytes --] On May 29 21:37, Achim Gratz wrote: > > The new SHA512 checksums are rather lengthy. Could we switch them to > Base64 (perhaps the URL and file safe variant) instead of the current > hex encoding instead (maybe with an SHA512: prefix if we want to support > both)? Not for the time being. Sombody would have to add code to setup as well as upset to make this work. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-05-31 10:24 ` setup Corinna Vinschen @ 2015-06-01 18:11 ` Achim Gratz 2015-06-01 18:34 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-01 18:11 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > On May 29 21:37, Achim Gratz wrote: >> >> The new SHA512 checksums are rather lengthy. Could we switch them to >> Base64 (perhaps the URL and file safe variant) instead of the current >> hex encoding instead (maybe with an SHA512: prefix if we want to support >> both)? > > Not for the time being. Sombody would have to add code to setup as well > as upset to make this work. Since it seems you plan to use libcrypto from OpenSSL anyway: https://gist.github.com/barrysteyn/7308212#file-base64decode-c Otherwise I'd prefer something less obtuse, there is C++ code with compatible license or I could roll my own. For upset, I don't know what interface you're using. For the object interface you'd simply switch from hexdigest() to b64digest() and for the functional interface from sha512_hex() to sha512_base64(). Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-01 18:11 ` setup Achim Gratz @ 2015-06-01 18:34 ` Corinna Vinschen 2015-06-01 19:49 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-01 18:34 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1572 bytes --] On Jun 1 20:11, Achim Gratz wrote: > Corinna Vinschen writes: > > On May 29 21:37, Achim Gratz wrote: > >> > >> The new SHA512 checksums are rather lengthy. Could we switch them to > >> Base64 (perhaps the URL and file safe variant) instead of the current > >> hex encoding instead (maybe with an SHA512: prefix if we want to support > >> both)? > > > > Not for the time being. Sombody would have to add code to setup as well > > as upset to make this work. > > Since it seems you plan to use libcrypto from OpenSSL anyway: > https://gist.github.com/barrysteyn/7308212#file-base64decode-c I don't. Why do you think so? > Otherwise I'd prefer something less obtuse, there is C++ code with > compatible license or I could roll my own. > > For upset, I don't know what interface you're using. For the object > interface you'd simply switch from hexdigest() to b64digest() and for > the functional interface from sha512_hex() to sha512_base64(). I'm not a perl person. Upset is using some kind of use Digest::SHA; [...] my $ctx = Digest::SHA->new(512); [...] } elsif ($f eq "$d0/sha512.sum") { This was a drop-in replacement for Digest::MD5. Anything else I'll let somebody do who better knows perl. Apart from this, the sah512.sum files are not only generated for the Cygwin release dir, they are generated on the sourceware ftp area system-wide. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-01 18:34 ` setup Corinna Vinschen @ 2015-06-01 19:49 ` Achim Gratz 2015-06-01 21:06 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-01 19:49 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: >> Since it seems you plan to use libcrypto from OpenSSL anyway: >> https://gist.github.com/barrysteyn/7308212#file-base64decode-c > > I don't. Why do you think so? Because of the openssl branch in Git. > This was a drop-in replacement for Digest::MD5. Anything else I'll > let somebody do who better knows perl. THen the call $ctx->hexdigest() would need to be replaced by $ctx->b64digest(). > Apart from this, the sah512.sum files are not only generated for the > Cygwin release dir, they are generated on the sourceware ftp area > system-wide. I'm only talking about SHA512 in the setup.ini files, though. The sha512.sum files have no bearing on that. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-01 19:49 ` setup Achim Gratz @ 2015-06-01 21:06 ` Corinna Vinschen 2015-06-02 5:17 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-01 21:06 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1310 bytes --] On Jun 1 21:49, Achim Gratz wrote: > Corinna Vinschen writes: > >> Since it seems you plan to use libcrypto from OpenSSL anyway: > >> https://gist.github.com/barrysteyn/7308212#file-base64decode-c > > > > I don't. Why do you think so? > > Because of the openssl branch in Git. Oh, that was a bad idea. I won't remove the branch for historical reasons, but we don't really want to link agaionst OpenSSL when we already link against gcrypt. > > This was a drop-in replacement for Digest::MD5. Anything else I'll > > let somebody do who better knows perl. > > THen the call $ctx->hexdigest() would need to be replaced by > $ctx->b64digest(). No, it would need to support this additionally. > > Apart from this, the sah512.sum files are not only generated for the > > Cygwin release dir, they are generated on the sourceware ftp area > > system-wide. > > I'm only talking about SHA512 in the setup.ini files, though. The > sha512.sum files have no bearing on that. So what exactly is the problem with the longer checksums? I don't see any problem here, especially when the file is being compressed anyway. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-01 21:06 ` setup Corinna Vinschen @ 2015-06-02 5:17 ` Achim Gratz 2015-06-02 8:29 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-02 5:17 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: >> Because of the openssl branch in Git. > > Oh, that was a bad idea. I won't remove the branch for historical > reasons, but we don't really want to link agaionst OpenSSL when we > already link against gcrypt. OK, makes sense. >> > This was a drop-in replacement for Digest::MD5. Anything else I'll >> > let somebody do who better knows perl. >> >> THen the call $ctx->hexdigest() would need to be replaced by >> $ctx->b64digest(). > > No, it would need to support this additionally. Why? In any case it'd be easy enough to make it switchable. >> > Apart from this, the sah512.sum files are not only generated for the >> > Cygwin release dir, they are generated on the sourceware ftp area >> > system-wide. >> >> I'm only talking about SHA512 in the setup.ini files, though. The >> sha512.sum files have no bearing on that. > > So what exactly is the problem with the longer checksums? I don't > see any problem here, especially when the file is being compressed > anyway. I'm not concerned about the size (although I note that even if compression recognized it's looking at an SHA512 it would at best be able to compress it down to 64 bytes anyway). The difference between 128 and 86 (or 88 padded) characters is pretty noticeable on screen. It's probably a bit unusual that I look at setup files so often (since I generate my own), but it still disturbes me. The other thing (but that really is unrelated, as I could take care of it on other ways) is that I still plan to move to a repository format that allows multiple versions to be active at the same time (via different setup.ini files pointing into the same repo hierarchy) and I was thinking of using the SHA512 to do directory balancing like it's done for the object directory in Git. ~ (2001) du -sh /mnt/mirror/cygwin 52G /mnt/mirror/cygwin ~ (2002) find /mnt/mirror/cygwin -type f |wc 44778 44778 3340383 ~ (2003) find /mnt/mirror/cygwin -type d | wc 11079 11079 602401 ~ (2004) ls -d /mnt/mirror/cygwin/x86/release/* | wc 2071 2071 89831 Using a 6bit key into a two level tree would seem to provide a better balance for this structure than 8bit or I could go single level with a 12bit key. It needs to provide optimum performance over network, so the optimum likely is at a different point than a local file system. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-02 5:17 ` setup Achim Gratz @ 2015-06-02 8:29 ` Corinna Vinschen 2015-06-02 16:24 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-02 8:29 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 3121 bytes --] On Jun 2 07:17, Achim Gratz wrote: > Corinna Vinschen writes: > >> Because of the openssl branch in Git. > > > > Oh, that was a bad idea. I won't remove the branch for historical > > reasons, but we don't really want to link agaionst OpenSSL when we > > already link against gcrypt. > > OK, makes sense. > > >> > This was a drop-in replacement for Digest::MD5. Anything else I'll > >> > let somebody do who better knows perl. > >> > >> THen the call $ctx->hexdigest() would need to be replaced by > >> $ctx->b64digest(). > > > > No, it would need to support this additionally. > > Why? In any case it'd be easy enough to make it switchable. Transition period. We should do changes like that in two steps, first updating to a setup which handles the new checksums, then changing the generation of checksums to the new method. Like we did with MD5->SHA512. It's smoother that way. It's also certainly not a bad idea to continue supporting the older checksum methods. You're very likely not the only person creating his/her own setup.ini files and every change may break a script elsewhere :) > >> > Apart from this, the sah512.sum files are not only generated for the > >> > Cygwin release dir, they are generated on the sourceware ftp area > >> > system-wide. > >> > >> I'm only talking about SHA512 in the setup.ini files, though. The > >> sha512.sum files have no bearing on that. > > > > So what exactly is the problem with the longer checksums? I don't > > see any problem here, especially when the file is being compressed > > anyway. > > I'm not concerned about the size (although I note that even if > compression recognized it's looking at an SHA512 it would at best be > able to compress it down to 64 bytes anyway). The difference between > 128 and 86 (or 88 padded) characters is pretty noticeable on screen. > It's probably a bit unusual that I look at setup files so often (since I > generate my own), You seem to be doing this on a regular basis with your own scripts. Is that right? Would you think your own method is just hacked to work, or do you think your own ini creation scripts are clean enough to release them to the public? I'm asking because, right now, we're relying on a convoluted perl script set which is hard to understand (at least for non-perl guys), is missing comments, has no maintainer, and above all, has a questionable license. Upset is a beast. It handles ini file creation as well as creating the package information for https://cygwin.com/cgi-bin2/package-grep.cgi, as well as the package upload post-processing. I would very much appreciate if we could split these tasks into separate, independent scripts or tools under GPL or BSD license. So, what about your scripts? Do you think they could be used as a start? Do you have, perhaps, fun to rewrite the upset functionality in a maintainable form and *gasp* maintain it? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-02 8:29 ` setup Corinna Vinschen @ 2015-06-02 16:24 ` Achim Gratz 2015-06-03 8:19 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-02 16:24 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: >> Why? In any case it'd be easy enough to make it switchable. > > Transition period. We should do changes like that in two steps, first > updating to a setup which handles the new checksums, then changing the > generation of checksums to the new method. Like we did with > MD5->SHA512. It's smoother that way. It's also certainly not a bad > idea to continue supporting the older checksum methods. You're very > likely not the only person creating his/her own setup.ini files and > every change may break a script elsewhere :) Yes, but in this case you're really talking about the setup.exe side, not the generation of the setup via upset or genini (which I thought you were talking about, since that's the part implemented in Perl). If we use the SHA512: prefix then both hex and base64 encoded checksum are distinguishable easily and we might even throw in MD5 just for fun. I don't see off-hand why either upset or genini should be capable of emitting both styles into the same setup.ini, but it is easy enough to do this per setup.ini file creation by adding an option to specify which digest you want. > You seem to be doing this on a regular basis with your own scripts. Is > that right? Would you think your own method is just hacked to work, or > do you think your own ini creation scripts are clean enough to release > them to the public? No, it's fairly sophisticated already, but not yet feature-complete and requires a few clean-ups (don't they always?). > I'm asking because, right now, we're relying on a convoluted perl script > set which is hard to understand (at least for non-perl guys), is missing > comments, has no maintainer, and above all, has a questionable license. What I have is also mostly a Perl script plus a few CMD files around it, so that part likely wouldn't change. > Upset is a beast. It handles ini file creation as well as creating the > package information for https://cygwin.com/cgi-bin2/package-grep.cgi, > as well as the package upload post-processing. Since I can't even look at upset, I use genini for the initial setup.ini creation for any local packages (I have many of those). The rest of the process is taking multiple setup.ini files and rewrites them into a new one, then installs the necessary files into a staging area, which can optionally be cleaned of files not being referenced anymore after an update or change of packages to install. The staging area is then replicated onto the real installation directory. This is under control of a setup.conf file that looks more or less exactly like the name implies. You define your package sources and their dependency order (should they provide the same package, which is what happens for packages that I carry local patches for), and then list the packages you want to have available (you can inject new group names so it's easier to install via setup.exe). Dependencies are pulled in automatically and you can specify that you want an experimental or previous package instead of current. It can also automatically pull devel and debug packages into groups so that they do not clutter the install for non-developer installs and are by default available on those install that are for developers. You can also blacklist a package in some package source if you want to reverse the order just for one package or if there's a packaging error in an external package source. > I would very much appreciate if we could split these tasks into separate, > independent scripts or tools under GPL or BSD license. Well, if we keep this as a task for Perl it would make most sense to put those things into Perl modules, one per task. If a standalone script is preferred in some place we can always fatpack those modules again. > So, what about your scripts? Do you think they could be used as a > start? Do you have, perhaps, fun to rewrite the upset functionality in > a maintainable form and *gasp* maintain it? As I said, I'm coming at this from a different angle: the need to integrate multiple package sources and make a new install hierarchy that can be used locally, without being connected to the Internet and without carrying all the baggage around that a full mirror would have. Plus, I need the install to be fully scriptable to install for users that have no admin rights. The features that I'm missing is explicit version pinning (unfortunately I'll have to do some extensive changes under the hood to get rid of the current prev-curr-test structure, didn't think of that early enough) and the ability to keep different versions of an installation around. I sort of can do that already if I'm careful enough not to clean away files that are still needed, but it's too fragile even for very sparse usage, so I use the staging directory for release tests at the moment. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-02 16:24 ` setup Achim Gratz @ 2015-06-03 8:19 ` Corinna Vinschen 2015-06-03 8:31 ` setup Marco Atzeri ` (2 more replies) 0 siblings, 3 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-06-03 8:19 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 3079 bytes --] On Jun 2 18:24, Achim Gratz wrote: > Corinna Vinschen writes: > >> Why? In any case it'd be easy enough to make it switchable. > > > > Transition period. We should do changes like that in two steps, first > > updating to a setup which handles the new checksums, then changing the > > generation of checksums to the new method. Like we did with > > MD5->SHA512. It's smoother that way. It's also certainly not a bad > > idea to continue supporting the older checksum methods. You're very > > likely not the only person creating his/her own setup.ini files and > > every change may break a script elsewhere :) > > Yes, but in this case you're really talking about the setup.exe side, Yes, sure. > not the generation of the setup via upset or genini (which I thought you > were talking about, since that's the part implemented in Perl). If we > use the SHA512: prefix then both hex and base64 encoded checksum are > distinguishable easily and we might even throw in MD5 just for fun. There's no reason for upset to create more than one style of checksum. MD5, SHA512, SHA512/base64, whatever. What I'm talking about is the client side. The setup tool should understand all types of checksums used. Right now it identifies MD5 vs. SHA512 just by the length of the checksum. > > You seem to be doing this on a regular basis with your own scripts. Is > > that right? Would you think your own method is just hacked to work, or > > do you think your own ini creation scripts are clean enough to release > > them to the public? > > No, it's fairly sophisticated already, but not yet feature-complete and > requires a few clean-ups (don't they always?). > > > I'm asking because, right now, we're relying on a convoluted perl script > > set which is hard to understand (at least for non-perl guys), is missing > > comments, has no maintainer, and above all, has a questionable license. > > What I have is also mostly a Perl script plus a few CMD files around it, > so that part likely wouldn't change. CMD is out of the question, this running on a Linux box. > > Upset is a beast. It handles ini file creation as well as creating the > > package information for https://cygwin.com/cgi-bin2/package-grep.cgi, > > as well as the package upload post-processing. > > Since I can't even look at upset, I use genini for the initial setup.ini I forgot that the upset repo has been hidden from the public for a long time, and these days there isn't even a repo since it has been deleted :-P If you're willing to take a look into these upset perl scripts, please apply for a normal ssh account on sourceware.org via https://sourceware.org/cgi-bin/pdw/ps_form.cgi Use my private email address as approver. And make sure to use *another* public SSH key for shell access than the one you're using for Cygwin uploads to avoid confusion. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-03 8:19 ` setup Corinna Vinschen @ 2015-06-03 8:31 ` Marco Atzeri 2015-06-03 11:07 ` setup Corinna Vinschen 2015-06-03 16:51 ` setup Achim Gratz 2015-06-03 17:04 ` setup Yaakov Selkowitz 2 siblings, 1 reply; 125+ messages in thread From: Marco Atzeri @ 2015-06-03 8:31 UTC (permalink / raw) To: cygwin-apps On 6/3/2015 10:19 AM, Corinna Vinschen wrote: > > If you're willing to take a look into these upset perl scripts, please > apply for a normal ssh account on sourceware.org via > > https://sourceware.org/cgi-bin/pdw/ps_form.cgi > > Use my private email address as approver. And make sure to use > *another* public SSH key for shell access than the one you're using > for Cygwin uploads to avoid confusion. Why ? I am using the same KEY for both and never had an issue. > Thanks, > Corinna Regards Marco ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-03 8:31 ` setup Marco Atzeri @ 2015-06-03 11:07 ` Corinna Vinschen 2015-06-03 12:45 ` setup Eric Blake 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-03 11:07 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 904 bytes --] On Jun 3 10:31, Marco Atzeri wrote: > On 6/3/2015 10:19 AM, Corinna Vinschen wrote: > > > > >If you're willing to take a look into these upset perl scripts, please > >apply for a normal ssh account on sourceware.org via > > > > https://sourceware.org/cgi-bin/pdw/ps_form.cgi > > > >Use my private email address as approver. And make sure to use > >*another* public SSH key for shell access than the one you're using > >for Cygwin uploads to avoid confusion. > > Why ? > I am using the same KEY for both and never had an issue. I had problems with the home directory, but maybe that was just me. I think Eric had some problem as well, but maybe that's just a problem for the cygwin with admin rights, I'm not sure. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-03 11:07 ` setup Corinna Vinschen @ 2015-06-03 12:45 ` Eric Blake 0 siblings, 0 replies; 125+ messages in thread From: Eric Blake @ 2015-06-03 12:45 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1251 bytes --] On 06/03/2015 05:07 AM, Corinna Vinschen wrote: > On Jun 3 10:31, Marco Atzeri wrote: >> On 6/3/2015 10:19 AM, Corinna Vinschen wrote: >> >>> >>> If you're willing to take a look into these upset perl scripts, please >>> apply for a normal ssh account on sourceware.org via >>> >>> https://sourceware.org/cgi-bin/pdw/ps_form.cgi >>> >>> Use my private email address as approver. And make sure to use >>> *another* public SSH key for shell access than the one you're using >>> for Cygwin uploads to avoid confusion. >> >> Why ? >> I am using the same KEY for both and never had an issue. > > I had problems with the home directory, but maybe that was just me. I > think Eric had some problem as well, but maybe that's just a problem for > the cygwin with admin rights, I'm not sure. Same key for both personal login and package upload is fine. The only time two keys are needed is if you are granted admin rights as the 'cygwin' user. In that case, package upload and admin login need different keys, but it is then up to you whether your personal login shares with the upload key or with the admin access. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 604 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-03 8:19 ` setup Corinna Vinschen 2015-06-03 8:31 ` setup Marco Atzeri @ 2015-06-03 16:51 ` Achim Gratz 2015-06-03 17:04 ` setup Yaakov Selkowitz 2 siblings, 0 replies; 125+ messages in thread From: Achim Gratz @ 2015-06-03 16:51 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > There's no reason for upset to create more than one style of checksum. > MD5, SHA512, SHA512/base64, whatever. What I'm talking about is the > client side. The setup tool should understand all types of checksums > used. Right now it identifies MD5 vs. SHA512 just by the length of > the checksum. OK, that's easy, then. Just check for 86 and 88 characters for the non-padded and padded variant of the base64 digest. > CMD is out of the question, this running on a Linux box. The scripts just set up the environment and then call to Perl. It's really a lot easier to do in (ba)sh than in cmd, so that doesn't need to worry you. Again, the origin of those scripts was a different task, so I'd like to spend some time on discussion of the feature set you're envisioning for this. > I forgot that the upset repo has been hidden from the public for a long > time, and these days there isn't even a repo since it has been deleted :-P > > If you're willing to take a look into these upset perl scripts, please > apply for a normal ssh account on sourceware.org via > > https://sourceware.org/cgi-bin/pdw/ps_form.cgi > > Use my private email address as approver. And make sure to use > *another* public SSH key for shell access than the one you're using > for Cygwin uploads to avoid confusion. I'll do that, but expect no comment before Perl 5.22 gets out the door. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf microQ V2.22R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-03 8:19 ` setup Corinna Vinschen 2015-06-03 8:31 ` setup Marco Atzeri 2015-06-03 16:51 ` setup Achim Gratz @ 2015-06-03 17:04 ` Yaakov Selkowitz 2015-06-03 17:11 ` setup Achim Gratz 2 siblings, 1 reply; 125+ messages in thread From: Yaakov Selkowitz @ 2015-06-03 17:04 UTC (permalink / raw) To: cygwin-apps On Wed, 2015-06-03 at 10:19 +0200, Corinna Vinschen wrote: > On Jun 2 18:24, Achim Gratz wrote: > > Corinna Vinschen writes: > > > Upset is a beast. It handles ini file creation as well as creating the > > > package information for https://cygwin.com/cgi-bin2/package-grep.cgi, > > > as well as the package upload post-processing. > > > > Since I can't even look at upset, I use genini for the initial setup.ini > > I forgot that the upset repo has been hidden from the public for a long > time, and these days there isn't even a repo since it has been deleted :-P > > If you're willing to take a look into these upset perl scripts, Anyone looking at the sources would be tainted thereby. Because we are concerned about the licensing of upset, AFAICS we are going to have to clean-room reverse-engineer this. -- Yaakov ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-03 17:04 ` setup Yaakov Selkowitz @ 2015-06-03 17:11 ` Achim Gratz 2015-06-08 12:56 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-03 17:11 UTC (permalink / raw) To: cygwin-apps Yaakov Selkowitz writes: > Anyone looking at the sources would be tainted thereby. Because we are > concerned about the licensing of upset, AFAICS we are going to have to > clean-room reverse-engineer this. So what exactly does upset that genini doesn't when it comes to generating setup.ini and what other functions does it do in the maintenance of the download area (which I assume is the part where it gets hairy)? Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-03 17:11 ` setup Achim Gratz @ 2015-06-08 12:56 ` Corinna Vinschen 2015-06-08 13:49 ` setup Corinna Vinschen 2015-06-08 18:52 ` setup Achim Gratz 0 siblings, 2 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-06-08 12:56 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 866 bytes --] On Jun 3 19:11, Achim Gratz wrote: > Yaakov Selkowitz writes: > > Anyone looking at the sources would be tainted thereby. Because we are > > concerned about the licensing of upset, AFAICS we are going to have to > > clean-room reverse-engineer this. > > So what exactly does upset that genini doesn't when it comes to > generating setup.ini and what other functions does it do in the > maintenance of the download area (which I assume is the part where it > gets hairy)? Basically upset generates setup.ini and setup.bz2 files and a makefile signs the files. But additionally upset also creates the data for the package search and it does the uploading from the package staging area. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 12:56 ` setup Corinna Vinschen @ 2015-06-08 13:49 ` Corinna Vinschen 2015-06-08 18:34 ` setup Yaakov Selkowitz 2015-06-08 18:52 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-08 13:49 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1302 bytes --] On Jun 8 14:56, Corinna Vinschen wrote: > On Jun 3 19:11, Achim Gratz wrote: > > Yaakov Selkowitz writes: > > > Anyone looking at the sources would be tainted thereby. Because we are > > > concerned about the licensing of upset, AFAICS we are going to have to > > > clean-room reverse-engineer this. > > > > So what exactly does upset that genini doesn't when it comes to > > generating setup.ini and what other functions does it do in the > > maintenance of the download area (which I assume is the part where it > > gets hairy)? > > Basically upset generates setup.ini and setup.bz2 files and a makefile > signs the files. But additionally upset also creates the data for the > package search and it does the uploading from the package staging area. ...it also handles the "autodep" and "incver_ifdep" stuff in setup.hint. "autodep" (and "noautodep") seems to have a bug, though, as we discussed a while back. It seems to miss adding some of the required dependencies. "incver_ifdep" is still used in _update-info-dir unfortunately. It would be nice to make this a perpetual postinstall script instead. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 13:49 ` setup Corinna Vinschen @ 2015-06-08 18:34 ` Yaakov Selkowitz 2015-06-08 18:47 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Yaakov Selkowitz @ 2015-06-08 18:34 UTC (permalink / raw) To: cygwin-apps On Mon, 2015-06-08 at 15:49 +0200, Corinna Vinschen wrote: > On Jun 8 14:56, Corinna Vinschen wrote: > > On Jun 3 19:11, Achim Gratz wrote: > > > Yaakov Selkowitz writes: > > > > Anyone looking at the sources would be tainted thereby. Because we are > > > > concerned about the licensing of upset, AFAICS we are going to have to > > > > clean-room reverse-engineer this. > > > > > > So what exactly does upset that genini doesn't when it comes to > > > generating setup.ini and what other functions does it do in the > > > maintenance of the download area (which I assume is the part where it > > > gets hairy)? > > > > Basically upset generates setup.ini and setup.bz2 files and a makefile > > signs the files. But additionally upset also creates the data for the > > package search and it does the uploading from the package staging area. > > ...it also handles the "autodep" and "incver_ifdep" stuff in setup.hint. > > "autodep" (and "noautodep") seems to have a bug, though, as we discussed > a while back. It seems to miss adding some of the required dependencies. Do we really need autodep anymore? > "incver_ifdep" is still used in _update-info-dir unfortunately. It would > be nice to make this a perpetual postinstall script instead. Ack. -- Yaakov ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 18:34 ` setup Yaakov Selkowitz @ 2015-06-08 18:47 ` Corinna Vinschen 2015-06-08 19:05 ` setup Yaakov Selkowitz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-08 18:47 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1737 bytes --] On Jun 8 13:34, Yaakov Selkowitz wrote: > On Mon, 2015-06-08 at 15:49 +0200, Corinna Vinschen wrote: > > On Jun 8 14:56, Corinna Vinschen wrote: > > > On Jun 3 19:11, Achim Gratz wrote: > > > > Yaakov Selkowitz writes: > > > > > Anyone looking at the sources would be tainted thereby. Because we are > > > > > concerned about the licensing of upset, AFAICS we are going to have to > > > > > clean-room reverse-engineer this. > > > > > > > > So what exactly does upset that genini doesn't when it comes to > > > > generating setup.ini and what other functions does it do in the > > > > maintenance of the download area (which I assume is the part where it > > > > gets hairy)? > > > > > > Basically upset generates setup.ini and setup.bz2 files and a makefile > > > signs the files. But additionally upset also creates the data for the > > > package search and it does the uploading from the package staging area. > > > > ...it also handles the "autodep" and "incver_ifdep" stuff in setup.hint. > > > > "autodep" (and "noautodep") seems to have a bug, though, as we discussed > > a while back. It seems to miss adding some of the required dependencies. > > Do we really need autodep anymore? IDK. autodep is only used to create the auto-dependency to the "cygwin" package, except for the two "noautodep" packages _update-info-dir and _autorebase. The reason, afair, was to make sure the dependency chain created by setup when (un-)installing in the right order makes sense. If you think we don't need it anymore, please explain. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 18:47 ` setup Corinna Vinschen @ 2015-06-08 19:05 ` Yaakov Selkowitz 2015-06-08 19:35 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Yaakov Selkowitz @ 2015-06-08 19:05 UTC (permalink / raw) To: cygwin-apps On Mon, 2015-06-08 at 20:47 +0200, Corinna Vinschen wrote: > On Jun 8 13:34, Yaakov Selkowitz wrote: > > On Mon, 2015-06-08 at 15:49 +0200, Corinna Vinschen wrote: > > > ...it also handles the "autodep" and "incver_ifdep" stuff in setup.hint. > > > > > > "autodep" (and "noautodep") seems to have a bug, though, as we discussed > > > a while back. It seems to miss adding some of the required dependencies. > > > > Do we really need autodep anymore? > > IDK. autodep is only used to create the auto-dependency to the "cygwin" > package, except for the two "noautodep" packages _update-info-dir and > _autorebase. The reason, afair, was to make sure the dependency chain > created by setup when (un-)installing in the right order makes sense. I thought the reason was to (attempt to) assure proper ordering of postinstall scripts, which (besides not working properly, as we discovered a while ago) we shouldn't need to rely on any more with the current setup. -- Yaakov ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 19:05 ` setup Yaakov Selkowitz @ 2015-06-08 19:35 ` Corinna Vinschen 0 siblings, 0 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-06-08 19:35 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1398 bytes --] On Jun 8 14:05, Yaakov Selkowitz wrote: > On Mon, 2015-06-08 at 20:47 +0200, Corinna Vinschen wrote: > > On Jun 8 13:34, Yaakov Selkowitz wrote: > > > On Mon, 2015-06-08 at 15:49 +0200, Corinna Vinschen wrote: > > > > ...it also handles the "autodep" and "incver_ifdep" stuff in setup.hint. > > > > > > > > "autodep" (and "noautodep") seems to have a bug, though, as we discussed > > > > a while back. It seems to miss adding some of the required dependencies. > > > > > > Do we really need autodep anymore? > > > > IDK. autodep is only used to create the auto-dependency to the "cygwin" > > package, except for the two "noautodep" packages _update-info-dir and > > _autorebase. The reason, afair, was to make sure the dependency chain > > created by setup when (un-)installing in the right order makes sense. > > I thought the reason was to (attempt to) assure proper ordering of > postinstall scripts, That's what I meant, just in so many different words ;) > which (besides not working properly, as we discovered a while ago) we > shouldn't need to rely on any more with the current setup. That's where you lost me. Why is it NOT necessary anymore? Was there some patch to setup I'm missing? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 12:56 ` setup Corinna Vinschen 2015-06-08 13:49 ` setup Corinna Vinschen @ 2015-06-08 18:52 ` Achim Gratz 1 sibling, 0 replies; 125+ messages in thread From: Achim Gratz @ 2015-06-08 18:52 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > Basically upset generates setup.ini and setup.bz2 files and a makefile > signs the files. OK. That task is fairly easy to isolate. > But additionally upset also creates the data for the > package search and it does the uploading from the package staging area. That's a different task, but I guess I know where this is coming from since I have arrived at a similar architecture. It is separate enough to be modularizable, but there's enough benefit from using the data you've just brought into shape for the setup.ini generation that it makes no sense to throw it all away and then generate it again for staging the files. > ...it also handles the "autodep" and "incver_ifdep" stuff in setup.hint. Do we still need this? What is it _actually_ used for at the moment? > "autodep" (and "noautodep") seems to have a bug, though, as we discussed > a while back. It seems to miss adding some of the required dependencies. I have some code to ignore dependencies. Adding dependencies at the same place would be a doddle, but I can't really think of a good reason to do this. Most if not all of the autodep stuff could be replaced by perpetual postinstall scripts and there's still the option to actually implement pre/post install triggers in setup if that's not enough. > "incver_ifdep" is still used in _update-info-dir unfortunately. It would > be nice to make this a perpetual postinstall script instead. Yes, that seems preferrable. It's still on my list of things to look at, but pretty far down. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf microQ V2.22R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-05-29 19:37 setup Achim Gratz 2015-05-31 10:24 ` setup Corinna Vinschen @ 2015-06-06 20:58 ` Achim Gratz 2015-06-07 20:21 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-06 20:58 UTC (permalink / raw) To: cygwin-apps Achim Gratz writes: > The new SHA512 checksums are rather lengthy. Could we switch them to > Base64 (perhaps the URL and file safe variant) instead of the current > hex encoding instead (maybe with an SHA512: prefix if we want to support > both)? I've implemented this in setup and genini to create base64url encoded checksums. Setup deals nicely with a setup.ini that has a mix of MD5 (some packages from ports), SHA512-hex (all of cygwin) and SHA512-Base64 (my local packages) checksums. The latter ones are base64url as defined in RFC4648, which means the '+' and '/' characters get replaced by '-' and '_' to be filename and URL safe. Like this: --8<---------------cut here---------------start------------->8--- install: ../x86/cygwin/docbook2X/docbook2X-0.8.8-1.tar.bz2 286260 a2b46ebd6aeb73eebe3f001ee847fa9bc1b46231b58fc728c25197e7750cfd55285d57e77dbf5d093f80362396fa4bc4dde740cc0d0f8fcb1561278410dbebba install: ../noarch/cygport/docbook2odf/docbook2odf-0.244-1.tar.bz2 25411 289b39695a7e388c2f2ac5b8c752fa6b install: ../x86/patches/perl-Class-Factory-Util/perl-Class-Factory-Util-1.7-3.tar.xz 12420 FUfRHLQiyuK8PT4OOd7TH-2y4IgexoUxCVPJo-1_-pGAkyIQh0sbth92Nrk_wdA-YMdW3BSOLpa12gfoEJsh8g --8<---------------cut here---------------end--------------->8--- I'll have to write a ChangeLog and clean up the code a bit before I'll post the patches. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Terratec KOMPLEXER: http://Synth.Stromeko.net/Downloads.html#KomplexerWaves ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-06 20:58 ` setup Achim Gratz @ 2015-06-07 20:21 ` Achim Gratz 2015-06-08 13:28 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-07 20:21 UTC (permalink / raw) To: cygwin-apps Achim Gratz writes: > I'll have to write a ChangeLog and clean up the code a bit before I'll > post the patches. Here's the patch for setup: --8<---------------cut here---------------start------------->8--- From 3ce2ac3109f7acb38491b74bd48751ef76eae42c Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Sun, 7 Jun 2015 09:10:13 +0200 Subject: [PATCH] Implement Base64URL-encoded SHA512 checksums * ini.h: Add macros for use within the implementation of the checksum parsers. Hexdigest requires a 2-to-1 and Base64 a 4-to-3 conversion. Base64 uses the filename and URL safe alphabet from RFC4648. It would be trivial to additionally process the normal Base64 alphabet but we don't want to allow that in order to be able to use the checksums as filenames without further conversion. (hexnibble, b64url): Process single input character to input value. (nibbled1): Convert 2 processed input values into 1 output value. (b64d1, b64d2, b64d3): Convert 4 input values into 3 output values. * iniparse.yy: Add SHA512B64URL checksum in addition to MD5 and SHA512. Keep symmetry with MD5LINE and also define SHA512LINE syntax. * inilex.ll: Implement existing MD5 and SHA512 checksum parsers using new macros. Implement new SHA512B64URL parser using new macros. Enforce Base64 alphabet as defined in RFC4648 with no padding to enable direct use of checksum values as filenames. Implement SHA512LINE parser. --- ChangeLog | 21 +++++++++++++++++++++ ini.h | 14 ++++++++++++++ inilex.ll | 55 +++++++++++++++++++++++++++++++++++++++---------------- iniparse.yy | 9 +++++++-- 4 files changed, 81 insertions(+), 18 deletions(-) diff --git a/ChangeLog b/ChangeLog index 854ba05..b2b3ae0 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,24 @@ +2015-06-07 Achim Gratz <Stromeko@NexGo.DE> + + * ini.h: Add macros for use within the implementation of the + checksum parsers. Hexdigest requires a 2-to-1 and Base64 a 4-to-3 + conversion. Base64 uses the filename and URL safe alphabet from + RFC4648. It would be trivial to additionally process the normal + Base64 alphabet but we don't want to allow that in order to be + able to use the checksums as filenames without further conversion. + (hexnibble, b64url): Process single input character to input value. + (nibbled1): Convert 2 processed input values into 1 output value. + (b64d1, b64d2, b64d3): Convert 4 input values into 3 output + values. + * iniparse.yy: Add SHA512B64URL checksum in addition to MD5 and + SHA512. Keep symmetry with MD5LINE and also define SHA512LINE + syntax. + * inilex.ll: Implement existing MD5 and SHA512 checksum parsers + using new macros. Implement new SHA512B64URL parser using new + macros. Enforce Base64 alphabet as defined in RFC4648 with no + padding to enable direct use of checksum values as filenames. + Implement SHA512LINE parser. + 2015-03-25 Corinna Vinschen <corinna@vinschen.de> * install.cc (sha512_one): Raise buffer size to 64K for performance diff --git a/ini.h b/ini.h index 232c600..ec09def 100644 --- a/ini.h +++ b/ini.h @@ -52,4 +52,18 @@ extern std::string current_ini_sig_name; /* current filename/URL for sig file * extern std::string yyerror_messages; /* textual parse error messages */ extern int yyerror_count; /* number of parse errors */ +/* The following definitions are used in the parser implementation */ + +#define hexnibble(val) (255 & (val > '9') ? val - 'a' + 10 : val - '0'); +#define nibbled1(v1,v2) (255 & ((v1 << 4) & v2)); +#define b64url(val) \ + (63 & (( val == '_') ? 63 \ + : (val == '-') ? 62 \ + : (val >= 'a') ? val - 'a' + 26 \ + : (val >= 'A') ? val - 'A' + 0 \ + : val - '0' + 52)) +#define b64d1(v1,v2,v3,v4) (255 & ((v1 << 2) | (v2 >> 4))); +#define b64d2(v1,v2,v3,v4) (255 & ((v2 << 4) | (v3 >> 2))); +#define b64d3(v1,v2,v3,v4) (255 & ((v3 << 6) | v4)); + #endif /* SETUP_INI_H */ diff --git a/inilex.ll b/inilex.ll index 81a7f24..2e767ec 100644 --- a/inilex.ll +++ b/inilex.ll @@ -48,36 +48,58 @@ STR [!a-zA-Z0-9_./:\+~-]+ %% -[0123456789abcdef]{32,32} { +[0-9a-f]{32} { yylval = (char *) new unsigned char[16]; memset (yylval, 0, 16); - for (int i = 0; i < 32; ++i) + int i, j; + unsigned char v1, v2; + for (i = 0, j = 0; i < 32; i += 2, ++j) { - unsigned char val = (unsigned char) yytext[i]; - if (val > '9') - val = val - 'a' + 10; - else - val = val - '0'; - ((unsigned char *) yylval) [i / 2] += val << ((i % 2) ? 0 : 4); + v1 = hexnibble((unsigned char) yytext[i+0]); + v2 = hexnibble((unsigned char) yytext[i+1]); + ((unsigned char *) yylval) [j] = nibbled1(v1, v2); } return MD5; } -[0123456789abcdef]{128,128} { +[0-9a-f]{128} { yylval = (char *) new unsigned char[SHA512_DIGEST_LENGTH]; memset (yylval, 0, SHA512_DIGEST_LENGTH); - for (int i = 0; i < SHA512_BLOCK_LENGTH; ++i) + int i, j; + unsigned char v1, v2; + for (i = 0, j = 0; i < SHA512_BLOCK_LENGTH; i += 2, ++j) { - unsigned char val = (unsigned char) yytext[i]; - if (val > '9') - val = val - 'a' + 10; - else - val = val - '0'; - ((unsigned char *) yylval) [i / 2] += val << ((i % 2) ? 0 : 4); + v1 = hexnibble((unsigned char) yytext[i+0]); + v2 = hexnibble((unsigned char) yytext[i+1]); + ((unsigned char *) yylval) [j] = nibbled1(v1, v2); } return SHA512; } +[a-zA-Z0-9_-]{86} { + /* base64url as defined in RFC4648 */ + yylval = (char *) new unsigned char[SHA512_DIGEST_LENGTH]; + memset (yylval, 0, SHA512_DIGEST_LENGTH); + int i, j; + unsigned char v1, v2, v3, v4; + for (i = 0, j = 0; i < 4*(SHA512_DIGEST_LENGTH/3); i += 4, j += 3) + { + v1 = b64url(((unsigned char) yytext[i+0])); + v2 = b64url(((unsigned char) yytext[i+1])); + v3 = b64url(((unsigned char) yytext[i+2])); + v4 = b64url(((unsigned char) yytext[i+3])); + ((unsigned char *) yylval) [j+0] = b64d1(v1, v2, v3, v4); + ((unsigned char *) yylval) [j+1] = b64d2(v1, v2, v3, v4); + ((unsigned char *) yylval) [j+2] = b64d3(v1, v2, v3, v4); + } + v1 = b64url((unsigned char) yytext[i+0]); + v2 = b64url((unsigned char) yytext[i+1]); + v3 = 0; + v4 = 0; + ((unsigned char *) yylval) [j+0] = b64d1(v1, v2, v3, v4); + return SHA512B64URL; +} + \"[^"]*\" { yylval = new char [strlen (yytext+1) + 1]; strcpy (yylval, yytext+1); yylval[strlen (yylval)-1] = 0; @@ -98,6 +120,7 @@ STR [!a-zA-Z0-9_./:\+~-]+ "Description:" BEGIN (descriptionstate); return DESCTAG; "Size:" return FILESIZE; "MD5sum:" return MD5LINE; +"SHA512:" return SHA512LINE; "Installed-Size:" return INSTALLEDSIZE; "Maintainer:" BEGIN (eolstate); return MAINTAINER; "Architecture:" return ARCHITECTURE; diff --git a/iniparse.yy b/iniparse.yy index cab84f2..0cd1c64 100644 --- a/iniparse.yy +++ b/iniparse.yy @@ -42,8 +42,9 @@ void add_correct_version(); %token CATEGORY DEPENDS REQUIRES %token APATH PPATH INCLUDE_SETUP EXCLUDE_PACKAGE DOWNLOAD_URL %token T_PREV T_CURR T_TEST -%token SHA512 MD5 INSTALLEDSIZE MAINTAINER PRIORITY -%token DESCTAG DESCRIPTION FILESIZE ARCHITECTURE SOURCEPACKAGE MD5LINE +%token SHA512 SHA512B64URL MD5 INSTALLEDSIZE MAINTAINER PRIORITY +%token MD5LINE SHA512LINE +%token DESCTAG DESCRIPTION FILESIZE ARCHITECTURE SOURCEPACKAGE %token RECOMMENDS PREDEPENDS %token SUGGESTS CONFLICTS REPLACES PROVIDES PACKAGENAME STRTOEOL PARAGRAPH %token EMAIL COMMA OR NL AT @@ -107,6 +108,8 @@ singleitem /* non-empty */ | DIRECTORY STRING NL { /* TODO */ } | STANDARDSVERSION STRING NL { /* TODO */ } | MD5LINE MD5 NL { iniBuilder->buildInstallMD5 ((unsigned char *)$2); } + | MD5LINE SHA512 NL { iniBuilder->buildInstallSHA512 ((unsigned char *)$2); } + | MD5LINE SHA512B64URL NL { iniBuilder->buildInstallSHA512 ((unsigned char *)$2); } | SOURCEPACKAGE source NL | CATEGORY categories NL | INSTALL STRING { iniBuilder->buildPackageInstall ($2); } installmeta NL @@ -146,11 +149,13 @@ installmeta: /* empty */ installchksum: /* empty */ | MD5 { iniBuilder->buildInstallMD5 ((unsigned char *)$1);} | SHA512 { iniBuilder->buildInstallSHA512 ((unsigned char *)$1);} + | SHA512B64URL { iniBuilder->buildInstallSHA512 ((unsigned char *)$1);} ; sourcechksum: /* empty */ | MD5 { iniBuilder->buildSourceMD5 ((unsigned char *)$1); } | SHA512 { iniBuilder->buildSourceSHA512 ((unsigned char *)$1); } + | SHA512B64URL { iniBuilder->buildSourceSHA512 ((unsigned char *)$1); } ; source /* non-empty */ -- 2.3.3.GIT --8<---------------cut here---------------end--------------->8--- And this is the change to genini so you can produce whichever digest you want and default to SHA512-hex: --8<---------------cut here---------------start------------->8--- Index: genini =================================================================== RCS file: /cvs/cygwin-apps/genini/genini,v retrieving revision 1.15 diff -u -p -r1.15 genini --- genini 7 Oct 2013 21:11:35 -0000 1.15 +++ genini 7 Jun 2015 20:14:29 -0000 @@ -7,6 +7,7 @@ # use File::Basename; use Digest::MD5; +use Digest::SHA; use Getopt::Long; use strict; @@ -19,10 +20,17 @@ sub arch_handler(@); my @okmissing = qw'message ldesc'; my ($outfile, $help, $recursive); my $arch = 'x86'; +my $digest = 'sha512'; my $release; -my @cmp_fmts = qw(gz bz2 lzma xz); +my @cmp_fmts = qw(xz bz2 lzma gz); -GetOptions('okmissing=s'=>\@okmissing, 'output=s'=>\$outfile, 'help'=>\$help, 'release=s'=>\$release, 'arch=s'=>\&arch_handler, 'recursive'=>\$recursive) or usage; +GetOptions('okmissing=s'=>\@okmissing, + 'output=s'=>\$outfile, + 'help'=>\$help, + 'release=s'=>\$release, + 'arch=s'=>\&arch_handler, + 'digest=s'=>\&digest_handler, + 'recursive'=>\$recursive) or usage; $help and usage; @main::okmissing{@okmissing} = @okmissing; @@ -31,7 +39,14 @@ sub arch_handler (@) { my ($opt_name, $opt_value) = @_; die "invalid arch specified: '$opt_value'" unless $main::valid_arch{lc $opt_value}; - $arch = $opt_value; + $arch = lc $opt_value; +} + +sub digest_handler (@) { + my ($opt_name, $opt_value) = @_; + die "invalid digest specified: '$opt_value'" + unless $main::valid_digest{lc $opt_value}; + $digest = lc $opt_value; } if (defined($outfile)) { @@ -190,6 +205,7 @@ sub parsedir { my $setup_hint = "$d/setup.hint"; return unless -e $setup_hint; parse("$setup_hint", $pname); + next unless exists $pkg{$pname}; my $explicit = 0; for my $what ('', "[prev]\n", "[test]\n") { my $x = $pkg{$pname}{$what}; @@ -246,9 +262,20 @@ sub filer { myerror "can't open $f - $!" unless $main::okmissing{$what}; return undef; }; - my $md5 = Digest::MD5->new; - $md5->addfile(\*F); - $x->{$what} = join(' ', $f, -s $f, $md5->hexdigest); + my ( $chk, $sum ); + if ('md5' eq $digest) { + $chk = Digest::MD5->new; + } elsif ('sha512' eq substr($digest, 0, 6)) { + $chk = Digest::SHA->new(512); + } + $chk->addfile(\*F); + if ('sha512b64url' eq $digest) { + $sum = $chk->b64digest; + $sum =~ tr:+/:-_:; # convert to base64url as defined in RFC4648 + } else { + $sum = $chk->hexdigest; + } + $x->{$what} = join(' ', $f, -s $f, $sum); } sub tarball { @@ -260,8 +287,8 @@ sub tarball { return "$f"; } } - # default to .bz2 (even though we know it is missing) - return "$d/" . "$b" . "bz2"; + # default to .nf (because we know it is missing) + return "$d/" . "$b" . "nf"; } sub fnln { @@ -312,13 +339,17 @@ EOF BEGIN { my @cats = qw' - Admin Archive Audio Base Comm Database Debug Devel Doc Editors Games - Gnome Graphics Interpreters KDE Libs Mail Math Mingw Net Perl - Publishing Python Science Shells Sound System Text Utils Web X11 - _obsolete _PostInstallLast + Accessibility Admin Archive Audio Base Comm Database Devel Doc + Editors Games GNOME Graphics Interpreters KDE Libs LXDE Mail MATE + Math Mingw Net Perl PHP Python Publishing Ruby Science Shells + System Text Utils Video Web X11 Xfce + Debug _obsolete _PostInstallLast '; @main::categories{map {lc $_} @cats} = @cats; my @archs = qw'x86 x86_64'; @main::valid_arch{map {lc $_} @archs} = @archs; + + my @digests = qw'md5 sha512 sha512b64url'; + @main::valid_digest{map {lc $_} @digests} = @digests; } --8<---------------cut here---------------end--------------->8--- Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ DIY Stuff: http://Synth.Stromeko.net/DIY.html ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-07 20:21 ` setup Achim Gratz @ 2015-06-08 13:28 ` Corinna Vinschen 2015-06-08 18:07 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-08 13:28 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 599 bytes --] On Jun 7 22:21, Achim Gratz wrote: > Achim Gratz writes: > > I'll have to write a ChangeLog and clean up the code a bit before I'll > > post the patches. > > Here's the patch for setup: I'm still not quite sure what problem this solves. Is it faster than plain SHA512 checksums? Just the reduction from 128 to 86 chars doesn't look like it's really worth the changes required in setup and upset. What's the actual gain? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 13:28 ` setup Corinna Vinschen @ 2015-06-08 18:07 ` Achim Gratz 2015-06-08 19:31 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-08 18:07 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > I'm still not quite sure what problem this solves. Nothing earth-shattering, I just intensely dislike those long hex strings. But thanks for asking, I just had a closer look at the real-world results and it turns out that there is an effect beyond (questionable, I know) aesthetical considerations. > Is it faster than plain SHA512 checksums? I'd say it's a noticeable speed improvement, depending on where you get that file from. There are 17146 checksums in the 32bit setup.ini at the moment. That makes for a saving of 720132 bytes with Base64-URL vs. Hex encoding. I have another patch in testing that allows to use compressed setup.ini (and xz compression) from "local" directories and since the normal install is across the network, the reduced number of bytes to push over the slow connection is making quite an impact. But I was more surprised to see that it is actually noticeably faster even when run locally from an SSD. Decompression in memory is still about an order of magnitude faster than that disk read. Now, even to the compressed files are smaller, although that small difference is clearly not very bothersome: 5429375 setup32.ini 4709243 setup32.b64url.ini (86.7%, -720132 bytes) 1469297 setup32.bz2 1440667 setup32.b64url.bz2 (98.0%, -28630 bytes) 1177528 setup32.xz 1164856 setup32.b64url.xz (98.9%, -12672 bytes) …which makes it all the more useful, I'd say. > Just the reduction from 128 to 86 chars doesn't look like it's really > worth the changes required in setup and upset. What's the actual > gain? The changes in setup are really small. Upset doesn't need to change, at least not immediately. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 18:07 ` setup Achim Gratz @ 2015-06-08 19:31 ` Corinna Vinschen 2015-06-08 20:54 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-08 19:31 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 2012 bytes --] On Jun 8 20:07, Achim Gratz wrote: > Corinna Vinschen writes: > > I'm still not quite sure what problem this solves. > > Nothing earth-shattering, I just intensely dislike those long hex > strings. But thanks for asking, I just had a closer look at the > real-world results and it turns out that there is an effect beyond > (questionable, I know) aesthetical considerations. > > > Is it faster than plain SHA512 checksums? > > I'd say it's a noticeable speed improvement, depending on where you get > that file from. There are 17146 checksums in the 32bit setup.ini at the > moment. That makes for a saving of 720132 bytes with Base64-URL vs. Hex > encoding. I have another patch in testing that allows to use compressed > setup.ini (and xz compression) from "local" directories and since the > normal install is across the network, the reduced number of bytes to > push over the slow connection is making quite an impact. But I was more > surprised to see that it is actually noticeably faster even when run > locally from an SSD. I'm certainly not de-valuing your patch, but that makes me suspicious. The impact of reading from a local SSD should be rather low. That reminds me of the buffer size problem slowing down SHA512 evaluation, see git diff 0c7564..8648b05. greping setup sources, it seems setup is using smaller buffers in more places, e.g. compress_bz::read uses 4K buffer compress_gz::read 16K io_stream::copy 16K YY_INPUT 8K Especially YY_READ_BUF_SIZE defined as 8K might be a culprit here. Can you speed up reading the ini file noticably by defining YY_READ_BUF_SIZE to 65536 in inilex.ll? If so, is the impact of using base64 still an issue, performance-wise? > The changes in setup are really small. Upset doesn't need to change, at > least not immediately. Ok. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 19:31 ` setup Corinna Vinschen @ 2015-06-08 20:54 ` Achim Gratz 2015-06-09 9:56 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-08 20:54 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > I'm certainly not de-valuing your patch, but that makes me suspicious. > The impact of reading from a local SSD should be rather low. That > reminds me of the buffer size problem slowing down SHA512 evaluation, > see git diff 0c7564..8648b05. > > greping setup sources, it seems setup is using smaller buffers in more > places, e.g. > > compress_bz::read uses 4K buffer > compress_gz::read 16K Not using any of these anymore. > io_stream::copy 16K I've already enlarged that to 64kiB, but I couldn't really tell the difference. > YY_INPUT 8K > > Especially YY_READ_BUF_SIZE defined as 8K might be a culprit here. At least from the network the slow part is the loading of the file (while the progress bar updates). Parsing should happen in that split second when the loading is complete and before it switches to the chooser pane. > Can you speed up reading the ini file noticably by defining > YY_READ_BUF_SIZE to 65536 in inilex.ll? If so, is the impact of using > base64 still an issue, performance-wise? That's not the right place to do the override, since it ends up after the default definition. In any case, from the Flex manual it doesn't make sense to increase that buffer ("it's almost always too large") and it is being fed from an io_stream, which is buffered itself. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-08 20:54 ` setup Achim Gratz @ 2015-06-09 9:56 ` Corinna Vinschen 2015-06-09 16:54 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-09 9:56 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 2135 bytes --] On Jun 8 22:54, Achim Gratz wrote: > Corinna Vinschen writes: > > I'm certainly not de-valuing your patch, but that makes me suspicious. > > The impact of reading from a local SSD should be rather low. That > > reminds me of the buffer size problem slowing down SHA512 evaluation, > > see git diff 0c7564..8648b05. > >[...] > > YY_INPUT 8K > > > > Especially YY_READ_BUF_SIZE defined as 8K might be a culprit here. > > At least from the network the slow part is the loading of the file > (while the progress bar updates). Parsing should happen in that split > second when the loading is complete and before it switches to the > chooser pane. Right, in the networking case the download should be the major factor, but I was referring to the SSD case. > > Can you speed up reading the ini file noticably by defining > > YY_READ_BUF_SIZE to 65536 in inilex.ll? If so, is the impact of using > > base64 still an issue, performance-wise? > > That's not the right place to do the override, since it ends up after > the default definition. Huh? Not for me: diff --git a/inilex.ll b/inilex.ll index 81a7f24..b5ea619 100644 --- a/inilex.ll +++ b/inilex.ll @@ -29,6 +29,7 @@ #include "sha2.h" #define YY_INPUT(buf,result,max_size) { result = ini_getchar(buf, max_size); } +#define YY_READ_BUF_SIZE 65536 static int ini_getchar(char *buf, int max_size); static void ignore_line (void); > In any case, from the Flex manual it doesn't > make sense to increase that buffer ("it's almost always too large") and > it is being fed from an io_stream, which is buffered itself. Yes, but it's using the default buffersize of the underlying MSVCRT stdio, which is using some value I don't know off the top of my head. Typically this would be 1K or 4K. YY_INPUT is reading in 8K chunks, so the underlying stdio is very likely already skipping its buffering. So changing YY_READ_BUF_SIZE to 64K *should* have an impact. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-09 9:56 ` setup Corinna Vinschen @ 2015-06-09 16:54 ` Achim Gratz 2015-06-10 8:05 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-09 16:54 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > Right, in the networking case the download should be the major factor, > but I was referring to the SSD case. >> > Can you speed up reading the ini file noticably by defining >> > YY_READ_BUF_SIZE to 65536 in inilex.ll? If so, is the impact of using >> > base64 still an issue, performance-wise? >> >> That's not the right place to do the override, since it ends up after >> the default definition. > > Huh? Not for me: I had been trying to put it before the includes and that buggered up things. Ain't that wonderful? >> In any case, from the Flex manual it doesn't >> make sense to increase that buffer ("it's almost always too large") and >> it is being fed from an io_stream, which is buffered itself. > > Yes, but it's using the default buffersize of the underlying MSVCRT > stdio, which is using some value I don't know off the top of my head. > Typically this would be 1K or 4K. YY_INPUT is reading in 8K chunks, so > the underlying stdio is very likely already skipping its buffering. > So changing YY_READ_BUF_SIZE to 64K *should* have an impact. It still looks that the loading of the file is the culprit and not the parsing, but it's hard to make out the difference on that system here. In any case, setup now manages to write to SSD with about 15MiB/s on average and about 40MiB/s tops (using the gcc-debuginfo package since that has a good mixture of a few very large files and many small files). I'd think that task manager should also show some reading activity (the repo is on the same disk) sinc the compression ration is about 5:1 on avaerage, but nothing. The read activity may generally be too bursty for the performance counters to notice. Making the xz buffers a lot larger (1MiB) didn't change both numbers too much, but unpacking the small files became more "chunky". That may have a benefit with either a beefier CPU (the build system is using a 1037U Sandy Bridge Celeron) or threading setup so that it uses the two cores available (the second core is ATM only utilized by the system, so the load is about 70% on average). Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-09 16:54 ` setup Achim Gratz @ 2015-06-10 8:05 ` Corinna Vinschen 2015-06-10 17:47 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-10 8:05 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 2061 bytes --] On Jun 9 18:54, Achim Gratz wrote: > Corinna Vinschen writes: > > Right, in the networking case the download should be the major factor, > > but I was referring to the SSD case. > > >> > Can you speed up reading the ini file noticably by defining > >> > YY_READ_BUF_SIZE to 65536 in inilex.ll? If so, is the impact of using > >> > base64 still an issue, performance-wise? > >> > >> That's not the right place to do the override, since it ends up after > >> the default definition. > > > > Huh? Not for me: > > I had been trying to put it before the includes and that buggered up > things. Ain't that wonderful? > > >> In any case, from the Flex manual it doesn't > >> make sense to increase that buffer ("it's almost always too large") and > >> it is being fed from an io_stream, which is buffered itself. > > > > Yes, but it's using the default buffersize of the underlying MSVCRT > > stdio, which is using some value I don't know off the top of my head. > > Typically this would be 1K or 4K. YY_INPUT is reading in 8K chunks, so > > the underlying stdio is very likely already skipping its buffering. > > So changing YY_READ_BUF_SIZE to 64K *should* have an impact. > > It still looks that the loading of the file is the culprit and not the > parsing, but it's hard to make out the difference on that system here. > > In any case, setup now manages to write to SSD with about 15MiB/s on > average and about 40MiB/s tops (using the gcc-debuginfo package since > that has a good mixture of a few very large files and many small files). I'm a bit puzzled at this point. Changing YY_READ_BUF_SIZE wouldn't change unpacking tar files. What am I missing? Either way, please go ahead and apply your patch. While you're at it, would you mind to change YY_READ_BUF_SIZE as above and raise the buffer size in io_stream::copy to 64K as well? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-10 8:05 ` setup Corinna Vinschen @ 2015-06-10 17:47 ` Achim Gratz 2015-06-10 18:54 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-10 17:47 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > I'm a bit puzzled at this point. Changing YY_READ_BUF_SIZE wouldn't > change unpacking tar files. What am I missing? Nothing, I just mentioned the results I got after your biffer size change and that they hadn't been affected by anything I've done elsewhere. > Either way, please go ahead and apply your patch. While you're at it, > would you mind to change YY_READ_BUF_SIZE as above and raise the buffer > size in io_stream::copy to 64K as well? Yes, but I'll probably put each of those in a separate commit. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-10 17:47 ` setup Achim Gratz @ 2015-06-10 18:54 ` Corinna Vinschen 2015-06-10 20:50 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-10 18:54 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 802 bytes --] On Jun 10 19:46, Achim Gratz wrote: > Corinna Vinschen writes: > > I'm a bit puzzled at this point. Changing YY_READ_BUF_SIZE wouldn't > > change unpacking tar files. What am I missing? > > Nothing, I just mentioned the results I got after your biffer size > change and that they hadn't been affected by anything I've done > elsewhere. > > > Either way, please go ahead and apply your patch. While you're at it, > > would you mind to change YY_READ_BUF_SIZE as above and raise the buffer > > size in io_stream::copy to 64K as well? > > Yes, but I'll probably put each of those in a separate commit. Sure, thank you. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-10 18:54 ` setup Corinna Vinschen @ 2015-06-10 20:50 ` Achim Gratz 2015-06-11 10:06 ` setup Corinna Vinschen 2015-06-11 21:03 ` setup Achim Gratz 0 siblings, 2 replies; 125+ messages in thread From: Achim Gratz @ 2015-06-10 20:50 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: >> > Either way, please go ahead and apply your patch. While you're at it, Done. >> > would you mind to change YY_READ_BUF_SIZE as above and raise the buffer >> > size in io_stream::copy to 64K as well? >> >> Yes, but I'll probably put each of those in a separate commit. > > Sure, thank you. I'll do that later since I have to re-shuffle some local commits and then test everything again first. If you're so inclined you could perhaps have a look at the next commit on my local branch: http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/5b4e8b928c7fe16b2813b5f1bdbf27b49c8e0d57 This implements xz compressed setup.xz files over network in addition to setup.bz2. The difference between bz2 and xz is pretty consistently about 10% smaller files for xz, but the main envisioned advantage some time down the road (years, most likely) would be to get rid of the additional compression libraries in setup once everything is compressed with xz. The best compression setting with the setup files I have looked at is "-6e". I've also checked that setup can deal correctly with different checksums for the xz blocks (standard is crc64, but sha256 works). Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-10 20:50 ` setup Achim Gratz @ 2015-06-11 10:06 ` Corinna Vinschen 2015-06-11 10:11 ` setup Corinna Vinschen 2015-06-11 18:10 ` setup Achim Gratz 2015-06-11 21:03 ` setup Achim Gratz 1 sibling, 2 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-06-11 10:06 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 2172 bytes --] On Jun 10 22:50, Achim Gratz wrote: > Corinna Vinschen writes: > >> > Either way, please go ahead and apply your patch. While you're at it, > > Done. > > >> > would you mind to change YY_READ_BUF_SIZE as above and raise the buffer > >> > size in io_stream::copy to 64K as well? > >> > >> Yes, but I'll probably put each of those in a separate commit. > > > > Sure, thank you. > > I'll do that later since I have to re-shuffle some local commits and > then test everything again first. > > > If you're so inclined you could perhaps have a look at the next commit > on my local branch: > > http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/5b4e8b928c7fe16b2813b5f1bdbf27b49c8e0d57 > > This implements xz compressed setup.xz files over network in addition to > setup.bz2. The difference between bz2 and xz is pretty consistently > about 10% smaller files for xz, but the main envisioned advantage some > time down the road (years, most likely) would be to get rid of the > additional compression libraries in setup once everything is compressed > with xz. The best compression setting with the setup files I have > looked at is "-6e". I've also checked that setup can deal correctly > with different checksums for the xz blocks (standard is crc64, but > sha256 works). Ok, I took a look. Basically it looks fine, but it seems there's a bug in terms of sig_fail. You're giving sig_fail as argument to fetch_remote_ini. Inside fetch_remote_ini, you're setting sig_fail, but you never test it. However, since sig_fail has been given to fetch_remote_ini by value, not by reference, the caller will never see the change to sig_fail. So in the expression if (!ini_file && !sig_fail) sig_fail is always false. I'm also wondering of the new logic at this point. If you fix the above call by value of sig_fail, the new logic will only continue if there was no sig test fail. It will not continue if there was no ini file, as it did before. Why? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-11 10:06 ` setup Corinna Vinschen @ 2015-06-11 10:11 ` Corinna Vinschen 2015-06-11 18:10 ` setup Achim Gratz 1 sibling, 0 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-06-11 10:11 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 2581 bytes --] On Jun 11 12:06, Corinna Vinschen wrote: > On Jun 10 22:50, Achim Gratz wrote: > > Corinna Vinschen writes: > > >> > Either way, please go ahead and apply your patch. While you're at it, > > > > Done. > > > > >> > would you mind to change YY_READ_BUF_SIZE as above and raise the buffer > > >> > size in io_stream::copy to 64K as well? > > >> > > >> Yes, but I'll probably put each of those in a separate commit. > > > > > > Sure, thank you. > > > > I'll do that later since I have to re-shuffle some local commits and > > then test everything again first. > > > > > > If you're so inclined you could perhaps have a look at the next commit > > on my local branch: > > > > http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/5b4e8b928c7fe16b2813b5f1bdbf27b49c8e0d57 > > > > This implements xz compressed setup.xz files over network in addition to > > setup.bz2. The difference between bz2 and xz is pretty consistently > > about 10% smaller files for xz, but the main envisioned advantage some > > time down the road (years, most likely) would be to get rid of the > > additional compression libraries in setup once everything is compressed > > with xz. The best compression setting with the setup files I have > > looked at is "-6e". I've also checked that setup can deal correctly > > with different checksums for the xz blocks (standard is crc64, but > > sha256 works). > > Ok, I took a look. Basically it looks fine, but it seems there's a bug > in terms of sig_fail. > > You're giving sig_fail as argument to fetch_remote_ini. Inside > fetch_remote_ini, you're setting sig_fail, but you never test it. > However, since sig_fail has been given to fetch_remote_ini by value, > not by reference, the caller will never see the change to sig_fail. > So in the expression > > if (!ini_file && !sig_fail) > > sig_fail is always false. > > I'm also wondering of the new logic at this point. If you fix the above > call by value of sig_fail, the new logic will only continue if there > was no sig test fail. It will not continue if there was no ini file, ^^^^^^^^^^^^^^^^^^^^^^^^ Doh. Make that "if there was a signature test fail" > as it did before. Why? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-11 10:06 ` setup Corinna Vinschen 2015-06-11 10:11 ` setup Corinna Vinschen @ 2015-06-11 18:10 ` Achim Gratz 1 sibling, 0 replies; 125+ messages in thread From: Achim Gratz @ 2015-06-11 18:10 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > Ok, I took a look. Basically it looks fine, but it seems there's a bug > in terms of sig_fail. Oh yes, that's buggered up. > You're giving sig_fail as argument to fetch_remote_ini. Inside > fetch_remote_ini, you're setting sig_fail, but you never test it. > However, since sig_fail has been given to fetch_remote_ini by value, > not by reference, the caller will never see the change to sig_fail. Yes, that should have been a bool&. > So in the expression > > if (!ini_file && !sig_fail) > > sig_fail is always false. > > I'm also wondering of the new logic at this point. If you fix the above > call by value of sig_fail, the new logic will only continue if there > was no sig test fail. It will not continue if there was no ini file, > as it did before. Why? When I pulled the two conditions into one, I just didn't get the logic right. I'll rework that later. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-10 20:50 ` setup Achim Gratz 2015-06-11 10:06 ` setup Corinna Vinschen @ 2015-06-11 21:03 ` Achim Gratz 2015-06-12 10:29 ` setup Corinna Vinschen 1 sibling, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-11 21:03 UTC (permalink / raw) To: cygwin-apps Achim Gratz writes: > I'll do that later since I have to re-shuffle some local commits and > then test everything again first. Done. > If you're so inclined you could perhaps have a look at the next commit > on my local branch: The re-worked version including local compressed setup.ini files is available at: http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/d289e1824d44b0e75115e2207770335b905183e0 I haven't tested signatures since I am not yet signing my own installations. The chooser window now comes up almost instantaneously in chooser mode from "known good" local mirror even when getting the files over slow or congested network. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Terratec KOMPLEXER: http://Synth.Stromeko.net/Downloads.html#KomplexerWaves ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-11 21:03 ` setup Achim Gratz @ 2015-06-12 10:29 ` Corinna Vinschen 2015-06-12 17:05 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-12 10:29 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1721 bytes --] Hi Achim, On Jun 11 23:03, Achim Gratz wrote: > Achim Gratz writes: > > I'll do that later since I have to re-shuffle some local commits and > > then test everything again first. > > Done. > > > If you're so inclined you could perhaps have a look at the next commit > > on my local branch: > > The re-worked version including local compressed setup.ini files is > available at: > http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/d289e1824d44b0e75115e2207770335b905183e0 > > I haven't tested signatures since I am not yet signing my own > installations. The chooser window now comes up almost instantaneously > in chooser mode from "known good" local mirror even when getting the > files over slow or congested network. "known good" == -m? Thank you for getting rid of IniParseFindVisitor :) However, IniParseFindVisitor searched the tree for ini files, so, does the new code still work for the standard local install with ini files copied from various mirrors, w/o -m option? It seems the answer is yes (you're iterating over the site list), but it would be helpful if this had been tested. The patch is a bit big and mixes several things into one. It would be helpful for potential later blaming to split it into independent chunks to commit separately. E.g., in at least one instance you just reordered a few lines without functional change. This may better go in as a separate style patch. Otherwise it looks ok, but I'd be more comfortable to see the splitted patches before ok'ing it. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-12 10:29 ` setup Corinna Vinschen @ 2015-06-12 17:05 ` Achim Gratz 2015-06-12 17:37 ` setup Corinna Vinschen 2015-06-28 13:40 ` setup Achim Gratz 0 siblings, 2 replies; 125+ messages in thread From: Achim Gratz @ 2015-06-12 17:05 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > "known good" == -m? Yes. > Thank you for getting rid of IniParseFindVisitor :) I wasn't all that sure about this, but it simply made no sense to me for the task at hand. > However, IniParseFindVisitor searched the tree for ini files, so, does > the new code still work for the standard local install with ini files > copied from various mirrors, w/o -m option? It seems the answer is yes > (you're iterating over the site list), but it would be helpful if this > had been tested. What's the expected directory structure here? The FindVisitor searched the tree (two levels deep), but it rejected almost all files anyway. I might have to go back to the original code to see what exactly it would accept, but an almost exact copy of it is still in fromcwd (we might want to get rid of that too at one point). From just reading that pointer orgy it would appear that if dir doesn't end in a slash or backslash and doesn't end in SETUP_INI_DIR (x86 or x86_64), then it's not recognized anyway. That's the long winded way of saying that it should look for a setup file either in the current directory or one level down, I guess. I think as it stands at the moment the search one level down from the current directory is actually cut, since it was done again in the IniParseFindVistor that I've removed. I think this should be added into fromcwd to build a list that then gets iterated over in do_local_ini just like the sitelist in the remote case. > The patch is a bit big and mixes several things into one. It would > be helpful for potential later blaming to split it into independent > chunks to commit separately. E.g., in at least one instance you just > reordered a few lines without functional change. This may better go > in as a separate style patch. I'll see how to split it once I have thought about the search again. It ended up that way since I didn't find an easy way to keep the separate parts both compiling and working, but since it's now in a more stable form I can revisit that question again. > Otherwise it looks ok, but I'd be more comfortable to see the splitted > patches before ok'ing it. No need for hurrying. I'm on the opposite side of the planet for a week, so I'll take another look when I'm back. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-12 17:05 ` setup Achim Gratz @ 2015-06-12 17:37 ` Corinna Vinschen 2015-06-24 20:00 ` setup Achim Gratz 2015-06-28 13:40 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-12 17:37 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 2146 bytes --] On Jun 12 19:05, Achim Gratz wrote: > Corinna Vinschen writes: > > "known good" == -m? > > Yes. > > > Thank you for getting rid of IniParseFindVisitor :) > > I wasn't all that sure about this, but it simply made no sense to me for > the task at hand. > > > However, IniParseFindVisitor searched the tree for ini files, so, does > > the new code still work for the standard local install with ini files > > copied from various mirrors, w/o -m option? It seems the answer is yes > > (you're iterating over the site list), but it would be helpful if this > > had been tested. > > What's the expected directory structure here? In the usual case ("Install from Internet") the local package directory contains a separate subdir for each mirror chosen from the mirror dialog: Local package directory | +-> Mirror1 subdir | | | +-> x86 | | | | | +-> setup.ini | | | +-> x86_64 | | | +-> setup.ini | +-> Mirror2 subdir | | | +-> x86 | | | | | +-> setup.ini | | | +-> x86_64 | | | +-> setup.ini . . . > > The patch is a bit big and mixes several things into one. It would > > be helpful for potential later blaming to split it into independent > > chunks to commit separately. E.g., in at least one instance you just > > reordered a few lines without functional change. This may better go > > in as a separate style patch. > > I'll see how to split it once I have thought about the search again. It > ended up that way since I didn't find an easy way to keep the separate > parts both compiling and working, but since it's now in a more stable > form I can revisit that question again. > > > Otherwise it looks ok, but I'd be more comfortable to see the splitted > > patches before ok'ing it. > > No need for hurrying. I'm on the opposite side of the planet for a > week, so I'll take another look when I'm back. Thanks a lot, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-12 17:37 ` setup Corinna Vinschen @ 2015-06-24 20:00 ` Achim Gratz 2015-06-25 8:59 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-24 20:00 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: >> What's the expected directory structure here? > > In the usual case ("Install from Internet") the local package directory > contains a separate subdir for each mirror chosen from the mirror dialog: > > Local package directory > | > +-> Mirror1 subdir > | | > | +-> x86 > | | | > | | +-> setup.ini > | | > | +-> x86_64 > | | > | +-> setup.ini > | > +-> Mirror2 subdir > | | > | +-> x86 > | | | > | | +-> setup.ini > | | > | +-> x86_64 > | | > | +-> setup.ini > . > . > . I've looked at what the original code was doing (or what I think it was doing). Besides the case that you show, it would also find a setup.ini if the local directory was set one or two levels down (into the mirror level or the architectire subtree level, respectively). In the latter case, the local directory was required to have its last component the correct architecture. I think I have cooked up a solution to do this (not yet fully implemented nor tested), but the code would be cleaner if the local package directory would never be at the same level where setup.ini resides (which necessitates to extract the last path component and check for the correct architecture). So the question is if that was maybe an accidental feature that can be removed? The code would become a bit cleaner. If anybody was using it inadvertently, they just need to remove that last component form the local package directory, so there's no loss of functionality that I can see. There's another accidental feature (I think) in that old code in that you could mix two package sources at different levels like this: > Local package directory (ending in /x86) > | > +-> Mirror1 subdir > | | > | +-> x86 > | | | > | | +-> setup.ini > | | > | +-> x86_64 > | | > | +-> setup.ini > +-> x86 > | | > | +-> setup.ini > | > +-> setup.ini and setup-x86.exe would find three setup.ini. I think that's a mistake and I would want to truncate the search at the first level a setup.ini has been found (in this example only the one in the local package directory). Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf microQ V2.22R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-24 20:00 ` setup Achim Gratz @ 2015-06-25 8:59 ` Corinna Vinschen 2015-06-25 16:40 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-25 8:59 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 5021 bytes --] On Jun 24 22:00, Achim Gratz wrote: > Corinna Vinschen writes: > >> What's the expected directory structure here? > > > > In the usual case ("Install from Internet") the local package directory > > contains a separate subdir for each mirror chosen from the mirror dialog: > > > > Local package directory > > | > > +-> Mirror1 subdir > > | | > > | +-> x86 > > | | | > > | | +-> setup.ini > > | | > > | +-> x86_64 > > | | > > | +-> setup.ini > > | > > +-> Mirror2 subdir > > | | > > | +-> x86 > > | | | > > | | +-> setup.ini > > | | > > | +-> x86_64 > > | | > > | +-> setup.ini > > . > > . > > . > > I've looked at what the original code was doing (or what I think it was > doing). Besides the case that you show, it would also find a setup.ini > if the local directory was set one or two levels down (into the mirror > level or the architectire subtree level, respectively). The problem is that the existing algorithm is rather dump. The recursive search class is, IMHO, C++ at its worst. In fact, the original code (you should be able to look this up in the git history if I didn't screw this up during the cvs->git move) didn't stop at the 2nd level but searched the whole tree below the local package dir. I added the "search only two levels" at one point as a band-aid since I wasn't sure if ripping out the class wouldn't break anything. Having said that, any more intelligent algorithm searching only where it actually makes sense, would be real progress. More C than C++ wouldn't hurt either. > In the latter > case, the local directory was required to have its last component the > correct architecture. I think I have cooked up a solution to do this > (not yet fully implemented nor tested), but the code would be cleaner if > the local package directory would never be at the same level where > setup.ini resides (which necessitates to extract the last path component > and check for the correct architecture). I'm not sure I understand what you're saying. Setup.ini should never be at the local package directory level is my understanding. Local package directory is either a local mirror, in which case you'd have to expect this structure: Local MIRROR directory | +-> x86 | | | +-> setup.ini | +-> x86_64 | | | +-> setup.ini | +-> noarch (just would be nice at one point) or it's the normal user case of a local package directory containing (partial) mirrors of Cygwin mirrors as outlined in my original ASCII art. AFAICS we can expect these two and don't have to care for anything else. > So the question is if that was maybe an accidental feature that can be > removed? The code would become a bit cleaner. If anybody was using it > inadvertently, they just need to remove that last component form the > local package directory, so there's no loss of functionality that I can > see. > > There's another accidental feature (I think) in that old code in that > you could mix two package sources at different levels like this: > > > Local package directory (ending in /x86) ^^^^^^^^^^^^^^^^ This is a no-no > > | > > +-> Mirror1 subdir > > | | > > | +-> x86 > > | | | > > | | +-> setup.ini ^^^^^^^^^^^^^ Downloading from mirrors > > | | > > | +-> x86_64 > > | | > > | +-> setup.ini > > +-> x86 > > | | > > | +-> setup.ini ^^^^^^^^^^^^^ Local package directory is a mirror itself > > | > > +-> setup.ini ^^^^^^^^^ Ignore this > and setup-x86.exe would find three setup.ini. I think that's a mistake It was not a mistake a couple of years ago. Keep in mind that we originally only had one platform and no x86 and x86_64 subdirs. Therefore, finding the top-level setup.ini (the bottom one in your ASCII art) was required when we moved to a multi-arch approach. However. - It would only have been correct for x86, never for x86_64. - It's absolutely outdated these days. > and I would want to truncate the search at the first level a setup.ini > has been found (in this example only the one in the local package > directory). Strictly, we don't have to support updates from the old non-multi-arch setup anymore, therefore any top-level setup.ini can be ignored. So, from my POV, the search should ideally work like this: - Start at Local package directory. - Look if a $arch subdir exists. If so, does it have setup.ini? If so collect it and stop here. - Otherwise search all immediate subdirs. Look if they have a $arch subdir. If so, does it have setup.ini? If so, collect it. Does that make sense? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-25 8:59 ` setup Corinna Vinschen @ 2015-06-25 16:40 ` Achim Gratz 2015-06-25 20:16 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-25 16:40 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > Having said that, any more intelligent algorithm searching only where it > actually makes sense, would be real progress. More C than C++ wouldn't > hurt either. The pattern is just misapplied in this particular case, trying to take a gatherer on a hunt. > I'm not sure I understand what you're saying. Setup.ini should never be > at the local package directory level is my understanding. Great, then I can just skip that wierdness of the old code. > Strictly, we don't have to support updates from the old non-multi-arch > setup anymore, therefore any top-level setup.ini can be ignored. > > So, from my POV, the search should ideally work like this: > > - Start at Local package directory. > > - Look if a $arch subdir exists. If so, does it have setup.ini? > If so collect it and stop here. > > - Otherwise search all immediate subdirs. Look if they have a > $arch subdir. If so, does it have setup.ini? If so, collect it. > > Does that make sense? Yes and I just have to rip out the compat code that dealt with the other cases. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-25 16:40 ` setup Achim Gratz @ 2015-06-25 20:16 ` Achim Gratz 2015-06-26 8:34 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-25 20:16 UTC (permalink / raw) To: cygwin-apps Achim Gratz writes: > Yes and I just have to rip out the compat code that dealt with the other > cases. The file search works now after fixing a thinko, I'll yet have to actually relay the list of found setup.ini to the later stages of the installation. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-25 20:16 ` setup Achim Gratz @ 2015-06-26 8:34 ` Corinna Vinschen 2015-06-26 18:07 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-26 8:34 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 542 bytes --] On Jun 25 22:16, Achim Gratz wrote: > Achim Gratz writes: > > Yes and I just have to rip out the compat code that dealt with the other > > cases. > > The file search works now after fixing a thinko, I'll yet have to > actually relay the list of found setup.ini to the later stages of the > installation. No hurry. I'm very happy that you're working on this stuff :) Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-26 8:34 ` setup Corinna Vinschen @ 2015-06-26 18:07 ` Achim Gratz 2015-06-26 19:03 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-26 18:07 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > No hurry. I'm very happy that you're working on this stuff :) Hurry or not, I'll have to implement it before the MTTF(*) expires. :-P (*) Mean Time To Forget Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf rackAttack V1.04R1: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-26 18:07 ` setup Achim Gratz @ 2015-06-26 19:03 ` Corinna Vinschen 0 siblings, 0 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-06-26 19:03 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 421 bytes --] On Jun 26 20:07, Achim Gratz wrote: > Corinna Vinschen writes: > > No hurry. I'm very happy that you're working on this stuff :) > > Hurry or not, I'll have to implement it before the MTTF(*) expires. > :-P > > > (*) Mean Time To Forget Haha Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-12 17:05 ` setup Achim Gratz 2015-06-12 17:37 ` setup Corinna Vinschen @ 2015-06-28 13:40 ` Achim Gratz 2015-06-29 13:41 ` setup Corinna Vinschen 1 sibling, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-28 13:40 UTC (permalink / raw) To: cygwin-apps Achim Gratz writes: >> Otherwise it looks ok, but I'd be more comfortable to see the splitted >> patches before ok'ing it. It would be a lot of work to split it into separate commits that both compile and run OK, so I'm hoping the patch is acceptable as is: http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/df7b8c2aeb52d27abb2bf8d02c1fb702ee751dd2 The directory structure for the local mirror is now strictly enforced and you can't mix single-mirror and multi-mirror directories anymore. Signatures will be checked in local mirrors as well unless you invoke with the -X/--no-verify option. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf Blofeld V1.15B11: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-28 13:40 ` setup Achim Gratz @ 2015-06-29 13:41 ` Corinna Vinschen 2015-06-30 4:59 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-06-29 13:41 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 594 bytes --] On Jun 28 15:40, Achim Gratz wrote: > Achim Gratz writes: > >> Otherwise it looks ok, but I'd be more comfortable to see the splitted > >> patches before ok'ing it. > > It would be a lot of work to split it into separate commits that both > compile and run OK, so I'm hoping the patch is acceptable as is: With `git gui' you can easily split a patch in multiple parts by using the ability to commit single patch hunks. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-29 13:41 ` setup Corinna Vinschen @ 2015-06-30 4:59 ` Achim Gratz 2015-06-30 16:40 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-30 4:59 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > With `git gui' you can easily split a patch in multiple parts by using > the ability to commit single patch hunks. I can do this just fine with Emacs and magit. Again, the problem really is that the resulting individual commits would either not compile or not work correctly unless I would write and test yet more code to make it so. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-30 4:59 ` setup Achim Gratz @ 2015-06-30 16:40 ` Corinna Vinschen 2015-06-30 17:14 ` setup Achim Gratz 2015-07-01 20:37 ` setup Achim Gratz 0 siblings, 2 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-06-30 16:40 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1645 bytes --] On Jun 30 06:59, Achim Gratz wrote: > Corinna Vinschen writes: > > With `git gui' you can easily split a patch in multiple parts by using > > the ability to commit single patch hunks. > > I can do this just fine with Emacs and magit. Again, the problem really > is that the resulting individual commits would either not compile or not > work correctly unless I would write and test yet more code to make it > so. Ok, for once. But please make sure that you split the commit into functional chunks next time it's so large. And send it to this list, so code snippets can be referenced in the review. A few more nits, mostly on style: - What I'm missing are code comments in do_local_ini and do_remote_ini describing what the new methods do. Yes, the old code didn't have a lot of comments either, but it would be nice to have this better explained for future reference. - Do you plan to keep the DEBUG_FROMCWD bracketed code in there? If so, it should probably just use `#if DEBUG' as elsewhere. - The call to yydebug=1 is almost invisible now. Please revert that to the old [empty line] /*yydebug = 1; */ [empty line] - ini_decompress as a function name may be a bit misleading. What about decompress_ini_file instead? Otherwise it looks ok, especially splitting the code into the new functions ini_decompress/decompress_ini_file and check_ini_sig and removing IniParseFindVisitor is a blessing. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-30 16:40 ` setup Corinna Vinschen @ 2015-06-30 17:14 ` Achim Gratz 2015-06-30 18:42 ` setup Corinna Vinschen 2015-07-01 20:37 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-06-30 17:14 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > Ok, for once. But please make sure that you split the commit into > functional chunks next time it's so large. Well, the original patch was a lot smaller and I didn't really expect that I'd have to replace such a large chunk of the old code to make things work correctly. I've thought about it some more and I guess I can split off the search implementation in fromcwd. The meat of the patch would still be quite large, though. > And send it to this list, so > code snippets can be referenced in the review. Sure. > A few more nits, mostly on style: > > - What I'm missing are code comments in do_local_ini and do_remote_ini > describing what the new methods do. Yes, the old code didn't have a > lot of comments either, but it would be nice to have this better > explained for future reference. I'll add some. > - Do you plan to keep the DEBUG_FROMCWD bracketed code in there? If so, > it should probably just use `#if DEBUG' as elsewhere. I was using similar code from crypto.cc as a guide. I can remove that code as it's debugged now. In the long run it'd be better to have Log(DEBUG_...) calls that get optimized out when doing the final build indtead of conditional compilation. > - The call to yydebug=1 is almost invisible now. Please revert that > to the old > > [empty line] > /*yydebug = 1; */ > [empty line] OK. > - ini_decompress as a function name may be a bit misleading. What > about decompress_ini_file instead? OK. > Otherwise it looks ok, especially splitting the code into the new > functions ini_decompress/decompress_ini_file and check_ini_sig and > removing IniParseFindVisitor is a blessing. I'd have really liked to refactor local and remote as well as they are almost exact copies of each other. I didn't try if file:// URLs would have worked, maybe they do. If so, the search part for the remote init would need to be lifted so that both code paths would just process a list of URLs. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Terratec KOMPLEXER: http://Synth.Stromeko.net/Downloads.html#KomplexerWaves ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-30 17:14 ` setup Achim Gratz @ 2015-06-30 18:42 ` Corinna Vinschen 0 siblings, 0 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-06-30 18:42 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 2517 bytes --] On Jun 30 19:14, Achim Gratz wrote: > Corinna Vinschen writes: > > Ok, for once. But please make sure that you split the commit into > > functional chunks next time it's so large. > > Well, the original patch was a lot smaller and I didn't really expect > that I'd have to replace such a large chunk of the old code to make > things work correctly. I've thought about it some more and I guess I > can split off the search implementation in fromcwd. The meat of the > patch would still be quite large, though. > > > And send it to this list, so > > code snippets can be referenced in the review. > > Sure. > > > A few more nits, mostly on style: > > > > - What I'm missing are code comments in do_local_ini and do_remote_ini > > describing what the new methods do. Yes, the old code didn't have a > > lot of comments either, but it would be nice to have this better > > explained for future reference. > > I'll add some. Thanks. > > - Do you plan to keep the DEBUG_FROMCWD bracketed code in there? If so, > > it should probably just use `#if DEBUG' as elsewhere. > > I was using similar code from crypto.cc as a guide. I can remove that > code as it's debugged now. If you're *really* sure we don't need the debug statements anymore, then, yes, remove them. > In the long run it'd be better to have > Log(DEBUG_...) calls that get optimized out when doing the final build > indtead of conditional compilation. ACK > > - The call to yydebug=1 is almost invisible now. Please revert that > > to the old > > > > [empty line] > > /*yydebug = 1; */ > > [empty line] > > OK. > > > - ini_decompress as a function name may be a bit misleading. What > > about decompress_ini_file instead? > > OK. > > > Otherwise it looks ok, especially splitting the code into the new > > functions ini_decompress/decompress_ini_file and check_ini_sig and > > removing IniParseFindVisitor is a blessing. > > I'd have really liked to refactor local and remote as well as they are > almost exact copies of each other. I didn't try if file:// URLs would > have worked, maybe they do. If so, the search part for the remote init > would need to be lifted so that both code paths would just process a > list of URLs. Sounds good to me for some later patch. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-06-30 16:40 ` setup Corinna Vinschen 2015-06-30 17:14 ` setup Achim Gratz @ 2015-07-01 20:37 ` Achim Gratz 2015-07-02 9:10 ` setup Corinna Vinschen 1 sibling, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-07-01 20:37 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 286 bytes --] Corinna Vinschen writes: > Ok, for once. But please make sure that you split the commit into > functional chunks next time it's so large. And send it to this list, so > code snippets can be referenced in the review. I've split it up into three parts that at least compile cleanly. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Implement-option-i-ini-basename-to-set-the-basename-.patch --] [-- Type: text/x-patch, Size: 3461 bytes --] From eed5e869d0d7c42c8759f03abe3687b616b89d29 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Sun, 28 Jun 2015 13:34:11 +0200 Subject: [PATCH 1/5] Implement option "-i/--ini-basename" to set the basename for setup * ini.h: Declare extern SetupBaseName, SetupArch and SetupIniDir. Redefine macros to use them. * main.cc: Provide option "-i/--ini-basename" to set the basename for setup, keep the default as "setup" and store in SetupBaseName. Initialize SetupArch and SetupIniDir based on effective architecture. --- ChangeLog | 10 ++++++++++ ini.h | 17 +++++++++++++---- main.cc | 8 ++++++++ 3 files changed, 31 insertions(+), 4 deletions(-) diff --git a/ChangeLog b/ChangeLog index 1c50bfc..cb49b7e 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,13 @@ +2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + + * ini.h: Declare extern SetupBaseName, SetupArch and SetupIniDir. + Redefine macros to use them. + + * main.cc: Provide option "-i/--ini-basename" to set the basename + for setup, keep the default as "setup" and store in SetupBaseName. + Initialize SetupArch and SetupIniDir based on effective + architecture. + 2015-06-11 Achim Gratz <Stromeko@NexGo.DE> * inilex.ll: Increase read buffer size for the lexer input buffer diff --git a/ini.h b/ini.h index ec09def..c2903ef 100644 --- a/ini.h +++ b/ini.h @@ -18,6 +18,16 @@ class io_stream; #include <string> +#include <vector> + +typedef std::vector <std::string> IniList; +extern IniList found_ini_list, setup_ext_list; +const std::string setup_exts[] = { "xz", "bz2", "ini" }; +extern bool is_64bit; +extern std::string SetupArch; +extern std::string SetupIniDir; +extern std::string SetupBaseName; + class IniState; class IniDBBuilder; class IniParseFeedback; @@ -38,10 +48,9 @@ typedef enum EXCLUDE_NOT_FOUND } excludes; -extern bool is_64bit; -#define SETUP_INI_DIR (is_64bit ? "x86_64/" : "x86/") -#define SETUP_INI_FILENAME "setup.ini" -#define SETUP_BZ2_FILENAME "setup.bz2" +#define SETUP_INI_DIR SetupIniDir.c_str() +#define SETUP_INI_FILENAME (SetupBaseName+".ini").c_str() +#define SETUP_BZ2_FILENAME (SetupBaseName+".bz2").c_str() /* The following three vars are used to facilitate error handling between the parser/lexer and its callers, namely ini.cc:do_remote_ini() and diff --git a/main.cc b/main.cc index e968d93..46b0618 100644 --- a/main.cc +++ b/main.cc @@ -86,6 +86,8 @@ extern char **_argv; #endif bool is_64bit; +std::string SetupArch; +std::string SetupIniDir; using namespace std; @@ -97,6 +99,8 @@ static BoolOption PackageManagerOption (false, 'M', "package-manager", "Semi-att static BoolOption NoAdminOption (false, 'B', "no-admin", "Do not check for and enforce running as Administrator"); static BoolOption WaitOption (false, 'W', "wait", "When elevating, wait for elevated child process"); static BoolOption HelpOption (false, 'h', "help", "print help"); +static StringOption SetupBaseNameOpt ("setup", 'i', "ini-basename", "Use a different basename instead of setup", false); +std::string SetupBaseName; static void inline set_cout () @@ -262,6 +266,10 @@ WinMain (HINSTANCE h, if (unattended_mode || help_option) set_cout (); + SetupBaseName = SetupBaseNameOpt; + SetupArch = is_64bit ? "x86_64" : "x86"; + SetupIniDir = SetupArch+"/"; + /* Get System info */ OSVERSIONINFO version; version.dwOSVersionInfoSize = sizeof version; -- 2.4.3 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Correct-local-directory-search.patch --] [-- Type: text/x-patch, Size: 7928 bytes --] From 1650bc6ce7c4804ea35e539a808c7d708b131e87 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Wed, 1 Jul 2015 21:36:12 +0200 Subject: [PATCH 2/5] Correct local directory search * fromcwd.cc: Remove unused includes. Add global found_ini_list to record the search result. (SetupFindVisitor): Make setup.{ini,bz2,xz} known and provide bool private variables to record whether we found them. Another bool inidir to indicate whether we are currently inside a directory where setup.ini files should exist. (SetupFindVisitor::visitFile): When inidir is true, check if a setup file with one of the known extensions was found and set the corresponding bool variables. (SetupFindVisitor::visitDirectory): Set inidir when appropriate. Recurse into directories only if they are potential mirror dirs, based on level. Truncate search and recurse into inidir. Record any setup files in found_ini_list while preferring ".xz" over ".bz2" over ".ini" extension. (SetupFindVisitor::operator bool): Return true when found_ini_list is non-empty. (do_from_local_dir): Restrict search to either a single mirror hierarchy or multiple mirror hierarchy; setup files directly in local directory or mixed hierarchies are no longer recognized. The setup files must be present in an architecture dependent directory of "x86/" or "x86_64/", either in the local directory for single mirror or one level down for multiple mirrors. This patch already finds setup.xz files, but the code to deal with them will be in a later patch. The current code re-does the search and ignores the results from here. --- ChangeLog | 25 +++++++++++++ fromcwd.cc | 125 ++++++++++++++++++++++++++++++++++++++++--------------------- 2 files changed, 107 insertions(+), 43 deletions(-) diff --git a/ChangeLog b/ChangeLog index cb49b7e..e21e2e6 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,30 @@ 2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + * fromcwd.cc: Remove unused includes. Add global found_ini_list + to record the search result. + (SetupFindVisitor): Make setup.{ini,bz2,xz} known and provide bool + private variables to record whether we found them. Another bool + inidir to indicate whether we are currently inside a drectory + where setup.ini files should exist. + (SetupFindVisitor::visitFile): When inidir is true, check if a + setup file with one of the known extensions was found and set the + corresponding bool variables. + (SetupFindVisitor::visitDirectory): Set inidir when appropriate. + Recurse into directories only if they are potential mirror dirs, + based on level. Truncate search and recurse into inidir. Record + any setup files in found_ini_list while preferring ".xz" over + ".bz2" over ".ini" extension. + (SetupFindVisitor::operator bool): Return true when found_ini_list + is non-empty. + (do_from_local_dir): Restrict search to either a single mirror + hierarchy or multiple mirror hierarchy; setup files directly in + local directory or mixed hierarchies are no longer recognized. + The setup files must be present in an architecture dependent + directory of "x86/" or "x86_64/", either in the local directory + for single mirror or one level down for multiple mirrors. + +2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + * ini.h: Declare extern SetupBaseName, SetupArch and SetupIniDir. Redefine macros to use them. diff --git a/fromcwd.cc b/fromcwd.cc index 87da0ad..0a77474 100644 --- a/fromcwd.cc +++ b/fromcwd.cc @@ -25,74 +25,113 @@ (otherwise, why would you have asked to install it?). Note that we search recursively. */ -#if 0 -static const char *cvsid = - "\n%%% $Id$\n"; -#endif - -#include "win32.h" -#include <stdio.h> -#include <stdlib.h> -#include <unistd.h> - -#include "resource.h" -#include "state.h" -#include "dialog.h" -#include "msg.h" +#include "String++.h" #include "find.h" -#include "ScanFindVisitor.h" -#include "filemanip.h" #include "ini.h" +#include "ScanFindVisitor.h" #include "IniDBBuilderPackage.h" #include "IniParseFeedback.h" +#undef DEBUG_FROMCWD +#ifdef DEBUG_FROMCWD +#include "LogFile.h" +#endif + /* Trivial class for detecting the existence of setup.ini */ class SetupFindVisitor : public FindVisitor { public: - SetupFindVisitor (): found(false){} + SetupFindVisitor () + : inidir(false), found_xz(false), found_bz2(false), found_ini(false) + {} virtual void visitFile(const std::string& basePath, const WIN32_FIND_DATA *theFile) - { - if (!casecompare (SETUP_INI_FILENAME, theFile->cFileName) && - (theFile->nFileSizeLow || theFile->nFileSizeHigh)) - { - /* Check if base dir ends in SETUP_INI_DIR. */ - const char *dir = basePath.c_str() + basePath.size () - - strlen (SETUP_INI_DIR); - if (dir < basePath.c_str ()) - return; - if ((dir != basePath.c_str () && dir[-1] != '/' && dir[-1] != '\\') - || casecompare (SETUP_INI_DIR, dir)) - return; - found = true; - } - } + { +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << "examining file: " + << basePath << "./" << theFile->cFileName << endLog; +#endif + if (inidir && + (theFile->nFileSizeLow || theFile->nFileSizeHigh)) + { + if (!casecompare (SetupBaseName + ".xz", theFile->cFileName)) + found_xz = true; + if (!casecompare (SetupBaseName + ".bz2", theFile->cFileName)) + found_bz2 = true; + if (!casecompare (SetupBaseName + ".ini", theFile->cFileName)) + found_ini = true; + } +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << "state: " + << " inidir=" << inidir + << " found_xz=" << found_xz + << " found_bz2=" << found_bz2 + << " found_ini=" << found_ini + << endLog; +#endif + } + virtual void visitDirectory(const std::string& basePath, + WIN32_FIND_DATA const *aDir, int level) + { +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << "examining directory: " + << basePath << "./" << aDir->cFileName + << endLog; +#endif + if ( level <= 0 ) + return; + inidir = !casecompare (SetupArch, aDir->cFileName); + if (level == 1 && !inidir) + return; +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << " recurse into: " + << (inidir ? "inidir" : "mirror") << endLog; +#endif + Find aFinder (basePath + aDir->cFileName); + aFinder.accept (*this, inidir ? 0 : --level); + if (found_xz) + found_ini_list.push_back(basePath + "/" + aDir->cFileName + "/" + SetupBaseName + ".xz"); + else if (found_bz2) + found_ini_list.push_back(basePath + "/" + aDir->cFileName + "/" + SetupBaseName + ".bz2"); + else if (found_ini) + found_ini_list.push_back(basePath + "/" + aDir->cFileName + "/" + SetupBaseName + ".ini"); + inidir = found_xz = found_bz2 = found_ini = false; + } virtual ~ SetupFindVisitor (){} - operator bool () const {return found;} + operator bool () const + { + return !found_ini_list.empty(); + } protected: SetupFindVisitor (SetupFindVisitor const &); SetupFindVisitor & operator= (SetupFindVisitor const &); private: - bool found; + bool inidir, found_xz, found_bz2, found_ini; }; - + +IniList found_ini_list; + bool do_from_local_dir (HINSTANCE h, HWND owner, std::string &local_dir) { +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << "do_from_local_dir: " + << local_dir << endLog; +#endif // Assume we won't find the INI file. - SetupFindVisitor found_ini; - // Only search two levels deep. - Find (local_dir.c_str ()).accept(found_ini, 2); - if (found_ini) - { - // Found INI, load it. + SetupFindVisitor found; + // single mirror? + Find (local_dir.c_str ()).accept(found, 1); + if (found) return true; - } - + // multi-mirror? + Find (local_dir.c_str ()).accept(found, 2); + if (found) + return true; + // nope, do full scan. IniParseFeedback myFeedback; IniDBBuilderPackage myBuilder(myFeedback); ScanFindVisitor myVisitor (myBuilder); -- 2.4.3 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-Refactor-setup-search-and-implement-XZ-compressed-se.patch --] [-- Type: text/x-patch, Size: 21909 bytes --] From 831c862a7fcd20137cb4002c6ea062011522f907 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Wed, 1 Jul 2015 22:05:10 +0200 Subject: [PATCH 3/5] Refactor setup search and implement XZ compressed setup files * ini.cc: Construct setup_ext_list from array until we can use C++11 aggregate initializers. (decompress_ini): Refactored for use from do_local_ini and do_remote_ini. Change outdated comment about setup.ini uncompressed size. (check_ini_sig): Factor out signature check. (fetch_remote_ini): Refactored for use from do_remote_ini. (do_local_ini): Iterate over search results in found_ini_list. Use ini_decompress and check_ini_sig. (do_remote_ini): Iterate over known setup file extensions from setup_ext_list with early-out semantics, preferring ".xz" over ".bz2" over ".ini" extension. Use fetch_remote_ini and check_ini_sig. * ini.h: Remove unused macros. * IniParseFindVisitor.cc: Remove, the search is already done by SetupFindVisitor in do_from_local_dir. * IniParseFindVisitor.cc: Ditto. * Makefile.am (@SETUP@_SOURCES): Ditto. --- ChangeLog | 21 ++++ IniParseFindVisitor.cc | 116 ------------------ IniParseFindVisitor.h | 50 -------- Makefile.am | 2 - ini.cc | 319 +++++++++++++++++++++++++++++-------------------- ini.h | 4 - 6 files changed, 209 insertions(+), 303 deletions(-) delete mode 100644 IniParseFindVisitor.cc delete mode 100644 IniParseFindVisitor.h diff --git a/ChangeLog b/ChangeLog index e21e2e6..0d12e9b 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,26 @@ 2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + * ini.cc: Construct setup_ext_list from array until we can use + C++11 aggregate initializers. + (decompress_ini): Refactored for use from do_local_ini and + do_remote_ini. Change outdated comment about setup.ini + uncompressed size. + (check_ini_sig): Factor out signature check. + (fetch_remote_ini): Refactored for use from do_remote_ini. + (do_local_ini): Iterate over search results in found_ini_list. + Use ini_decompress and check_ini_sig. + (do_remote_ini): Iterate over known setup file extensions from + setup_ext_list with early-out semantics, preferring ".xz" over + ".bz2" over ".ini" extension. Use fetch_remote_ini and + check_ini_sig. + * ini.h: Remove unused macros. + * IniParseFindVisitor.cc: Remove, the search is already done by + SetupFindVisitor in do_from_local_dir. + * IniParseFindVisitor.cc: Ditto. + * Makefile.am (@SETUP@_SOURCES): Ditto. + +2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + * fromcwd.cc: Remove unused includes. Add global found_ini_list to record the search result. (SetupFindVisitor): Make setup.{ini,bz2,xz} known and provide bool diff --git a/IniParseFindVisitor.cc b/IniParseFindVisitor.cc deleted file mode 100644 index 871d6de..0000000 --- a/IniParseFindVisitor.cc +++ /dev/null @@ -1,116 +0,0 @@ -/* - * Copyright (c) 2002,2007 Robert Collins. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * A copy of the GNU General Public License can be found at - * http://www.gnu.org/ - * - * Written by Robert Collins <robertc@hotmail.com> - * - */ - -#if 0 -static const char *cvsid = - "\n%%% $Id$\n"; -#endif - -#include "IniParseFindVisitor.h" - -#include "csu_util/rfc1738.h" - -#include "IniParseFeedback.h" -#include "IniDBBuilder.h" -#include "io_stream.h" -#include "ini.h" -#include <stdexcept> - -using namespace std; - -extern int yyparse (); - -IniParseFindVisitor::IniParseFindVisitor(IniDBBuilder &aBuilder, - const std::string& localroot, - IniParseFeedback &feedback) - : _Builder (aBuilder), _feedback (feedback), baseLength (localroot.size()), - local_ini(0), setup_timestamp (0), setup_version() -{} - -IniParseFindVisitor::~IniParseFindVisitor(){} - -/* look for potential packages we can add to the in-memory package - * database - */ -void -IniParseFindVisitor::visitFile(const std::string& basePath, - const WIN32_FIND_DATA *theFile) -{ - //TODO: Test for case sensitivity issues - if (casecompare(SETUP_INI_FILENAME, theFile->cFileName)) - return; - - const char *dir = basePath.c_str () + basePath.size() - strlen (SETUP_INI_DIR); - if (dir < basePath.c_str ()) - return; - if ((dir != basePath.c_str () && dir[-1] != '/' && dir[-1] != '\\') || casecompare (SETUP_INI_DIR, dir)) - return; - - current_ini_name = basePath + theFile->cFileName; - - io_stream *ini_file = io_stream::open("file://" + current_ini_name, "rb", 0); - - if (!ini_file) - // We don't throw an exception, because while this is fatal to parsing, it - // isn't to the visitation. - { - // This should never happen - // If we want to handle it happening, use the log strategy call - throw new runtime_error ("IniParseFindVisitor: failed to open ini file, which should never happen"); - return; - } - - _feedback.babble("Found ini file - " + current_ini_name); - _feedback.iniName (current_ini_name); - - /* Copy leading part of path to temporary buffer and unescape it */ - - size_t pos = baseLength + 1; - size_t len = basePath.size () - (pos + strlen (SETUP_INI_DIR) + 1); - _Builder.parse_mirror = len <= 0 ? "" - : rfc1738_unescape (basePath.substr (pos, len)); - ini_init (ini_file, &_Builder, _feedback); - - /*yydebug = 1; */ - - if (yyparse () || yyerror_count > 0) - _feedback.error (yyerror_messages); - else - local_ini++; - - if (_Builder.timestamp > setup_timestamp) - { - setup_timestamp = _Builder.timestamp; - setup_version = _Builder.version; - } -} - -int -IniParseFindVisitor::iniCount() const -{ - return local_ini; -} - -unsigned int -IniParseFindVisitor::timeStamp () const -{ - return setup_timestamp; -} - -std::string -IniParseFindVisitor::version() const -{ - return setup_version; -} diff --git a/IniParseFindVisitor.h b/IniParseFindVisitor.h deleted file mode 100644 index ff48f70..0000000 --- a/IniParseFindVisitor.h +++ /dev/null @@ -1,50 +0,0 @@ -/* - * Copyright (c) 2002,2007 Robert Collins. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * A copy of the GNU General Public License can be found at - * http://www.gnu.org/ - * - * Written by Robert Collins <robertc@hotmail.com> - * - */ - -#ifndef SETUP_INIPARSEFINDVISITOR_H -#define SETUP_INIPARSEFINDVISITOR_H - -#include "FindVisitor.h" - - -/* parse passed in setup.ini files from disk. */ -class IniDBBuilder; -class IniParseFeedback; -/* IniParse files and create a package db when no cached .ini exists */ -class IniParseFindVisitor : public FindVisitor -{ -public: - IniParseFindVisitor (IniDBBuilder &aBuilder, - const std::string& localroot, - IniParseFeedback &); - virtual void visitFile(const std::string& basePath, const WIN32_FIND_DATA *); - virtual ~ IniParseFindVisitor (); - - unsigned int timeStamp() const; - std::string version() const; - int iniCount() const; -protected: - IniParseFindVisitor (IniParseFindVisitor const &); - IniParseFindVisitor & operator= (IniParseFindVisitor const &); -private: - IniDBBuilder &_Builder; - IniParseFeedback &_feedback; - unsigned int baseLength; - int local_ini; - unsigned int setup_timestamp; - std::string setup_version; -}; - -#endif /* SETUP_INIPARSEFINDVISITOR_H */ diff --git a/Makefile.am b/Makefile.am index 88330d5..e9263cc 100644 --- a/Makefile.am +++ b/Makefile.am @@ -163,8 +163,6 @@ inilint_SOURCES = \ iniparse.yy \ IniParseFeedback.cc \ IniParseFeedback.h \ - IniParseFindVisitor.cc \ - IniParseFindVisitor.h \ install.cc \ io_stream.cc \ io_stream.h \ diff --git a/ini.cc b/ini.cc index e90f98c..92642b3 100644 --- a/ini.cc +++ b/ini.cc @@ -39,7 +39,6 @@ #include "mount.h" #include "site.h" #include "find.h" -#include "IniParseFindVisitor.h" #include "IniParseFeedback.h" #include "io_stream.h" @@ -57,11 +56,12 @@ extern ThreeBarProgressPage Progress; unsigned int setup_timestamp = 0; std::string ini_setup_version; -std::string current_ini_sig_name; +IniList setup_ext_list( setup_exts, setup_exts+3); // TODO: use C++11x initializer lists static BoolOption NoVerifyOption (false, 'X', "no-verify", "Don't verify setup.ini signatures"); extern int yyparse (); + /*extern int yydebug;*/ class GuiParseFeedback : public IniParseFeedback @@ -119,16 +119,145 @@ private: unsigned int lastpct; }; +static io_stream* +decompress_ini (io_stream *ini_file) +{ + // Replace the current compressed setup stream with its decompressed + // version. Which decompressor to use is determined by file magic. + io_stream *compressed_stream = compress::decompress (ini_file); + if (!compressed_stream) + { + /* This isn't a valid compressed file. */ + // TODO: if we could determine that the original input stream + // was an uncompressed file to begin with we could always try to + // uncompress and skip the corresponding check. + delete compressed_stream; + delete ini_file; + ini_file = NULL; + } + else + { + /* Decompress the entire file in memory. This has the advantage + that input_stream->get_size() will work during parsing and + we'll have an accurate status bar. Also, we can't seek + compressed streams, so when we write out a local cached copy + of the .ini file below, we'd otherwise have to delete this + stream and uncompress it again from the start, which is + wasteful. The current uncompressed size of the setup.ini + file as of 2015 is about 5 MiB, so this is not a great deal + of memory. */ + io_stream *uncompressed = new io_stream_memory (); + /* Note that the decompress io_stream now "owns" the underlying + compressed io_stream instance, so it need not be deleted + explicitly. */ + if ((io_stream::copy (compressed_stream, uncompressed) != 0) || + (compressed_stream->error() != 0)) + { + /* There was a problem decompressing compressed_stream. */ + Log (LOG_PLAIN) << + "Warning: Error code " << compressed_stream->error() << + " occurred while uncompressing " << current_ini_name << + " - possibly truncated or corrupt file. " + " Retrying with uncompressed version." << endLog; + delete uncompressed; + ini_file = NULL; + } + else + { + ini_file = uncompressed; + ini_file->seek (0, IO_SEEK_SET); + } + } + return ini_file; +} + +static io_stream* +check_ini_sig (io_stream* ini_file, io_stream* ini_sig_file, + bool& sig_fail, const char* site, const char* sig_name, HWND owner) +{ + // Unless the NoVerifyOption is set, check the signature for the + // current setup and record the result. On a failed signature check + // the streams are invalidated so even if we tried to read in the + // setup anyway there's be nothing to parse. + if (!NoVerifyOption && ini_file) + { + if (!ini_sig_file) + { + note (owner, IDS_SETUPINI_MISSING, sig_name, site); + delete ini_file; + ini_file = NULL; + sig_fail = true; + } + else if (!verify_ini_file_sig (ini_file, ini_sig_file, owner)) + { + note (owner, IDS_SIG_INVALID, sig_name, site); + delete ini_sig_file; + ini_sig_file = NULL; + delete ini_file; + ini_file = NULL; + sig_fail = true; + } + } + return ini_file; +} + static int -do_local_ini (HWND) +do_local_ini (HWND owner) { + size_t ini_count = 0; GuiParseFeedback myFeedback; - IniDBBuilderPackage findBuilder(myFeedback); - IniParseFindVisitor myVisitor (findBuilder, local_dir, myFeedback); - Find (local_dir).accept(myVisitor, 2); // Only search two levels deep. - setup_timestamp = myVisitor.timeStamp(); - ini_setup_version = myVisitor.version(); - return myVisitor.iniCount(); + IniDBBuilderPackage aBuilder(myFeedback); + io_stream *ini_file, *ini_sig_file; + // iterate over all setup files found in do_from_local_dir + for (IniList::const_iterator n = found_ini_list.begin(); + n != found_ini_list.end(); ++n) + { + bool sig_fail = false; + std::string current_ini_ext, current_ini_name, current_ini_sig_name; + + current_ini_name = *n; + current_ini_sig_name = current_ini_name + ".sig"; + current_ini_ext = current_ini_name.substr(current_ini_name.rfind(".") + 1); + ini_sig_file = io_stream::open("file://" + current_ini_sig_name, "rb", 0); + ini_file = io_stream::open("file://" + current_ini_name, "rb", 0); + ini_file = check_ini_sig (ini_file, ini_sig_file, sig_fail, + "localdir", current_ini_sig_name.c_str(), owner); + // did we find a compressed setup? + if (ini_file && + !(casecompare (current_ini_ext, "xz") && + casecompare (current_ini_ext, "bz2"))) + ini_file = decompress_ini(ini_file); + if (!ini_file || sig_fail) + { + // no setup found or signature invalid + note (owner, IDS_SETUPINI_MISSING, SetupBaseName.c_str(), + "localdir"); + } + else + { + // grok information from setup + myFeedback.babble("Found ini file - " + current_ini_name); + myFeedback.iniName (current_ini_name); + aBuilder.parse_mirror = ""; + ini_init (ini_file, &aBuilder, myFeedback); + + /*yydebug = 1; */ + + if (yyparse () || yyerror_count > 0) + myFeedback.error (yyerror_messages); + else + ++ini_count; + + if (aBuilder.timestamp > setup_timestamp) + { + setup_timestamp = aBuilder.timestamp; + ini_setup_version = aBuilder.version; + } + delete ini_file; + ini_file = NULL; + } + } + return ini_count; } static int @@ -140,146 +269,75 @@ do_remote_ini (HWND owner) io_stream *ini_file, *ini_sig_file; /* FIXME: Get rid of this io_stream pointer travesty. The need to - explicitly delete these things is ridiculous. Note that the - decompress io_stream "owns" the underlying compressed io_stream - instance, so it should not be deleted explicitly. */ + explicitly delete these things is ridiculous. */ + // iterate over all sites for (SiteList::const_iterator n = site_list.begin(); n != site_list.end(); ++n) { bool sig_fail = false; - /* First try to fetch the .bz2 compressed ini file. */ - current_ini_name = n->url + SETUP_INI_DIR + SETUP_BZ2_FILENAME; - current_ini_sig_name = n->url + SETUP_INI_DIR + SETUP_BZ2_FILENAME + ".sig"; - ini_file = get_url_to_membuf (current_ini_name, owner); - if (!NoVerifyOption) - ini_sig_file = get_url_to_membuf (current_ini_sig_name, owner); - if (!NoVerifyOption && ini_file && !ini_sig_file) + std::string current_ini_ext, current_ini_name, current_ini_sig_name; + // iterate over known extensions for setup + for (IniList::const_iterator ext = setup_ext_list.begin(); + ext != setup_ext_list.end(); + ext++) { - note (owner, IDS_SETUPINI_MISSING, current_ini_sig_name.c_str(), n->url.c_str()); - delete ini_file; - ini_file = NULL; - sig_fail = true; + current_ini_ext = *ext; + current_ini_name = n->url + SetupIniDir + SetupBaseName + "." + current_ini_ext; + current_ini_sig_name = current_ini_name + ".sig"; + ini_sig_file = get_url_to_membuf (current_ini_sig_name, owner); + ini_file = get_url_to_membuf (current_ini_name, owner); + ini_file = check_ini_sig (ini_file, ini_sig_file, sig_fail, + n->url.c_str(), current_ini_sig_name.c_str(), owner); + // stop searching as soon as we find a setup file + if (ini_file) + break; } - else if (!NoVerifyOption && ini_file && !verify_ini_file_sig (ini_file, ini_sig_file, owner)) + // did we find a compressed setup? + if (ini_file && + !(casecompare (current_ini_ext, "xz") && + casecompare (current_ini_ext, "bz2"))) + ini_file = decompress_ini(ini_file); + if (!ini_file || sig_fail) { - note (owner, IDS_SIG_INVALID, current_ini_sig_name.c_str(), n->url.c_str()); - delete ini_file; - ini_file = NULL; - delete ini_sig_file; - ini_sig_file = NULL; - sig_fail = true; + // no setup found or signature invalid + note (owner, IDS_SETUPINI_MISSING, SetupBaseName.c_str(), n->url.c_str()); } - if (ini_file) + else { - /* Decompress the entire file in memory right now. This has the - advantage that input_stream->get_size() will work during parsing - and we'll have an accurate status bar. Also, we can't seek - bz2 streams, so when it comes time to write out a local cached - copy of the .ini file below, we'd otherwise have to delete this - stream and uncompress it again from the start, which is wasteful. - The current uncompresed size of the .ini file as of 2007 is less - than 600 kB, so this is not a great deal of memory. */ - io_stream *bz2_stream = compress::decompress (ini_file); - if (!bz2_stream) - { - /* This isn't a valid bz2 file. */ - delete ini_file; - ini_file = NULL; - } + // grok information from setup + myFeedback.iniName (current_ini_name); + aBuilder.parse_mirror = n->url; + ini_init (ini_file, &aBuilder, myFeedback); + + /*yydebug = 1; */ + + if (yyparse () || yyerror_count > 0) + myFeedback.error (yyerror_messages); else { - io_stream *uncompressed = new io_stream_memory (); - - if ((io_stream::copy (bz2_stream, uncompressed) != 0) || - (bz2_stream->error() != 0)) - { - /* There was a problem decompressing bz2. */ - ini_file = NULL; - Log (LOG_PLAIN) << - "Warning: Error code " << bz2_stream->error() << " occurred while uncompressing " << - current_ini_name << " - possibly truncated or corrupt bzip2" - " file. Retrying with uncompressed version." << endLog; - delete bz2_stream; - delete uncompressed; - } - else + /* save known-good setup.ini locally */ + const std::string fp = "file://" + local_dir + "/" + + rfc1738_escape_part (n->url) + + "/" + SetupIniDir + SetupBaseName + ".ini"; + io_stream::mkpath_p (PATH_TO_FILE, fp, 0); + if (io_stream *out = io_stream::open (fp, "wb", 0)) { - delete bz2_stream; - ini_file = uncompressed; ini_file->seek (0, IO_SEEK_SET); + if (io_stream::copy (ini_file, out) != 0) + io_stream::remove (fp); + delete out; } + ++ini_count; } - } - - if (!ini_file) - { - /* Try to look for a plain .ini file because one of the following - happened above: - - there was no .bz2 file found on the mirror. - - the .bz2 file didn't look like a valid bzip2 file. - - there was an error during bzip2 decompression. */ - current_ini_name = n->url + SETUP_INI_DIR + SETUP_INI_FILENAME; - current_ini_sig_name = n->url + SETUP_INI_DIR + SETUP_INI_FILENAME + ".sig"; - ini_file = get_url_to_membuf (current_ini_name, owner); - if (!NoVerifyOption) - ini_sig_file = get_url_to_membuf (current_ini_sig_name, owner); - - if (!NoVerifyOption && ini_file && !ini_sig_file) - { - note (owner, IDS_SETUPINI_MISSING, current_ini_sig_name.c_str(), n->url.c_str()); - delete ini_file; - ini_file = NULL; - sig_fail = true; - } - else if (!NoVerifyOption && ini_file && !verify_ini_file_sig (ini_file, ini_sig_file, owner)) - { - note (owner, IDS_SIG_INVALID, current_ini_sig_name.c_str(), n->url.c_str()); - delete ini_file; - ini_file = NULL; - delete ini_sig_file; - ini_sig_file = NULL; - sig_fail = true; - } - } - - if (!ini_file) - { - if (!sig_fail) - note (owner, IDS_SETUPINI_MISSING, SETUP_INI_FILENAME, n->url.c_str()); - continue; - } - - myFeedback.iniName (current_ini_name); - aBuilder.parse_mirror = n->url; - ini_init (ini_file, &aBuilder, myFeedback); - - /*yydebug = 1; */ - - if (yyparse () || yyerror_count > 0) - myFeedback.error (yyerror_messages); - else - { - /* save known-good setup.ini locally */ - const std::string fp = "file://" + local_dir + "/" + - rfc1738_escape_part (n->url) + - "/" + SETUP_INI_DIR + SETUP_INI_FILENAME; - io_stream::mkpath_p (PATH_TO_FILE, fp, 0); - if (io_stream *out = io_stream::open (fp, "wb", 0)) + if (aBuilder.timestamp > setup_timestamp) { - ini_file->seek (0, IO_SEEK_SET); - if (io_stream::copy (ini_file, out) != 0) - io_stream::remove (fp); - delete out; + setup_timestamp = aBuilder.timestamp; + ini_setup_version = aBuilder.version; } - ++ini_count; - } - if (aBuilder.timestamp > setup_timestamp) - { - setup_timestamp = aBuilder.timestamp; - ini_setup_version = aBuilder.version; + delete ini_file; + ini_file = NULL; } - delete ini_file; } return ini_count; } @@ -374,4 +432,3 @@ do_ini (HINSTANCE h, HWND owner) DWORD threadID; CreateThread (NULL, 0, do_ini_thread_reflector, context, 0, &threadID); } - diff --git a/ini.h b/ini.h index c2903ef..164e3d2 100644 --- a/ini.h +++ b/ini.h @@ -48,10 +48,6 @@ typedef enum EXCLUDE_NOT_FOUND } excludes; -#define SETUP_INI_DIR SetupIniDir.c_str() -#define SETUP_INI_FILENAME (SetupBaseName+".ini").c_str() -#define SETUP_BZ2_FILENAME (SetupBaseName+".bz2").c_str() - /* The following three vars are used to facilitate error handling between the parser/lexer and its callers, namely ini.cc:do_remote_ini() and IniParseFindVisitor::visitFile(). */ -- 2.4.3 [-- Attachment #5: Type: text/plain, Size: 199 bytes --] Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-07-01 20:37 ` setup Achim Gratz @ 2015-07-02 9:10 ` Corinna Vinschen 2015-07-02 21:03 ` setup Achim Gratz 2015-07-11 21:31 ` setup Achim Gratz 0 siblings, 2 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-07-02 9:10 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 6180 bytes --] Hi Achim, On Jul 1 22:37, Achim Gratz wrote: > Corinna Vinschen writes: > > Ok, for once. But please make sure that you split the commit into > > functional chunks next time it's so large. And send it to this list, so > > code snippets can be referenced in the review. > > I've split it up into three parts that at least compile cleanly. Thanks! > +extern IniList found_ini_list, setup_ext_list; > +const std::string setup_exts[] = { "xz", "bz2", "ini" }; See (*) below. > @@ -97,6 +99,8 @@ static BoolOption PackageManagerOption (false, 'M', "package-manager", "Semi-att > static BoolOption NoAdminOption (false, 'B', "no-admin", "Do not check for and enforce running as Administrator"); > static BoolOption WaitOption (false, 'W', "wait", "When elevating, wait for elevated child process"); > static BoolOption HelpOption (false, 'h', "help", "print help"); > +static StringOption SetupBaseNameOpt ("setup", 'i', "ini-basename", "Use a different basename instead of setup", false); The word setup needs quoting of some sort I think. An example wouldn't hurt either. What about "Use a different ini file basename instead of \"setup\", e.g. \"foo\".ini" ? > + (theFile->nFileSizeLow || theFile->nFileSizeHigh)) > + { > + if (!casecompare (SetupBaseName + ".xz", theFile->cFileName)) > + found_xz = true; > + if (!casecompare (SetupBaseName + ".bz2", theFile->cFileName)) > + found_bz2 = true; > + if (!casecompare (SetupBaseName + ".ini", theFile->cFileName)) > + found_ini = true; > + } (*) This puzzles me a bit. You're keeping arrays and lists in terms of the file suffix (setup_ext, setup_ext_list), but you don't use the information here and elsewhere. The setup_exts array already contains the suffixes and constitutes an order. If you extend the array of strings to a struct, you could just run a loop over it in places like the above and generalize the suffix handling. For instance... struct ini_suffix_t { char *suffix; bool needs_decompressing; [...] } ini_suffix_t ini_suffixes[] = { { "xz", true }, { "bz2", true }, { "ini", false }, { NULL, false } }; bool found_ini[]; // <- just as example for (i = 0; ini_suffixes[i].suffix; ++i) if (!casecompare (SetupBaseName + "." + ini_suffixes[i].suffix, ...) found_ini[i] = true; This would work OOTB, without C++11x initializer lists, and you could drop setup_ext_list entirely. This would also simplify things if there's a new compression we want to support. > + if (found_xz) > + found_ini_list.push_back(basePath + "/" + aDir->cFileName + "/" + SetupBaseName + ".xz"); > + else if (found_bz2) > + found_ini_list.push_back(basePath + "/" + aDir->cFileName + "/" + SetupBaseName + ".bz2"); > + else if (found_ini) > + found_ini_list.push_back(basePath + "/" + aDir->cFileName + "/" + SetupBaseName + ".ini"); Same kind of loop here. for (i = 0; ini_suffixes[i].suffix; ++i) if (found_ini[i]) found_ini_list.push_back (... + ini_suffixes[i].suffix); Ideally found_ini_list keeps a pointer into ini_suffixes as well so you don't have to extract the suffix again at (**). > Subject: [PATCH 3/5] Refactor setup search and implement XZ compressed setup > files > > * ini.cc: Construct setup_ext_list from array until we can use > C++11 aggregate initializers. > (decompress_ini): Refactored for use from do_local_ini and > do_remote_ini. Change outdated comment about setup.ini > uncompressed size. > (check_ini_sig): Factor out signature check. > (fetch_remote_ini): Refactored for use from do_remote_ini. > (do_local_ini): Iterate over search results in found_ini_list. > Use ini_decompress and check_ini_sig. ^^^^^^^^^^^^^^ missing name change > + // Unless the NoVerifyOption is set, check the signature for the > + // current setup and record the result. On a failed signature check > + // the streams are invalidated so even if we tried to read in the > + // setup anyway there's be nothing to parse. I'd prefer /* */ for multiline comments, but that's used pretty inconsistently anyway, so, never mind. > + IniDBBuilderPackage aBuilder(myFeedback); > + io_stream *ini_file, *ini_sig_file; > + // iterate over all setup files found in do_from_local_dir > + for (IniList::const_iterator n = found_ini_list.begin(); > + n != found_ini_list.end(); ++n) > + { > + bool sig_fail = false; > + std::string current_ini_ext, current_ini_name, current_ini_sig_name; > + > + current_ini_name = *n; > + current_ini_sig_name = current_ini_name + ".sig"; > + current_ini_ext = current_ini_name.substr(current_ini_name.rfind(".") + 1); (**) See above. > + ini_sig_file = io_stream::open("file://" + current_ini_sig_name, "rb", 0); > + ini_file = io_stream::open("file://" + current_ini_name, "rb", 0); > + ini_file = check_ini_sig (ini_file, ini_sig_file, sig_fail, > + "localdir", current_ini_sig_name.c_str(), owner); The existing code is inconsistently formatted, but for new code it would be nice if we could try to be more consistent. Always prepend a space to a left parenthesis, please. > + // did we find a compressed setup? > + if (ini_file && > + !(casecompare (current_ini_ext, "xz") && > + casecompare (current_ini_ext, "bz2"))) > + ini_file = decompress_ini(ini_file); (*) And this could simply use the information from ini_suffixes if "current_ini_ext" wouldn't be the actual extension, but rather a pointer into ini_suffixes. Kind of like if (ini_file && current_ini_ext->needs_decompressing) ini_file = decompress_ini (ini_file); Sorry if that's a lot. It just occured to me while reading your code. I'm not adamant about the structural change I outlined above, but to me it seems better to do it that way. What do you think? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-07-02 9:10 ` setup Corinna Vinschen @ 2015-07-02 21:03 ` Achim Gratz 2015-07-03 10:14 ` setup Corinna Vinschen 2015-07-11 21:31 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-07-02 21:03 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > (*) This puzzles me a bit. You're keeping arrays and lists in terms of > the file suffix (setup_ext, setup_ext_list), but you don't use the > information here and elsewhere. THat vector is a relatively late addition to the code when the rest was already using literals as before. It's something that I want to clean up, but not necessarily right now. As you noted, I'd probably need to use a more structured data type. > I'd prefer /* */ for multiline comments, but that's used pretty > inconsistently anyway, so, never mind. If you insist… but setup is C++ and not C anyway. > The existing code is inconsistently formatted, but for new code it would > be nice if we could try to be more consistent. Always prepend a space > to a left parenthesis, please. I'll check that. > Sorry if that's a lot. It just occured to me while reading your code. > I'm not adamant about the structural change I outlined above, but to > me it seems better to do it that way. What do you think? It will just have to wait a bit, I think. But yes, these are all good suggestions. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf microQ V2.22R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-07-02 21:03 ` setup Achim Gratz @ 2015-07-03 10:14 ` Corinna Vinschen 0 siblings, 0 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-07-03 10:14 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1609 bytes --] On Jul 2 23:03, Achim Gratz wrote: > Corinna Vinschen writes: > > (*) This puzzles me a bit. You're keeping arrays and lists in terms of > > the file suffix (setup_ext, setup_ext_list), but you don't use the > > information here and elsewhere. > > THat vector is a relatively late addition to the code when the rest was > already using literals as before. It's something that I want to clean > up, but not necessarily right now. As you noted, I'd probably need to > use a more structured data type. > > > I'd prefer /* */ for multiline comments, but that's used pretty > > inconsistently anyway, so, never mind. > > If you insist… but setup is C++ and not C anyway. Not so much insisting, just a wish. > > The existing code is inconsistently formatted, but for new code it would > > be nice if we could try to be more consistent. Always prepend a space > > to a left parenthesis, please. > > I'll check that. > > > Sorry if that's a lot. It just occured to me while reading your code. > > I'm not adamant about the structural change I outlined above, but to > > me it seems better to do it that way. What do you think? > > It will just have to wait a bit, I think. But yes, these are all good > suggestions. Cool. If you want to check in the current state (plus formatting tweaks), feel free. It would just be nice if you would consider to make the structural changes at one point. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-07-02 9:10 ` setup Corinna Vinschen 2015-07-02 21:03 ` setup Achim Gratz @ 2015-07-11 21:31 ` Achim Gratz 2015-07-13 9:44 ` setup Corinna Vinschen 1 sibling, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-07-11 21:31 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 346 bytes --] Corinna Vinschen writes: > Sorry if that's a lot. It just occured to me while reading your code. > I'm not adamant about the structural change I outlined above, but to > me it seems better to do it that way. What do you think? I've implemented your suggestions except the struct stuff. I've not as thoroughly tested it as last time, though. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Implement-option-i-ini-basename-to-set-the-basename-.patch --] [-- Type: text/x-patch, Size: 3489 bytes --] From 296c4272b13d8fdd3a16bb231fc2d2bfe3ba6c29 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Sun, 28 Jun 2015 13:34:11 +0200 Subject: [PATCH 1/6] Implement option "-i/--ini-basename" to set the basename for setup * ini.h: Declare extern SetupBaseName, SetupArch and SetupIniDir. Redefine macros to use them. * main.cc: Provide option "-i/--ini-basename" to set the basename for setup, keep the default as "setup" and store in SetupBaseName. Initialize SetupArch and SetupIniDir based on effective architecture. --- ChangeLog | 10 ++++++++++ ini.h | 17 +++++++++++++---- main.cc | 8 ++++++++ 3 files changed, 31 insertions(+), 4 deletions(-) diff --git a/ChangeLog b/ChangeLog index 1c50bfc..cb49b7e 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,13 @@ +2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + + * ini.h: Declare extern SetupBaseName, SetupArch and SetupIniDir. + Redefine macros to use them. + + * main.cc: Provide option "-i/--ini-basename" to set the basename + for setup, keep the default as "setup" and store in SetupBaseName. + Initialize SetupArch and SetupIniDir based on effective + architecture. + 2015-06-11 Achim Gratz <Stromeko@NexGo.DE> * inilex.ll: Increase read buffer size for the lexer input buffer diff --git a/ini.h b/ini.h index ec09def..1969da9 100644 --- a/ini.h +++ b/ini.h @@ -18,6 +18,16 @@ class io_stream; #include <string> +#include <vector> + +typedef std::vector <std::string> IniList; +extern IniList found_ini_list, setup_ext_list; +const std::string setup_exts[] = { "xz", "bz2", "ini" }; +extern bool is_64bit; +extern std::string SetupArch; +extern std::string SetupIniDir; +extern std::string SetupBaseName; + class IniState; class IniDBBuilder; class IniParseFeedback; @@ -38,10 +48,9 @@ typedef enum EXCLUDE_NOT_FOUND } excludes; -extern bool is_64bit; -#define SETUP_INI_DIR (is_64bit ? "x86_64/" : "x86/") -#define SETUP_INI_FILENAME "setup.ini" -#define SETUP_BZ2_FILENAME "setup.bz2" +#define SETUP_INI_DIR (SetupIniDir.c_str ()) +#define SETUP_INI_FILENAME ((SetupBaseName+".ini").c_str ()) +#define SETUP_BZ2_FILENAME ((SetupBaseName+".bz2").c_str ()) /* The following three vars are used to facilitate error handling between the parser/lexer and its callers, namely ini.cc:do_remote_ini() and diff --git a/main.cc b/main.cc index e968d93..b0d78eb 100644 --- a/main.cc +++ b/main.cc @@ -86,6 +86,8 @@ extern char **_argv; #endif bool is_64bit; +std::string SetupArch; +std::string SetupIniDir; using namespace std; @@ -97,6 +99,8 @@ static BoolOption PackageManagerOption (false, 'M', "package-manager", "Semi-att static BoolOption NoAdminOption (false, 'B', "no-admin", "Do not check for and enforce running as Administrator"); static BoolOption WaitOption (false, 'W', "wait", "When elevating, wait for elevated child process"); static BoolOption HelpOption (false, 'h', "help", "print help"); +static StringOption SetupBaseNameOpt ("setup", 'i', "ini-basename", "Use a different basename, e.g. \"foo\", instead of \"setup\"", false); +std::string SetupBaseName; static void inline set_cout () @@ -262,6 +266,10 @@ WinMain (HINSTANCE h, if (unattended_mode || help_option) set_cout (); + SetupBaseName = SetupBaseNameOpt; + SetupArch = is_64bit ? "x86_64" : "x86"; + SetupIniDir = SetupArch+"/"; + /* Get System info */ OSVERSIONINFO version; version.dwOSVersionInfoSize = sizeof version; -- 2.4.5 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Correct-local-directory-search.patch --] [-- Type: text/x-patch, Size: 8560 bytes --] From 35addac4a65a1d70ab02b8d175f71efa83cf74ba Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Wed, 1 Jul 2015 21:36:12 +0200 Subject: [PATCH 2/6] Correct local directory search * fromcwd.cc: Remove unused includes. Add global found_ini_list to record the search result. (SetupFindVisitor): Make setup.{ini,bz2,xz} known and provide bool private variables to record whether we found them. Another bool inidir to indicate whether we are currently inside a directory where setup.ini files should exist. (SetupFindVisitor::visitFile): When inidir is true, check if a setup file with one of the known extensions was found and set the corresponding bool variables. (SetupFindVisitor::visitDirectory): Set inidir when appropriate. Recurse into directories only if they are potential mirror dirs, based on level. Truncate search and recurse into inidir. Record any setup files in found_ini_list while preferring ".xz" over ".bz2" over ".ini" extension. (SetupFindVisitor::operator bool): Return true when found_ini_list is non-empty. (do_from_local_dir): Restrict search to either a single mirror hierarchy or multiple mirror hierarchy; setup files directly in local directory or mixed hierarchies are no longer recognized. The setup files must be present in an architecture dependent directory of "x86/" or "x86_64/", either in the local directory for single mirror or one level down for multiple mirrors. This patch already finds setup.xz files, but the code to deal with them will be in a later patch. The current code re-does the search and ignores the results from here. --- ChangeLog | 25 ++++++++++ fromcwd.cc | 154 ++++++++++++++++++++++++++++++++++++++++++------------------- 2 files changed, 132 insertions(+), 47 deletions(-) diff --git a/ChangeLog b/ChangeLog index cb49b7e..e21e2e6 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,30 @@ 2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + * fromcwd.cc: Remove unused includes. Add global found_ini_list + to record the search result. + (SetupFindVisitor): Make setup.{ini,bz2,xz} known and provide bool + private variables to record whether we found them. Another bool + inidir to indicate whether we are currently inside a drectory + where setup.ini files should exist. + (SetupFindVisitor::visitFile): When inidir is true, check if a + setup file with one of the known extensions was found and set the + corresponding bool variables. + (SetupFindVisitor::visitDirectory): Set inidir when appropriate. + Recurse into directories only if they are potential mirror dirs, + based on level. Truncate search and recurse into inidir. Record + any setup files in found_ini_list while preferring ".xz" over + ".bz2" over ".ini" extension. + (SetupFindVisitor::operator bool): Return true when found_ini_list + is non-empty. + (do_from_local_dir): Restrict search to either a single mirror + hierarchy or multiple mirror hierarchy; setup files directly in + local directory or mixed hierarchies are no longer recognized. + The setup files must be present in an architecture dependent + directory of "x86/" or "x86_64/", either in the local directory + for single mirror or one level down for multiple mirrors. + +2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + * ini.h: Declare extern SetupBaseName, SetupArch and SetupIniDir. Redefine macros to use them. diff --git a/fromcwd.cc b/fromcwd.cc index 87da0ad..edb67a6 100644 --- a/fromcwd.cc +++ b/fromcwd.cc @@ -25,77 +25,137 @@ (otherwise, why would you have asked to install it?). Note that we search recursively. */ -#if 0 -static const char *cvsid = - "\n%%% $Id$\n"; -#endif - -#include "win32.h" - -#include <stdio.h> -#include <stdlib.h> -#include <unistd.h> -#include "resource.h" -#include "state.h" -#include "dialog.h" -#include "msg.h" +#include "String++.h" #include "find.h" -#include "ScanFindVisitor.h" -#include "filemanip.h" #include "ini.h" +#include "ScanFindVisitor.h" #include "IniDBBuilderPackage.h" #include "IniParseFeedback.h" +#define DEBUG_FROMCWD +#ifdef DEBUG_FROMCWD +#include "LogFile.h" +#endif + /* Trivial class for detecting the existence of setup.ini */ class SetupFindVisitor : public FindVisitor { public: - SetupFindVisitor (): found(false){} - virtual void visitFile(const std::string& basePath, - const WIN32_FIND_DATA *theFile) - { - if (!casecompare (SETUP_INI_FILENAME, theFile->cFileName) && - (theFile->nFileSizeLow || theFile->nFileSizeHigh)) - { - /* Check if base dir ends in SETUP_INI_DIR. */ - const char *dir = basePath.c_str() + basePath.size () - - strlen (SETUP_INI_DIR); - if (dir < basePath.c_str ()) - return; - if ((dir != basePath.c_str () && dir[-1] != '/' && dir[-1] != '\\') - || casecompare (SETUP_INI_DIR, dir)) - return; - found = true; - } - } + SetupFindVisitor () : inidir (false) + { + found_ini.resize (setup_ext_list.size ()); + found_ini.assign (setup_ext_list.size (), false); + } + virtual void visitFile (const std::string& basePath, + const WIN32_FIND_DATA *theFile) + { +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << "examining file: " + << basePath << "./" << theFile->cFileName << endLog; +#endif + if (inidir && + (theFile->nFileSizeLow || theFile->nFileSizeHigh)) + { +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << "checking extension: "; +#endif + std::vector<bool>::iterator fi = found_ini.begin (); + for (std::vector<std::string>::const_iterator ext = setup_ext_list.begin (); + ext != setup_ext_list.end (); + ext++, fi++) + { +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << *ext << " "; +#endif + if (!casecompare (SetupBaseName + "." + *ext, theFile->cFileName)) + *fi = true; + } +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << endLog; +#endif + } +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << "state: " + << " inidir=" << inidir + << " found_xz=" << found_ini[0] + << " found_bz2=" << found_ini[1] + << " found_ini=" << found_ini[2] + << endLog; +#endif + } + virtual void visitDirectory (const std::string& basePath, + WIN32_FIND_DATA const *aDir, int level) + { +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << "examining directory: " + << basePath << "./" << aDir->cFileName + << endLog; +#endif + if (level <= 0) + return; + inidir = !casecompare (SetupArch, aDir->cFileName); + if (level == 1 && !inidir) + return; +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << " recurse into: " + << (inidir ? "inidir" : "mirror") << endLog; +#endif + Find aFinder (basePath + aDir->cFileName); + aFinder.accept (*this, inidir ? 0 : --level); + std::vector<bool>::const_iterator fi = found_ini.begin (); + for (std::vector<std::string>::const_iterator ext = setup_ext_list.begin (); + ext != setup_ext_list.end (); + ext++, fi++) + { + if (*fi) + found_ini_list.push_back (basePath + "/" + aDir->cFileName + "/" + + SetupBaseName + "." + *ext); + } + found_ini.assign (setup_ext_list.size (), false); + } virtual ~ SetupFindVisitor (){} - operator bool () const {return found;} + operator bool () const + { + return !found_ini_list.empty (); + } protected: SetupFindVisitor (SetupFindVisitor const &); SetupFindVisitor & operator= (SetupFindVisitor const &); private: - bool found; + bool inidir; + std::vector<bool> found_ini; }; - + +IniList found_ini_list; + bool do_from_local_dir (HINSTANCE h, HWND owner, std::string &local_dir) { +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << "do_from_local_dir: " + << local_dir << endLog; +#endif // Assume we won't find the INI file. - SetupFindVisitor found_ini; - // Only search two levels deep. - Find (local_dir.c_str ()).accept(found_ini, 2); - if (found_ini) - { - // Found INI, load it. + SetupFindVisitor found; + // single mirror? + Find (local_dir.c_str ()).accept (found, 1); + if (found) return true; - } - + // multi-mirror? + Find (local_dir.c_str ()).accept (found, 2); + if (found) + return true; + // nope, do full scan. +#ifdef DEBUG_FROMCWD + Log (LOG_PLAIN) << " starting full scan from " + << local_dir << endLog; +#endif IniParseFeedback myFeedback; - IniDBBuilderPackage myBuilder(myFeedback); + IniDBBuilderPackage myBuilder (myFeedback); ScanFindVisitor myVisitor (myBuilder); - Find(local_dir.c_str ()).accept(myVisitor); + Find (local_dir.c_str ()).accept (myVisitor); return false; } -- 2.4.5 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-Refactor-setup-search-and-implement-XZ-compressed-se.patch --] [-- Type: text/x-patch, Size: 24897 bytes --] From c843bf7d1e10f4d2e3d63807410298aca8cf63c4 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Sat, 11 Jul 2015 21:58:30 +0200 Subject: [PATCH 3/6] Refactor setup search and implement XZ compressed setup files * ini.cc: Construct setup_ext_list from array until we can use C++11 aggregate initializers. Clean up function parenthesis. (decompress_ini): Refactored for use from do_local_ini and do_remote_ini. Change outdated comment about setup.ini uncompressed size. (check_ini_sig): Factor out signature check. (fetch_remote_ini): Refactored for use from do_remote_ini. (do_local_ini): Iterate over search results in found_ini_list. Use decompress_ini and check_ini_sig. (do_remote_ini): Iterate over known setup file extensions from setup_ext_list with early-out semantics, preferring ".xz" over ".bz2" over ".ini" extension. Use fetch_remote_ini and check_ini_sig. * ini.h: Remove unused macros. * IniParseFindVisitor.cc: Remove, the search is already done by SetupFindVisitor in do_from_local_dir. * IniParseFindVisitor.cc: Ditto. * Makefile.am (@SETUP@_SOURCES): Ditto. --- ChangeLog | 21 +++ IniParseFindVisitor.cc | 116 ----------------- IniParseFindVisitor.h | 50 ------- Makefile.am | 2 - ini.cc | 345 ++++++++++++++++++++++++++++--------------------- ini.h | 4 - 6 files changed, 217 insertions(+), 321 deletions(-) delete mode 100644 IniParseFindVisitor.cc delete mode 100644 IniParseFindVisitor.h diff --git a/ChangeLog b/ChangeLog index e21e2e6..5fc1c86 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,26 @@ 2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + * ini.cc: Construct setup_ext_list from array until we can use + C++11 aggregate initializers. + (decompress_ini): Refactored for use from do_local_ini and + do_remote_ini. Change outdated comment about setup.ini + uncompressed size. + (check_ini_sig): Factor out signature check. + (fetch_remote_ini): Refactored for use from do_remote_ini. + (do_local_ini): Iterate over search results in found_ini_list. + Use decompress_ini and check_ini_sig. + (do_remote_ini): Iterate over known setup file extensions from + setup_ext_list with early-out semantics, preferring ".xz" over + ".bz2" over ".ini" extension. Use fetch_remote_ini and + check_ini_sig. + * ini.h: Remove unused macros. + * IniParseFindVisitor.cc: Remove, the search is already done by + SetupFindVisitor in do_from_local_dir. + * IniParseFindVisitor.cc: Ditto. + * Makefile.am (@SETUP@_SOURCES): Ditto. + +2015-07-01 Achim Gratz <Stromeko@NexGo.DE> + * fromcwd.cc: Remove unused includes. Add global found_ini_list to record the search result. (SetupFindVisitor): Make setup.{ini,bz2,xz} known and provide bool diff --git a/IniParseFindVisitor.cc b/IniParseFindVisitor.cc deleted file mode 100644 index 871d6de..0000000 --- a/IniParseFindVisitor.cc +++ /dev/null @@ -1,116 +0,0 @@ -/* - * Copyright (c) 2002,2007 Robert Collins. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * A copy of the GNU General Public License can be found at - * http://www.gnu.org/ - * - * Written by Robert Collins <robertc@hotmail.com> - * - */ - -#if 0 -static const char *cvsid = - "\n%%% $Id$\n"; -#endif - -#include "IniParseFindVisitor.h" - -#include "csu_util/rfc1738.h" - -#include "IniParseFeedback.h" -#include "IniDBBuilder.h" -#include "io_stream.h" -#include "ini.h" -#include <stdexcept> - -using namespace std; - -extern int yyparse (); - -IniParseFindVisitor::IniParseFindVisitor(IniDBBuilder &aBuilder, - const std::string& localroot, - IniParseFeedback &feedback) - : _Builder (aBuilder), _feedback (feedback), baseLength (localroot.size()), - local_ini(0), setup_timestamp (0), setup_version() -{} - -IniParseFindVisitor::~IniParseFindVisitor(){} - -/* look for potential packages we can add to the in-memory package - * database - */ -void -IniParseFindVisitor::visitFile(const std::string& basePath, - const WIN32_FIND_DATA *theFile) -{ - //TODO: Test for case sensitivity issues - if (casecompare(SETUP_INI_FILENAME, theFile->cFileName)) - return; - - const char *dir = basePath.c_str () + basePath.size() - strlen (SETUP_INI_DIR); - if (dir < basePath.c_str ()) - return; - if ((dir != basePath.c_str () && dir[-1] != '/' && dir[-1] != '\\') || casecompare (SETUP_INI_DIR, dir)) - return; - - current_ini_name = basePath + theFile->cFileName; - - io_stream *ini_file = io_stream::open("file://" + current_ini_name, "rb", 0); - - if (!ini_file) - // We don't throw an exception, because while this is fatal to parsing, it - // isn't to the visitation. - { - // This should never happen - // If we want to handle it happening, use the log strategy call - throw new runtime_error ("IniParseFindVisitor: failed to open ini file, which should never happen"); - return; - } - - _feedback.babble("Found ini file - " + current_ini_name); - _feedback.iniName (current_ini_name); - - /* Copy leading part of path to temporary buffer and unescape it */ - - size_t pos = baseLength + 1; - size_t len = basePath.size () - (pos + strlen (SETUP_INI_DIR) + 1); - _Builder.parse_mirror = len <= 0 ? "" - : rfc1738_unescape (basePath.substr (pos, len)); - ini_init (ini_file, &_Builder, _feedback); - - /*yydebug = 1; */ - - if (yyparse () || yyerror_count > 0) - _feedback.error (yyerror_messages); - else - local_ini++; - - if (_Builder.timestamp > setup_timestamp) - { - setup_timestamp = _Builder.timestamp; - setup_version = _Builder.version; - } -} - -int -IniParseFindVisitor::iniCount() const -{ - return local_ini; -} - -unsigned int -IniParseFindVisitor::timeStamp () const -{ - return setup_timestamp; -} - -std::string -IniParseFindVisitor::version() const -{ - return setup_version; -} diff --git a/IniParseFindVisitor.h b/IniParseFindVisitor.h deleted file mode 100644 index ff48f70..0000000 --- a/IniParseFindVisitor.h +++ /dev/null @@ -1,50 +0,0 @@ -/* - * Copyright (c) 2002,2007 Robert Collins. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * A copy of the GNU General Public License can be found at - * http://www.gnu.org/ - * - * Written by Robert Collins <robertc@hotmail.com> - * - */ - -#ifndef SETUP_INIPARSEFINDVISITOR_H -#define SETUP_INIPARSEFINDVISITOR_H - -#include "FindVisitor.h" - - -/* parse passed in setup.ini files from disk. */ -class IniDBBuilder; -class IniParseFeedback; -/* IniParse files and create a package db when no cached .ini exists */ -class IniParseFindVisitor : public FindVisitor -{ -public: - IniParseFindVisitor (IniDBBuilder &aBuilder, - const std::string& localroot, - IniParseFeedback &); - virtual void visitFile(const std::string& basePath, const WIN32_FIND_DATA *); - virtual ~ IniParseFindVisitor (); - - unsigned int timeStamp() const; - std::string version() const; - int iniCount() const; -protected: - IniParseFindVisitor (IniParseFindVisitor const &); - IniParseFindVisitor & operator= (IniParseFindVisitor const &); -private: - IniDBBuilder &_Builder; - IniParseFeedback &_feedback; - unsigned int baseLength; - int local_ini; - unsigned int setup_timestamp; - std::string setup_version; -}; - -#endif /* SETUP_INIPARSEFINDVISITOR_H */ diff --git a/Makefile.am b/Makefile.am index 88330d5..e9263cc 100644 --- a/Makefile.am +++ b/Makefile.am @@ -163,8 +163,6 @@ inilint_SOURCES = \ iniparse.yy \ IniParseFeedback.cc \ IniParseFeedback.h \ - IniParseFindVisitor.cc \ - IniParseFindVisitor.h \ install.cc \ io_stream.cc \ io_stream.h \ diff --git a/ini.cc b/ini.cc index e90f98c..571ee2d 100644 --- a/ini.cc +++ b/ini.cc @@ -39,7 +39,6 @@ #include "mount.h" #include "site.h" #include "find.h" -#include "IniParseFindVisitor.h" #include "IniParseFeedback.h" #include "io_stream.h" @@ -57,11 +56,12 @@ extern ThreeBarProgressPage Progress; unsigned int setup_timestamp = 0; std::string ini_setup_version; -std::string current_ini_sig_name; +IniList setup_ext_list ( setup_exts, setup_exts+3); // TODO: use C++11x initializer lists static BoolOption NoVerifyOption (false, 'X', "no-verify", "Don't verify setup.ini signatures"); extern int yyparse (); + /*extern int yydebug;*/ class GuiParseFeedback : public IniParseFeedback @@ -73,7 +73,7 @@ public: Progress.SetText3 (""); Progress.SetText4 ("Progress:"); } - virtual void progress(unsigned long const pos, unsigned long const max) + virtual void progress (unsigned long const pos, unsigned long const max) { if (!max) /* length not known or eof */ @@ -87,199 +87,247 @@ public: /* Log (LOG_BABBLE) << lastpct << "% (" << pos << " of " << max << " bytes of ini file read)" << endLog; */ } - Progress.SetBar1(pos, max); + Progress.SetBar1 (pos, max); static char buf[100]; - sprintf(buf, "%d %% (%ldk/%ldk)", lastpct, pos/1000, max/1000); - Progress.SetText3(buf); + sprintf (buf, "%d %% (%ldk/%ldk)", lastpct, pos/1000, max/1000); + Progress.SetText3 (buf); } virtual void iniName (const std::string& name) { Progress.SetText1 ("Parsing..."); - Progress.SetText2 (name.c_str()); + Progress.SetText2 (name.c_str ()); Progress.SetText3 (""); } - virtual void babble(const std::string& message)const + virtual void babble (const std::string& message)const { Log (LOG_BABBLE) << message << endLog; } virtual void warning (const std::string& message)const { - mbox (0, message.c_str(), "Warning", 0); + mbox (0, message.c_str (), "Warning", 0); } - virtual void error(const std::string& message)const + virtual void error (const std::string& message)const { - mbox (0, message.c_str(), "Parse Errors", 0); + mbox (0, message.c_str (), "Parse Errors", 0); } virtual ~ GuiParseFeedback () { - Progress.SetText4("Package:"); + Progress.SetText4 ("Package:"); } private: unsigned int lastpct; }; -static int -do_local_ini (HWND) +static io_stream* +decompress_ini (io_stream *ini_file) { - GuiParseFeedback myFeedback; - IniDBBuilderPackage findBuilder(myFeedback); - IniParseFindVisitor myVisitor (findBuilder, local_dir, myFeedback); - Find (local_dir).accept(myVisitor, 2); // Only search two levels deep. - setup_timestamp = myVisitor.timeStamp(); - ini_setup_version = myVisitor.version(); - return myVisitor.iniCount(); + // Replace the current compressed setup stream with its decompressed + // version. Which decompressor to use is determined by file magic. + io_stream *compressed_stream = compress::decompress (ini_file); + if (!compressed_stream) + { + /* This isn't a known compression format or an uncompressed file + stream. Pass it on in case it was uncompressed, it will + generate a parser error if it was some unknown format. */ + delete compressed_stream; + } + else + { + /* Decompress the entire file in memory. This has the advantage + that input_stream->get_size () will work during parsing and + we'll have an accurate status bar. Also, we can't seek + compressed streams, so when we write out a local cached copy + of the .ini file below, we'd otherwise have to delete this + stream and uncompress it again from the start, which is + wasteful. The current uncompressed size of the setup.ini + file as of 2015 is about 5 MiB, so this is not a great deal + of memory. */ + io_stream *uncompressed = new io_stream_memory (); + /* Note that the decompress io_stream now "owns" the underlying + compressed io_stream instance, so it need not be deleted + explicitly. */ + if ((io_stream::copy (compressed_stream, uncompressed) != 0) || + (compressed_stream->error () != 0)) + { + /* There was a problem decompressing compressed_stream. */ + Log (LOG_PLAIN) << + "Warning: Error code " << compressed_stream->error () << + " occurred while uncompressing " << current_ini_name << + " - possibly truncated or corrupt file. " << endLog; + delete uncompressed; + ini_file = NULL; + } + else + { + ini_file = uncompressed; + ini_file->seek (0, IO_SEEK_SET); + } + } + return ini_file; } -static int -do_remote_ini (HWND owner) +static io_stream* +check_ini_sig (io_stream* ini_file, io_stream* ini_sig_file, + bool& sig_fail, const char* site, const char* sig_name, HWND owner) { - size_t ini_count = 0; - GuiParseFeedback myFeedback; - IniDBBuilderPackage aBuilder(myFeedback); - io_stream *ini_file, *ini_sig_file; - - /* FIXME: Get rid of this io_stream pointer travesty. The need to - explicitly delete these things is ridiculous. Note that the - decompress io_stream "owns" the underlying compressed io_stream - instance, so it should not be deleted explicitly. */ - - for (SiteList::const_iterator n = site_list.begin(); - n != site_list.end(); ++n) + /* Unless the NoVerifyOption is set, check the signature for the + current setup and record the result. On a failed signature check + the streams are invalidated so even if we tried to read in the + setup anyway there's be nothing to parse. */ + if (!NoVerifyOption && ini_file) { - bool sig_fail = false; - /* First try to fetch the .bz2 compressed ini file. */ - current_ini_name = n->url + SETUP_INI_DIR + SETUP_BZ2_FILENAME; - current_ini_sig_name = n->url + SETUP_INI_DIR + SETUP_BZ2_FILENAME + ".sig"; - ini_file = get_url_to_membuf (current_ini_name, owner); - if (!NoVerifyOption) - ini_sig_file = get_url_to_membuf (current_ini_sig_name, owner); - if (!NoVerifyOption && ini_file && !ini_sig_file) + if (!ini_sig_file) { - note (owner, IDS_SETUPINI_MISSING, current_ini_sig_name.c_str(), n->url.c_str()); + note (owner, IDS_SETUPINI_MISSING, sig_name, site); delete ini_file; ini_file = NULL; sig_fail = true; } - else if (!NoVerifyOption && ini_file && !verify_ini_file_sig (ini_file, ini_sig_file, owner)) + else if (!verify_ini_file_sig (ini_file, ini_sig_file, owner)) { - note (owner, IDS_SIG_INVALID, current_ini_sig_name.c_str(), n->url.c_str()); - delete ini_file; - ini_file = NULL; + note (owner, IDS_SIG_INVALID, sig_name, site); delete ini_sig_file; ini_sig_file = NULL; + delete ini_file; + ini_file = NULL; sig_fail = true; } + } + return ini_file; +} + +static int +do_local_ini (HWND owner) +{ + size_t ini_count = 0; + GuiParseFeedback myFeedback; + IniDBBuilderPackage aBuilder (myFeedback); + io_stream *ini_file, *ini_sig_file; + // iterate over all setup files found in do_from_local_dir + for (IniList::const_iterator n = found_ini_list.begin (); + n != found_ini_list.end (); ++n) + { + bool sig_fail = false; + std::string current_ini_ext, current_ini_name, current_ini_sig_name; + + current_ini_name = *n; + current_ini_sig_name = current_ini_name + ".sig"; + current_ini_ext = current_ini_name.substr (current_ini_name.rfind (".") + 1); + ini_sig_file = io_stream::open ("file://" + current_ini_sig_name, "rb", 0); + ini_file = io_stream::open ("file://" + current_ini_name, "rb", 0); + ini_file = check_ini_sig (ini_file, ini_sig_file, sig_fail, + "localdir", current_ini_sig_name.c_str (), owner); if (ini_file) + ini_file = decompress_ini (ini_file); + if (!ini_file || sig_fail) { - /* Decompress the entire file in memory right now. This has the - advantage that input_stream->get_size() will work during parsing - and we'll have an accurate status bar. Also, we can't seek - bz2 streams, so when it comes time to write out a local cached - copy of the .ini file below, we'd otherwise have to delete this - stream and uncompress it again from the start, which is wasteful. - The current uncompresed size of the .ini file as of 2007 is less - than 600 kB, so this is not a great deal of memory. */ - io_stream *bz2_stream = compress::decompress (ini_file); - if (!bz2_stream) - { - /* This isn't a valid bz2 file. */ - delete ini_file; - ini_file = NULL; - } + // no setup found or signature invalid + note (owner, IDS_SETUPINI_MISSING, SetupBaseName.c_str (), + "localdir"); + } + else + { + // grok information from setup + myFeedback.babble ("Found ini file - " + current_ini_name); + myFeedback.iniName (current_ini_name); + aBuilder.parse_mirror = ""; + ini_init (ini_file, &aBuilder, myFeedback); + + /*yydebug = 1; */ + + if (yyparse () || yyerror_count > 0) + myFeedback.error (yyerror_messages); else - { - io_stream *uncompressed = new io_stream_memory (); + ++ini_count; - if ((io_stream::copy (bz2_stream, uncompressed) != 0) || - (bz2_stream->error() != 0)) - { - /* There was a problem decompressing bz2. */ - ini_file = NULL; - Log (LOG_PLAIN) << - "Warning: Error code " << bz2_stream->error() << " occurred while uncompressing " << - current_ini_name << " - possibly truncated or corrupt bzip2" - " file. Retrying with uncompressed version." << endLog; - delete bz2_stream; - delete uncompressed; - } - else - { - delete bz2_stream; - ini_file = uncompressed; - ini_file->seek (0, IO_SEEK_SET); - } + if (aBuilder.timestamp > setup_timestamp) + { + setup_timestamp = aBuilder.timestamp; + ini_setup_version = aBuilder.version; } + delete ini_file; + ini_file = NULL; } + } + return ini_count; +} - if (!ini_file) +static int +do_remote_ini (HWND owner) +{ + size_t ini_count = 0; + GuiParseFeedback myFeedback; + IniDBBuilderPackage aBuilder (myFeedback); + io_stream *ini_file, *ini_sig_file; + + /* FIXME: Get rid of this io_stream pointer travesty. The need to + explicitly delete these things is ridiculous. */ + + // iterate over all sites + for (SiteList::const_iterator n = site_list.begin (); + n != site_list.end (); ++n) + { + bool sig_fail = false; + std::string current_ini_ext, current_ini_name, current_ini_sig_name; + // iterate over known extensions for setup + for (IniList::const_iterator ext = setup_ext_list.begin (); + ext != setup_ext_list.end (); + ext++) { - /* Try to look for a plain .ini file because one of the following - happened above: - - there was no .bz2 file found on the mirror. - - the .bz2 file didn't look like a valid bzip2 file. - - there was an error during bzip2 decompression. */ - current_ini_name = n->url + SETUP_INI_DIR + SETUP_INI_FILENAME; - current_ini_sig_name = n->url + SETUP_INI_DIR + SETUP_INI_FILENAME + ".sig"; + current_ini_ext = *ext; + current_ini_name = n->url + SetupIniDir + SetupBaseName + "." + current_ini_ext; + current_ini_sig_name = current_ini_name + ".sig"; + ini_sig_file = get_url_to_membuf (current_ini_sig_name, owner); ini_file = get_url_to_membuf (current_ini_name, owner); - if (!NoVerifyOption) - ini_sig_file = get_url_to_membuf (current_ini_sig_name, owner); - - if (!NoVerifyOption && ini_file && !ini_sig_file) - { - note (owner, IDS_SETUPINI_MISSING, current_ini_sig_name.c_str(), n->url.c_str()); - delete ini_file; - ini_file = NULL; - sig_fail = true; - } - else if (!NoVerifyOption && ini_file && !verify_ini_file_sig (ini_file, ini_sig_file, owner)) - { - note (owner, IDS_SIG_INVALID, current_ini_sig_name.c_str(), n->url.c_str()); - delete ini_file; - ini_file = NULL; - delete ini_sig_file; - ini_sig_file = NULL; - sig_fail = true; - } + ini_file = check_ini_sig (ini_file, ini_sig_file, sig_fail, + n->url.c_str (), current_ini_sig_name.c_str (), owner); + // stop searching as soon as we find a setup file + if (ini_file) + break; } - - if (!ini_file) + if (ini_file) + ini_file = decompress_ini (ini_file); + if (!ini_file || sig_fail) { - if (!sig_fail) - note (owner, IDS_SETUPINI_MISSING, SETUP_INI_FILENAME, n->url.c_str()); - continue; + // no setup found or signature invalid + note (owner, IDS_SETUPINI_MISSING, SetupBaseName.c_str (), n->url.c_str ()); } - - myFeedback.iniName (current_ini_name); - aBuilder.parse_mirror = n->url; - ini_init (ini_file, &aBuilder, myFeedback); - - /*yydebug = 1; */ - - if (yyparse () || yyerror_count > 0) - myFeedback.error (yyerror_messages); else { - /* save known-good setup.ini locally */ - const std::string fp = "file://" + local_dir + "/" + - rfc1738_escape_part (n->url) + - "/" + SETUP_INI_DIR + SETUP_INI_FILENAME; - io_stream::mkpath_p (PATH_TO_FILE, fp, 0); - if (io_stream *out = io_stream::open (fp, "wb", 0)) + // grok information from setup + myFeedback.iniName (current_ini_name); + aBuilder.parse_mirror = n->url; + ini_init (ini_file, &aBuilder, myFeedback); + + /*yydebug = 1; */ + + if (yyparse () || yyerror_count > 0) + myFeedback.error (yyerror_messages); + else { - ini_file->seek (0, IO_SEEK_SET); - if (io_stream::copy (ini_file, out) != 0) - io_stream::remove (fp); - delete out; + /* save known-good setup.ini locally */ + const std::string fp = "file://" + local_dir + "/" + + rfc1738_escape_part (n->url) + + "/" + SetupIniDir + SetupBaseName + ".ini"; + io_stream::mkpath_p (PATH_TO_FILE, fp, 0); + if (io_stream *out = io_stream::open (fp, "wb", 0)) + { + ini_file->seek (0, IO_SEEK_SET); + if (io_stream::copy (ini_file, out) != 0) + io_stream::remove (fp); + delete out; + } + ++ini_count; } - ++ini_count; - } - if (aBuilder.timestamp > setup_timestamp) - { - setup_timestamp = aBuilder.timestamp; - ini_setup_version = aBuilder.version; + if (aBuilder.timestamp > setup_timestamp) + { + setup_timestamp = aBuilder.timestamp; + ini_setup_version = aBuilder.version; + } + delete ini_file; + ini_file = NULL; } - delete ini_file; } return ini_count; } @@ -296,7 +344,7 @@ do_ini_thread (HINSTANCE h, HWND owner) if (ini_count == 0) return false; - if (get_root_dir ().c_str()) + if (get_root_dir ().c_str ()) { io_stream::mkpath_p (PATH_TO_DIR, "cygfile:///etc/setup", 0755); @@ -332,7 +380,7 @@ do_ini_thread (HINSTANCE h, HWND owner) } } - LogBabblePrintf (".ini setup_version is %s, our setup_version is %s", ini_setup_version.size() ? + LogBabblePrintf (".ini setup_version is %s, our setup_version is %s", ini_setup_version.size () ? ini_setup_version.c_str () : "(null)", setup_version); if (ini_setup_version.size ()) @@ -346,21 +394,21 @@ do_ini_thread (HINSTANCE h, HWND owner) } static DWORD WINAPI -do_ini_thread_reflector(void* p) +do_ini_thread_reflector (void* p) { HANDLE *context; context = (HANDLE*)p; try { - bool succeeded = do_ini_thread((HINSTANCE)context[0], (HWND)context[1]); + bool succeeded = do_ini_thread ((HINSTANCE)context[0], (HWND)context[1]); // Tell the progress page that we're done downloading - Progress.PostMessageNow(WM_APP_SETUP_INI_DOWNLOAD_COMPLETE, 0, succeeded); + Progress.PostMessageNow (WM_APP_SETUP_INI_DOWNLOAD_COMPLETE, 0, succeeded); } - TOPLEVEL_CATCH("ini"); + TOPLEVEL_CATCH ("ini"); - ExitThread(0); + ExitThread (0); } static HANDLE context[2]; @@ -374,4 +422,3 @@ do_ini (HINSTANCE h, HWND owner) DWORD threadID; CreateThread (NULL, 0, do_ini_thread_reflector, context, 0, &threadID); } - diff --git a/ini.h b/ini.h index 1969da9..164e3d2 100644 --- a/ini.h +++ b/ini.h @@ -48,10 +48,6 @@ typedef enum EXCLUDE_NOT_FOUND } excludes; -#define SETUP_INI_DIR (SetupIniDir.c_str ()) -#define SETUP_INI_FILENAME ((SetupBaseName+".ini").c_str ()) -#define SETUP_BZ2_FILENAME ((SetupBaseName+".bz2").c_str ()) - /* The following three vars are used to facilitate error handling between the parser/lexer and its callers, namely ini.cc:do_remote_ini() and IniParseFindVisitor::visitFile(). */ -- 2.4.5 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: 0004-fromcwd.cc-Remove-debugging-output.patch --] [-- Type: text/x-patch, Size: 3173 bytes --] From 7788675cb2cfc547de7ead3ff5ae9839ae531506 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Sat, 11 Jul 2015 23:24:11 +0200 Subject: [PATCH 4/6] fromcwd.cc: Remove debugging output --- fromcwd.cc | 45 --------------------------------------------- 1 file changed, 45 deletions(-) diff --git a/fromcwd.cc b/fromcwd.cc index edb67a6..d571b02 100644 --- a/fromcwd.cc +++ b/fromcwd.cc @@ -34,13 +34,6 @@ #include "IniDBBuilderPackage.h" #include "IniParseFeedback.h" -#define DEBUG_FROMCWD -#ifdef DEBUG_FROMCWD -#include "LogFile.h" -#endif - -/* Trivial class for detecting the existence of setup.ini */ - class SetupFindVisitor : public FindVisitor { public: @@ -52,57 +45,27 @@ public: virtual void visitFile (const std::string& basePath, const WIN32_FIND_DATA *theFile) { -#ifdef DEBUG_FROMCWD - Log (LOG_PLAIN) << "examining file: " - << basePath << "./" << theFile->cFileName << endLog; -#endif if (inidir && (theFile->nFileSizeLow || theFile->nFileSizeHigh)) { -#ifdef DEBUG_FROMCWD - Log (LOG_PLAIN) << "checking extension: "; -#endif std::vector<bool>::iterator fi = found_ini.begin (); for (std::vector<std::string>::const_iterator ext = setup_ext_list.begin (); ext != setup_ext_list.end (); ext++, fi++) { -#ifdef DEBUG_FROMCWD - Log (LOG_PLAIN) << *ext << " "; -#endif if (!casecompare (SetupBaseName + "." + *ext, theFile->cFileName)) *fi = true; } -#ifdef DEBUG_FROMCWD - Log (LOG_PLAIN) << endLog; -#endif } -#ifdef DEBUG_FROMCWD - Log (LOG_PLAIN) << "state: " - << " inidir=" << inidir - << " found_xz=" << found_ini[0] - << " found_bz2=" << found_ini[1] - << " found_ini=" << found_ini[2] - << endLog; -#endif } virtual void visitDirectory (const std::string& basePath, WIN32_FIND_DATA const *aDir, int level) { -#ifdef DEBUG_FROMCWD - Log (LOG_PLAIN) << "examining directory: " - << basePath << "./" << aDir->cFileName - << endLog; -#endif if (level <= 0) return; inidir = !casecompare (SetupArch, aDir->cFileName); if (level == 1 && !inidir) return; -#ifdef DEBUG_FROMCWD - Log (LOG_PLAIN) << " recurse into: " - << (inidir ? "inidir" : "mirror") << endLog; -#endif Find aFinder (basePath + aDir->cFileName); aFinder.accept (*this, inidir ? 0 : --level); std::vector<bool>::const_iterator fi = found_ini.begin (); @@ -134,10 +97,6 @@ IniList found_ini_list; bool do_from_local_dir (HINSTANCE h, HWND owner, std::string &local_dir) { -#ifdef DEBUG_FROMCWD - Log (LOG_PLAIN) << "do_from_local_dir: " - << local_dir << endLog; -#endif // Assume we won't find the INI file. SetupFindVisitor found; // single mirror? @@ -149,10 +108,6 @@ do_from_local_dir (HINSTANCE h, HWND owner, std::string &local_dir) if (found) return true; // nope, do full scan. -#ifdef DEBUG_FROMCWD - Log (LOG_PLAIN) << " starting full scan from " - << local_dir << endLog; -#endif IniParseFeedback myFeedback; IniDBBuilderPackage myBuilder (myFeedback); ScanFindVisitor myVisitor (myBuilder); -- 2.4.5 [-- Attachment #6: Type: text/plain, Size: 202 bytes --] Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-07-11 21:31 ` setup Achim Gratz @ 2015-07-13 9:44 ` Corinna Vinschen 2015-07-14 19:57 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-07-13 9:44 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1305 bytes --] Hi Achim, looks good to me. Please apply with two minor changes: On Jul 11 23:31, Achim Gratz wrote: > [...] > diff --git a/ini.cc b/ini.cc > index e90f98c..571ee2d 100644 > --- a/ini.cc > +++ b/ini.cc > @@ -39,7 +39,6 @@ > #include "mount.h" > #include "site.h" > #include "find.h" > -#include "IniParseFindVisitor.h" > #include "IniParseFeedback.h" > > #include "io_stream.h" > @@ -57,11 +56,12 @@ extern ThreeBarProgressPage Progress; > > unsigned int setup_timestamp = 0; > std::string ini_setup_version; > -std::string current_ini_sig_name; > +IniList setup_ext_list ( setup_exts, setup_exts+3); // TODO: use C++11x initializer lists Can the constant 3 go away (sizeof setup_exts / sizeof *setup_exts)? > [...] > + myFeedback.iniName (current_ini_name); > + aBuilder.parse_mirror = ""; > + ini_init (ini_file, &aBuilder, myFeedback); > + ^^^^^^^ Extra whitespaces, `git am' complains about, in the line preceeding the "yydebug = 1" comments in both, do_local_ini and do_remote_ini. > + /*yydebug = 1; */ > + > [...] > - > - /*yydebug = 1; */ > - Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-07-13 9:44 ` setup Corinna Vinschen @ 2015-07-14 19:57 ` Achim Gratz 0 siblings, 0 replies; 125+ messages in thread From: Achim Gratz @ 2015-07-14 19:57 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > looks good to me. Please apply with two minor changes: Pushed. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Terratec KOMPLEXER: http://Synth.Stromeko.net/Downloads.html#KomplexerWaves ^ permalink raw reply [flat|nested] 125+ messages in thread
* setup @ 2015-07-28 19:19 Achim Gratz 2015-07-29 12:51 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-07-28 19:19 UTC (permalink / raw) To: cygwin-apps With the latest round of changes I'm now down to two local change-sets for setup: 1. Implement an option that installs exactly the requested packages, no matter what packages are already installed. This leaves an installation in the same state as a new install from scratch would have, modulo any customizations in /etc. All package versions will be reset to current, missing packages get installed and additional packages removed. I use this to ensure that automated installs end up with the same packages no matter the history of the existing installation. Specifically I implemented it so that I can switch installations from one option to another (currently there are four install options: command line, GUI, developer and developer w/ debuginfo) without manual intervention. http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/37bbf074f6333624be361f2acd07dfefeacd5a83 2. Implement an option that pulls in source packages automatically. This is taken from a patch offered by David Hoke on this list about a year ago (ChangeLog is missing at the moment). http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/0c6a81eec41765ef7864682b7a038c7237c21c16 Both changes have been tested for a while, albeit in a somewhat restricted environment. Let me know what you think, I'd love to get these upstream. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-07-28 19:19 setup Achim Gratz @ 2015-07-29 12:51 ` Corinna Vinschen 2015-08-01 15:52 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-07-29 12:51 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1605 bytes --] On Jul 28 21:19, Achim Gratz wrote: > > With the latest round of changes I'm now down to two local change-sets > for setup: > > 1. Implement an option that installs exactly the requested packages, no > matter what packages are already installed. This leaves an installation > in the same state as a new install from scratch would have, modulo any > customizations in /etc. All package versions will be reset to current, > missing packages get installed and additional packages removed. I use > this to ensure that automated installs end up with the same packages no > matter the history of the existing installation. Specifically I > implemented it so that I can switch installations from one option to > another (currently there are four install options: command line, GUI, > developer and developer w/ debuginfo) without manual intervention. > > http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/37bbf074f6333624be361f2acd07dfefeacd5a83 This one is missing a ChangeLog hint that it tries to cleanup the code slightly (extra else branch for upgrade == true), otherwise ok for commit. > 2. Implement an option that pulls in source packages automatically. > This is taken from a patch offered by David Hoke on this list about a > year ago (ChangeLog is missing at the moment). > > http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/0c6a81eec41765ef7864682b7a038c7237c21c16 Ok for commit. Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-07-29 12:51 ` setup Corinna Vinschen @ 2015-08-01 15:52 ` Achim Gratz 2015-08-03 18:03 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-01 15:52 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > On Jul 28 21:19, Achim Gratz wrote: >> http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/37bbf074f6333624be361f2acd07dfefeacd5a83 > > This one is missing a ChangeLog hint that it tries to cleanup the code > slightly (extra else branch for upgrade == true), otherwise ok for commit. Done with that change. >> http://repo.or.cz/w/cygwin-setup/local.git/commitdiff/0c6a81eec41765ef7864682b7a038c7237c21c16 > > Ok for commit. Done with a ChangeLog entry. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-01 15:52 ` setup Achim Gratz @ 2015-08-03 18:03 ` Achim Gratz 2015-08-03 19:07 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-03 18:03 UTC (permalink / raw) To: cygwin-apps I've just found out that MD5 checksums get parsed OK, but wind up as solid zeros in the package database in setup. I have no idea how this is even possible, since the exact same code works just fine for SHA512 checksums. I didn't check yet if this actually works in the release version of setup. An extra pair of eyeballs would be appreciated. I don't see how my changes could be responsible and I was sure I'd checked it before… :-( Also, I want to clean up the parser code a bit and fix a very old formatting bug found while debugging the problem with the MD5 checksums. Another thing: it seems that the SHA512 digest is checked for existence by checking that the first byte is non-zero. Is that actually a guarantee that the SHA2 standard makes or is it just very unlikely to encounter 0x00 as the first byte? Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-03 18:03 ` setup Achim Gratz @ 2015-08-03 19:07 ` Achim Gratz 2015-08-03 20:02 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-03 19:07 UTC (permalink / raw) To: cygwin-apps Achim Gratz writes: > I've just found out that MD5 checksums get parsed OK, but wind up as > solid zeros in the package database in setup. I have no idea how this > is even possible, since the exact same code works just fine for SHA512 > checksums. I didn't check yet if this actually works in the release > version of setup. An extra pair of eyeballs would be appreciated. I > don't see how my changes could be responsible and I was sure I'd checked > it before… :-( Found it, I feel stupid now… > Also, I want to clean up the parser code a bit and fix a very old > formatting bug found while debugging the problem with the MD5 checksums. In preparation. > Another thing: it seems that the SHA512 digest is checked for existence > by checking that the first byte is non-zero. Is that actually a > guarantee that the SHA2 standard makes or is it just very unlikely to > encounter 0x00 as the first byte? If the SHA512 checksum is all zeros in the package DB it doesn't get checked, which is actually the reason I didn't catch the error earlier. Have to fix that also. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-03 19:07 ` setup Achim Gratz @ 2015-08-03 20:02 ` Achim Gratz 2015-08-03 20:29 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-03 20:02 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 54 bytes --] Here are the four patches to make setup work again. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Fix-MD5sum.str-formatting-bug.patch --] [-- Type: text/x-patch, Size: 1314 bytes --] From c7ad9a642a51753344a841c1718efd963af82505 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Mon, 3 Aug 2015 20:15:51 +0200 Subject: [PATCH 1/4] Fix MD5sum.str () formatting bug * csu_util/MD5Sum.cc (str): The stream modifiers are not sticky and must be applied for each conversion. --- ChangeLog | 5 +++++ csu_util/MD5Sum.cc | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7d14458..f8cd2b4 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2015-08-03 Achim Gratz <Stromeko@NexGo.DE> + + * csu_util/MD5Sum.cc (str): The stream modifiers are not sticky + and must be applied for each conversion. + 2015-08-01 David Hoke <d_hoke@hotmail.com> * download.cc (IncludeSource): New command line switch diff --git a/csu_util/MD5Sum.cc b/csu_util/MD5Sum.cc index 1d9c362..596089d 100644 --- a/csu_util/MD5Sum.cc +++ b/csu_util/MD5Sum.cc @@ -85,9 +85,9 @@ MD5Sum::str() const { std::ostringstream hexdigest; - hexdigest << std::hex << std::setfill('0') << std::setw(2); for (int i=0; i<16; ++i ) - hexdigest << static_cast<unsigned int>(digest[i]); + hexdigest << std::hex << std::setfill('0') << std::setw(2) + << static_cast<unsigned int>(digest[i]); return hexdigest.str(); } -- 2.4.6 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Fix-bug-in-nibbled1-and-use-character-types-througho.patch --] [-- Type: text/x-patch, Size: 2472 bytes --] From 7faf6f57472ee0247c757f12aff56faf2c2c4919 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Mon, 3 Aug 2015 21:04:10 +0200 Subject: [PATCH 2/4] Fix bug in nibbled1 and use character types throughout * ini.h (nibbled1): Fix a thinko in this macro that made the result always zero. (hexnibble, nibbled1, b64url, b64d1, b64d2, b64d3): Make all calculation constants dealing with characters character literals. Remove trailing semicolons. --- ChangeLog | 7 +++++++ ini.h | 20 ++++++++++---------- 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/ChangeLog b/ChangeLog index f8cd2b4..ef3a733 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,12 @@ 2015-08-03 Achim Gratz <Stromeko@NexGo.DE> + * ini.h (nibbled1): Fix a thinko in this macro that made the result always zero. + (hexnibble, nibbled1, b64url, b64d1, b64d2, b64d3): Make all + calculation constants dealing with characters character literals. + Remove trailing semicolons. + +2015-08-03 Achim Gratz <Stromeko@NexGo.DE> + * csu_util/MD5Sum.cc (str): The stream modifiers are not sticky and must be applied for each conversion. diff --git a/ini.h b/ini.h index 164e3d2..7f6db22 100644 --- a/ini.h +++ b/ini.h @@ -59,16 +59,16 @@ extern int yyerror_count; /* number of parse errors */ /* The following definitions are used in the parser implementation */ -#define hexnibble(val) (255 & (val > '9') ? val - 'a' + 10 : val - '0'); -#define nibbled1(v1,v2) (255 & ((v1 << 4) & v2)); +#define hexnibble(val) ('\xff' & (val > '9') ? val - 'a' + 10 : val - '0') +#define nibbled1(v1,v2) ('\xff' & ((v1 << 4) | v2)) #define b64url(val) \ - (63 & (( val == '_') ? 63 \ - : (val == '-') ? 62 \ - : (val >= 'a') ? val - 'a' + 26 \ - : (val >= 'A') ? val - 'A' + 0 \ - : val - '0' + 52)) -#define b64d1(v1,v2,v3,v4) (255 & ((v1 << 2) | (v2 >> 4))); -#define b64d2(v1,v2,v3,v4) (255 & ((v2 << 4) | (v3 >> 2))); -#define b64d3(v1,v2,v3,v4) (255 & ((v3 << 6) | v4)); + ('\x3f' & (( val == '_') ? '\x3f' \ + : (val == '-') ? '\x3e' \ + : (val >= 'a') ? val - 'a' + '\x1a' \ + : (val >= 'A') ? val - 'A' + '\x00' \ + : val - '0' + '\x34')) +#define b64d1(v1,v2,v3,v4) ('\xff' & ((v1 << 2) | (v2 >> 4))) +#define b64d2(v1,v2,v3,v4) ('\xff' & ((v2 << 4) | (v3 >> 2))) +#define b64d3(v1,v2,v3,v4) ('\xff' & ((v3 << 6) | v4)) #endif /* SETUP_INI_H */ -- 2.4.6 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-If-MD5-checksum-is-missing-then-SHA512-must-always-b.patch --] [-- Type: text/x-patch, Size: 1621 bytes --] From ffbee617a0943ef08d250b3d65a1a46a9b7fd9e1 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Mon, 3 Aug 2015 21:19:53 +0200 Subject: [PATCH 3/4] If MD5 checksum is missing then SHA512 must always be checked * install.cc (chksum_one): If MD5 checksum is not available, then SHA512 must always be checked even if just against the zeroed digest as initialized. Otherwise a package with no checksum would be treated as valid. --- ChangeLog | 7 +++++++ install.cc | 6 +++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/ChangeLog b/ChangeLog index ef3a733..4d19736 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,12 @@ 2015-08-03 Achim Gratz <Stromeko@NexGo.DE> + * install.cc (chksum_one): If MD5 checksum is not available, then + SHA512 must always be checked even if just against the zeroed + digest as initialized. Otherwise a package with no checksum would + be treated as valid. + +2015-08-03 Achim Gratz <Stromeko@NexGo.DE> + * ini.h (nibbled1): Fix a thinko in this macro that made the result always zero. (hexnibble, nibbled1, b64url, b64d1, b64d2, b64d3): Make all calculation constants dealing with characters character literals. diff --git a/install.cc b/install.cc index 4274715..1e69564 100644 --- a/install.cc +++ b/install.cc @@ -1044,8 +1044,8 @@ chksum_one (const packagesource& pkgsource) { if (!pkgsource.Cached ()) return; - if (pkgsource.sha512sum[0]) - sha512_one (pkgsource); - else if (pkgsource.md5.isSet()) + if (pkgsource.md5.isSet()) md5_one (pkgsource); + else + sha512_one (pkgsource); } -- 2.4.6 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: 0004-Simplify-parser-both-SHA512-checksum-types-can-parse.patch --] [-- Type: text/x-patch, Size: 3872 bytes --] From 4d0695ecf313e9eede3f6693ea9426e0ce674517 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Mon, 3 Aug 2015 21:23:37 +0200 Subject: [PATCH 4/4] Simplify parser, both SHA512 checksum types can parse to the same token * inilex.ll: Introduce HEX and B64 definitions, use them in the rules section. Parse both SHA512 and SHA512-Base64URL checksums to the SHA512 token. * iniparse.yy (packagedata): Remove all occurences of the SHA512B64URL token. --- ChangeLog | 8 ++++++++ inilex.ll | 10 ++++++---- iniparse.yy | 7 ++----- 3 files changed, 16 insertions(+), 9 deletions(-) diff --git a/ChangeLog b/ChangeLog index 4d19736..cf5cbda 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +2015-08-03 Achim Gratz <ASSI <Stromeko@NexGo.DE>> + + * inilex.ll: Introduce HEX and B64 definitions, use them in the + rules section. Parse both SHA512 and SHA512-Base64URL checksums + to the SHA512 token. + * iniparse.yy (packagedata): Remove all occurences of the + SHA512B64URL token. + 2015-08-03 Achim Gratz <Stromeko@NexGo.DE> * install.cc (chksum_one): If MD5 checksum is not available, then diff --git a/inilex.ll b/inilex.ll index 86100ae..67764f0 100644 --- a/inilex.ll +++ b/inilex.ll @@ -46,10 +46,12 @@ static void ignore_line (void); %x eolstate STR [!a-zA-Z0-9_./:\+~-]+ +HEX [0-9a-f] +B64 [a-zA-Z0-9_-] %% -[0-9a-f]{32} { +{HEX}{32} { yylval = (char *) new unsigned char[16]; memset (yylval, 0, 16); int i, j; @@ -63,7 +65,7 @@ STR [!a-zA-Z0-9_./:\+~-]+ return MD5; } -[0-9a-f]{128} { +{HEX}{128} { yylval = (char *) new unsigned char[SHA512_DIGEST_LENGTH]; memset (yylval, 0, SHA512_DIGEST_LENGTH); int i, j; @@ -77,7 +79,7 @@ STR [!a-zA-Z0-9_./:\+~-]+ return SHA512; } -[a-zA-Z0-9_-]{86} { +{B64}{86} { /* base64url as defined in RFC4648 */ yylval = (char *) new unsigned char[SHA512_DIGEST_LENGTH]; memset (yylval, 0, SHA512_DIGEST_LENGTH); @@ -98,7 +100,7 @@ STR [!a-zA-Z0-9_./:\+~-]+ v3 = 0; v4 = 0; ((unsigned char *) yylval) [j+0] = b64d1(v1, v2, v3, v4); - return SHA512B64URL; + return SHA512; } \"[^"]*\" { yylval = new char [strlen (yytext+1) + 1]; diff --git a/iniparse.yy b/iniparse.yy index 442135d..5fd6c3f 100644 --- a/iniparse.yy +++ b/iniparse.yy @@ -42,8 +42,8 @@ void add_correct_version(); %token CATEGORY DEPENDS REQUIRES %token APATH PPATH INCLUDE_SETUP EXCLUDE_PACKAGE DOWNLOAD_URL %token T_PREV T_CURR T_TEST -%token SHA512 SHA512B64URL MD5 INSTALLEDSIZE MAINTAINER PRIORITY -%token MD5LINE SHA512LINE +%token INSTALLEDSIZE MAINTAINER PRIORITY +%token MD5 MD5LINE SHA512 SHA512LINE %token DESCTAG DESCRIPTION FILESIZE ARCHITECTURE SOURCEPACKAGE %token RECOMMENDS PREDEPENDS %token SUGGESTS CONFLICTS REPLACES PROVIDES PACKAGENAME STRTOEOL PARAGRAPH @@ -109,7 +109,6 @@ singleitem /* non-empty */ | STANDARDSVERSION STRING NL { /* TODO */ } | MD5LINE MD5 NL { iniBuilder->buildInstallMD5 ((unsigned char *)$2); } | SHA512LINE SHA512 NL { iniBuilder->buildInstallSHA512 ((unsigned char *)$2); } - | SHA512LINE SHA512B64URL NL { iniBuilder->buildInstallSHA512 ((unsigned char *)$2); } | SOURCEPACKAGE source NL | CATEGORY categories NL | INSTALL STRING { iniBuilder->buildPackageInstall ($2); } installmeta NL @@ -149,13 +148,11 @@ installmeta: /* empty */ installchksum: /* empty */ | MD5 { iniBuilder->buildInstallMD5 ((unsigned char *)$1);} | SHA512 { iniBuilder->buildInstallSHA512 ((unsigned char *)$1);} - | SHA512B64URL { iniBuilder->buildInstallSHA512 ((unsigned char *)$1);} ; sourcechksum: /* empty */ | MD5 { iniBuilder->buildSourceMD5 ((unsigned char *)$1); } | SHA512 { iniBuilder->buildSourceSHA512 ((unsigned char *)$1); } - | SHA512B64URL { iniBuilder->buildSourceSHA512 ((unsigned char *)$1); } ; source /* non-empty */ -- 2.4.6 [-- Attachment #6: Type: text/plain, Size: 197 bytes --] Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptations for KORG EX-800 and Poly-800MkII V0.9: http://Synth.Stromeko.net/Downloads.html#KorgSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-03 20:02 ` setup Achim Gratz @ 2015-08-03 20:29 ` Corinna Vinschen 2015-08-03 20:31 ` setup Achim Gratz 2015-08-03 21:20 ` setup Achim Gratz 0 siblings, 2 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-08-03 20:29 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 654 bytes --] On Aug 3 22:02, Achim Gratz wrote: > > Here are the four patches to make setup work again. Trusting that you tested them, I only have a minor nit in terms of the ChangeLog: > * ini.h (nibbled1): Fix a thinko in this macro that made the result always zero. ^^^^^^^^^^^^^^^^ line too long. > (hexnibble, nibbled1, b64url, b64d1, b64d2, b64d3): Make all > calculation constants dealing with characters character literals. characters character? :) Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-03 20:29 ` setup Corinna Vinschen @ 2015-08-03 20:31 ` Achim Gratz 2015-08-03 20:40 ` setup Corinna Vinschen 2015-08-03 21:20 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-03 20:31 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: >> (hexnibble, nibbled1, b64url, b64d1, b64d2, b64d3): Make all >> calculation constants dealing with characters character literals. > > characters character? :) How'd you say that? Sorry, it's been a long day. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-03 20:31 ` setup Achim Gratz @ 2015-08-03 20:40 ` Corinna Vinschen 2015-08-03 20:54 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-08-03 20:40 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 563 bytes --] On Aug 3 22:31, Achim Gratz wrote: > Corinna Vinschen writes: > >> (hexnibble, nibbled1, b64url, b64d1, b64d2, b64d3): Make all > >> calculation constants dealing with characters character literals. > > > > characters character? :) > > How'd you say that? Sorry, it's been a long day. Replace numerical constants with character literals. Alternatively, what about using UCHAR_MAX? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-03 20:40 ` setup Corinna Vinschen @ 2015-08-03 20:54 ` Achim Gratz 0 siblings, 0 replies; 125+ messages in thread From: Achim Gratz @ 2015-08-03 20:54 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > Replace numerical constants with character literals. Sure, but only those that should represent characters. The shift lengths for instance need to be kept as integers. > Alternatively, what about using UCHAR_MAX? There's an implicit assumption that the character is 8bits wide, so using that constant seems to conceal that. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-03 20:29 ` setup Corinna Vinschen 2015-08-03 20:31 ` setup Achim Gratz @ 2015-08-03 21:20 ` Achim Gratz 2015-08-04 18:35 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-03 21:20 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > On Aug 3 22:02, Achim Gratz wrote: >> >> Here are the four patches to make setup work again. > > Trusting that you tested them, I only have a minor nit in terms of > the ChangeLog: Pushed. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-03 21:20 ` setup Achim Gratz @ 2015-08-04 18:35 ` Achim Gratz 2015-08-05 8:08 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-04 18:35 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 692 bytes --] Achim Gratz writes: > Pushed. I've found another (and hopefully the last, except for the mostly dormant gzip code) 16kiB buffer to be increased to 64kiB. Also, when installing from a local directory without a setup.ini file there actually is a need to skip the checksum validation. For this to work we also need to record whether we actually set a checksum and testing the first byte for zero is not correct for doing this. As detailed in the design for Bitcoin, there is an exponetnially decreasing probability for the leading bits to all be zero. Current Cygwin has more than 80 packages with the first eight bits or more all zero (I've seen 15 leading zero bits in my local mirror). [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Increase-read-buffer-size-for-MD5-checksumming-to-64.patch --] [-- Type: text/x-patch, Size: 1290 bytes --] From 9de3a769ae38b1e54ec99e5f105f832a74becfab Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Tue, 4 Aug 2015 19:39:23 +0200 Subject: [PATCH 1/2] Increase read buffer size for MD5 checksumming to 64kiB * install.cc (md5_one): Change buffer size from 16kiB to 64kiB for faster reading. --- ChangeLog | 7 ++++++- install.cc | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/ChangeLog b/ChangeLog index 355fd23..334bd9a 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,4 +1,9 @@ -2015-08-03 Achim Gratz <ASSI <Stromeko@NexGo.DE>> +2015-08-04 Achim Gratz <Stromeko@NexGo.DE> + + * install.cc (md5_one): Change buffer size from 16kiB to 64kiB for + faster reading. + +2015-08-03 Achim Gratz <Stromeko@NexGo.DE> * inilex.ll: Introduce HEX and B64 definitions, use them in the rules section. Parse both SHA512 and SHA512-Base64URL checksums diff --git a/install.cc b/install.cc index 1e69564..8c1589b 100644 --- a/install.cc +++ b/install.cc @@ -1010,7 +1010,7 @@ md5_one (const packagesource& pkgsource) Progress.SetText4 ("Progress:"); Progress.SetBar1 (0); - unsigned char buffer[16384]; + unsigned char buffer[64 * 1024]; ssize_t count; while ((count = thefile->read (buffer, sizeof (buffer))) > 0) { -- 2.4.6 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Properly-record-SHA512-checksum-presence-and-skip-va.patch --] [-- Type: text/x-patch, Size: 4447 bytes --] From d2f2501580391154fbd5331a932271ce17880680 Mon Sep 17 00:00:00 2001 From: Achim Gratz <Stromeko@Stromeko.DE> Date: Tue, 4 Aug 2015 19:48:56 +0200 Subject: [PATCH 2/2] Properly record SHA512 checksum presence and skip validation for ad-hoc installs * package_source.h (packagesource): Add boolean member variable sha512_isSet to record whether an SHA512 checksum has been set. (packagesource): Initialize sha512_isSet to false. * IniDBBuilderPackage.cc (buildInstallSHA512, buildSourceSHA512): Only set the SHA512 checksum when it was previously unset like it is done for MD5 checksums. That will generally be the checksum recorded on the package line in setup.ini, any further checksums in separate lines will thus be ignored. * install.cc (chksum_one): Conditionalize the comparison of the SHA512 checksum on whether or not it was previously set. Check SHA512 checksum first since it is the default now. This is necessary for ad-hoc installs from local disk without a setup.ini file. Output a warning when the checksum was not be verified because neither a MD5 nor a SHA512 checksum was set. --- ChangeLog | 19 +++++++++++++++++++ IniDBBuilderPackage.cc | 8 ++++++-- install.cc | 8 ++++++-- package_source.h | 2 ++ 4 files changed, 33 insertions(+), 4 deletions(-) diff --git a/ChangeLog b/ChangeLog index 334bd9a..03f91b4 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,24 @@ 2015-08-04 Achim Gratz <Stromeko@NexGo.DE> + * package_source.h (packagesource): Add boolean member variable + sha512_isSet to record whether an SHA512 checksum has been set. + (packagesource): Initialize sha512_isSet to false. + + * IniDBBuilderPackage.cc (buildInstallSHA512, buildSourceSHA512): + Only set the SHA512 checksum when it was previously unset like it + is done for MD5 checksums. That will generally be the checksum + recorded on the package line in setup.ini, any further checksums + in separate lines will thus be ignored. + + * install.cc (chksum_one): Conditionalize the comparison of the + SHA512 checksum on whether or not it was previously set. Check + SHA512 checksum first since it is the default now. This is + necessary for ad-hoc installs from local disk without a setup.ini + file. Output a warning when the checksum was not be verified + because neither a MD5 nor a SHA512 checksum was set. + +2015-08-04 Achim Gratz <Stromeko@NexGo.DE> + * install.cc (md5_one): Change buffer size from 16kiB to 64kiB for faster reading. diff --git a/IniDBBuilderPackage.cc b/IniDBBuilderPackage.cc index 7ee2af4..b41955a 100644 --- a/IniDBBuilderPackage.cc +++ b/IniDBBuilderPackage.cc @@ -268,15 +268,19 @@ IniDBBuilderPackage::buildInstallSize (const std::string &size) void IniDBBuilderPackage::buildInstallSHA512 (unsigned char const *sha512) { - if (sha512) + if (sha512 && cbpv.source()->sha512_isSet) { memcpy (cbpv.source()->sha512sum, sha512, sizeof cbpv.source()->sha512sum); + cbpv.source()->sha512_isSet = true; + } } void IniDBBuilderPackage::buildSourceSHA512 (unsigned char const *sha512) { - if (sha512) + if (sha512 && cbpv.source()->sha512_isSet) { memcpy (cspv.source()->sha512sum, sha512, sizeof cspv.source()->sha512sum); + cbpv.source()->sha512_isSet = true; + } } void diff --git a/install.cc b/install.cc index 8c1589b..a5c4b10 100644 --- a/install.cc +++ b/install.cc @@ -1044,8 +1044,12 @@ chksum_one (const packagesource& pkgsource) { if (!pkgsource.Cached ()) return; - if (pkgsource.md5.isSet()) + if (pkgsource.sha512_isSet) + sha512_one (pkgsource); + else if (pkgsource.md5.isSet()) md5_one (pkgsource); else - sha512_one (pkgsource); + Log (LOG_BABBLE) << "No checksum recorded for " << pkgsource.Base () + << ", cannot determine integrity of package!" + << endLog; } diff --git a/package_source.h b/package_source.h index 79d357b..997ccf8 100644 --- a/package_source.h +++ b/package_source.h @@ -62,6 +62,7 @@ public: _installedSize (0) { memset (sha512sum, 0, sizeof sha512sum); + sha512_isSet = false; }; /* how big is the source file */ size_t size; @@ -107,6 +108,7 @@ public: virtual void set_canonical (char const *); virtual void set_cached (const std::string& ); unsigned char sha512sum[SHA512_DIGEST_LENGTH]; + bool sha512_isSet; MD5Sum md5; typedef std::vector <site> sitestype; sitestype sites; -- 2.4.6 [-- Attachment #4: Type: text/plain, Size: 188 bytes --] Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ SD adaptation for Waldorf microQ V2.22R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-04 18:35 ` setup Achim Gratz @ 2015-08-05 8:08 ` Corinna Vinschen 2015-08-05 15:42 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-08-05 8:08 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 2588 bytes --] On Aug 4 20:34, Achim Gratz wrote: > Achim Gratz writes: > > Pushed. > > I've found another (and hopefully the last, except for the mostly > dormant gzip code) 16kiB buffer to be increased to 64kiB. Also, when > installing from a local directory without a setup.ini file there > actually is a need to skip the checksum validation. For this to work we > also need to record whether we actually set a checksum and testing the > first byte for zero is not correct for doing this. As detailed in the > design for Bitcoin, there is an exponetnially decreasing probability for > the leading bits to all be zero. Current Cygwin has more than 80 > packages with the first eight bits or more all zero (I've seen 15 > leading zero bits in my local mirror). > > >From 9de3a769ae38b1e54ec99e5f105f832a74becfab Mon Sep 17 00:00:00 2001 > From: Achim Gratz <...> > Date: Tue, 4 Aug 2015 19:39:23 +0200 > Subject: [PATCH 1/2] Increase read buffer size for MD5 checksumming to 64kiB > > * install.cc (md5_one): Change buffer size from 16kiB to 64kiB for > faster reading. Acked. > >From d2f2501580391154fbd5331a932271ce17880680 Mon Sep 17 00:00:00 2001 > From: Achim Gratz <...> > Date: Tue, 4 Aug 2015 19:48:56 +0200 > Subject: [PATCH 2/2] Properly record SHA512 checksum presence and skip > validation for ad-hoc installs > > diff --git a/IniDBBuilderPackage.cc b/IniDBBuilderPackage.cc > index 7ee2af4..b41955a 100644 > --- a/IniDBBuilderPackage.cc > +++ b/IniDBBuilderPackage.cc > @@ -268,15 +268,19 @@ IniDBBuilderPackage::buildInstallSize (const std::string &size) > void > IniDBBuilderPackage::buildInstallSHA512 (unsigned char const *sha512) > { > - if (sha512) > + if (sha512 && cbpv.source()->sha512_isSet) { > memcpy (cbpv.source()->sha512sum, sha512, sizeof cbpv.source()->sha512sum); > + cbpv.source()->sha512_isSet = true; > + } This looks wrong. if (...&& cbpv.source()->sha512_isSet) ... cbpv.source()->sha512_isSet = true; If the flag is set you set it again? > } > > void > IniDBBuilderPackage::buildSourceSHA512 (unsigned char const *sha512) > { > - if (sha512) > + if (sha512 && cbpv.source()->sha512_isSet) { > memcpy (cspv.source()->sha512sum, sha512, sizeof cspv.source()->sha512sum); > + cbpv.source()->sha512_isSet = true; > + } > } Same here. Am I missing something? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-05 8:08 ` setup Corinna Vinschen @ 2015-08-05 15:42 ` Achim Gratz 2015-08-06 10:03 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-05 15:42 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > Same here. Am I missing something? Nothing, but I missed to copy the negation from the corresponding code for MD5 even though the description of the patch explicitly makes note of the correct logic. It's fixed already in my repo since I finally updated all of Cygwin at work today. --8<---------------cut here---------------start------------->8--- Modified IniDBBuilderPackage.cc diff --git a/IniDBBuilderPackage.cc b/IniDBBuilderPackage.cc index b41955a..ff92ec2 100644 --- a/IniDBBuilderPackage.cc +++ b/IniDBBuilderPackage.cc @@ -268,7 +268,7 @@ IniDBBuilderPackage::buildInstallSize (const std::string &size) void IniDBBuilderPackage::buildInstallSHA512 (unsigned char const *sha512) { - if (sha512 && cbpv.source()->sha512_isSet) { + if (sha512 && !cbpv.source()->sha512_isSet) { memcpy (cbpv.source()->sha512sum, sha512, sizeof cbpv.source()->sha512sum); cbpv.source()->sha512_isSet = true; } @@ -277,7 +277,7 @@ IniDBBuilderPackage::buildInstallSHA512 (unsigned char const *sha512) void IniDBBuilderPackage::buildSourceSHA512 (unsigned char const *sha512) { - if (sha512 && cbpv.source()->sha512_isSet) { + if (sha512 && !cbpv.source()->sha512_isSet) { memcpy (cspv.source()->sha512sum, sha512, sizeof cspv.source()->sha512sum); cbpv.source()->sha512_isSet = true; } --8<---------------cut here---------------end--------------->8--- OK for push? Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-05 15:42 ` setup Achim Gratz @ 2015-08-06 10:03 ` Corinna Vinschen 2015-08-06 15:58 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-08-06 10:03 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1748 bytes --] On Aug 5 17:41, Achim Gratz wrote: > Corinna Vinschen writes: > > Same here. Am I missing something? > > Nothing, but I missed to copy the negation from the corresponding code > for MD5 even though the description of the patch explicitly makes note > of the correct logic. It's fixed already in my repo since I finally > updated all of Cygwin at work today. > > --8<---------------cut here---------------start------------->8--- > Modified IniDBBuilderPackage.cc > diff --git a/IniDBBuilderPackage.cc b/IniDBBuilderPackage.cc > index b41955a..ff92ec2 100644 > --- a/IniDBBuilderPackage.cc > +++ b/IniDBBuilderPackage.cc > @@ -268,7 +268,7 @@ IniDBBuilderPackage::buildInstallSize (const std::string &size) > void > IniDBBuilderPackage::buildInstallSHA512 (unsigned char const *sha512) > { > - if (sha512 && cbpv.source()->sha512_isSet) { > + if (sha512 && !cbpv.source()->sha512_isSet) { > memcpy (cbpv.source()->sha512sum, sha512, sizeof cbpv.source()->sha512sum); > cbpv.source()->sha512_isSet = true; > } > @@ -277,7 +277,7 @@ IniDBBuilderPackage::buildInstallSHA512 (unsigned char const *sha512) > void > IniDBBuilderPackage::buildSourceSHA512 (unsigned char const *sha512) > { > - if (sha512 && cbpv.source()->sha512_isSet) { > + if (sha512 && !cbpv.source()->sha512_isSet) { > memcpy (cspv.source()->sha512sum, sha512, sizeof cspv.source()->sha512sum); > cbpv.source()->sha512_isSet = true; > } > --8<---------------cut here---------------end--------------->8--- > > OK for push? Yes, thank you. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-06 10:03 ` setup Corinna Vinschen @ 2015-08-06 15:58 ` Achim Gratz 2015-08-07 19:47 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-06 15:58 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: >> OK for push? > > Yes, thank you. Done. I would consider this a release candidate. Some more testing with interactive and ad-hoc installs would be useful, though. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-06 15:58 ` setup Achim Gratz @ 2015-08-07 19:47 ` Corinna Vinschen 2015-08-07 21:05 ` setup Warren Young 2015-08-08 5:22 ` setup Achim Gratz 0 siblings, 2 replies; 125+ messages in thread From: Corinna Vinschen @ 2015-08-07 19:47 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 992 bytes --] On Aug 6 17:57, Achim Gratz wrote: > Corinna Vinschen writes: > >> OK for push? > > > > Yes, thank you. > > Done. > > I would consider this a release candidate. Some more testing with > interactive and ad-hoc installs would be useful, though. How do you want to handle this? We don't have real provisions for setup.exe release candidates. Maybe we should change how we provide setup updates in future? I have a vague idea that setup should ideally be part of the Cygwin distro package set. So setup updates itself, and it would be possible to handle public "test" releases. The issue of overwriting setup while setup is running could be worked around by setup first copying itself to a intermediate filename (e.g. .setup.exe) and then exce'ing that copy. Other ideas how we could handle this? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-07 19:47 ` setup Corinna Vinschen @ 2015-08-07 21:05 ` Warren Young 2015-08-10 8:44 ` setup Corinna Vinschen 2015-08-08 5:22 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Warren Young @ 2015-08-07 21:05 UTC (permalink / raw) To: cygwin-apps On Aug 7, 2015, at 1:47 PM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > > On Aug 6 17:57, Achim Gratz wrote: >> >> I would consider this a release candidate. Some more testing with >> interactive and ad-hoc installs would be useful, though. > > I have a vague idea that setup should ideally be part of the Cygwin > distro package set. So setup updates itself, and it would be possible > to handle public "test" releases. > > The issue of overwriting setup while setup is running could be worked > around by setup first copying itself to a intermediate filename (e.g. > .setup.exe) and then exce'ing that copy. You’d have to couple this either with a MoveFileEx(…, MOVEFILE_DELAY_UNTIL_REBOOT) call or a background task that replaces /bin/setup.exe with %LOCALAPPDATA%/Temp/setup-v$next.exe. This would be a nice point to give setup.exe a CLI language for installing packages in yum/apt-get fashion, too. (Yes, yes, I know, SHTDI.) ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-07 21:05 ` setup Warren Young @ 2015-08-10 8:44 ` Corinna Vinschen 2015-08-10 15:02 ` setup Warren Young 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-08-10 8:44 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1735 bytes --] On Aug 7 15:05, Warren Young wrote: > On Aug 7, 2015, at 1:47 PM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote: > > > > On Aug 6 17:57, Achim Gratz wrote: > >> > >> I would consider this a release candidate. Some more testing with > >> interactive and ad-hoc installs would be useful, though. > > > > I have a vague idea that setup should ideally be part of the Cygwin > > distro package set. So setup updates itself, and it would be possible > > to handle public "test" releases. > > > > The issue of overwriting setup while setup is running could be worked > > around by setup first copying itself to a intermediate filename (e.g. > > .setup.exe) and then exce'ing that copy. > > You’d have to couple this either with a MoveFileEx(…, > MOVEFILE_DELAY_UNTIL_REBOOT) call or a background task that replaces > /bin/setup.exe with %LOCALAPPDATA%/Temp/setup-v$next.exe. Why? The idea is that setup is called, performs a CopyFile of itself and then exec's its copy. If the copyfile fails with a sharing violation or something like that, the copy is in use, which means, setup is already running. However, if the copy already exists and can be overwritten. it's just some older copy. I don't think there's a necessity to remove the copy at all. > This would be a nice point to give setup.exe a CLI language for > installing packages in yum/apt-get fashion, too. (Yes, yes, I know, > SHTDI.) I like the idea in genral, but I'd also like to write SHTDI in all uppercase. Well, it already is, but you get my meaning... Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-10 8:44 ` setup Corinna Vinschen @ 2015-08-10 15:02 ` Warren Young 2015-08-11 7:48 ` setup Corinna Vinschen 0 siblings, 1 reply; 125+ messages in thread From: Warren Young @ 2015-08-10 15:02 UTC (permalink / raw) To: cygwin-apps > On Aug 7 15:05, Warren Young wrote: >> >> You’d have to couple this either with a MoveFileEx(…, >> MOVEFILE_DELAY_UNTIL_REBOOT) call or a background task that replaces >> /bin/setup.exe with %LOCALAPPDATA%/Temp/setup-v$next.exe. > > Why? I was assuming a world where setup.exe lives in /bin and you can call it from Bash. (In such a world, I hope it gets renamed to something less generic, like cygpkg.) I was further assuming that setup-*.tar.xz would be packaged the same as any other Cygwin package, unpacking directly into /bin. Thus, the MoveFileEx() call would be the one setup.exe normally offers, where it sees you’re trying to replace an in-use executable. I was offering the background task alternative as a way around that. However, the rest of your reply gives me a different idea. > The idea is that setup is called, performs a CopyFile of itself > and then exec's its copy. If you mean that it always copies itself to some kind of temp space and execs it from there, that feels kind of wasteful, since you’re trading my design that copies only when required for one that copies every time. But what if we invert it? Why couldn't setup-*.tar.xz package unpack into /tmp? The running setup.exe would exec it there with a flag that causes the second setup.exe instance to copy itself back into /bin. Then there is only a copy made when setup.exe is actually being replaced, and /bin/setup.exe is kept up-to-date. ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-10 15:02 ` setup Warren Young @ 2015-08-11 7:48 ` Corinna Vinschen 2015-08-11 19:53 ` setup Warren Young 0 siblings, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-08-11 7:48 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 1829 bytes --] On Aug 10 09:02, Warren Young wrote: > > On Aug 7 15:05, Warren Young wrote: > >> > >> You’d have to couple this either with a MoveFileEx(…, > >> MOVEFILE_DELAY_UNTIL_REBOOT) call or a background task that replaces > >> /bin/setup.exe with %LOCALAPPDATA%/Temp/setup-v$next.exe. > > > > Why? > > I was assuming a world where setup.exe lives in /bin and you can call > it from Bash. (In such a world, I hope it gets renamed to something > less generic, like cygpkg.) I was further assuming that > setup-*.tar.xz would be packaged the same as any other Cygwin package, > unpacking directly into /bin. That was the idea. > Thus, the MoveFileEx() call would be the one setup.exe normally > offers, where it sees you’re trying to replace an in-use executable. > > I was offering the background task alternative as a way around that. > However, the rest of your reply gives me a different idea. > > > The idea is that setup is called, performs a CopyFile of itself and > > then exec's its copy. > > If you mean that it always copies itself to some kind of temp space > and execs it from there, that feels kind of wasteful, since you’re > trading my design that copies only when required for one that copies > every time. Setup is not called multiple times per second. Copy and restart take not much time, and the restart part is done already anyway as soon as setup elevates itself. Why not simply elevate (or just start if already elevated) the copy called .setup-${arch}.exe? Note the leading dot. This copy never has the problem to be overwritten by unpacking, and the rest of setup does not need any changes. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-11 7:48 ` setup Corinna Vinschen @ 2015-08-11 19:53 ` Warren Young 0 siblings, 0 replies; 125+ messages in thread From: Warren Young @ 2015-08-11 19:53 UTC (permalink / raw) To: cygwin-apps On Aug 11, 2015, at 1:47 AM, Corinna Vinschen wrote: > > On Aug 10 09:02, Warren Young wrote: > >> If you mean that it always copies itself to some kind of temp space >> and execs it from there, that feels kind of wasteful > > ...the restart part is done already anyway as soon as > setup elevates itself. Sold. :) ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-07 19:47 ` setup Corinna Vinschen 2015-08-07 21:05 ` setup Warren Young @ 2015-08-08 5:22 ` Achim Gratz 2015-08-10 9:18 ` setup Corinna Vinschen 2015-08-10 14:44 ` setup Warren Young 1 sibling, 2 replies; 125+ messages in thread From: Achim Gratz @ 2015-08-08 5:22 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: >> I would consider this a release candidate. Some more testing with >> interactive and ad-hoc installs would be useful, though. > > How do you want to handle this? We don't have real provisions for > setup.exe release candidates. True. At the moment it'd probably mean that some other folks on this list try it and give their GTG. They'd either have to build it themselves or maybe use the AppVeyor build from Jon Turney. > Maybe we should change how we provide setup updates in future? > > I have a vague idea that setup should ideally be part of the Cygwin > distro package set. So setup updates itself, and it would be possible > to handle public "test" releases. We could add a new key to setup.ini that indicates the existence of a test version and have setup ask if the user would maybe want to use that instead. Ideally setup would then download and exec it if the user choses to run it. I don't have code ready to do that, though… Another problem that needs to be solved is to know how much exposure the new setup.exe really had to decide if it's good for release. > The issue of overwriting setup while setup is running could be worked > around by setup first copying itself to a intermediate filename (e.g. > .setup.exe) and then exce'ing that copy. > > Other ideas how we could handle this? As said before, the idea that setup.exe needs to be entirely self-contained is both an advantage and limiting what it can do. I haven't had time yet to check, but a minimal Cygwin install system w/ busybox might not be too large compared to the connectivity demands of todays' Windows itself. Here setup.exe would just need to unpack the current image of the install system and provide the GUI to pick the packages and control the actual installation. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-08 5:22 ` setup Achim Gratz @ 2015-08-10 9:18 ` Corinna Vinschen 2015-08-10 16:33 ` setup Achim Gratz 2015-08-10 14:44 ` setup Warren Young 1 sibling, 1 reply; 125+ messages in thread From: Corinna Vinschen @ 2015-08-10 9:18 UTC (permalink / raw) To: cygwin-apps [-- Attachment #1: Type: text/plain, Size: 3021 bytes --] On Aug 8 07:22, Achim Gratz wrote: > Corinna Vinschen writes: > >> I would consider this a release candidate. Some more testing with > >> interactive and ad-hoc installs would be useful, though. > > > > How do you want to handle this? We don't have real provisions for > > setup.exe release candidates. > > True. At the moment it'd probably mean that some other folks on this > list try it and give their GTG. They'd either have to build it > themselves or maybe use the AppVeyor build from Jon Turney. Link? I'm using setup with the -m option exclusively so I'm mostly interested that this still works. However, you might want to make a quick list what scenarios you want to have tested as well. I try to do that over the next couple of days and hopfully other maintainers try, too. > > Maybe we should change how we provide setup updates in future? > > > > I have a vague idea that setup should ideally be part of the Cygwin > > distro package set. So setup updates itself, and it would be possible > > to handle public "test" releases. > > We could add a new key to setup.ini that indicates the existence of a > test version and have setup ask if the user would maybe want to use that > instead. Ideally setup would then download and exec it if the user > choses to run it. IIUC, you mean that the current method of downloading setup only from sware should be maintained? What kind of "key" are you talking about? > I don't have code ready to do that, though… Another > problem that needs to be solved is to know how much exposure the new > setup.exe really had to decide if it's good for release. That's a general problem. I'm not sure how much exposure the Cygwin test DLLs really have, either. I'm reasonable sure that some of you maintainers are using them, but otherwise it's a bit of a black hole. > > The issue of overwriting setup while setup is running could be worked > > around by setup first copying itself to a intermediate filename (e.g. > > .setup.exe) and then exce'ing that copy. > > > > Other ideas how we could handle this? > > As said before, the idea that setup.exe needs to be entirely > self-contained is both an advantage and limiting what it can do. I > haven't had time yet to check, but a minimal Cygwin install system w/ > busybox might not be too large compared to the connectivity demands of > todays' Windows itself. Here setup.exe would just need to unpack the > current image of the install system and provide the GUI to pick the > packages and control the actual installation. Not sure I follow. How would such an install system look like? I have a vague notion that busybox could run the scripts, but there are installation path issues and requirements of some postinstall scripts which might not be suffiecently handled by busybox. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat [-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-10 9:18 ` setup Corinna Vinschen @ 2015-08-10 16:33 ` Achim Gratz 0 siblings, 0 replies; 125+ messages in thread From: Achim Gratz @ 2015-08-10 16:33 UTC (permalink / raw) To: cygwin-apps Corinna Vinschen writes: > Link? I'd think Jon needs to give his OK for me to post it or he should post it himself. > I'm using setup with the -m option exclusively so I'm mostly > interested that this still works. That's pretty extensively tested by now since that is what I use for the installations at work. > However, you might want to make a quick list what scenarios you want > to have tested as well. I try to do that over the next couple of days > and hopfully other maintainers try, too. Installs from the internet (with and without proxy) and ad-hoc installs of single packages and package groups from a local directory that doesn't have a setup.ini. > IIUC, you mean that the current method of downloading setup only from > sware should be maintained? What kind of "key" are you talking about? Currently the integrity of setup.exe and therefore the whole installation hinges on it being downloaded from sware via https, so that's a strong reason to keep it that way. The "key" I'm talking about was that in addition to "setup-version:" we could have an additional "setup-trial-version" or something like that would inform setup that a successor is waiting in the wings. > That's a general problem. I'm not sure how much exposure the Cygwin > test DLLs really have, either. I'm reasonable sure that some of you > maintainers are using them, but otherwise it's a bit of a black hole. You should at least be able to find out how often they were downloaded. For setup trial versions this isn't enough, so they'd have to contact sourceware and HEAD/GET something that indicated success or failure (and some script to extract that out of the logs on sourceware). It also must not be sneaky about this, ideally the user consents before the trial version gets downloaded at all. > Not sure I follow. How would such an install system look like? > I have a vague notion that busybox could run the scripts, but there > are installation path issues and requirements of some postinstall > scripts which might not be suffiecently handled by busybox. I have done a few manual installations from a running Cygwin onto a "cold" installation on the same machine, mainly to switch out the Cygwin DLL. You need to mount the "other" Cygwin somewhere (I use /mnt/cygwin or something like that) and then redirect the install root there (tar -C /mnt/cygwin). The postinstall scripts on the other hand must be started from within the new Cygwin installation anyway. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-08 5:22 ` setup Achim Gratz 2015-08-10 9:18 ` setup Corinna Vinschen @ 2015-08-10 14:44 ` Warren Young 2015-08-10 16:40 ` setup Achim Gratz 1 sibling, 1 reply; 125+ messages in thread From: Warren Young @ 2015-08-10 14:44 UTC (permalink / raw) To: cygwin-apps On Aug 7, 2015, at 11:22 PM, Achim Gratz <Stromeko@nexgo.de> wrote: > > a minimal Cygwin install system w/ busybox Busybox isn’t going to be usable in its normal sense without cygwin1.dll to map symlinks to argv[0] so that Busybox can tell how it was called, and thus which function to provide. But that’s a side issue, because I don’t see why you think you need Busybox in the first place, unless it is to be able to run postinst scripts. Can’t you just say that that setup-*.tar.xz package never has a postinst script, so that all you need to do is un-tar it? In that sense, setup.exe is already bootstrapped: it has everything within itself to be able to replace itself. ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-10 14:44 ` setup Warren Young @ 2015-08-10 16:40 ` Achim Gratz 2015-08-10 16:45 ` setup Warren Young 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-10 16:40 UTC (permalink / raw) To: cygwin-apps Warren Young writes: > But that’s a side issue, because I don’t see why you think you need > Busybox in the first place, unless it is to be able to run postinst > scripts. Can’t you just say that that setup-*.tar.xz package never > has a postinst script, so that all you need to do is un-tar it? The idea was that you have a fairly complete POSIX environment with as small a footprint as possible, then use that to bootstrap a more conventionally styled Cygwin install. > In that sense, setup.exe is already bootstrapped: it has everything > within itself to be able to replace itself. I do like the concept of a self-contained executable, but teaching setup.exe new tricks is likely a larger effort than adding a script here and there to an install system. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Terratec KOMPLEXER: http://Synth.Stromeko.net/Downloads.html#KomplexerWaves ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-10 16:40 ` setup Achim Gratz @ 2015-08-10 16:45 ` Warren Young 2015-08-10 17:00 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Warren Young @ 2015-08-10 16:45 UTC (permalink / raw) To: cygwin-apps On Aug 10, 2015, at 10:40 AM, Achim Gratz wrote: > > Warren Young writes: > >> In that sense, setup.exe is already bootstrapped: it has everything >> within itself to be able to replace itself. > > I do like the concept of a self-contained executable, but teaching > setup.exe new tricks is likely a larger effort than adding a script here > and there to an install system. Isn’t the whole point of this discussion that setup.exe already knows all the tricks it needs to in order to do what we want here, except for the “replace setup.exe in place” issue I’ve brought up separately? Why do you need a self-contained POSIX environment to replace setup.exe? ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-10 16:45 ` setup Warren Young @ 2015-08-10 17:00 ` Achim Gratz 2015-08-10 17:09 ` setup Warren Young 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-10 17:00 UTC (permalink / raw) To: cygwin-apps Warren Young writes: > Isn’t the whole point of this discussion that setup.exe already knows > all the tricks it needs to in order to do what we want here, except > for the “replace setup.exe in place” issue I’ve brought up separately? Well, setup.exe today has a lot of probably bitrotted features that Cygwin never used (even if some of it looks quite interesting and useful) and there are other things it doesn't do even though it should probably be doing it. > Why do you need a self-contained POSIX environment to replace > setup.exe? At the moment I'm just thinking whether that might be a more sustainable way forward. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-10 17:00 ` setup Achim Gratz @ 2015-08-10 17:09 ` Warren Young 2015-08-10 18:03 ` setup Achim Gratz 0 siblings, 1 reply; 125+ messages in thread From: Warren Young @ 2015-08-10 17:09 UTC (permalink / raw) To: cygwin-apps On Aug 10, 2015, at 11:00 AM, Achim Gratz wrote: > > Warren Young writes: >> Isn’t the whole point of this discussion that setup.exe already knows >> all the tricks it needs to in order to do what we want here, except >> for the “replace setup.exe in place” issue I’ve brought up separately? > > Well, setup.exe today has a lot of probably bitrotted features that > Cygwin never used (even if some of it looks quite interesting and > useful) I don’t think there is any call for setup.exe to be highly backwards-compatible. It isn’t asked to deal with old package repos, very old versions of setup.ini, etc. Therefore, I would say that whoever is in charge of this code base should simply get rid of the bitrotted features. If any of these obsolete features are needed again in the future, they can be pulled out of the SCM history. > there are other things it doesn't do even though it should > probably be doing it. Okay, but... >> Why do you need a self-contained POSIX environment to replace >> setup.exe? > > At the moment I'm just thinking whether that might be a more sustainable > way forward. Replacing core infrastructure like this almost never works out smoothly, and usually fails outright. It’s far better to remediate the code base in-place, if at all possible. There have been a bunch of attempts in the past at replacing setup.exe. At least 3, that I can think of. All have fizzled. ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-10 17:09 ` setup Warren Young @ 2015-08-10 18:03 ` Achim Gratz 2015-08-10 18:41 ` setup Warren Young 0 siblings, 1 reply; 125+ messages in thread From: Achim Gratz @ 2015-08-10 18:03 UTC (permalink / raw) To: cygwin-apps Warren Young writes: > Replacing core infrastructure like this almost never works out > smoothly, and usually fails outright. It’s far better to remediate > the code base in-place, if at all possible. That's my sentiment as well, but that doesn't preclude the exploration of alternatives. > There have been a bunch of attempts in the past at replacing > setup.exe. At least 3, that I can think of. All have fizzled. These were? Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2015-08-10 18:03 ` setup Achim Gratz @ 2015-08-10 18:41 ` Warren Young 0 siblings, 0 replies; 125+ messages in thread From: Warren Young @ 2015-08-10 18:41 UTC (permalink / raw) To: cygwin-apps On Aug 10, 2015, at 12:03 PM, Achim Gratz wrote: > >> There have been a bunch of attempts in the past at replacing >> setup.exe. At least 3, that I can think of. All have fizzled. > > These were? The Debian and Red Hat packaging systems have both been ported to Cygwin. Those could be used along with a bare-bones setup.exe to bootstrap Cygwin, after which setup.exe would no longer be needed. https://github.com/transcode-open/apt-cyg https://cygwin.com/ml/cygwin-announce/2003-05/msg00001.html Then there was a pretty GUI installer someone made many years ago; I believe we were still in the 1.5.x line at the time. I don’t remember enough about it to find it via Google again, but I do remember a few posts to the mailing list with positive responses. Then the developer disappeared and no one took up the code base. That’s part of what I mean about the difficulty of replacing key infrastructure. Without either a change at the core or an overwhelming attack from outside, there just isn’t enough reason for someone to try to adopt something nonstandard. Without that user base, there isn’t enough drive to continue development, so the project fizzles. Consider the rise of Ubuntu-based Linuxes, replacing the various the RPM-based ones. That didn’t happen purely because Ubuntu was “better.” A necessary part of this was Shuttleworth pouring millions of dollars from a really lucky IPO into the project. As proof, consider all of the Ubuntu clones that have gone nowhere, despite being “better” in some way. The closest thing I can think of to success in the area you propose to tackle is mintty, which stepped into a gap between Windows Console and Cygwin/X + rxvt. It didn’t try to replace either one, exactly, so it didn’t have to succeed by first making the other disappear. “Better and separate” beats “better.” ^ permalink raw reply [flat|nested] 125+ messages in thread
* setup @ 2002-03-04 19:28 Robert Collins 2002-03-04 23:38 ` setup Michael A Chase 2002-03-05 8:31 ` setup Brian Keener 0 siblings, 2 replies; 125+ messages in thread From: Robert Collins @ 2002-03-04 19:28 UTC (permalink / raw) To: cygwin-apps I'll be looking into the reports on the setup-20020225 beta today/tonight. I've caught one regression myself - there's no src only capability any more :[. How'd that happen? :}. Anyway, I've an idea for a UI change to support it more effectively: add a bin column, and the default is bin on, src off, but the user can choose any combination (with bin and src off = keep/skip) Thoughts? Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2002-03-04 19:28 setup Robert Collins @ 2002-03-04 23:38 ` Michael A Chase 2002-03-05 8:31 ` setup Brian Keener 1 sibling, 0 replies; 125+ messages in thread From: Michael A Chase @ 2002-03-04 23:38 UTC (permalink / raw) To: Robert Collins, cygwin-apps ----- Original Message ----- From: "Robert Collins" <robert.collins@itdomain.com.au> To: <cygwin-apps@cygwin.com> Sent: Monday, March 04, 2002 19:28 Subject: setup I'll be looking into the reports on the setup-20020225 beta today/tonight. I've caught one regression myself - there's no src only capability any more :[. How'd that happen? :}. Anyway, I've an idea for a UI change to support it more effectively: add a bin column, and the default is bin on, src off, but the user can choose any combination (with bin and src off = keep/skip) ====== That sounds more intuitive than the src checkbox. Will it be in the normal CVS update or will you produce a new snapshot? -- Mac :}) ** I normally forward private questions to the appropriate mail list. ** Ask Smarter: http://www.tuxedo.org/~esr/faqs/smart-questions.html Give a hobbit a fish and he eats fish for a day. Give a hobbit a ring and he eats fish for an age. ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: setup 2002-03-04 19:28 setup Robert Collins 2002-03-04 23:38 ` setup Michael A Chase @ 2002-03-05 8:31 ` Brian Keener 1 sibling, 0 replies; 125+ messages in thread From: Brian Keener @ 2002-03-05 8:31 UTC (permalink / raw) To: cygwin-apps Robert Collins wrote: > Anyway, I've an idea for a UI change to support it more effectively: add > a bin column, and the default is bin on, src off, but the user can > choose any combination (with bin and src off = keep/skip Sounds intriguing but also I'm afraid somewhat confusing as I now have the version selection column (keep/skip, Reinstall, Retrieve, 1.4.3, 1.4.4, uninstall ....) And then I have the source column which dictates if I want the source for the selected version if I am changing version or I can select source only for the current version. Now you add a bin column and then I could have in the version column 1.4.4 (which is not the current installed version) but have bin and source off which is essentially do nothing (sounds confusing). How about an alternate approach in terms of get rid of the Source column and only use the version columns as follows: keep/skip, uninstall, 1.4.4 bin&src, 1.4.4 bin, 1.4.4 src, reinstall bin&src, reinstall bin, reinstall src, retrieve bin&src, retrieve bin, retrieve src ... Just an alternate thought. ^ permalink raw reply [flat|nested] 125+ messages in thread
* RE: Setup @ 2002-02-20 23:10 Robert Collins 0 siblings, 0 replies; 125+ messages in thread From: Robert Collins @ 2002-02-20 23:10 UTC (permalink / raw) To: Gary R. Van Sickle, cygwin-apps > -----Original Message----- > From: Gary R. Van Sickle [mailto:g.r.vansickle@worldnet.att.net] > Feature freeze feature freeze feature freeze ;-) HEAD is open for business, as per ~ 3 emails back. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Setup @ 2002-02-18 6:12 Robert Collins 2002-02-18 13:06 ` Setup Brian Keener 0 siblings, 1 reply; 125+ messages in thread From: Robert Collins @ 2002-02-18 6:12 UTC (permalink / raw) To: cygwin-apps Ok, final feedback and bug-killing time folk. A big thanks to Michael, Jason, Pavel, Gary, and everyone else who has helped make this latest incarnation of setup.exe the best yet. Chuck, your fix is in there :}. Chris, thank you for humouring me with regards to that... 300 line changelogs tend to make me grumpy :}. I've branched 'setup200202' with the string patch + Michaels' patchs + Jason's off-by-one patch. The version number on the new setup will be 2.194.2.x where x will be version specific bug fixes. No, and I really mean it this time :] wholesale changes to the branch please. CVS HEAD is no open for development again. Cheers, Rob P.S. Michael : A Changelog is a beautiful thing :}. ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 6:12 Setup Robert Collins @ 2002-02-18 13:06 ` Brian Keener 2002-02-18 17:26 ` Setup Robert Collins 2002-02-18 18:48 ` Setup Robert Collins 0 siblings, 2 replies; 125+ messages in thread From: Brian Keener @ 2002-02-18 13:06 UTC (permalink / raw) To: cygwin-apps Robert Collins wrote: > Ok, final feedback and bug-killing time folk. I just built from CVS and the big thing I noticed in the past and still seems to be there has to do with doing an "Install from Local Directory" after doing the Download. It appears the download is working fine but when I select Install from Local Directory if I select any package(s) for reinstall then setup seems to uninstall just fine but the Reinstall doesn't seem to do anything - my log doesn't seem to show any creations but it shows alot of unlinks for the removals. I discovered this while researching another problem dealing with install from local directory where setup doesn't seem to install the last package selected for upgrade (as opposed to reinstall mentioned above). It seems to download it fine or if I select install from the internet that also seems to work - I've been trying to debug it but my skills are beginner at best. I can neither confirm or deny the upgrade problem at the moment but I just had the reinstall happen so I can confirm that one. The other couple of things I've noticed are probably open for preference so I will throw them out for general consensus: 1) If I have a group of packages installed and I select the Test option to see what packages are available for in Test should all the packages I have installed that do not have a Test Version show Uninstall or should they should keep?? It is my contention that thy should show keep and not uninstall - unless I explicitly request uninstall. 2) If I am doing and "Install from Local Directory" and the Source file does not exist in the Local path should the option for Source only or the Sources checkbox even be available?? It is my contention again that if the Sources cannot be found then the option should not exist. Those are the few things I found - if you need a copy of my setup.log.full - just let me know. I'll keep trying on the Reinstall to get a handle on what gives but if anyone has any clues you might want to fix it as opposed to waiting on me. bk ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 13:06 ` Setup Brian Keener @ 2002-02-18 17:26 ` Robert Collins 2002-02-18 20:05 ` Setup Christopher Faylor 2002-02-18 20:18 ` Setup Brian Keener 2002-02-18 18:48 ` Setup Robert Collins 1 sibling, 2 replies; 125+ messages in thread From: Robert Collins @ 2002-02-18 17:26 UTC (permalink / raw) To: bkeener, cygwin-apps === ----- Original Message ----- From: "Brian Keener" <bkeener@thesoftwaresource.com> > Robert Collins wrote: > > Ok, final feedback and bug-killing time folk. > > The other couple of things I've noticed are probably open for preference so I > will throw them out for general consensus: > > 1) If I have a group of packages installed and I select the > Test option to see what packages are available for in Test should all the > packages I have installed that do not have a Test Version show Uninstall or > should they should keep?? It is my contention that thy should show keep and > not uninstall - unless I explicitly request uninstall. I have emailed trying to start a discussion on this at least twice. My contention is that this is a factor for setup.ini to take care of, not setup.exe. It's easier to add entries than have complex logic for removing them when they aren't appropriate. > 2) If I am doing and "Install from Local Directory" and the Source file does > not exist in the Local path should the option for Source only or the Sources > checkbox even be available?? It is my contention again that if the Sources > cannot be found then the option should not exist. Patches gratefully accepted. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 17:26 ` Setup Robert Collins @ 2002-02-18 20:05 ` Christopher Faylor 2002-02-18 20:19 ` Setup Robert Collins 2002-02-18 20:18 ` Setup Brian Keener 1 sibling, 1 reply; 125+ messages in thread From: Christopher Faylor @ 2002-02-18 20:05 UTC (permalink / raw) To: cygwin-apps On Tue, Feb 19, 2002 at 12:27:26PM +1100, Robert Collins wrote: >>1) If I have a group of packages installed and I select the Test option >>to see what packages are available for in Test should all the packages >>I have installed that do not have a Test Version show Uninstall or >>should they should keep?? It is my contention that thy should show >>keep and not uninstall - unless I explicitly request uninstall. > >I have emailed trying to start a discussion on this at least twice. My >contention is that this is a factor for setup.ini to take care of, not >setup.exe. It's easier to add entries than have complex logic for >removing them when they aren't appropriate. If the question is "Should 'upset' add a dummy Test entry for every case where there is no such thing?" then the answer, IMO, is no. I think the same applies for the case where an initial release of a product is marked test. Setting up a dummy "Current" which is the same as "Test" would defeat the purpose of "Test". I think that the bottom line is that setup.exe should NEVER default to Uninstall. Uninstall should only be on when the user specifically selects it. Anything else is, IMO, surprising and dangerous. cgf ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:05 ` Setup Christopher Faylor @ 2002-02-18 20:19 ` Robert Collins 2002-02-18 20:31 ` Setup Brian Keener 2002-02-18 20:46 ` Setup Christopher Faylor 0 siblings, 2 replies; 125+ messages in thread From: Robert Collins @ 2002-02-18 20:19 UTC (permalink / raw) To: cygwin-apps === ----- Original Message ----- From: "Christopher Faylor" <cgf@redhat.com> To: <cygwin-apps@cygwin.com> Sent: Tuesday, February 19, 2002 3:05 PM Subject: Re: Setup > On Tue, Feb 19, 2002 at 12:27:26PM +1100, Robert Collins wrote: > If the question is "Should 'upset' add a dummy Test entry for every case > where there is no such thing?" then the answer, IMO, is no. I think the > same applies for the case where an initial release of a product is > marked test. Setting up a dummy "Current" which is the same as "Test" > would defeat the purpose of "Test". For the first case, I think the answer is yes, for the second, it *should* be no (because, as you say, it would defeat the purpose of test). Otherwise we need a *new* mechanism to tell setup.exe when a package is replaced from current to test - that is that no test version exists, and that when moving to test, the current version should be removed. > I think that the bottom line is that setup.exe should NEVER default to > Uninstall. Uninstall should only be on when the user specifically selects > it. Anything else is, IMO, surprising and dangerous. I agree that the user should be warned before automated uninstalls happen. Thats not ever been the case though in the gui. Setup doesn't *default* to uninstall. Two things have to happen: The user has to select Test (which means 'give me a testing distribution'). Their has to be no valid testing version for that package. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:19 ` Setup Robert Collins @ 2002-02-18 20:31 ` Brian Keener 2002-02-18 20:34 ` Setup Robert Collins 2002-02-18 20:46 ` Setup Christopher Faylor 1 sibling, 1 reply; 125+ messages in thread From: Brian Keener @ 2002-02-18 20:31 UTC (permalink / raw) To: Robert Collins, cygwin-apps Robert Collins wrote: > Setup doesn't *default* to uninstall. Two things have to happen: > The user has to select Test (which means 'give me a testing > distribution'). > Their has to be no valid testing version for that package. But if I select test (which means 'give me a testing package not a testing distribution') and the only test version available is a test version of vim that I want to try and everything else defaults to uninstall and I say next what do I have when I get done - nada - nothing and lets do it again because I wasn't paying attention and had no idea that setup would uninstall my entire system because I wanted to try one test version. Hummmmm - sounds pretty nasty to me. bk ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:31 ` Setup Brian Keener @ 2002-02-18 20:34 ` Robert Collins 2002-02-18 20:44 ` Setup Brian Keener 0 siblings, 1 reply; 125+ messages in thread From: Robert Collins @ 2002-02-18 20:34 UTC (permalink / raw) To: bkeener, cygwin-apps === ----- Original Message ----- From: "Brian Keener" <bkeener@thesoftwaresource.com> > But if I select test (which means 'give me a testing package not a testing > distribution') No it doesn't. Clicking on a package version is how one selects per package versions. > and the only test version available is a test version of vim > that I want to try and everything else defaults to uninstall and I say next > what do I have when I get done - nada - nothing and lets do it again because I > wasn't paying attention and had no idea that setup would uninstall my entire > system because I wanted to try one test version. > > Hummmmm - sounds pretty nasty to me. Exactly why I've been raising this! Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:34 ` Setup Robert Collins @ 2002-02-18 20:44 ` Brian Keener 2002-02-18 20:50 ` Setup Robert Collins 0 siblings, 1 reply; 125+ messages in thread From: Brian Keener @ 2002-02-18 20:44 UTC (permalink / raw) To: cygwin-apps Robert Collins wrote: > > But if I select test (which means 'give me a testing package not a > testing > > distribution') > > No it doesn't. Clicking on a package version is how one selects per > package versions. > That's where our view differs because I see the radio buttons as a way of selecting the maximum version I want considered and if that version does not exist then the default is the next version down (Test -> Current -> Prev depending on Radio button selected) and if the version selected is what is installed the default is keep and if it is not installed at all then the default is skip. ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:44 ` Setup Brian Keener @ 2002-02-18 20:50 ` Robert Collins 2002-02-19 8:53 ` Setup Brian Keener 0 siblings, 1 reply; 125+ messages in thread From: Robert Collins @ 2002-02-18 20:50 UTC (permalink / raw) To: bkeener, cygwin-apps === ----- Original Message ----- From: "Brian Keener" <bkeener@thesoftwaresource.com> To: <cygwin-apps@cygwin.com> Sent: Tuesday, February 19, 2002 3:44 PM Subject: Re: Setup > Robert Collins wrote: > > > But if I select test (which means 'give me a testing package not a > > testing > > > distribution') > > > > No it doesn't. Clicking on a package version is how one selects per > > package versions. > > > That's where our view differs because I see the radio buttons as a way of > selecting the maximum version I want considered and if that version does not > exist then the default is the next version down (Test -> Current -> Prev > depending on Radio button selected) and if the version selected is what is > installed the default is keep and if it is not installed at all then the > default is skip. Well, I'll consider a patch to setup to change it's behaviour. At the moment no limitation of maximum version is present, regardless of the prev/curr/test state. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:50 ` Setup Robert Collins @ 2002-02-19 8:53 ` Brian Keener 2002-02-24 3:03 ` Setup Robert Collins 0 siblings, 1 reply; 125+ messages in thread From: Brian Keener @ 2002-02-19 8:53 UTC (permalink / raw) To: cygwin-apps Robert Collins wrote: > > That's where our view differs because I see the radio buttons as a way > of > > selecting the maximum version I want considered and if that version > does not > > exist then the default is the next version down (Test -> Current -> > Prev > > depending on Radio button selected) and if the version selected is > what is > > installed the default is keep and if it is not installed at all then > the > > default is skip. > > Well, I'll consider a patch to setup to change it's behaviour. At the > moment no limitation of maximum version is present, regardless of the > prev/curr/test state. I catch the drift on the patches and the same with the other two we discussed. At one point in time when I was still submitting changes to setup, when I still understood the code before the major rewrite :-) - don't get me wrong this is good stuff - I thought we had it where Test, Curr, and Prev only set the maximum version we wanted to look at. But on the down side we never had it fully functional where it would handle multiple versions simply because they existed on disk and this is a very good new feature. I would not want to loose itt. But what I do propose is that we think of Prev as the last stable version, we think of Current as the current stable and we think of test as a development version - which could be changing rapidly. There may be any number of versions resident on my disk that are in between the control points and they should all be available for me to choose from in the chooser. But the maintainer dictates via setup.ini which version is the Prev stable, Curr stable and experimental (Test) or any combination thereof. With this in mind the Prev, Curr, and Test buttons should simply set the maximum version (with the primary choice being the Current Stable for the inexperience that simply want to use the product) that I want installed of any given package and then I still have the ability to select any other major or intermediate version (these amy or may not be a Prev, Curr, or Test) for a package as well. Essentially if I have the Current radio selected and a Prev is installed and there is a new Curr it will select that current version for install and if on another package I have a Test version installed and there is a Curr version available then it will select the current version for install (thus deinstalling the Test version) and if by some quirk I have a Test installed, the Current Radio button selected but there is no Curr version it will select the Prev and if there is no Prev version it will simply select keep (although I'm not sure I like this - but I like an uninstall option even less). I would think we also need some mechanism (because of the ability to mix and match versions) to give the user the ability to select the default actions for all packages listed - for example: 1) I want to *keep/skip* (depending on if installed or not) all and then simply upgrade one or two - no matter what is available 2) I want to *reinstall* whatever I currently have installed - no matter what is available 3) I want to install *sources* for everything I have installed - nothing more - nothing less and the really nasty option I want to *uninstall* everything with a big fat warning. I am essentially describing a global for the chooser that lets me pick from the standard chooser options but for all packages. I can then pick and choose below that for the individual package. I am sure categories will to some degree get in the way of this but I am sure that some workable solution could be found. Another nice touch and clarification in chooser might be to expand the text on the version number in the display for those packages where I might have 5 versions available for install. As I am clicking through the versions on a given package when I get to a version that is the Prev, Curr or Test if the text shows this - ie 1.4.5 (Curr) or 1.4.3 (Prev) or 1.4.4. In this case 1.4.4 is just an available intermediate. This is my $.02 worth and the world of setup and chooser as I see and would code if I could - don't get me wrong I'm still trying but the going is slow - got my hands in too many pots (and some have hot water) and setup is a tough teacher for learning C++. Bk ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-19 8:53 ` Setup Brian Keener @ 2002-02-24 3:03 ` Robert Collins 0 siblings, 0 replies; 125+ messages in thread From: Robert Collins @ 2002-02-24 3:03 UTC (permalink / raw) To: bkeener, cygwin-apps ----- Original Message ----- From: "Brian Keener" <bkeener@thesoftwaresource.com> > I catch the drift on the patches and the same with the other two we discussed. > At one point in time when I was still submitting changes to setup, when I still > understood the code before the major rewrite :-) It should be a lot easier to understand now, and it's starting to really come together now. > Another nice touch and clarification in chooser might be to expand the text on > the version number in the display for those packages where I might have 5 > versions available for install. As I am clicking through the versions on a > given package when I get to a version that is the Prev, Curr or Test if the > text shows this - ie 1.4.5 (Curr) or 1.4.3 (Prev) or 1.4.4. In this case 1.4.4 > is just an available intermediate. Nice idea, patches accepted :} (Hint: start with the PackagePickLine class). I've added to the TODO list. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:19 ` Setup Robert Collins 2002-02-18 20:31 ` Setup Brian Keener @ 2002-02-18 20:46 ` Christopher Faylor 2002-02-18 20:56 ` Setup Christopher Faylor ` (2 more replies) 1 sibling, 3 replies; 125+ messages in thread From: Christopher Faylor @ 2002-02-18 20:46 UTC (permalink / raw) To: cygwin-apps On Tue, Feb 19, 2002 at 03:19:43PM +1100, Robert Collins wrote: > >=== >----- Original Message ----- >From: "Christopher Faylor" <cgf@redhat.com> >To: <cygwin-apps@cygwin.com> >Sent: Tuesday, February 19, 2002 3:05 PM >Subject: Re: Setup > > >> On Tue, Feb 19, 2002 at 12:27:26PM +1100, Robert Collins wrote: >>If the question is "Should 'upset' add a dummy Test entry for every >>case where there is no such thing?" then the answer, IMO, is no. I >>think the same applies for the case where an initial release of a >>product is marked test. Setting up a dummy "Current" which is the same >>as "Test" would defeat the purpose of "Test". > >For the first case, I think the answer is yes, for the second, it >*should* be no (because, as you say, it would defeat the purpose of >test). > >Otherwise we need a *new* mechanism to tell setup.exe when a package is >replaced from current to test - that is that no test version exists, and >that when moving to test, the current version should be removed. Ok. We need a new mechanism. We also need a mechanism that says "remove this package" and I don't think that the mechanism is to just move the package to "prev". Maybe the mechanism is as simple as just having a setup.hint like: setup.hint: curr uninstall "This package is now obsolete" (wouldn't it be cool to have a "bubble" appear with the above words when you moved the mouse cursor over the package name?) Actually, in this case, where the package maintainer means to obviously uninstall something, I think it is acceptable for setup.exe to do so. >>I think that the bottom line is that setup.exe should NEVER default to >>Uninstall. Uninstall should only be on when the user specifically >>selects it. Anything else is, IMO, surprising and dangerous. > >I agree that the user should be warned before automated uninstalls >happen. Thats not ever been the case though in the gui. I've always considered it a bug when the word "Uninstall" shows up in the GUI and I didn't specifically click on it. >Setup doesn't *default* to uninstall. Two things have to happen: The >user has to select Test (which means 'give me a testing distribution'). >Their has to be no valid testing version for that package. If I click on Test, then a whole bunch of Installs show up on the screen. If you don't like the word "default" then I'll use "automatically switch". I don't think that setup.exe should automatically switch to Uninstall in any circumstances unless the package maintainer has specifically indicated that is the correct behavior. Somehow. When I click on test I assume it means "Give me all of the test versions". I think it is acceptable that if there are no test versions, then every other package shows up as "Skip". I don't think that we should assume that the correct behavior is Uninstall. Actually, this is one of the things that I hate about the current way setup works. I don't like the Prev/Curr/Test stuff that much. It feels like we're overloading functionality incorrectly but I don't have a real idea about how to fix it. cgf ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:46 ` Setup Christopher Faylor @ 2002-02-18 20:56 ` Christopher Faylor 2002-02-20 22:45 ` Setup Gary R. Van Sickle 2002-02-24 3:05 ` Setup Robert Collins 2 siblings, 0 replies; 125+ messages in thread From: Christopher Faylor @ 2002-02-18 20:56 UTC (permalink / raw) To: cygwin-apps On Mon, Feb 18, 2002 at 11:46:01PM -0500, Christopher Faylor wrote: >>Setup doesn't *default* to uninstall. Two things have to happen: The >>user has to select Test (which means 'give me a testing distribution'). >>Their has to be no valid testing version for that package. > >If I click on Test, then a whole bunch of Installs show up on the uninstalls >screen. If you don't like the word "default" then I'll use >"automatically switch". cgf ^ permalink raw reply [flat|nested] 125+ messages in thread
* RE: Setup 2002-02-18 20:46 ` Setup Christopher Faylor 2002-02-18 20:56 ` Setup Christopher Faylor @ 2002-02-20 22:45 ` Gary R. Van Sickle 2002-02-20 23:00 ` Setup Christopher Faylor 2002-02-24 3:05 ` Setup Robert Collins 2 siblings, 1 reply; 125+ messages in thread From: Gary R. Van Sickle @ 2002-02-20 22:45 UTC (permalink / raw) To: cygwin-apps > -----Original Message----- > From: cygwin-apps-owner@cygwin.com > [mailto:cygwin-apps-owner@cygwin.com]On Behalf Of Christopher Faylor [snip] > (wouldn't it be cool to have a "bubble" appear with the above words when > you moved the mouse cursor over the package name?) > NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Feature freeze feature freeze feature freeze ;-) No, for Setup 3.0 or whatever the release-after-next will be numbered, I think we almost need to have something like that. In particular I think we need to do the "popup-text-tooltip-when-the-description-text-goes-off-the-screen" trick, since pretty much every description overflows the allotted real-estate. -- Gary R. Van Sickle Brewer. Patriot. ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-20 22:45 ` Setup Gary R. Van Sickle @ 2002-02-20 23:00 ` Christopher Faylor 0 siblings, 0 replies; 125+ messages in thread From: Christopher Faylor @ 2002-02-20 23:00 UTC (permalink / raw) To: cygwin-apps On Wed, Feb 20, 2002 at 11:17:05PM -0600, Gary R. Van Sickle wrote: >> -----Original Message----- >> From: cygwin-apps-owner@cygwin.com >> [mailto:cygwin-apps-owner@cygwin.com]On Behalf Of Christopher Faylor > >[snip] > >> (wouldn't it be cool to have a "bubble" appear with the above words when >> you moved the mouse cursor over the package name?) >> > >NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Um. I wasn't suggesting this for this release. I understand what feature freeze means. Speaking of chilling... cgf ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:46 ` Setup Christopher Faylor 2002-02-18 20:56 ` Setup Christopher Faylor 2002-02-20 22:45 ` Setup Gary R. Van Sickle @ 2002-02-24 3:05 ` Robert Collins 2002-02-24 9:07 ` Setup Christopher Faylor 2 siblings, 1 reply; 125+ messages in thread From: Robert Collins @ 2002-02-24 3:05 UTC (permalink / raw) To: cygwin-apps === ----- Original Message ----- From: "Christopher Faylor" <cgf@redhat.com> > >Otherwise we need a *new* mechanism to tell setup.exe when a package is > >replaced from current to test - that is that no test version exists, and > >that when moving to test, the current version should be removed. > > Ok. We need a new mechanism. We also need a mechanism that says > "remove this package" and I don't think that the mechanism is to just > move the package to "prev". Maybe the mechanism is as simple as just > having a setup.hint like: > > setup.hint: > curr uninstall "This package is now obsolete" > > (wouldn't it be cool to have a "bubble" appear with the above words when > you moved the mouse cursor over the package name?) > > Actually, in this case, where the package maintainer means to obviously > uninstall something, I think it is acceptable for setup.exe to do so. Versioned conflicts will do this, which is a wishlist feature. Someone offered to do them, but I don't think a patch has arrived yet. > I don't think that setup.exe should automatically switch to Uninstall > in any circumstances unless the package maintainer has specifically > indicated that is the correct behavior. Somehow. I agree. My point has been (from the beginning) about *where* that intent gets indicated. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-24 3:05 ` Setup Robert Collins @ 2002-02-24 9:07 ` Christopher Faylor 0 siblings, 0 replies; 125+ messages in thread From: Christopher Faylor @ 2002-02-24 9:07 UTC (permalink / raw) To: cygwin-apps On Sun, Feb 24, 2002 at 10:04:32PM +1100, Robert Collins wrote: >> I don't think that setup.exe should automatically switch to Uninstall >> in any circumstances unless the package maintainer has specifically >> indicated that is the correct behavior. Somehow. > >I agree. My point has been (from the beginning) about *where* that >intent gets indicated. No, you're not agreeing if you think that a setup.ini file can be crafted in such a way that an "Uninstall" will show up without the package maintainer explicitly somehow saying "I want this package to be uninstalled". And, not making a test version of a package available is not "explicit". cgf ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 17:26 ` Setup Robert Collins 2002-02-18 20:05 ` Setup Christopher Faylor @ 2002-02-18 20:18 ` Brian Keener 2002-02-18 20:30 ` Setup Robert Collins 1 sibling, 1 reply; 125+ messages in thread From: Brian Keener @ 2002-02-18 20:18 UTC (permalink / raw) To: Robert Collins, cygwin-apps Robert Collins wrote: > I have emailed trying to start a discussion on this at least twice. My > contention is that this is a factor for setup.ini to take care of, not > setup.exe. It's easier to add entries than have complex logic for > removing them when they aren't appropriate. > So what you're saying is just looking at Current vs Previous - if there was no setup.ini entry for current and there was only an entry for previous then with the default display set to current the selected default action for the theoretical package would be uninstall (just as I saw on the Test) - correct? And in addition if as a maintainer I didn't want this to happen then I should make sure I have a prev,curr and test in setup.ini even if they all are the same file name. This bears the question (if my previous assumption is correct) what if my setup.ini has a prev, curr, and test file defined but I am doing an "Install from Local Directory" and the test version does not exist on my system. I would assume that since I defined a test and even though is does not exist that my default action would be keep - true? bk ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 20:18 ` Setup Brian Keener @ 2002-02-18 20:30 ` Robert Collins 0 siblings, 0 replies; 125+ messages in thread From: Robert Collins @ 2002-02-18 20:30 UTC (permalink / raw) To: bkeener, cygwin-apps === ----- Original Message ----- From: "Brian Keener" <bkeener@thesoftwaresource.com> To: "Robert Collins" <robert.collins@itdomain.com.au>; <cygwin-apps@sources.redhat.com> Sent: Tuesday, February 19, 2002 3:18 PM Subject: Re: Setup > Robert Collins wrote: > > I have emailed trying to start a discussion on this at least twice. My > > contention is that this is a factor for setup.ini to take care of, not > > setup.exe. It's easier to add entries than have complex logic for > > removing them when they aren't appropriate. > > > So what you're saying is just looking at Current vs Previous - if there was no > setup.ini entry for current and there was only an entry for previous then with > the default display set to current the selected default action for the > theoretical package would be uninstall (just as I saw on the Test) - correct? No. There is a pretty big difference (IMO) between prev, current and test. Prev - distribution with old versions kept around for convenience sake. Current - distribution with the latest stable versions. Test - distribution with versions where things break, major changes occur etc. Any version of a package can be selected without the use of the prev/curr/test buttons - they are designed to make wholesale changes to the environment. Prev-Current should be minor - to allow backing out of a minor revision change that introduced a new bug. Current-Test *may* be very major. we have currently in the 'default' picking logic: packageversion * trustp (trusts const t) const { return t == TRUST_PREV ? (prev ? prev : (curr ? curr : installed)) : t == TRUST_CURR ? (curr ? curr : installed) : exp; } IOW, for prev, choose an explicit prev, then a current, and then an installed. for current, choose a current, and then an installed. for test (exp), choose the explicit experimental version. > And in addition if as a maintainer I didn't want this to happen then I should > make sure I have a prev,curr and test in setup.ini even if they all are the > same file name. That is one way, it is very manual however. What I'm suggesting is that *in the absence* of prev:,curr:,test: tags in setup.hint, that upset always generate a [test] entry - just like it automatically generates a [prev] entry when there are two versions in the directory. > This bears the question (if my previous assumption is correct) what if my > setup.ini has a prev, curr, and test file defined but I am doing an "Install > from Local Directory" and the test version does not exist on my system. I > would assume that since I defined a test and even though is does not exist that > my default action would be keep - true? I'm not sure :}. I'm not sure that that would be appropriate behaivour in the general case. We really need versioned conflicts: and depends: lines to make this all work correctly. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 13:06 ` Setup Brian Keener 2002-02-18 17:26 ` Setup Robert Collins @ 2002-02-18 18:48 ` Robert Collins 2002-02-18 19:50 ` Setup Robert Collins 1 sibling, 1 reply; 125+ messages in thread From: Robert Collins @ 2002-02-18 18:48 UTC (permalink / raw) To: bkeener, cygwin-apps === ----- Original Message ----- From: "Brian Keener" <bkeener@thesoftwaresource.com> To: <cygwin-apps@sources.redhat.com> Sent: Tuesday, February 19, 2002 8:06 AM Subject: Re: Setup > Robert Collins wrote: > > Ok, final feedback and bug-killing time folk. > > I just built from CVS and the big thing I noticed in the past and still seems > to be there has to do with doing an "Install from Local Directory" after doing I've found the cause, and stopped the silent failure (fix is in setup200202). I'm working on correcting the behaviour now. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 18:48 ` Setup Robert Collins @ 2002-02-18 19:50 ` Robert Collins 2002-02-22 5:38 ` Setup Brian Keener 0 siblings, 1 reply; 125+ messages in thread From: Robert Collins @ 2002-02-18 19:50 UTC (permalink / raw) To: bkeener, cygwin-apps === ----- Original Message ----- From: "Robert Collins" <robert.collins@itdomain.com.au> > I've found the cause, and stopped the silent failure (fix is in > setup200202). I'm working on correcting the behaviour now. And a complete fix is in CVS now. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-18 19:50 ` Setup Robert Collins @ 2002-02-22 5:38 ` Brian Keener 2002-02-22 14:46 ` Setup Robert Collins 0 siblings, 1 reply; 125+ messages in thread From: Brian Keener @ 2002-02-22 5:38 UTC (permalink / raw) To: cygwin-apps Robert Collins wrote: > And a complete fix is in CVS now. > I just tried setup from cvs and rebuilt and I still get the null file on the last file I try to install from my local directory. Actually it was a reinstall of zlib and it failed with the null error and then when I went back into setup it showed it as a new install and when I selected to install it - I still got the Null error. I then went and did an install from the internet and zlib showed as needing to be installed and then installed just fine. It might be something with my version since I did a full cvs update but I didnot get the patch for test to default to keep if already installed. That appears to have been applied to the branch and so I applied it to mine manually. So maybe something else got skipped as well. Bk ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-22 5:38 ` Setup Brian Keener @ 2002-02-22 14:46 ` Robert Collins 2002-02-22 19:15 ` Setup Brian Keener 0 siblings, 1 reply; 125+ messages in thread From: Robert Collins @ 2002-02-22 14:46 UTC (permalink / raw) To: bkeener, cygwin-apps === ----- Original Message ----- From: "Brian Keener" <bkeener@thesoftwaresource.com> To: <cygwin-apps@sources.redhat.com> Sent: Friday, February 22, 2002 12:53 PM Subject: Re: Setup > Robert Collins wrote: > > And a complete fix is in CVS now. > > > > I just tried setup from cvs and rebuilt and I still get the null file on the > last file I try to install from my local directory. Actually it was a > reinstall of zlib and it failed with the null error and then when I went back > into setup it showed it as a new install and when I selected to install it - I > still got the Null error. I then went and did an install from the internet and > zlib showed as needing to be installed and then installed just fine. > > It might be something with my version since I did a full cvs update but I > didnot get the patch for test to default to keep if already installed. That > appears to have been applied to the branch and so I applied it to mine > manually. So maybe something else got skipped as well. What TAG did you build from? setup200202 works fine for me, and doesn't uninstall installed packages when test is selected. Like Chris I feel the solution is wrong, but I can't articulate (yet) how the model is wrong, and so I've followed the (apparent) consensus. If you're looking at HEAD, bugfixes are going in there as well, but the test mode behaviour is cosmetic IMO, as opposed to the silent failure from local directories! Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-22 14:46 ` Setup Robert Collins @ 2002-02-22 19:15 ` Brian Keener 2002-02-22 23:04 ` Setup Christopher Faylor 2002-02-24 2:38 ` Setup Robert Collins 0 siblings, 2 replies; 125+ messages in thread From: Brian Keener @ 2002-02-22 19:15 UTC (permalink / raw) To: cygwin-apps Robert Collins wrote: > What TAG did you build from? setup200202 works fine for me, and doesn't > uninstall installed packages when test is selected. Like Chris I feel Built from HEAD - which is why I guess the fix for the Test/Uninstall wasn't there, but once I applied the patch for the Test/Uninstall fix to my copy then Test/Keep as opposed to Test/Uninstall works great. > he solution is wrong, but I can't articulate (yet) how the model is > wrong, and so I've followed the (apparent) consensus. I don't know why that doesn't seem right to you. It appears to me that I will never have an installation that is comprised solely of all Test packages. I will always be testing a few packages but the bulk of the system will be the current Stable versions. That would naturally be the nature of the beast as working with all test versions would be to cumbersome to find where something was failing. That said if I have to work with Current Stable versions while I am testing experimental packages why would I want the default on the installed packages to be Uninstall - I would want to keep those packages so I would still have a working system. I also do not particularly want to click on 20 packages to say keep these instead of uninstalling. Makes perfect sense to me. As to the NULL file when reinstalling - that patch was applied to HEAD and not just setup200202 wasn't it or was the full fix only committed to setup200202? I do seem to have that fix (since I get the message about the Null file - no silent failure) or at least part of it but it does still seem to have a problem with the last file that you might have selected. At first I thought it only had the problem if for some reason setup determined it needed the method replace_one but even after it removed zlib when I said reinstall and then I tried to just install zlib it still failed and would not install zlib. It just doesn't seem to like the last file. ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-22 19:15 ` Setup Brian Keener @ 2002-02-22 23:04 ` Christopher Faylor 2002-02-24 2:38 ` Setup Robert Collins 1 sibling, 0 replies; 125+ messages in thread From: Christopher Faylor @ 2002-02-22 23:04 UTC (permalink / raw) To: cygwin-apps On Fri, Feb 22, 2002 at 08:48:04PM -0500, Brian Keener wrote: >Robert Collins wrote: >> What TAG did you build from? setup200202 works fine for me, and doesn't >> uninstall installed packages when test is selected. Like Chris I feel > >Built from HEAD - which is why I guess the fix for the Test/Uninstall wasn't >there, but once I applied the patch for the Test/Uninstall fix to my copy then >Test/Keep as opposed to Test/Uninstall works great. > >> he solution is wrong, but I can't articulate (yet) how the model is >> wrong, and so I've followed the (apparent) consensus. > >I don't know why that doesn't seem right to you. I think Robert was saying that the whole prev/curr/test model feels wrong. But, perhaps uncharacteristically, neither of us has an opinion on how to change it. :-) cgf ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-22 19:15 ` Setup Brian Keener 2002-02-22 23:04 ` Setup Christopher Faylor @ 2002-02-24 2:38 ` Robert Collins 2002-02-27 10:20 ` Setup Brian Keener 1 sibling, 1 reply; 125+ messages in thread From: Robert Collins @ 2002-02-24 2:38 UTC (permalink / raw) To: bkeener, cygwin-apps ----- Original Message ----- From: "Brian Keener" <bkeener@thesoftwaresource.com> > > he solution is wrong, but I can't articulate (yet) how the model is > > wrong, and so I've followed the (apparent) consensus. > > I don't know why that doesn't seem right to you. It appears to me that I will > never have an installation that is comprised solely of all Test packages. [Devil's advocate mode on] Why not? Debian has one, and it works great. > I > will always be testing a few packages but the bulk of the system will be the > current Stable versions. That would naturally be the nature of the beast as > working with all test versions would be to cumbersome to find where something > was failing. That said if I have to work with Current Stable versions while I > am testing experimental packages why would I want the default on the installed > packages to be Uninstall - I would want to keep those packages so I would still > have a working system. I also do not particularly want to click on 20 packages > to say keep these instead of uninstalling. Makes perfect sense to me. I have never said that I want every package to uninstall. I have explained that that behaviour is a SIDE EFFECT of the behaviour I want, which is for system smarts about the intention of maintainers to be removed from setup, and made explicit. This gives greater flexability, and the potential for quicker changes - because setup.exe won't be part of the change process. However, that was not what I was referring to in saying that the solution is wrong. I mean that the whole x= prev/curr/test y=version model is wrong, and because THAT is wrong, the GUI and engine behaviour is confusing (because it has multiple, reasonable interpretations). > As to the NULL file when reinstalling - that patch was applied to HEAD and not ... > tried to just install zlib it still failed and would not install zlib. It just > doesn't seem to like the last file. I'll have a look-see. Rob ^ permalink raw reply [flat|nested] 125+ messages in thread
* Re: Setup 2002-02-24 2:38 ` Setup Robert Collins @ 2002-02-27 10:20 ` Brian Keener 0 siblings, 0 replies; 125+ messages in thread From: Brian Keener @ 2002-02-27 10:20 UTC (permalink / raw) To: cygwin-apps Robert Collins wrote: > > As to the NULL file when reinstalling - that patch was applied to HEAD > and not > ... > > tried to just install zlib it still failed and would not install zlib. > It just > doesn't seem to like the last file. > > I'll have a look-see. > I tried the setup-20020225.exe and also updated my sources from cvs HEAD and this problem appears to be fixed. I installed one and only one file as a reinstall and it appeared to work just like it should. It also appears (and I haven't looked real close yet) that you have improved what packages show and do not show depending on what installation/download method I chose. Good job. Bk ^ permalink raw reply [flat|nested] 125+ messages in thread
end of thread, other threads:[~2015-08-11 19:53 UTC | newest] Thread overview: 125+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-05-29 19:37 setup Achim Gratz 2015-05-31 10:24 ` setup Corinna Vinschen 2015-06-01 18:11 ` setup Achim Gratz 2015-06-01 18:34 ` setup Corinna Vinschen 2015-06-01 19:49 ` setup Achim Gratz 2015-06-01 21:06 ` setup Corinna Vinschen 2015-06-02 5:17 ` setup Achim Gratz 2015-06-02 8:29 ` setup Corinna Vinschen 2015-06-02 16:24 ` setup Achim Gratz 2015-06-03 8:19 ` setup Corinna Vinschen 2015-06-03 8:31 ` setup Marco Atzeri 2015-06-03 11:07 ` setup Corinna Vinschen 2015-06-03 12:45 ` setup Eric Blake 2015-06-03 16:51 ` setup Achim Gratz 2015-06-03 17:04 ` setup Yaakov Selkowitz 2015-06-03 17:11 ` setup Achim Gratz 2015-06-08 12:56 ` setup Corinna Vinschen 2015-06-08 13:49 ` setup Corinna Vinschen 2015-06-08 18:34 ` setup Yaakov Selkowitz 2015-06-08 18:47 ` setup Corinna Vinschen 2015-06-08 19:05 ` setup Yaakov Selkowitz 2015-06-08 19:35 ` setup Corinna Vinschen 2015-06-08 18:52 ` setup Achim Gratz 2015-06-06 20:58 ` setup Achim Gratz 2015-06-07 20:21 ` setup Achim Gratz 2015-06-08 13:28 ` setup Corinna Vinschen 2015-06-08 18:07 ` setup Achim Gratz 2015-06-08 19:31 ` setup Corinna Vinschen 2015-06-08 20:54 ` setup Achim Gratz 2015-06-09 9:56 ` setup Corinna Vinschen 2015-06-09 16:54 ` setup Achim Gratz 2015-06-10 8:05 ` setup Corinna Vinschen 2015-06-10 17:47 ` setup Achim Gratz 2015-06-10 18:54 ` setup Corinna Vinschen 2015-06-10 20:50 ` setup Achim Gratz 2015-06-11 10:06 ` setup Corinna Vinschen 2015-06-11 10:11 ` setup Corinna Vinschen 2015-06-11 18:10 ` setup Achim Gratz 2015-06-11 21:03 ` setup Achim Gratz 2015-06-12 10:29 ` setup Corinna Vinschen 2015-06-12 17:05 ` setup Achim Gratz 2015-06-12 17:37 ` setup Corinna Vinschen 2015-06-24 20:00 ` setup Achim Gratz 2015-06-25 8:59 ` setup Corinna Vinschen 2015-06-25 16:40 ` setup Achim Gratz 2015-06-25 20:16 ` setup Achim Gratz 2015-06-26 8:34 ` setup Corinna Vinschen 2015-06-26 18:07 ` setup Achim Gratz 2015-06-26 19:03 ` setup Corinna Vinschen 2015-06-28 13:40 ` setup Achim Gratz 2015-06-29 13:41 ` setup Corinna Vinschen 2015-06-30 4:59 ` setup Achim Gratz 2015-06-30 16:40 ` setup Corinna Vinschen 2015-06-30 17:14 ` setup Achim Gratz 2015-06-30 18:42 ` setup Corinna Vinschen 2015-07-01 20:37 ` setup Achim Gratz 2015-07-02 9:10 ` setup Corinna Vinschen 2015-07-02 21:03 ` setup Achim Gratz 2015-07-03 10:14 ` setup Corinna Vinschen 2015-07-11 21:31 ` setup Achim Gratz 2015-07-13 9:44 ` setup Corinna Vinschen 2015-07-14 19:57 ` setup Achim Gratz -- strict thread matches above, loose matches on Subject: below -- 2015-07-28 19:19 setup Achim Gratz 2015-07-29 12:51 ` setup Corinna Vinschen 2015-08-01 15:52 ` setup Achim Gratz 2015-08-03 18:03 ` setup Achim Gratz 2015-08-03 19:07 ` setup Achim Gratz 2015-08-03 20:02 ` setup Achim Gratz 2015-08-03 20:29 ` setup Corinna Vinschen 2015-08-03 20:31 ` setup Achim Gratz 2015-08-03 20:40 ` setup Corinna Vinschen 2015-08-03 20:54 ` setup Achim Gratz 2015-08-03 21:20 ` setup Achim Gratz 2015-08-04 18:35 ` setup Achim Gratz 2015-08-05 8:08 ` setup Corinna Vinschen 2015-08-05 15:42 ` setup Achim Gratz 2015-08-06 10:03 ` setup Corinna Vinschen 2015-08-06 15:58 ` setup Achim Gratz 2015-08-07 19:47 ` setup Corinna Vinschen 2015-08-07 21:05 ` setup Warren Young 2015-08-10 8:44 ` setup Corinna Vinschen 2015-08-10 15:02 ` setup Warren Young 2015-08-11 7:48 ` setup Corinna Vinschen 2015-08-11 19:53 ` setup Warren Young 2015-08-08 5:22 ` setup Achim Gratz 2015-08-10 9:18 ` setup Corinna Vinschen 2015-08-10 16:33 ` setup Achim Gratz 2015-08-10 14:44 ` setup Warren Young 2015-08-10 16:40 ` setup Achim Gratz 2015-08-10 16:45 ` setup Warren Young 2015-08-10 17:00 ` setup Achim Gratz 2015-08-10 17:09 ` setup Warren Young 2015-08-10 18:03 ` setup Achim Gratz 2015-08-10 18:41 ` setup Warren Young 2002-03-04 19:28 setup Robert Collins 2002-03-04 23:38 ` setup Michael A Chase 2002-03-05 8:31 ` setup Brian Keener 2002-02-20 23:10 Setup Robert Collins 2002-02-18 6:12 Setup Robert Collins 2002-02-18 13:06 ` Setup Brian Keener 2002-02-18 17:26 ` Setup Robert Collins 2002-02-18 20:05 ` Setup Christopher Faylor 2002-02-18 20:19 ` Setup Robert Collins 2002-02-18 20:31 ` Setup Brian Keener 2002-02-18 20:34 ` Setup Robert Collins 2002-02-18 20:44 ` Setup Brian Keener 2002-02-18 20:50 ` Setup Robert Collins 2002-02-19 8:53 ` Setup Brian Keener 2002-02-24 3:03 ` Setup Robert Collins 2002-02-18 20:46 ` Setup Christopher Faylor 2002-02-18 20:56 ` Setup Christopher Faylor 2002-02-20 22:45 ` Setup Gary R. Van Sickle 2002-02-20 23:00 ` Setup Christopher Faylor 2002-02-24 3:05 ` Setup Robert Collins 2002-02-24 9:07 ` Setup Christopher Faylor 2002-02-18 20:18 ` Setup Brian Keener 2002-02-18 20:30 ` Setup Robert Collins 2002-02-18 18:48 ` Setup Robert Collins 2002-02-18 19:50 ` Setup Robert Collins 2002-02-22 5:38 ` Setup Brian Keener 2002-02-22 14:46 ` Setup Robert Collins 2002-02-22 19:15 ` Setup Brian Keener 2002-02-22 23:04 ` Setup Christopher Faylor 2002-02-24 2:38 ` Setup Robert Collins 2002-02-27 10:20 ` Setup Brian Keener
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).