Re: cygport: patches welcome?

Re: cygport: patches welcome?

[Corinna Vinschen]

[Grr, I forgot to redirect this Mail.  This should have gone
 to the cygwin-apps list, actually]


On Jul 12 21:38, Eric Blake wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> According to Andrew Schulman on 7/12/2007 12:26 PM:
> > (2) Smarter update of config files.  In cygport's make_etc_defaults
> > function, we have a golden opportunity to fix a problem that's persisted for
> > a long time in Cygwin:  when a package's default config file (typically in
> > /etc) is updated, we don't currently have a standard way of automatically
> > and safely installing the new version, because just copying it into /etc
> > would destroy any of the user's existing customizations.  Our current best
> > practice is to put
> > 
> > [ -f /etc/file ] || cp /etc/defaults/etc/file /etc
> 
> But when coupled with preremove scripts, this is the right approach.
> Provided, of course, that the pre-remove does:
> 
> cmp -s /etc/file /etc/defaults/etc/file && rm /etc/file
> 
> or similar, which removes the file only if it is unmodified, so that the
> followon postinstall will only install the file if the user never touched
> it prior to the upgrade.

I'm still not sure how to handle situations where the default config
file in /etc/defaults/etc has changed between releases, but the user has
also changed the copied config file in /etc.  We have no mechanism and
no standarized way to handle this so far.

It looks like the postinstall script would have to create a copy of the
file within /etc/defaults/etc first.  This allows the postinstall script
in subsequent releases to recognize the fact that the config file has
changed.

We could create a similar naming convention as rpm (.cygnew, .cygorig)
and then we just need a way that setup notifies the user.

Two ideas come to mind.

- A postinstall script which has created a backup file returns a
  standarized return code (42 for instance).  Setup uses this return
  code to create a "backup files have been created by the following
  packages" list.

- We only allow this mechanism for the /etc directory.  After the
  postinstall scripts have finished, setup searches /etc and creates
  a list of backup files which it presents to the user.


Does that sound ok?  Any other ideas?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: cygport: patches welcome?

[Andrew Schulman]

> [Grr, I forgot to redirect this Mail.  This should have gone
>  to the cygwin-apps list, actually]

OK, but in that case the cygport README needs to be corrected, because it
says questions should go to the cygwin list.

> > > (2) Smarter update of config files.  In cygport's make_etc_defaults
> > > function, we have a golden opportunity to fix a problem that's persisted for
> > > a long time in Cygwin:  when a package's default config file (typically in
> > > /etc) is updated, we don't currently have a standard way of automatically
> > > and safely installing the new version, because just copying it into /etc
> > > would destroy any of the user's existing customizations.  Our current best
> > > practice is to put
> > > 
> > > [ -f /etc/file ] || cp /etc/defaults/etc/file /etc
> > 
> > But when coupled with preremove scripts, this is the right approach.
> > Provided, of course, that the pre-remove does:
> > 
> > cmp -s /etc/file /etc/defaults/etc/file && rm /etc/file
> > 
> > or similar, which removes the file only if it is unmodified, so that the
> > followon postinstall will only install the file if the user never touched
> > it prior to the upgrade.

Yes, this seems like a good approach, and cygport could easily do it, too.

> I'm still not sure how to handle situations where the default config
> file in /etc/defaults/etc has changed between releases, but the user has
> also changed the copied config file in /etc.  We have no mechanism and
> no standarized way to handle this so far.

When you say "handle", I think you mean notify the user, rather than try to
merge the changes.  Correct?

> It looks like the postinstall script would have to create a copy of the
> file within /etc/defaults/etc first.  This allows the postinstall script
> in subsequent releases to recognize the fact that the config file has
> changed.

I think that this won't be necessary if we use Dave's suggestion to remove
config files in preremove scripts if they haven't changed.  Then all the
postinstall script has to do is

if [ -f /etc/file ] ; then
  exit 32
else
  cp /etc/defaults/etc/file /etc
fi

(but with smarter exit handling).

> We could create a similar naming convention as rpm (.cygnew, .cygorig)
> and then we just need a way that setup notifies the user.
> 
> Two ideas come to mind.
> 
> - A postinstall script which has created a backup file returns a
>   standarized return code (42 for instance).  Setup uses this return
>   code to create a "backup files have been created by the following
>   packages" list.
>
> - We only allow this mechanism for the /etc directory.  After the
>   postinstall scripts have finished, setup searches /etc and creates
>   a list of backup files which it presents to the user.
> 
> Does that sound ok?  Any other ideas?

Debian asks the user what they want to do when it wants to update a config
file and finds that it's been modified.  But there's no mechanism for that
right now in setup, so we probably don't want to rely on adding it.

How about both:  postinstall scripts that weren't able to update a config
file because it was modified exit with signal 32 (or whatever; I suggest a
power of 2 so we can use bitmasks).  If any postinstall scripts exit with
32, then setup searches for .cygnew files in /etc and presents the list to
the user.  A side effect of this would be that each time a config file
can't be updated, setup would present the entire list of .cygnew files to
the user-- even if they were there from a previous installation, so the
user had already been notified about them before.  I think this would
probably be okay, maybe even good, as long as the user didn't get a new nag
screen *every* time they installed any new package-- only when a config
file couldn't be updated.

Anyway, no matter how we decide to handle it, it's clear that the logic
should go into cygport's make_etc_defaults function, so that packagers can
continue to just call

make_etc_defaults /etc/file

in their cygport scripts, and cygport will automatically bake the right
logic into their packages.

Andrew.

RE: cygport: patches welcome?

[Dave Korn]

On 13 July 2007 09:01, Corinna Vinschen wrote:

> I'm still not sure how to handle situations where the default config
> file in /etc/defaults/etc has changed between releases, but the user has
> also changed the copied config file in /etc.  We have no mechanism and
> no standarized way to handle this so far.

diff -u /etc/defaults/etc/prev-config /etc/config | patch --dry-run -p0 \
/etc/defaults/etc/new-config -o /etc/defaults/etc/config.mrg \
&& ( echo "Use the new one!" ; \
     mv /etc/defaults/etc/config.mrg /etc/config ) \
|| ( echo "You must manually update merge your config files! ; \
     cp /etc/defaults/etc/new-config /etc/config.new )

> Does that sound ok?  Any other ideas?

  I think it might be worth attempting an auto-merge like the above and use
the mechanism you describe in the fallback case.  The automerge procedure
needs more thinking out than the outline I've sketched above; we need to look
through the manifest, grep out the etc/defaults files, run 'file' on them to
verify that we only attempt to merge plain ascii text files, back up the old
default config files before unpacking the package tarball, etc. etc....


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: cygport: patches welcome?

[Corinna Vinschen]

On Jul 13 05:58, Andrew Schulman wrote:
> > [Grr, I forgot to redirect this Mail.  This should have gone
> >  to the cygwin-apps list, actually]
> 
> OK, but in that case the cygport README needs to be corrected, because it
> says questions should go to the cygwin list.

Actually I grabbed this thread to talk about a setup issue.  It's
not so much a cygport problem.

> > I'm still not sure how to handle situations where the default config
> > file in /etc/defaults/etc has changed between releases, but the user has
> > also changed the copied config file in /etc.  We have no mechanism and
> > no standarized way to handle this so far.
> 
> When you say "handle", I think you mean notify the user, rather than try to
> merge the changes.  Correct?


Notifying the user and creating .rpmnew/.rpmorig (s/rpm/cyg) files.

> > It looks like the postinstall script would have to create a copy of the
> > file within /etc/defaults/etc first.  This allows the postinstall script
> > in subsequent releases to recognize the fact that the config file has
> > changed.
> 
> I think that this won't be necessary if we use Dave's suggestion to remove
> config files in preremove scripts if they haven't changed.  Then all the
> postinstall script has to do is
> 
> if [ -f /etc/file ] ; then
>   exit 32
> else
>   cp /etc/defaults/etc/file /etc
> fi
> 
> (but with smarter exit handling).

Hmm, yes, that doesn't sound bad.

> How about both:  postinstall scripts that weren't able to update a config
> file because it was modified exit with signal 32 (or whatever; I suggest a
> power of 2 so we can use bitmasks).  If any postinstall scripts exit with
> 32, then setup searches for .cygnew files in /etc and presents the list to
> the user.  A side effect of this would be that each time a config file
> can't be updated, setup would present the entire list of .cygnew files to
> the user-- even if they were there from a previous installation, so the
> user had already been notified about them before.  I think this would
> probably be okay, maybe even good, as long as the user didn't get a new nag
> screen *every* time they installed any new package-- only when a config
> file couldn't be updated.

I agree.

> Anyway, no matter how we decide to handle it, it's clear that the logic
> should go into cygport's make_etc_defaults function, so that packagers can
> continue to just call
> 
> make_etc_defaults /etc/file
> 
> in their cygport scripts, and cygport will automatically bake the right
> logic into their packages.

That would be good but it's the second step.  The first step is to
figure out how we do it in setup and postinstall/preremove scripts,
if at all.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

Re: cygport: patches welcome?

[Corinna Vinschen]

On Jul 13 11:30, Dave Korn wrote:
> On 13 July 2007 09:01, Corinna Vinschen wrote:
> 
> > I'm still not sure how to handle situations where the default config
> > file in /etc/defaults/etc has changed between releases, but the user has
> > also changed the copied config file in /etc.  We have no mechanism and
> > no standarized way to handle this so far.
> 
> diff -u /etc/defaults/etc/prev-config /etc/config | patch --dry-run -p0 \
> /etc/defaults/etc/new-config -o /etc/defaults/etc/config.mrg \
> && ( echo "Use the new one!" ; \
>      mv /etc/defaults/etc/config.mrg /etc/config ) \
> || ( echo "You must manually update merge your config files! ; \
>      cp /etc/defaults/etc/new-config /etc/config.new )
> 
> > Does that sound ok?  Any other ideas?
> 
>   I think it might be worth attempting an auto-merge like the above and use
> the mechanism you describe in the fallback case.  The automerge procedure
> needs more thinking out than the outline I've sketched above; we need to look
> through the manifest, grep out the etc/defaults files, run 'file' on them to
> verify that we only attempt to merge plain ascii text files, back up the old
> default config files before unpacking the package tarball, etc. etc....

I'm a bit reluctant to present a generic auto-merge.  Consider the
situation where a new version of a package adds a new configuration
setting.  An auto-merge would pull in the default setting for this 
one, without the user knowing about this (RTFM, yes).  It might be
better to let the user decide how to proceed, wouldn't it?  Hmm,
on second thought...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

RE: cygport: patches welcome?

[Dave Korn]

On 13 July 2007 12:11, Corinna Vinschen wrote:

> On Jul 13 11:30, Dave Korn wrote:
>> On 13 July 2007 09:01, Corinna Vinschen wrote:
>> 
>>> I'm still not sure how to handle situations where the default config
>>> file in /etc/defaults/etc has changed between releases, but the user has
>>> also changed the copied config file in /etc.  We have no mechanism and
>>> no standarized way to handle this so far.
>> 
>> diff -u /etc/defaults/etc/prev-config /etc/config | patch --dry-run -p0 \
>> /etc/defaults/etc/new-config -o /etc/defaults/etc/config.mrg \
>> && ( echo "Use the new one!" ; \
>>      mv /etc/defaults/etc/config.mrg /etc/config ) \
>>>> ( echo "You must manually update merge your config files! ; \
>>      cp /etc/defaults/etc/new-config /etc/config.new )
>> 
>>> Does that sound ok?  Any other ideas?
>> 
>>   I think it might be worth attempting an auto-merge like the above and use
>> the mechanism you describe in the fallback case.  The automerge procedure
>> needs more thinking out than the outline I've sketched above; we need to
>> look through the manifest, grep out the etc/defaults files, run 'file' on
>> them to verify that we only attempt to merge plain ascii text files, back
>> up the old default config files before unpacking the package tarball, etc.
>> etc.... 
> 
> I'm a bit reluctant to present a generic auto-merge.  Consider the
> situation where a new version of a package adds a new configuration
> setting.  An auto-merge would pull in the default setting for this
> one, without the user knowing about this 


  You mean like, for instance, sshd introduces a new vital security-related
config option such as PrivSep and if the user doesn't know to turn it off
manually it gets enabled by default?  Sounds like a good thing to me ....!
(Plus it's not any different from the situation where the user installs the
package for the first time and doesn't bother to customize the config).

  I know the idea of automerge is scary.  I guess we should make sure there's
a very very easy roll-back mechanism that we can point users at if something
goes wrong and that would just restore their prior config exactly as it was
before the merge.

    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: cygport: patches welcome?

[Corinna Vinschen]

On Jul 13 12:17, Dave Korn wrote:
> On 13 July 2007 12:11, Corinna Vinschen wrote:
> > I'm a bit reluctant to present a generic auto-merge.  Consider the
> > situation where a new version of a package adds a new configuration
> > setting.  An auto-merge would pull in the default setting for this
> > one, without the user knowing about this 
> 
>   You mean like, for instance, sshd introduces a new vital security-related
> config option such as PrivSep and if the user doesn't know to turn it off
> manually it gets enabled by default?  Sounds like a good thing to me ....!

There are other packages which just change their default behaviour
because the upstream author thinks it's a good idea, not because it has
any security impact.

> (Plus it's not any different from the situation where the user installs the
> package for the first time and doesn't bother to customize the config).

Not quite.  The situation we're talking about is one in which the user
*did* care and changed the config file to his or her own needs.  When a
new release just merges in changes without notifying the user about
this, it's not exactly nice.

>   I know the idea of automerge is scary.

Yes, indeed.  Think of complex files like /etc/profile or /etc/csh.cshrc.
How do you make sure that a merge didn't just work (patch returned
without error), but that the merge actually had a still working result?
I think merging is not feasible for all files in /etc.

>     I guess we should make sure there's
> a very very easy roll-back mechanism that we can point users at if something
> goes wrong and that would just restore their prior config exactly as it was
> before the merge.

If we really do merges, a merge should always generate a copy of the
original file and setup should notify the user.  The way Andrew
described it would still be a good thing, even if an auto-merge went
fine.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

RE: cygport: patches welcome?

[Dave Korn]

On 13 July 2007 13:45, Corinna Vinschen wrote:

> On Jul 13 12:17, Dave Korn wrote:

>>   I know the idea of automerge is scary.
> 
> Yes, indeed.  Think of complex files like /etc/profile or /etc/csh.cshrc.
> How do you make sure that a merge didn't just work (patch returned
> without error), but that the merge actually had a still working result?
> I think merging is not feasible for all files in /etc.
> 
>>     I guess we should make sure there's
>> a very very easy roll-back mechanism that we can point users at if
>> something goes wrong and that would just restore their prior config
>> exactly as it was before the merge.
> 
> If we really do merges, a merge should always generate a copy of the
> original file and setup should notify the user.  The way Andrew
> described it would still be a good thing, even if an auto-merge went
> fine.

  I think you're right.  I reckon maybe we should only do automerges with the
collaboration of package maintainers, who would need to flag the config files
in their packages that are ok to merge in some way.

  Actually come to think of it I don't think even that would work.  Even the
simplest kind of VAR=VALUE config file could get messed up if the user's mods
involved re-ordering stuff, you'd end up with duplicated values.  Bah.  It's
probably not possible at all to make it really work right.


    cheers,
      DaveK
-- 
Can't think of a witty .sigline today....

Re: cygport: patches welcome?

[Corinna Vinschen]

On Jul 13 13:06, Corinna Vinschen wrote:
> On Jul 13 05:58, Andrew Schulman wrote:
> > I think that this won't be necessary if we use Dave's suggestion to remove
> > config files in preremove scripts if they haven't changed.  Then all the
> > postinstall script has to do is
> > 
> > if [ -f /etc/file ] ; then
> >   exit 32
> > else
> >   cp /etc/defaults/etc/file /etc
> > fi
> > 
> > (but with smarter exit handling).
> 
> Hmm, yes, that doesn't sound bad.
> 
> > How about both:  postinstall scripts that weren't able to update a config
> > file because it was modified exit with signal 32 (or whatever; I suggest a
> > power of 2 so we can use bitmasks).  If any postinstall scripts exit with
> > 32, then setup searches for .cygnew files in /etc and presents the list to
> > the user.  A side effect of this would be that each time a config file
> > can't be updated, setup would present the entire list of .cygnew files to
> > the user-- even if they were there from a previous installation, so the
> > user had already been notified about them before.  I think this would
> > probably be okay, maybe even good, as long as the user didn't get a new nag
> > screen *every* time they installed any new package-- only when a config
> > file couldn't be updated.
> 
> I agree.

Any comment from our setup gurus?


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

Re: cygport: patches welcome?

[Christopher Faylor]

On Mon, Jul 16, 2007 at 10:21:54AM +0200, Corinna Vinschen wrote:
>On Jul 13 13:06, Corinna Vinschen wrote:
>> On Jul 13 05:58, Andrew Schulman wrote:
>> > I think that this won't be necessary if we use Dave's suggestion to remove
>> > config files in preremove scripts if they haven't changed.  Then all the
>> > postinstall script has to do is
>> > 
>> > if [ -f /etc/file ] ; then
>> >   exit 32
>> > else
>> >   cp /etc/defaults/etc/file /etc
>> > fi
>> > 
>> > (but with smarter exit handling).
>> 
>> Hmm, yes, that doesn't sound bad.
>> 
>> > How about both:  postinstall scripts that weren't able to update a config
>> > file because it was modified exit with signal 32 (or whatever; I suggest a
>> > power of 2 so we can use bitmasks).  If any postinstall scripts exit with
>> > 32, then setup searches for .cygnew files in /etc and presents the list to
>> > the user.  A side effect of this would be that each time a config file
>> > can't be updated, setup would present the entire list of .cygnew files to
>> > the user-- even if they were there from a previous installation, so the
>> > user had already been notified about them before.  I think this would
>> > probably be okay, maybe even good, as long as the user didn't get a new nag
>> > screen *every* time they installed any new package-- only when a config
>> > file couldn't be updated.
>> 
>> I agree.
>
>Any comment from our setup gurus?

I have no comment other than to note that you can't "exit with signal 32".  You
can "exit 32" but you can't "kill -32 $$".

cgf