Re: autorebase and user-installed dynamic objects

Re: autorebase and user-installed dynamic objects

[Ken Brown]

[Redirected from the main Cygwin list.]

On 9/19/2021 2:18 AM, ASSI wrote:
> Ken Brown via Cygwin writes:
>> Looking into /usr/bin/rebaselst, I think I see the problem.  The
>> function rebase_user() greps the file /var/lib/rebase/user.d/kbrown
>> for the relevant suffixes,
> 
> That's how it was originally intended to work, IIRC (but the
> documentation indeed wrongly suggests to put paths there).  Obviously
> since we didn't have that situation before I never fully tested and
> completed this part.
> 
>> instead of looking for files in
>> /home/kbrown/.emacs.d/eln-cache. Shouldn't rebase_user use a variable
>> "userLocs" analogous to the variable "dynLocs" used by rebase_dyn()?
>> Or am I completely misunderstanding how this is supposed to work?
> 
> For reasons I don't exactly remember, I wanted to avoid that.  Probably
> because the user directory might not be available or accessible for the
> install user, but then obviously you'd just as likely have a problem
> with the actual rebasing also.  I've been mulling the idea of having
> user specific rebase databases on top of the system one (or more
> generally a hierarchy of rebase DB) several times and that's one of the
> reasons they might be needed.
> 
> Let's discuss how this can and should work on cygwin-apps.

A per-user database sounds like a good idea.

Ken

Re: autorebase and user-installed dynamic objects

[Achim Gratz]

Ken Brown via Cygwin-apps writes:
> A per-user database sounds like a good idea.

Well, the problem is how to maintain it.  So let's for the moment skip
that part and see if it would work when we pretend we'd already solved
that problem.  An ephemeral rebase is essentially a user-defined
database that gets thrown away immediately, so you can not re-use its
information.  We would need to replace emacs with a wrapper script that
checks a cookie file against the system wide rebase database and if the
latter is newer, trigger an ephemeral rebase of the emacs user
directory.  The same should be done for each compilation.  If that
works, then we can try to figure out how to not constantly rebase stuff
that doesn't need to get rebased again via some sort of user rebase
database.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: autorebase and user-installed dynamic objects

[Ken Brown]

On 9/19/2021 12:39 PM, Achim Gratz wrote:
> Ken Brown via Cygwin-apps writes:
>> A per-user database sounds like a good idea.
> 
> Well, the problem is how to maintain it.  So let's for the moment skip
> that part and see if it would work when we pretend we'd already solved
> that problem.  An ephemeral rebase is essentially a user-defined
> database that gets thrown away immediately, so you can not re-use its
> information.  We would need to replace emacs with a wrapper script that
> checks a cookie file against the system wide rebase database and if the
> latter is newer, trigger an ephemeral rebase of the emacs user
> directory.  The same should be done for each compilation.  If that
> works, then we can try to figure out how to not constantly rebase stuff
> that doesn't need to get rebased again via some sort of user rebase
> database.

I'm not sure the rebase of the emacs user directory has to be ephemeral.  First 
of all, I think we should make /var/lib/rebase/user.d/<username> work as 
documented.  Users could list their emacs user directory there, and people who 
build emacs themselves could also list their emacs build directory.  That way 
things could be kept reasonably stable in the long run via the autorebase 
postinstall script.  Implicit in this is that the results of all these rebases 
would be stored in the user's rebase database.

After each compilation, but before the new .eln file is loaded, emacs could call 
rebaselst with suitable arguments to rebase that new file (and add it to the 
user database).

Ken

Re: autorebase and user-installed dynamic objects

[Achim Gratz]

Ken Brown via Cygwin-apps writes:
> I'm not sure the rebase of the emacs user directory has to be
> ephemeral.

That would just be a POC to show that it works in a general sense.

> First of all, I think we should make /var/lib/rebase/user.d/<username>
> work as documented.

No, that won't help and I should actually remove that facility since it
can't be fixed.  The user directory can not assumed to be even
accessible when setup runs.  So anything involving a user directory must
be run under that user account, after setup has completed.  Anything
that can be done via setup should be done with packages (and those can
already use the dynpath.d facility).

> Users could list their emacs user directory there, and people who
> build emacs themselves could also list their emacs build directory.
> That way things could be kept reasonably stable in the long run via
> the autorebase postinstall script.  Implicit in this is that the
> results of all these rebases would be stored in the user's rebase
> database.

The question of where to store the result is not really important at the
moment.  We can just re-generate the intended result each time; if and
only if that works should we then optimize via caching.

> After each compilation, but before the new .eln file is loaded, emacs
> could call rebaselst with suitable arguments to rebase that new file
> (and add it to the user database).

Yes, something like that or something else with similar effect.  But
first we need to figure out if we can get it working at all.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: autorebase and user-installed dynamic objects

[Ken Brown]

On 9/19/2021 3:09 PM, Achim Gratz wrote:
> Ken Brown via Cygwin-apps writes:
>> First of all, I think we should make /var/lib/rebase/user.d/<username>
>> work as documented.
> 
> No, that won't help and I should actually remove that facility since it
> can't be fixed.  The user directory can not assumed to be even
> accessible when setup runs.  So anything involving a user directory must
> be run under that user account, after setup has completed.  Anything
> that can be done via setup should be done with packages (and those can
> already use the dynpath.d facility).

OK.

>> Users could list their emacs user directory there, and people who
>> build emacs themselves could also list their emacs build directory.
>> That way things could be kept reasonably stable in the long run via
>> the autorebase postinstall script.  Implicit in this is that the
>> results of all these rebases would be stored in the user's rebase
>> database.
> 
> The question of where to store the result is not really important at the
> moment.  We can just re-generate the intended result each time; if and
> only if that works should we then optimize via caching.

How about this as a starting point:

1. Modify rebaseall and rebaselst to recognize 'eln' as a suffix and to 
recognize the standard directory where emacs will install the system-wide .eln 
files (corresponding to the preloaded libraries).

2. Create a script /usr/bin/rebaselst_usr similar to rebaselst, with a few 
modifications:

a) All files are in the user's home directory:

l=${HOME}/.config/rebase
b=${HOME}/.cache/rebase
db=${l}/dynpath.d

b) Each user can create create files in ${db} containing the directories in 
which rebaselst_usr should look for libraries to be rebased.  In the case of 
emacs, I would probably provide a script that users can call to create the 
default ${db}/emacs, containing the line ${HOME}/.emacs.d/eln-cache.  I don't 
know if there are other packages that would want to do something similar.

c) Other obvious modifications/deletions, such as making the rebase ephemeral 
for now.

3. Run emacs from a script that calls rebaselst_usr before launching 
/usr/bin/emacs.  I'll have to think about naming issues.  It would be nice if 
the script could just be called "emacs", but then something else has to be renamed.

>> After each compilation, but before the new .eln file is loaded, emacs
>> could call rebaselst with suitable arguments to rebase that new file
>> (and add it to the user database).
> 
> Yes, something like that or something else with similar effect.  But
> first we need to figure out if we can get it working at all.

Actually, we could probably just have emacs call rebaselst_usr after each 
compilation.  Those .eln files that are already loaded won't be rebaseable, but 
so what?

Ken

Re: autorebase and user-installed dynamic objects

[ASSI]

Ken Brown via Cygwin-apps writes:
> 1. Modify rebaseall and rebaselst to recognize 'eln' as a suffix and
> to recognize the standard directory where emacs will install the
> system-wide .eln files (corresponding to the preloaded libraries).

That would be which directory?

> 2. Create a script /usr/bin/rebaselst_usr similar to rebaselst, with a
> few modifications:
>
> a) All files are in the user's home directory:
>
> l=${HOME}/.config/rebase
> b=${HOME}/.cache/rebase
> db=${l}/dynpath.d

…and there is this knotty problem.  The user has one home directory, but
she might be using many different hosts and both Cygwin architectures.
The rebase information is host and architecture specific.  I have no
good solution for that yet and I don't know if XDG provides for that
situation.  Things might have to go into /var/run/user or some similar
location instead.  Note that the architecture dependence of eln (or
really build dependence on a particular Emacs config if you want to be
precise) already is a problem for Emacs in general and I haven't paid
attention as to how they intend to solve that.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: autorebase and user-installed dynamic objects

[Ken Brown]

On 9/20/2021 1:58 AM, ASSI wrote:
> Ken Brown via Cygwin-apps writes:
>> 1. Modify rebaseall and rebaselst to recognize 'eln' as a suffix and
>> to recognize the standard directory where emacs will install the
>> system-wide .eln files (corresponding to the preloaded libraries).
> 
> That would be which directory?

When I build emacs from the upstream repo (without setting prefix) I see

   ELN_DESTDIR='/usr/local/lib/emacs/28.0.50/

So I guess they will be in a subdirectory of /usr/lib/emacs when I build emacs 
for Cygwin.

>> 2. Create a script /usr/bin/rebaselst_usr similar to rebaselst, with a
>> few modifications:
>>
>> a) All files are in the user's home directory:
>>
>> l=${HOME}/.config/rebase
>> b=${HOME}/.cache/rebase
>> db=${l}/dynpath.d
> 
> …and there is this knotty problem.  The user has one home directory, but
> she might be using many different hosts and both Cygwin architectures.
> The rebase information is host and architecture specific.  I have no
> good solution for that yet and I don't know if XDG provides for that
> situation.  Things might have to go into /var/run/user or some similar
> location instead.  Note that the architecture dependence of eln (or
> really build dependence on a particular Emacs config if you want to be
> precise) already is a problem for Emacs in general and I haven't paid
> attention as to how they intend to solve that.

I haven't noticed any discussion of that.  I don't know about dependence on a 
particular Emacs config.  But that was never an issue for elc files, so I doubt 
if it's an issue for eln files.  There's certainly architecture dependence, but 
I don't think that will be a problem for us.  If rebase encounters a library 
intended for a different architecture, I think it will skip it, just as it skips 
in-use files, won't it?

Ken

Re: autorebase and user-installed dynamic objects

[Ken Brown]

[-- Attachment #1: Type: text/plain, Size: 807 bytes --]

On 9/20/2021 8:57 AM, Ken Brown via Cygwin-apps wrote:
> On 9/20/2021 1:58 AM, ASSI wrote:
>> Ken Brown via Cygwin-apps writes:
>>> 2. Create a script /usr/bin/rebaselst_usr similar to rebaselst, with a
>>> few modifications:
>>>
>>> a) All files are in the user's home directory:
>>>
>>> l=${HOME}/.config/rebase
>>> b=${HOME}/.cache/rebase
>>> db=${l}/dynpath.d

I've taken a stab at this (attached).  The appropriate command, then, is 
'rebaselst_usr dyn rebase'.  Within emacs I would use something like

(if (eq system-type 'cygwin)
     (call-process "rebaselst_usr" nil
                   '(:file "<log file>")
                   nil "dyn" "rebase"))

I'm not sure yet where this would go so that it's called after compilation but 
before the compiled file is loaded.  I'll see if Eli can help.

Ken

[-- Attachment #2: rebaselst_usr --]
[-- Type: text/plain, Size: 4487 bytes --]

#!/bin/dash
export PATH=/bin
l=${HOME}/.config/rebase
b=${HOME}/.cache/rebase
eb="${b}/rebase_epoch"
db="${l}/dynpath.d"
for d in ${b} ${db} ; do
  if [ ! -d ${d} ] ; then
    echo "Directory ${d} does not exist, trying to re-create."
    mkdir -p ${d}
  fi
done

BaseAddress=''
if [ "x86_64" = $( uname -m ) ] ; then
  DefaultBaseAddress=0x400000000
else
  DefaultBaseAddress=0x070000000
fi    

rebuild="no"
noaslr="no"
verbose=""
doSuffixes='dll|so|oct|mex|eln'
exeSuffixes='exe'

dynPaths=$( cat ${db}/* | sort -u )
dynLocs=''
for d in ${dynPaths} ; do
  [ -d "${d}" ] && dynLocs="${dynLocs} ${d}"
done

usage () {
  echo "
rebaselst [-h | --help | [ <cmd> | <cmd1> <cmd2> ... <cmdn> ]]

Commands, will be executed in order:
------------------------------------

--verbose
	Run some commands with verbose output.

--rebuild
	Later commands will discard information in cache files and
	rebuild them from scratch.

--no-rebuild
	Keep and use information in cache files.  This is the default,
	can be used to cancel effect of a \"--rebuild\" earlier on the
	command line.

update
	Update all caches.

dyn
	Update cache for dynamic language modules and libraries.  This
	currently implies \"--rebuild\" since modules could have been
	installed by the user and the cache would be outdated.

--noaslr
	Remove ASLR and TSAware flags from dynamic objects.

rebase
	Rebase with the information in cache files (i.e does not imply
	an \"update\").

peflags
	Set PE flags on executables with the information in cache
	files (i.e does not imply an \"update\").

"
}

check_file () {
  if [ "$2" = "yes" -a  -e $1 ] ; then
    echo "removing $1"
    rm -f $1
  fi
  if [ ! -e "${eb}" ] ; then
      touch -d "@0"                          "${eb}" || \
	  touch -d "1980-01-01 00:00 UTC"    "${eb}" || \
	  touch -d "1999-05-03 07:29:06 UTC" "${eb}" || \
	  touch -d "2002-08-17 07:00 CEST"   "${eb}"
  fi
  if [ ! -e "$1" ] ; then
    echo "creating empty $1"
    touch -r "${eb}"  "$1"
  fi
  if [ "$2" != "nowrite" -a  -e $1 ] ; then
    chmod 644 "$1"
  fi
}

_tr () {
  local IFS s r
  IFS="/-+.,"
  for s in $1; do
    r="${r}_${s}"
  done
  echo "${r}"
}

update_file () {
  local IFS f g k l m
  f="$1"
  g="${f}.old"
  mv -f "${f}" "${g}"
  cat >"${f}" <<EOF
## autogenerated, do not edit!
EOF
    IFS='
'
  for k in $( grep -E '^# rebase_pkg' "${g}" | tr -c '[:alnum:]\n:' '_' ) ; do
    l=${k##*from:_}
    m=$(( ${l-0}+=1 ))
  done
  {
    m=0
    for k in $( grep -vE "^##" "${g}" ) ; do
      if [ -e "${k}" -a "${m}" = "0" ] ; then
    	echo "${k}"
      elif [ "#" = "${k%% rebase_pkg*}" ] ; then
	l=$( _tr "${k##*from: /}" )
	if [ "$(( ${l-0}-=1 ))" = "0" ] ; then
	  m=0
    	  echo "${k}"
	else
	  m=1
    	  echo "#${k}"
	fi
      elif [ ! "##" = "${k%%[^#]*}" ] ; then
    	echo "#${k}"
      fi
    done
  } >>"${f}"
  chmod 444 "${f}"
}

rebase_do () {
  local g
  g="${b}/rebase_all"
  echo "Rebasing with list ${g}, built from $@."
  cat $@ | grep -vE '^#' | sort -u >"${g}"
  if [ ! -e "/etc/rebase.db.i386" -a ! -e "/etc/rebase.db.x86_64" ] ; then
    BaseAddress="-b ${DefaultBaseAddress}"
  fi
  rebase ${BaseAddress} ${verbose} -n -O -T "${g}"
  if [ "noaslr" = "yes" ] ; then
    peflags ${verbose} -d0 -t0 -T "${g}"
  fi
  BaseAddress=''
}

peflags_do () {
  local g
  g="${b}/rebase_all_exe"
  echo "Setting PE flags with list ${g}, built from $@."
  cat $@ | grep -vE '^#|ddd\/Ddd\.exe$' | sort -u >"${g}"
  peflags ${verbose} -t1 -T "${g}"
}

rebase_dyn () {
  local g
  g="${b}/rebase_dyn"
  check_file "${g}" "yes"
  if [ "x" = "${dynLocs:-x}" ] ; then
    touch "${g}"
  else
    echo "Looking for dynamic language modules/libraries in:"
    for d in ${dynLocs} ; do echo "  ${d}" ; done
    echo "Updating rebase information for dynamic language modules/libraries ${g}."
    find ${dynLocs} -xdev -regextype posix-extended -regex ".*\.(${doSuffixes})$" -newer "${g}" >>"${g}"
  fi
  update_file "${g}"
}

if [ "$#" = "0" ] ; then
  set -- "--help"
fi
while [ $# -gt 0 ] ; do 
  case "$1" in
    --verbose )
      verbose=--verbose
      ;;
    --rebuild )
      rebuild=yes
      ;;
    --noaslr )
      noaslr=yes
      ;;
    --no-rebuild )
      rebuild=no
      ;;
    rebase )
      rebase_do "${b}/rebase_dyn"
      ;;
    peflags )
      peflags_do "${b}/rebase_exe"
      ;;
    update )
      shift
      set -- dummy dyn $@
      ;;
    dyn )
      rebase_dyn
      ;;
    -h|--help|* )
      usage
      exit 127
      ;;
  esac
  shift
done
exit 0

Re: autorebase and user-installed dynamic objects

[Achim Gratz]

Ken Brown via Cygwin-apps writes:
> I've taken a stab at this (attached).

I'm a bit short on time right now, but my plan is to remove the user.d
functions from rebaselst and instead implement an option that will make
it work on non-system files (e.g. user directory).  That should cut down
on code duplication I hope and make it easier to later take advantage of
an updated rebase that can work with more than one database.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: autorebase and user-installed dynamic objects

[Achim Gratz]

Achim Gratz writes:
> Ken Brown via Cygwin-apps writes:
>> I've taken a stab at this (attached).
>
> I'm a bit short on time right now, but my plan is to remove the user.d
> functions from rebaselst and instead implement an option that will make
> it work on non-system files (e.g. user directory).  That should cut down
> on code duplication I hope and make it easier to later take advantage of
> an updated rebase that can work with more than one database.

I have an interim update that works more or less like the current
implementation plus some fixes to be more resilient against filenames
containing spaces somewhere (I'd appreciate if somone would actually
test that this works), but allows to have more fine-grained control over
the locations or files to be rebased (changed documentation below).  The
.eln suffix is already included in the list of extensions.  This only
works on systems where the user directory is fully accessible to setup,
as it will still be done during the normal autorebase step.  Any
user-specific rebase that caters to those installations that do not
fulfill that requirement will have to wait for the requisite support in
rebase to appear.  Point setup to use this additional install source:

root=http://cygwin.stromeko.net/
$root/stage

and install the test package for _autorebase contained therein.

--8<---------------cut here---------------start------------->8---
_autorebase
===========

This package provides scripts to keep the Cygwin system properly
rebased.  By default this happens incrementally, which means only
dynamic objects that have been installed after the last run of rebase
will be considered and the current rebase map takes account of the
already rebased parts of the installation.  The scripts must be run by
the system administrator or from another account that has all the
necessary access rights.

Over time the rebase map can fragment.  By triggering a full rebase,
all dynamic objects on the system are treated as newly installed and
the existing rebase map is cleared before doing the rebase.  To
perform a full rebase, execute "rebase-trigger fullrebase".  Then shut
down Cygwin including any services you have installed and simply run
setup.exe.  The rebase will be performed even when the installation
did not get modified in any way.  Subsequent runs of setup.exe will
again rebase in incremental mode.

Some programs allow the user to create or install dynamic objects
after installation.  Since these are not part of an installed package,
they wouldn't be rebased automatically, as their location isn't known
to the package system. Entire subtrees to be searched for dynamic
objects can be advertised in /var/lib/rebase/dynpath.d/,
/var/lib/rebase/localpath.d/ and /var/lib/rebase/userpath.d/ for
packaged paths, locally provided paths and user paths, respectively.
The format of files in these directories is one absolute path per
line.  The list of suffixes that files need to have to be considered
dynamic objects in these locations is hardcoded as
"dll|so|eln|oct|mex".  Files that should be rebased, but do not have
one of these suffixes can be advertised in /var/lib/dynfile.d,
/var/lib/localfile.d and /var/lib/userfile.d for packaged files,
locally provided files and user files, respectively.  The format of
files in these directories is one absolute file name per line.
Packagers should name their files after the (main) application package
that creates the dynamic objects at the advertised location and must
not package anything in either the local*.d or user*.d directories.
The local system administrator should only place files in local*.d
directories and otherwise keep the naming scheme as used for packages.
The user*.d file names should start with the account name of the user
the advertised locations belong to.

The user running the installation must have full access rights to all
such paths and all such locations must be available (e.g. remote or
encrypted volumes must be mounted and unlocked, respectively).

The incremental rebase is controlled by the script /usr/bin/rebaselst.
Except for debugging purposes this script should not be run directly
by users.  Like any other attempt at rebasing it doesn't work
correctly on a live Cygwin system.
--8<---------------cut here---------------end--------------->8---


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: autorebase and user-installed dynamic objects

[Ken Brown]

On 9/25/2021 9:26 AM, Achim Gratz wrote:
> I have an interim update that works more or less like the current
> implementation plus some fixes to be more resilient against filenames
> containing spaces somewhere (I'd appreciate if somone would actually
> test that this works), but allows to have more fine-grained control over
> the locations or files to be rebased (changed documentation below).  The
> .eln suffix is already included in the list of extensions.  This only
> works on systems where the user directory is fully accessible to setup,
> as it will still be done during the normal autorebase step.  Any
> user-specific rebase that caters to those installations that do not
> fulfill that requirement will have to wait for the requisite support in
> rebase to appear.  Point setup to use this additional install source:
> 
> root=http://cygwin.stromeko.net/
> $root/stage
> 
> and install the test package for _autorebase contained therein.

I tried to install this with 'setup -X', but I'm getting a Download Incomplete 
pop-up saying that there was a download error for _autorebase-001090-0.1.

Ken

Re: autorebase and user-installed dynamic objects

[Achim Gratz]

Ken Brown via Cygwin-apps writes:
> I tried to install this with 'setup -X', but I'm getting a Download
> Incomplete pop-up saying that there was a download error for
> _autorebase-001090-0.1.

Can you try again, please?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: autorebase and user-installed dynamic objects

[Ken Brown]

On 9/25/2021 11:45 AM, Achim Gratz wrote:
> Ken Brown via Cygwin-apps writes:
>> I tried to install this with 'setup -X', but I'm getting a Download
>> Incomplete pop-up saying that there was a download error for
>> _autorebase-001090-0.1.
> 
> Can you try again, please?

Installs fine now, thanks.

I'll try it out with emacs over the next few days.

Ken

Re: autorebase and user-installed dynamic objects

[Achim Gratz]

Ken Brown via Cygwin-apps writes:
> Installs fine now, thanks.

Please install the latest update.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: autorebase and user-installed dynamic objects

[Ken Brown]

On 9/26/2021 4:07 AM, Achim Gratz wrote:
> Ken Brown via Cygwin-apps writes:
>> Installs fine now, thanks.
> 
> Please install the latest update.

I just tried to do a full rebase (by running 'rebase-trigger full'), but it 
seems that files from installed packages aren't being rebased.  Looking at 
/var/cache/rebase, it looks like the files in rebase_pkg aren't making it into 
rebase_all.

Ken

Re: autorebase and user-installed dynamic objects

[ASSI]

Ken Brown via Cygwin-apps writes:
> I just tried to do a full rebase (by running 'rebase-trigger full'),
> but it seems that files from installed packages aren't being rebased.
> Looking at /var/cache/rebase, it looks like the files in rebase_pkg
> aren't making it into rebase_all.

It should already do a full rebase when you re-install (the latest
version is 001091-0.1).  Please do that and post / paste:

ls -al /var/cache/rebase
/var/log/setup.log.full


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: autorebase and user-installed dynamic objects

[Ken Brown]

[-- Attachment #1: Type: text/plain, Size: 1999 bytes --]

On 9/27/2021 2:43 AM, ASSI wrote:
> Ken Brown via Cygwin-apps writes:
>> I just tried to do a full rebase (by running 'rebase-trigger full'),
>> but it seems that files from installed packages aren't being rebased.
>> Looking at /var/cache/rebase, it looks like the files in rebase_pkg
>> aren't making it into rebase_all.
> 
> It should already do a full rebase when you re-install (the latest
> version is 001091-0.1).

I just reinstalled it.

>  Please do that and post / paste:
> 
> ls -al /var/cache/rebase

$ ls -al /var/cache/rebase/
total 579
drwxr-xr-x+ 1 kbrown-admin None      0 2021-09-27 08:35 ./
drwxr-xr-x+ 1 kbrown-admin None      0 2021-09-27 08:35 ../
-rw-r--r--  1 kbrown-admin None    115 2021-09-27 08:35 fullrebase.done
-rw-r--r--  1 kbrown-admin None 138550 2021-09-27 08:35 rebase_all
-r--r--r--  1 kbrown-admin None  25689 2021-09-27 08:35 rebase_dyn
-rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_dyn.old
-r--r--r--  1 kbrown-admin None     31 2021-09-27 08:35 rebase_dyn_exe
-rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_dyn_exe.old
-rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_epoch
-r--r--r--  1 kbrown-admin None  55862 2021-09-27 08:35 rebase_exe
-rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_exe.old
-r--r--r--  1 kbrown-admin None 120902 2021-09-27 08:35 rebase_lst
-rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_lst.old
-r--r--r--  1 kbrown-admin None 236372 2021-09-27 08:35 rebase_pkg
-rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_pkg.old
-rw-r--r--  1 kbrown-admin None     64 2021-09-27 08:35 rebuild_always

Note that rebase_pkg is much bigger than rebase_all.

> /var/log/setup.log.full

Attached.  Note that there are a lot of complaints about non-existent .eln 
files.  I'm experimenting with native-compilation builds of emacs, and I know 
why these errors are occurring.  But I don't think that should be preventing the 
full rebase from working.

Ken



[-- Attachment #2: setup.log.full.xz --]
[-- Type: application/octet-stream, Size: 17848 bytes --]

Re: autorebase and user-installed dynamic objects

[ASSI]

Ken Brown via Cygwin-apps writes:
> $ ls -al /var/cache/rebase/
> total 579
> drwxr-xr-x+ 1 kbrown-admin None      0 2021-09-27 08:35 ./
> drwxr-xr-x+ 1 kbrown-admin None      0 2021-09-27 08:35 ../
> -rw-r--r--  1 kbrown-admin None    115 2021-09-27 08:35 fullrebase.done
> -rw-r--r--  1 kbrown-admin None 138550 2021-09-27 08:35 rebase_all
> -r--r--r--  1 kbrown-admin None  25689 2021-09-27 08:35 rebase_dyn
> -rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_dyn.old
> -r--r--r--  1 kbrown-admin None     31 2021-09-27 08:35 rebase_dyn_exe
> -rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_dyn_exe.old
> -rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_epoch
> -r--r--r--  1 kbrown-admin None  55862 2021-09-27 08:35 rebase_exe
> -rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_exe.old
> -r--r--r--  1 kbrown-admin None 120902 2021-09-27 08:35 rebase_lst
> -rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_lst.old
> -r--r--r--  1 kbrown-admin None 236372 2021-09-27 08:35 rebase_pkg
> -rw-r--r--  1 kbrown-admin None      0 1969-12-31 19:00 rebase_pkg.old
> -rw-r--r--  1 kbrown-admin None     64 2021-09-27 08:35 rebuild_always
>
> Note that rebase_pkg is much bigger than rebase_all.

That's normal, rebase_pkg contains a mix that later gets filtered and
split into rebase_lst and rebase_exe; rebase_all is the unification
(minus comment lines) of rebase_lst and rebase_dyn.

>> /var/log/setup.log.full
>
> Attached.  Note that there are a lot of complaints about non-existent
> .eln files.  I'm experimenting with native-compilation builds of
> emacs, and I know why these errors are occurring.  But I don't think
> that should be preventing the full rebase from working.

It doesn't, based on this information I'd say everything worked as it
should.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: autorebase and user-installed dynamic objects

[Ken Brown]

On 9/27/2021 9:21 AM, ASSI wrote:
> Ken Brown via Cygwin-apps writes:
>> Note that there are a lot of complaints about non-existent
>> .eln files.  I'm experimenting with native-compilation builds of
>> emacs, and I know why these errors are occurring.  But I don't think
>> that should be preventing the full rebase from working.
> 
> It doesn't, based on this information I'd say everything worked as it
> should.

I've fixed the error that was causing rebase to complain about non-existent .eln 
files, and emacs seems to be working great now (64-bit only).  It's possible 
that we'll find that little or nothing further needs to be done for the 64-bit case.

I'm thinking of putting out a test release of emacs so that we can get some more 
data.  In case you want to try it yourself before I get around to doing that, 
you can find the current version of the .cygport file on the playground branch 
of the source repo 
(https://cygwin.com/git-cygwin-packages/?p=git/cygwin-packages/emacs.git).

Ken