symlink problems with mac os

symlink problems with mac os

[Björn Raupach]

Hi there,

on macOS High Sierra (10.13) the kawa bash script does’t work with symbolic links.

Steps to reproduce:

1) Download and extract kawa-3.0.zip to a folder $KAWA_HOME

2) Open a terminal, create a symlink, execute symlink

$ ln -s $KAWA_HOME/bin/kawa kawa
$ ./kawa
readlink: illegal option — f
usage: readlink [-n] [file …]
Error: Could not find or load main class kawa.repl
$

Cause:

The readlink version of macOS isn’t the same as GNU/Linux one. The latter version does follow symlinks with -f. The macOS one does not support this feature.

I stumbled upon this while working on a port for MacPorts<https://www.macports.org>  Should make kawa easier to install and upgrade on macOS. (Of course it is already simple enough being a jar file and such…)

Found a related discussion on SO<https://stackoverflow.com/questions/1055671/how-can-i-get-the-behavior-of-gnus-readlink-f-on-a-mac>.


with kind regards,

Björn Raupach

Re: symlink problems with mac os

[Per Bothner]

On 10/22/2017 05:00 AM, Björn Raupach wrote:
> The readlink version of macOS isn’t the same as GNU/Linux one. The latter version does follow symlinks with -f. The macOS one does not support this feature.
> 
> I stumbled upon this while working on a port for MacPorts<https://www.macports.org>  Should make kawa easier to install and upgrade on macOS. (Of course it is already simple enough being a jar file and such…)

There is a Homebrew port fof Kawa which avoids the problem by patching
bin/kawa before installing it:

https://github.com/Homebrew/homebrew-core/blob/master/Formula/kawa.rb

Another option is to install GNU readlink.

I considered replacing readlink -f by realpath, but as Mac OS doesn't
have readlink either, I don't see how that would help.

We could make the script more complicated, so it can work with plain readlink,
perhaps based on https://github.com/mkropat/sh-realpath/blob/master/realpath.sh .
I'm not thrilled to complicate the script that much, but it may be better
than adding a dependency.

The reason for the readlink to handle the case that you install kawa
as a symlink for example /usr/local/bin/kawa -> /opt/kawa-3.0/bin/kawa .
I think that is useful to support - and it doesn't work without the -f.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: symlink problems with mac os

[Björn Raupach]

On 22. Oct 2017, at 14:53, Per Bothner <per@bothner.com<mailto:per@bothner.com>> wrote:

On 10/22/2017 05:00 AM, Björn Raupach wrote:
The readlink version of macOS isn’t the same as GNU/Linux one. The latter version does follow symlinks with -f. The macOS one does not support this feature.
I stumbled upon this while working on a port for MacPorts<https://www.macports.org>  Should make kawa easier to install and upgrade on macOS. (Of course it is already simple enough being a jar file and such…)

There is a Homebrew port fof Kawa which avoids the problem by patching
bin/kawa before installing it:

https://github.com/Homebrew/homebrew-core/blob/master/Formula/kawa.rb

Agreed. I would do a patch like this in MacPorts, too. Was just raising awareness of this minor issue if it was unnoticed before.


Another option is to install GNU readlink.

I considered replacing readlink -f by realpath, but as Mac OS doesn't
have readlink either, I don't see how that would help.

We could make the script more complicated, so it can work with plain readlink,
perhaps based on https://github.com/mkropat/sh-realpath/blob/master/realpath.sh .
I'm not thrilled to complicate the script that much, but it may be better
than adding a dependency.

Apache Maven does nice bash scripting to work around this issue. I am not much of a Bash programmer so I would need some time to figure this out. Here is the snippet:

## resolve links - $0 may be a link to Maven's home
PRG="$0"

# need this for relative symlinks
while [ -h "$PRG" ] ; do
  ls=`ls -ld "$PRG"`
  link=`expr "$ls" : '.*-> \(.*\)$'`
  if expr "$link" : '/.*' > /dev/null; then
    PRG="$link"
  else
    PRG="`dirname "$PRG"`/$link"
  fi
done


The reason for the readlink to handle the case that you install kawa
as a symlink for example /usr/local/bin/kawa -> /opt/kawa-3.0/bin/kawa .
I think that is useful to support - and it doesn't work without the -f.

The maven snippet above handles that case. I can link a link and still get maven to run. Would make it portable and doesn’t need any dependency.

Looking for patches? I have time to look at that and now I am curious enough to try and solve this. ;)

--
--Per Bothner
per@bothner.com<mailto:per@bothner.com>   http://per.bothner.com/

Re: symlink problems with mac os

[Per Bothner]

On 10/22/2017 06:04 AM, Björn Raupach wrote:

> There is a Homebrew port fof Kawa which avoids the problem by patching
> bin/kawa before installing it:
> 
> https://github.com/Homebrew/homebrew-core/blob/master/Formula/kawa.rb
> 
> Agreed. I would do a patch like this in MacPorts, too. Was just raising awareness of this minor issue if it was unnoticed before.

I've been vaguely aware of it.

> The reason for the readlink to handle the case that you install kawa
> as a symlink for example /usr/local/bin/kawa -> /opt/kawa-3.0/bin/kawa .
> I think that is useful to support - and it doesn't work without the -f.

On second though that use-case can be handled just as well with creating
an executable file /usr/local/bin/kawa containing:

exec /opt/kawa-3.0/bin/kawa "$@"

However, when doing a source build, and then executing from the build
directory (as the test-suite does) still has to traverse a symlink,
as configure creates a symlink from kawa.sh to kawa.sh.in.

> The maven snippet above handles that case. I can link a link and still get maven to run. Would make it portable and doesn’t need any dependency.

It would slow start up a little bit.

> Looking for patches? I have time to look at that and now I am curious enough to try and solve this. ;)

I think I want to leave kawa.sh.in as-is, but have configure
create kawa instead of kawa.sh.  That way we don't need a symlink in the normal case.
I'm trying this out now.

-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: symlink problems with mac os

[Björn Raupach]

> On 22. Oct 2017, at 19:40, Per Bothner <per@bothner.com> wrote:
> 
> On 10/22/2017 06:04 AM, Björn Raupach wrote:
> 
>> There is a Homebrew port fof Kawa which avoids the problem by patching
>> bin/kawa before installing it:
>> https://github.com/Homebrew/homebrew-core/blob/master/Formula/kawa.rb
>> Agreed. I would do a patch like this in MacPorts, too. Was just raising awareness of this minor issue if it was unnoticed before.
> 
> I've been vaguely aware of it.
> 
>> The reason for the readlink to handle the case that you install kawa
>> as a symlink for example /usr/local/bin/kawa -> /opt/kawa-3.0/bin/kawa .
>> I think that is useful to support - and it doesn't work without the -f.
> 
> On second though that use-case can be handled just as well with creating
> an executable file /usr/local/bin/kawa containing:
> 
> exec /opt/kawa-3.0/bin/kawa "$@“

Not sure about this. Probably wouldn’t help my case. What if I don’t install in /opt/kawa-3.0? 

> 
> However, when doing a source build, and then executing from the build
> directory (as the test-suite does) still has to traverse a symlink,
> as configure creates a symlink from kawa.sh to kawa.sh.in.
> 
>> The maven snippet above handles that case. I can link a link and still get maven to run. Would make it portable and doesn’t need any dependency.
> 
> It would slow start up a little bit.

I don’t think it matters much. Slow start is already caused by starting a JVM. Starting chicken scheme or mit scheme feels more snappier. But we are talking felt milliseconds here. Compared to other programs kawa already starts fast.

> 
>> Looking for patches? I have time to look at that and now I am curious enough to try and solve this. ;)
> 
> I think I want to leave kawa.sh.in as-is, but have configure
> create kawa instead of kawa.sh.  That way we don't need a symlink in the normal case.
> I'm trying this out now.

Alright. I will still try to copy the plain bash approach in your startup script. Just to see if I can get it to run.

Happy Sunday

> 
> -- 
> 	--Per Bothner
> per@bothner.com   http://per.bothner.com/

Re: symlink problems with mac os

[Per Bothner]

On 10/22/2017 11:15 AM, Björn Raupach wrote:
>> On 22. Oct 2017, at 19:40, Per Bothner <per@bothner.com> wrote:

>>> The reason for the readlink to handle the case that you install kawa
>>> as a symlink for example /usr/local/bin/kawa -> /opt/kawa-3.0/bin/kawa .
>>> I think that is useful to support - and it doesn't work without the -f.
>>
>> On second though that use-case can be handled just as well with creating
>> an executable file /usr/local/bin/kawa containing:
>>
>> exec /opt/kawa-3.0/bin/kawa "$@“
> 
> Not sure about this. Probably wouldn’t help my case. What if I don’t install in /opt/kawa-3.0?

That is just an example.  I'm talking about either
(a)  building from source (and then running in-place without doing 'make install') or
(b) unzip the binary distribution in some directory - for example /opt/kawa-3.0.

After that you might want to put 'kawa' in your PATH.  Instead if adding another entry
to your PATH variable, you can either:
(1) create a 'kawa' symlink from a directory in your PATH to (for example)
/opt/kawa-3.0/bin/kawa, or
(2) create a small shell file containing 'exec /opt/kawa-3.0/bin/kawa "$@“'
in a directory in your path.

Option (1) requires 'readlink -f' to work - but option (2) doesn't.

(Assuming I fix thing so $builddir/bin/kawa is no longer a symlink.)

-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: symlink problems with mac os

[Per Bothner]

On 10/22/2017 10:40 AM, Per Bothner wrote:
> I think I want to leave kawa.sh.in as-is, but have configure
> create kawa instead of kawa.sh.  That way we don't need a symlink in the normal case.
> I'm trying this out now.

I checked this change in. Please try it.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: symlink problems with mac os

[Björn Raupach]

Good morning Per,

> On 23. Oct 2017, at 00:10, Per Bothner <per@bothner.com> wrote:
> 
> On 10/22/2017 10:40 AM, Per Bothner wrote:
>> I think I want to leave kawa.sh.in as-is, but have configure
>> create kawa instead of kawa.sh.  That way we don't need a symlink in the normal case.
>> I'm trying this out now.
> 
> I checked this change in. Please try it.

Thanks for spending time on this. Like I said it is really a minor issue.

Unfortunately I couldn’t make it work. Might be a misunderstanding on my part.

Did a git pull and saw your recent change 93d42…

Went with the configure & make route and installed the kawa distribution in a directory (KAWA_HOME)

kawa/
├── bin
│   ├── kawa -> /Users/raupach/opt/kawa/share/kawa/bin/kawa
│   └── qexo
└── share
    ├── info
    │   ├── dir
    │   └── kawa.info <http://kawa.info/>
    ├── kawa
    │   ├── bin
    │   │   └── kawa
    │   └── lib
    │       └── kawa.jar
    └── man
        └── man1
            ├── kawa.1
            └── qexo.1

8 directories, 8 files
$ 

Now, If I do a ./kawa in the bin directory of KAWA_HOME I end up with the recent failure message:

$ ./kawa 
readlink: illegal option -- f
usage: readlink [-n] [file ...]
Error: Could not find or load main class kawa.repl
$ 

This hasn’t happened before the change. I checked with the binary installation and there you can do just a ./kawa without this error. (It does work in the share/kawa/bin directory though)

If I symlink to KAWA_HOME/bin/kawa the issue remains. Same readlink error message. 


> -- 
> 	--Per Bothner
> per@bothner.com   http://per.bothner.com/

Re: symlink problems with mac os

[Per Bothner]

On 10/22/2017 11:47 PM, Björn Raupach wrote:

> Thanks for spending time on this. Like I said it is really a minor issue.

True. Regardless, I prefer a cleaner/simpler file layout that minimizes symlinks.
If that can avoid a problem on Mac so much the better.

> Unfortunately I couldn’t make it work. Might be a misunderstanding on my part.

I forgot about this: there is a both a share/kawa/bin/kawa, and a bin/kawa
that is a link to the latter.  That seems ugly.  The reason is so that the
kawa script can just look for ../kawa/lib/kawa.jar, regardless of whether we:
(1) run-in-place in the build directory,
(2) run installed files (result of 'make install'), or
(3) run from an unzipped kawa.zip binary distribution.

I think there are better ways to solve this "multi-mode problem" - one is
for 'make install' to make a one-line patch to the kawa script as it is installed.

-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: symlink problems with mac os

[Björn Raupach]

> On 23. Oct 2017, at 18:55, Per Bothner <per@bothner.com> wrote:
> 
> On 10/22/2017 11:47 PM, Björn Raupach wrote:
> 
>> Thanks for spending time on this. Like I said it is really a minor issue.
> 
> True. Regardless, I prefer a cleaner/simpler file layout that minimizes symlinks.
> If that can avoid a problem on Mac so much the better.

I found a to my understanding cleaner approach that doesn't rely on installing coreutils, python or some other dependency. Just pure bash. Credits go to Apache Maven and Apache Tomcat. I didn’t come up with this, I just looked how they did it.

Below is a modified version of the kawa bash script. Everything after the last line wasn’t touched.

#!/bin/bash
thisfile=`command -v $0`
case "$thisfile" in
  "") echo "installation error - can't find path to $0"; exit -1 ;;
  /*) ;;
  *) thisfile="$PWD/$thisfile"  ;;
esac
while [ -h "$thisfile" ]; do
  ls=`ls -ld "$thisfile"`
  link=`expr "$ls" : '.*-> \(.*\)$'`
  if expr "$link" : '/.*' > /dev/null; then
    thisfile="$link"
  else
    thisfile=`dirname “$thisfile"`/"$link"
  fi
done
thisdir=`dirname "$thisfile"`
kawadir=`cd "$thisdir/.." >/dev/null; pwd`

Instead of using readlink -f they just parse the output of the ls command. To get to the kawadir directory they use cd instead of realpath. Works! 

Not a lawyer, so I am not sure if this copy&paste is some license infringement.

> 
>> Unfortunately I couldn’t make it work. Might be a misunderstanding on my part.
> 
> I forgot about this: there is a both a share/kawa/bin/kawa, and a bin/kawa
> that is a link to the latter.  That seems ugly.  The reason is so that the
> kawa script can just look for ../kawa/lib/kawa.jar, regardless of whether we:
> (1) run-in-place in the build directory,
> (2) run installed files (result of 'make install'), or
> (3) run from an unzipped kawa.zip binary distribution.
> 
> I think there are better ways to solve this "multi-mode problem" - one is
> for 'make install' to make a one-line patch to the kawa script as it is installed.
> 
> -- 
> 	--Per Bothner
> per@bothner.com   http://per.bothner.com/