SRFI 170 (POSIX API) for Kawa

SRFI 170 (POSIX API) for Kawa

[Lassi Kortela]

Is anyone working on implementing SRFI 170 for Kawa yet? Since Kawa is 
Java-based instead of C-based, and uses pathname objects as well as a 
per-thread parameter object to store the current directory, it would 
make for a good test case.

Re: SRFI 170 (POSIX API) for Kawa

[Duncan Mak]

Oh, this is interesting! I wonder if https://github.com/jnr/jnr-posix can
be used to implement this.


Duncan.

On Sat, Feb 22, 2020 at 4:46 AM Lassi Kortela <lassi@lassi.io> wrote:

> Is anyone working on implementing SRFI 170 for Kawa yet? Since Kawa is
> Java-based instead of C-based, and uses pathname objects as well as a
> per-thread parameter object to store the current directory, it would
> make for a good test case.
>


-- 
Duncan.

Re: SRFI 170 (POSIX API) for Kawa

[Per Bothner]

On Sat, Feb 22, 2020 at 4:46 AM Lassi Kortela <lassi@lassi.io> wrote:
  
> Is anyone working on implementing SRFI 170 for Kawa yet? Since Kawa is
> Java-based instead of C-based, and uses pathname objects as well as a
> per-thread parameter object to store the current directory, it would
> make for a good test case.

It might be interesting to see how much could be implemented in portable Java,
without digging into native (C/C++) code.  Kawa currently does not contain
native code, which means we can make a single cross-platform "binary release".

That doesn't mean I'll rule out having optional features that depend
on native code, but I would prefer to stick to no native code in the
default build.

There are no doubt large parts of SRFI-170 that could be implemented
without native code, but I haven't done the effort to see what the issues are.
If someone wants to look into to what extent SRFI-170 can be mapped into
Java APIs that would be interesting, but I'm not planning to do that
study myself.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: SRFI 170 (POSIX API) for Kawa

[Duncan Mak]

Hello all,

The other weekend, I took on Per's suggestion and began typing up parts of
the SRFI 170 API using Kawa Scheme.

It's still WIP, but you can see what I have here:
https://github.com/duncanmak/srfi-170/blob/kawa/srfi/kawa/lib/srfi/170/170.scm

From the module listing, you can see which Java classes I'm using:

(define-library (srfi 170)
  (import (kawa base)
          (srfi 9)
          (class java.lang ProcessHandle UnsupportedOperationException)
          (class java.util EnumSet Iterator)
          (class java.nio.file DirectoryStream DirectoryStream$Filter Files
Path Paths)
          (class java.nio.file.attribute
                 BasicFileAttributeView
                 FileAttribute
                 PosixFileAttributeView
                 PosixFilePermission
                 PosixFilePermissions)
          (class java.nio.channels FileChannel))

I haven't gotten around to hooking up with the test code, so there might be
some mistakes there.

it's pretty clear that we won't be able to support the entirety of the SRFI
170 API using just Java classes. If we want to achieve that, we'll probably
have to use JNR-Posix.

With JNR-Posix, Per's comments about shipping native code will become a
thing.

If there's interest in seeing this work completed, I can look at finishing
up the patch and try to write something back to tally which of the APIs can
be implemented with just Java, and which would require jnr-posix.

If there's even more interest, I could also look into doing a jnr-posix
implementation as well.


Duncan.

On Sat, Feb 22, 2020 at 11:40 AM Per Bothner <per@bothner.com> wrote:

> On Sat, Feb 22, 2020 at 4:46 AM Lassi Kortela <lassi@lassi.io> wrote:
>
> > Is anyone working on implementing SRFI 170 for Kawa yet? Since Kawa is
> > Java-based instead of C-based, and uses pathname objects as well as a
> > per-thread parameter object to store the current directory, it would
> > make for a good test case.
>
> It might be interesting to see how much could be implemented in portable
> Java,
> without digging into native (C/C++) code.  Kawa currently does not contain
> native code, which means we can make a single cross-platform "binary
> release".
>
> That doesn't mean I'll rule out having optional features that depend
> on native code, but I would prefer to stick to no native code in the
> default build.
>
> There are no doubt large parts of SRFI-170 that could be implemented
> without native code, but I haven't done the effort to see what the issues
> are.
> If someone wants to look into to what extent SRFI-170 can be mapped into
> Java APIs that would be interesting, but I'm not planning to do that
> study myself.
> --
>         --Per Bothner
> per@bothner.com   http://per.bothner.com/
>


-- 
Duncan.

Re: SRFI 170 (POSIX API) for Kawa

[Per Bothner]

On 7/14/20 2:10 PM, Duncan Mak wrote:
> it's pretty clear that we won't be able to support the entirety of the SRFI 170 API using just Java classes. If we want to achieve that, we'll probably have to use JNR-Posix.
> 
> With JNR-Posix, Per's comments about shipping native code will become a thing.

I don't mind an optional (configurable) dependency on jnr or jnr-posix, but I don't think
it should be a required or default dependency.

I don't have much feel for how popular or common JNR is - it seems to be
a project that is in semi-maintenance mode, and not on any kind of "standards track".
I haven't done anything with it.

That doesn't seem we can't make use of it, of course.

Please us and Duncan know (here) whether SRFI-170 or other Posix bindings would be
useful to you.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/

Re: SRFI 170 (POSIX API) for Kawa

[Arvydas Silanskas]

What are the optics for these working on windows (seeing how out-of-box
crossplatform is one of the selling points of JVM ecosystem)? Srfi text
mentions windows support, but seems the extent of that is left for the
implementations to decide

2020-07-15, tr, 07:16 Per Bothner <per@bothner.com> rašė:

> On 7/14/20 2:10 PM, Duncan Mak wrote:
> > it's pretty clear that we won't be able to support the entirety of the
> SRFI 170 API using just Java classes. If we want to achieve that, we'll
> probably have to use JNR-Posix.
> >
> > With JNR-Posix, Per's comments about shipping native code will become a
> thing.
>
> I don't mind an optional (configurable) dependency on jnr or jnr-posix,
> but I don't think
> it should be a required or default dependency.
>
> I don't have much feel for how popular or common JNR is - it seems to be
> a project that is in semi-maintenance mode, and not on any kind of
> "standards track".
> I haven't done anything with it.
>
> That doesn't seem we can't make use of it, of course.
>
> Please us and Duncan know (here) whether SRFI-170 or other Posix bindings
> would be
> useful to you.
> --
>         --Per Bothner
> per@bothner.com   http://per.bothner.com/
>

Re: SRFI 170 (POSIX API) for Kawa

[Jamison Hope]

On Wed, Jul 15, 2020 at 12:16 AM Per Bothner <per@bothner.com> wrote:
>
> On 7/14/20 2:10 PM, Duncan Mak wrote:
> > it's pretty clear that we won't be able to support the entirety of the SRFI 170 API using just Java classes. If we want to achieve that, we'll probably have to use JNR-Posix.
> >
> > With JNR-Posix, Per's comments about shipping native code will become a thing.
>
> I don't mind an optional (configurable) dependency on jnr or jnr-posix, but I don't think
> it should be a required or default dependency.
>
> I don't have much feel for how popular or common JNR is - it seems to be
> a project that is in semi-maintenance mode, and not on any kind of "standards track".
> I haven't done anything with it.
>
> That doesn't seem we can't make use of it, of course.

I agree it shouldn't be a required dependency.  But it would also be
handy to be able to use the SRFI in conjunction with a stock binary
Kawa release, and not have to do a custom configuration.

Per, can the srfi loading in ImportFromLibrary.java cross jar
boundaries?  If so, one option might be for SRFI97Map to map 170 to
some other named class that isn't included in kawa.jar, and then
provide a separately-compiled kawa-srfi170.jar or whatever that did
rely on JNR-Posix or other such native code.  Then if that other jar
is in the CLASSPATH, then (import (srfi 170)) loads/uses it, and if
not, then it doesn't work, or maybe we instead fall back on a
Java-only subset.

I've never used JNR.  My impression is that it's like JNA but..
better?  Or at least newer?  I used to use JNA to access some C
libraries, and it worked fine for what I was doing at the time.  Is
JNR-Posix the only viable option?

Duncan, can you post a listing of which SRFI-170 functions are / are
not doable using just standard Java with no third-party/native code?
It'll help to weigh the pros and cons of introducing a new dependency.

> Please us and Duncan know (here) whether SRFI-170 or other Posix bindings would be
> useful to you.

It's really a shell thing and not a POSIX thing per se, but I have
some scsh scripts lying around that made heavy use of process
pipelines and &&, which seem to have been omitted from SRFI-170.  Kawa
supports pipelines via pipe-process, but is there an equivalent to
scsh's &&?  As in:

(let ((echo (if dry-run? '(echo) '())))
  (&& (,@echo cmake ,@(cmake-opts) ,@opts ,src-dir)
      ,(if verbose?
           `(,@echo make -j ,ncpu VERBOSE=1)
           `(,@echo make -j ,ncpu)))

This almost works:
(define-syntax && (syntax-rules () ((_ e ...) (and (process-exit-ok? e) ...))))

(&& &`{date} &`{true}) => #t
(&& &`{date} &`{false}) => #f

but it loses the output of each command.  I'm guessing it would need a
validate-apply thing just like pipe-process in order to inject
"out-to: 'inherit" to each one, but I haven't figured out the right
incantation.


As for what is covered by SRFI-170, for the most part they aren't
really things I would use very often, beyond what's already available
in Kawa's file system interface.  But that's not a vote against
proceeding with this effort: I've contributed several SRFI
implementations to Kawa, and I hardly ever use any of them, either.

-- 
Jamison Hope

Re: SRFI 170 (POSIX API) for Kawa

[Per Bothner]

On 7/15/20 5:02 PM, Jamison Hope wrote:
> I agree [JNR] shouldn't be a required dependency.  But it would also be
> handy to be able to use the SRFI in conjunction with a stock binary
> Kawa release, and not have to do a custom configuration.
> 
> Per, can the srfi loading in ImportFromLibrary.java cross jar
> boundaries?

IIRC ImportFromLibrary.java doesn't know anything about jar boundaries.
It just searches in the classpath.

If so, one option might be for SRFI97Map to map 170 to
> some other named class that isn't included in kawa.jar, and then
> provide a separately-compiled kawa-srfi170.jar or whatever that did
> rely on JNR-Posix or other such native code.  Then if that other jar
> is in the CLASSPATH, then (import (srfi 170)) loads/uses it, and if
> not, then it doesn't work, or maybe we instead fall back on a
> Java-only subset.

Should be possible, as long as we make sure failure modes (error messages etc)
are reasonable.

There may be some interaction with Java 9 modules; it's been a while since I looked into those.

> I've never used JNR.  My impression is that it's like JNA but..
> better?  Or at least newer?

I believe newer.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/