wordexp() implementation without fork()

wordexp() implementation without fork()

[Joel Sherrill]

Hi

Having a single process wordexp() implementation has been on the RTEMS
POSIX wishlist for a while. I haven't been able to find one. The POSIX
method description
seems to hint that this implementation strategy is possible and the Linux
man page
uses the phrase "shell-like expansion".

Does anyone know of an implementation that doesn't use fork()? Or an
implementation
strategy based around some open source wildcard evaluation?

Or is this a POSIX method that is just unimplementable without multiple
processes?

Thanks.

--joel

Re: wordexp() implementation without fork()

[Sebastian Huber]

On 05/08/2020 17:35, Joel Sherrill wrote:

> Hi
>
> Having a single process wordexp() implementation has been on the RTEMS
> POSIX wishlist for a while. I haven't been able to find one. The POSIX
> method description
> seems to hint that this implementation strategy is possible and the Linux
> man page
> uses the phrase "shell-like expansion".
>
> Does anyone know of an implementation that doesn't use fork()? Or an
> implementation
> strategy based around some open source wildcard evaluation?
>
> Or is this a POSIX method that is just unimplementable without multiple
> processes?

Can't you re-use parts of the shell implementation, e.g. 
freebsd_wordexpcmd() in

https://github.com/freebsd/freebsd/blob/master/bin/sh/expand.c#L1507

Re: wordexp() implementation without fork()

[Brian Inglis]

On 2020-08-05 11:09, Sebastian Huber wrote:
> On 05/08/2020 17:35, Joel Sherrill wrote:
>> Having a single process wordexp() implementation has been on the RTEMS 
>> POSIX wishlist for a while. I haven't been able to find one. The POSIX 
>> method description seems to hint that this implementation strategy is
>> possible and the Linux man page uses the phrase "shell-like expansion".
>>
>> Does anyone know of an implementation that doesn't use fork()? Or an 
>> implementation strategy based around some open source wildcard evaluation?
>>
>> Or is this a POSIX method that is just unimplementable without multiple
>> processes?

> Can't you re-use parts of the shell implementation, e.g. freebsd_wordexpcmd() in
> 
> https://github.com/freebsd/freebsd/blob/master/bin/sh/expand.c#L1507

The main issue seems to be if command substitution is supported, it has to be
executed in a subshell, with the full capabilities of a POSIX shell:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/wordexp.html

See RATIONALE:

"While wordexp() could be implemented entirely as a library routine, it is
expected that most implementations run a shell in a subprocess to do the expansion."

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in IEC units and prefixes, physical quantities in SI.]

Re: wordexp() implementation without fork()

[Joel Sherrill]

On Fri, Aug 7, 2020 at 2:32 PM Brian Inglis <Brian.Inglis@systematicsw.ab.ca>
wrote:

> On 2020-08-05 11:09, Sebastian Huber wrote:
> > On 05/08/2020 17:35, Joel Sherrill wrote:
> >> Having a single process wordexp() implementation has been on the RTEMS
> >> POSIX wishlist for a while. I haven't been able to find one. The POSIX
> >> method description seems to hint that this implementation strategy is
> >> possible and the Linux man page uses the phrase "shell-like expansion".
> >>
> >> Does anyone know of an implementation that doesn't use fork()? Or an
> >> implementation strategy based around some open source wildcard
> evaluation?
> >>
> >> Or is this a POSIX method that is just unimplementable without multiple
> >> processes?
>
> > Can't you re-use parts of the shell implementation, e.g.
> freebsd_wordexpcmd() in
> >
> > https://github.com/freebsd/freebsd/blob/master/bin/sh/expand.c#L1507
>
> The main issue seems to be if command substitution is supported, it has to
> be
> executed in a subshell, with the full capabilities of a POSIX shell:
>
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/wordexp.html
>
> See RATIONALE:
>
> "While wordexp() could be implemented entirely as a library routine, it is
> expected that most implementations run a shell in a subprocess to do the
> expansion."
>

The rationale clearly states that it could be implemented without
processes. It
just has to do the right thing. Sebastian's suggestion to use expansion
code from
a shell would satisfy the requirement without using fork().

Following Sebastian's design suggestion, a compliant implementation could
be constructed in a single process environment.

That leads to the question of effort required and general utility.

From an RTEMS perspective, we have been nibbling at having all POSIX
APIs possible in a single process environment even if they are lightly used.
My general opinion is that unless called out by an embedded system focused
profile of POSIX or specifically asked for by a user, it is just compliance
box
checking for us. That means this method's order in our "desired list"
would likely be lower than something simpler or asked for.

Eshan asked about this being on his Google Summer of Code list. At
this point, it seems like too much work to undertake this late in the
summer.
There are other methods on the list to add or that require tests to
demonstrate
they provide the required basic functionality.

I think the summary is that I will update the RTEMS ticket for wordexp()
with the suggestions and a link to this discussion but Eshan probably should
not undertake this before the end of GSoC. Now if he wants to stick around
and contribute, this would be an option to pick at. But not on a deadline.
:)

Thanks everyone.

--joel


>
> --
> Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
>
> This email may be disturbing to some readers as it contains
> too much technical detail. Reader discretion is advised.
> [Data in IEC units and prefixes, physical quantities in SI.]
>