[octave ] LOADPATH recurses only one level of subdirectories

[octave ] LOADPATH recurses only one level of subdirectories

[Larrie Carr]

>Your attachments don't seem to be available, so I am not able to see the
>structure of your test.

>However, a test of my own indicates the functionality does work.  What I do 
>is
>create a file called .octaverc in my own home directory, with the contents

>++++++++
>LOADPATH=[LOADPATH,':~/.octave//];
>++++++++

>I then add my own private octave functions to the directory tree under
>~/.octave.  I tested that adding a new m-file more than one level down does
>make it available when octave is restarted, forcing the path to be
>re-traversed.

>Please realize that normally the LOADPATH is only recursed once, at the 
>time
>octave is loaded.  If you put new functions out there, you need to force 
>octave
>to re-read the path.  There may be other ways to do it, but the easiest is 
>to
>exit and restart.

The problem is not that is is NOT recursing, it just that it seems to 
recurse only 1 level down.  And I've looked over at the bug lists at 
octave.org and at cygwin looking for something related.

If the loadpath contains the path /blah/A//, I would expect the tool to find 
a file (say test.m) located /blah/A/B/C/D/test.m.  It does not.

But if the loadpath contains the path /blah/A/B/C//, the tool does recurse 1 
level through directory D to find the file.  That includes all combinations 
of restarting and setting the path (LOAD_PATH and DEFAULT_LOADPATH) and path 
/blah/A/B//.  I believe this does not match the documentation and user 
experiences.

In short, octave-forge is non-functional as it uses multiple subdirectories.

I've been looking through the sources and nothing has sprung up.  Did find a 
cut-and-paste code error in oct_file_in_path() in utils.cc where the string 
length is not properly checked.  Look like a cut and paste from 
fcn_file_in_path().  But this is for a different group.

int len = name.length();

if( len > 0) {
   if (len >2 && name [ len-4] == '.' && name [len-3] == 'o'
      && name[len-2] == 'c' && name [len-1] =='t') 



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[octave ] LOADPATH recurses only one level of subdirectories

[John W. Eaton]

On  9-Feb-2006, Larrie Carr wrote:

| In short, octave-forge is non-functional as it uses multiple subdirectories.

If that's true for everyone, then I'm surprised as I think you are the
first to report it.

| I've been looking through the sources and nothing has sprung up.  Did find a 
| cut-and-paste code error in oct_file_in_path() in utils.cc where the string 
| length is not properly checked.  Look like a cut and paste from 
| fcn_file_in_path().  But this is for a different group.

Probably the code you are looking for is the function do_subdir in
liboctave/kpse.cc.  This file contains a stripped-down version of the
kpathsearch library.  Most modifications were to remove TeX-specific
stuff and to convert it to use std::string instead of plain C strings
which historically leaked memory.  In any case, that function may use
an optimization to decide when to check for subdirectories.  The
optimization looks at the link count of the current directory.  If it
is 2, then the assumption is that the current directory does not
contain any subdirectories.  That seems to work fine for Unixy
systems.  Does that assumption not hold for Cygwin?  If so, then I
think the fix is fairly simple as there is also Windows-specific code
in that function.  Whether the optimization is performed depends on
what is #defined at compile time, so you'll probably have to do some
checking on a Cygwin system to see what is really going on.

| int len = name.length();
| 
| if( len > 0) {
|    if (len >2 && name [ len-4] == '.' && name [len-3] == 'o'
|       && name[len-2] == 'c' && name [len-1] =='t') 

I've fixed this in my sources.

If you find other bugs in Octave, please report them to
bug@octave.org.

Thanks,

jwe

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [octave ] LOADPATH recurses only one level of subdirectories

[Larrie Carr]

> | In short, octave-forge is non-functional as it uses multiple 
> subdirectories.
>
> If that's true for everyone, then I'm surprised as I think you are the
> first to report it.
>
Sorry about the tone - it's not functional for me, while everyone else is 
happy.  I've tried a clean new cygwin installation using setup and pulled 
the octave installation through there - no more easy stuff to do.  Digging 
through new C++ code makes my head hurt (I'm more of a VHDL/C kind of guy).

> Probably the code you are looking for is the function do_subdir in
> liboctave/kpse.cc.  This file contains a stripped-down version of the
> kpathsearch library.  Most modifications were to remove TeX-specific
> stuff and to convert it to use std::string instead of plain C strings
> which historically leaked memory.  In any case, that function may use
> an optimization to decide when to check for subdirectories.  The
> optimization looks at the link count of the current directory.  If it
> is 2, then the assumption is that the current directory does not
> contain any subdirectories.  That seems to work fine for Unixy
> systems.  Does that assumption not hold for Cygwin?  If so, then I
> think the fix is fairly simple as there is also Windows-specific code
> in that function.  Whether the optimization is performed depends on
> what is #defined at compile time, so you'll probably have to do some
> checking on a Cygwin system to see what is really going on.

thanks for the pointer - I will have a look. 



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[octave ] LOADPATH recurses only one level of subdirectories (on network drives)

[Larrie Carr]

John W. Eaton wrote
> Probably the code you are looking for is the function do_subdir in
> liboctave/kpse.cc.  This file contains a stripped-down version of the
> kpathsearch library.  Most modifications were to remove TeX-specific
> stuff and to convert it to use std::string instead of plain C strings
> which historically leaked memory.  In any case, that function may use
> an optimization to decide when to check for subdirectories.  The
> optimization looks at the link count of the current directory.  If it
> is 2, then the assumption is that the current directory does not
> contain any subdirectories.  That seems to work fine for Unixy
> systems.  Does that assumption not hold for Cygwin?  If so, then I
> think the fix is fairly simple as there is also Windows-specific code
> in that function.  Whether the optimization is performed depends on
> what is #defined at compile time, so you'll probably have to do some
> checking on a Cygwin system to see what is really going on.


Well, after some more experimenting, the problem appears related to using 
the recursive search feature of LOADPATH on a *network* drive.  That is,

If the path is located within a physically attached hard drive, octave 
operates as expected.  For instance, /cygdrive/c/test// works all the way 
down to /cygdrive/c/test/a/b/c/d/e/

If the path is located on a network drive created using windows "Map Network 
Drive" menu option under "My Computer", octave will only recurse down 1 
level of subdirectory.  For instance "/usr/share/octave/site/m// does not 
work.

And if I use a cygwin windows mapping instead 
"/cygdrive/x/cygwin/usr/share/octave/site/m//, it still does not work 
correctly and recurses down only one level.

In my installation, the entire cygwin root is located on a network drive 
(and no one else uses it).  Works just fine for everything else.

John, it you got any other hints, I would appreciate it as I'm diving in.

Larrie. 



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[octave ] LOADPATH recurses only one level of subdirectories (on network drives)

[Larrie Carr]

So the punch line is that octave will not work with network drives due to 
the difference on how "stat" returns the number of hard links.  Octave uses 
stat to determine if the directory is recusible.  But you can replicate the 
problem with using stat on the command line.

$stat -c "%h %f" /cygdrive/c/test
2 41c0

$stat -c "%h %f" /usr/share/octave
1 41ed

$stat -c "%h %f" /cygdrive/x/cygwin/usr/share/octave
1 41ed

$ls -l /usr/share/octave
total 0
drwxr-xr-x 1 larrie mkpasswd 0 Feb   8 23.31 2.1.72
drwxr-xr-x 1 larrie mkpasswd 0 Feb   8 23.31 site

The code checks for two links (the %h) given that a subdirectory should have 
a "." and a ".." entry.  But for some reason, network drives created using 
windows within cygwin report 1.

So I'm at the edge of my understanding - should cygwin be reporting 2 or 1 
or is octave using a method that works on every other system except cygwin.

Larrie.

Larrie Carr wrote
> John W. Eaton wrote
>> Probably the code you are looking for is the function do_subdir in
>> liboctave/kpse.cc.  This file contains a stripped-down version of the
>> kpathsearch library.  Most modifications were to remove TeX-specific
>> stuff and to convert it to use std::string instead of plain C strings
>> which historically leaked memory.  In any case, that function may use
>> an optimization to decide when to check for subdirectories.  The
>> optimization looks at the link count of the current directory.  If it
>> is 2, then the assumption is that the current directory does not
>> contain any subdirectories.  That seems to work fine for Unixy
>> systems.  Does that assumption not hold for Cygwin?  If so, then I
>> think the fix is fairly simple as there is also Windows-specific code
>> in that function.  Whether the optimization is performed depends on
>> what is #defined at compile time, so you'll probably have to do some
>> checking on a Cygwin system to see what is really going on.
>
>
> Well, after some more experimenting, the problem appears related to using 
> the recursive search feature of LOADPATH on a *network* drive.  That is,
>
> If the path is located within a physically attached hard drive, octave 
> operates as expected.  For instance, /cygdrive/c/test// works all the way 
> down to /cygdrive/c/test/a/b/c/d/e/
>
> If the path is located on a network drive created using windows "Map 
> Network Drive" menu option under "My Computer", octave will only recurse 
> down 1 level of subdirectory.  For instance "/usr/share/octave/site/m// 
> does not work.
>
> And if I use a cygwin windows mapping instead 
> "/cygdrive/x/cygwin/usr/share/octave/site/m//, it still does not work 
> correctly and recurses down only one level.
>
> In my installation, the entire cygwin root is located on a network drive 
> (and no one else uses it).  Works just fine for everything else.
>
> John, it you got any other hints, I would appreciate it as I'm diving in.
>
> Larrie.
>
> 



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [octave ] LOADPATH recurses only one level of subdirectories (on network drives)

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ugh - top-posting reformatted http://cygwin.com/acronyms/#TOFU

According to Larrie Carr on 2/11/2006 11:25 PM:
>>> Probably the code you are looking for is the function do_subdir in
>>> liboctave/kpse.cc.  This file contains a stripped-down version of the
>>> kpathsearch library.  Most modifications were to remove TeX-specific
>>> stuff and to convert it to use std::string instead of plain C strings
>>> which historically leaked memory.  In any case, that function may use
>>> an optimization to decide when to check for subdirectories.  The
>>> optimization looks at the link count of the current directory.  If it
>>> is 2, then the assumption is that the current directory does not
>>> contain any subdirectories.  That seems to work fine for Unixy
>>> systems.  Does that assumption not hold for Cygwin?  If so, then I
>>> think the fix is fairly simple as there is also Windows-specific code
>>> in that function.  Whether the optimization is performed depends on
>>> what is #defined at compile time, so you'll probably have to do some
>>> checking on a Cygwin system to see what is really going on.
>>

> So the punch line is that octave will not work with network drives due
> to the difference on how "stat" returns the number of hard links.
> Octave uses stat to determine if the directory is recusible.  But you
> can replicate the problem with using stat on the command line.
>
> $stat -c "%h %f" /cygdrive/c/test
> 2 41c0
>
> $stat -c "%h %f" /usr/share/octave
> 1 41ed
>
> $stat -c "%h %f" /cygdrive/x/cygwin/usr/share/octave
> 1 41ed
>
> $ls -l /usr/share/octave
> total 0
> drwxr-xr-x 1 larrie mkpasswd 0 Feb   8 23.31 2.1.72
> drwxr-xr-x 1 larrie mkpasswd 0 Feb   8 23.31 site
>
> The code checks for two links (the %h) given that a subdirectory should
> have a "." and a ".." entry.  But for some reason, network drives
> created using windows within cygwin report 1.

Because it is too expensive for cygwin to report an accurate link count
(stat() of the remote dir would effectively have to stat every file in
that dir to see how many subdirectories there were, which would be quite
expensive over the network).

>
> So I'm at the edge of my understanding - should cygwin be reporting 2 or
> 1 or is octave using a method that works on every other system except
> cygwin.

find has the same optimization of using the directory link count to decide
in advance how many subdirectories it should expect from readdir(), so
that it can quit the directory search early once all the subdirs have been
found.  However, find takes care that if the link count of a directory is
less than 2, then the optimization must not be performed.  POSIX does not
guarantee the link count of directories (and both mount points, and
systems that allow directory hard links [although cygwin does not allow
directory hard links] can mess up the traditional semantics), so it is a
bug in octave if it is mis-optimizing traversal in the presence of a
directory link count of 1.  It might make sense, though, for cygwin to set
the link count to 0 on remote directories (rather than 1), to make it
obvious that the link count really is unknown, but this still does not
take the blame off of octave for the mis-optimization.

- --
Life is short - so eat dessert first!

Eric Blake             ebb9@byu.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFD8ebB84KuGfSFAYARAuv+AKDLLPSP9oDYGeKqfj9aBQX0DZWHOACfSVL/
nPld93ctXKzMgZwN9h+I/CM=
=CslM
-----END PGP SIGNATURE-----

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [octave ] LOADPATH recurses only one level of subdirectories (on network drives)

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

According to Eric Blake on 2/14/2006 7:18 AM:
>>>
>>>The code checks for two links (the %h) given that a subdirectory should
>>>have a "." and a ".." entry.  But for some reason, network drives
>>>created using windows within cygwin report 1.
> 
> Because it is too expensive for cygwin to report an accurate link count
> (stat() of the remote dir would effectively have to stat every file in
> that dir to see how many subdirectories there were, which would be quite
> expensive over the network).

Following up on myself, more details about POSIX requirements on directory
link counts, and the implications to programs trying to optimize based on
link counts, can be found in this thread of the Austin group mailing list:
https://www.opengroup.org/sophocles/show_archive.tpl?source=L&listname=austin-group-l&pagesize=10&first=1&searchstring=link+count&zone=F

- --
Life is short - so eat dessert first!

Eric Blake             ebb9@byu.net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFD8eoL84KuGfSFAYARAvfEAJ9AYm3XZX+ufUO+OwQVMZEhY7WhwQCgn0ye
XcAInc+REW0G4CqWar4Cldw=
=hJKm
-----END PGP SIGNATURE-----

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: [octave ] LOADPATH recurses only one level of subdirectories (on network drives)

[John W. Eaton]

On 14-Feb-2006, Eric Blake wrote:

| so it is a
| bug in octave if it is mis-optimizing traversal in the presence of a
| directory link count of 1.  It might make sense, though, for cygwin to set
| the link count to 0 on remote directories (rather than 1), to make it
| obvious that the link count really is unknown, but this still does not
| take the blame off of octave for the mis-optimization.

If we are playing the blame game, then let's assign some of the blame
to the kpathsearch library, which is the basis for Octave's path
searching code.

jwe

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/