Ruby on Rails 2.0.2/Cygwin Bug

Ruby on Rails 2.0.2/Cygwin Bug

[Mike Boone]

Can anyone running Cygwin and Ruby on Rails comment on this problem?

Create a Rails 2.0.2 application and see if it fails with this error:
No such file or directory - /dev/urandom

I filed a bug on Rails, please read for more details.
http://dev.rubyonrails.org/ticket/10544

I'm not sure if this is Cygwin's fault or something peculiar about the
Rails code, though it looks reasonable.

Thanks,
Mike Boone.
http://boonedocks.net/mike/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Igor Peshansky]

On Wed, 19 Dec 2007, Mike Boone wrote:

> Can anyone running Cygwin and Ruby on Rails comment on this problem?
>
> Create a Rails 2.0.2 application and see if it fails with this error:
> No such file or directory - /dev/urandom
>
> I filed a bug on Rails, please read for more details.
> http://dev.rubyonrails.org/ticket/10544
>
> I'm not sure if this is Cygwin's fault or something peculiar about the
> Rails code, though it looks reasonable.

Well, /dev/urandom certainly exists in Cygwin.  I'm not a Ruby expert, but
considering that adding a puts() before the file open seems to fix it, I'd
look somewhere in the Ruby code.  But you can try running the original
under strace to see what exactly Ruby is trying to do with /dev/urandom.

Also note that if, for some reason, you end up with non-Cygwin Ruby *or* a
non-Cygwin DLL that Ruby gets its File library from, it will definitely
not know about /dev/urandom.

In short, strace is your friend.  In particular, it will tell you the
exact Windows error Ruby gets when trying to open the device, as well as
any path translations that Cygwin tries to do in that case.
	Igor
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_	    pechtcha@cs.nyu.edu | igor@watson.ibm.com
ZZZzz /,`.-'`'    -.  ;-;;,_		Igor Peshansky, Ph.D. (name changed!)
     |,4-  ) )-,_. ,\ (  `'-'		old name: Igor Pechtchanski
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

"That which is hateful to you, do not do to your neighbor.  That is the whole
Torah; the rest is commentary.  Go and study it." -- Rabbi Hillel

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Mike Boone]

On Dec 19, 2007 11:59 PM, Igor Peshansky <pechtcha@cs.nyu.edu> wrote:
> Well, /dev/urandom certainly exists in Cygwin.  I'm not a Ruby expert, but
> considering that adding a puts() before the file open seems to fix it, I'd
> look somewhere in the Ruby code.  But you can try running the original
> under strace to see what exactly Ruby is trying to do with /dev/urandom.

I ran strace ruby /usr/bin/rails test_app > strace.txt, and it
generated 54MB of text. It looks like the last 100 lines or so might
be relevant, but I don't know how to decipher them. The "No such file
or directory - /dev/urandom" occurs toward the bottom.

http://boonedocks.net/code/rails-urandom-strace.txt

I'd appreciate it if anyone is willing to have a look. I guess my next
step might be to rerun the strace with the puts() statement in there
and see what the difference is.

Thanks,
Mike.
http://boonedocks.net/mike/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Frodak Baksik]

On Dec 20, 2007 10:28 AM, Mike Boone  wrote:
> On Dec 19, 2007 11:59 PM, Igor Peshansky wrote:
> > Well, /dev/urandom certainly exists in Cygwin.  I'm not a Ruby expert, but
> > considering that adding a puts() before the file open seems to fix it, I'd
> > look somewhere in the Ruby code.  But you can try running the original
> > under strace to see what exactly Ruby is trying to do with /dev/urandom.
>
> I ran strace ruby /usr/bin/rails test_app > strace.txt, and it
> generated 54MB of text. It looks like the last 100 lines or so might
> be relevant, but I don't know how to decipher them. The "No such file
> or directory - /dev/urandom" occurs toward the bottom.
>
> http://boonedocks.net/code/rails-urandom-strace.txt
>
> I'd appreciate it if anyone is willing to have a look. I guess my next
> step might be to rerun the strace with the puts() statement in there
> and see what the difference is.
>
> Thanks,
> Mike.
> http://boonedocks.net/mike/
>
> --


Can you access /dev/urandom outside of ruby?  It might be a simpler
check to ensure that /dev/urandom is working.

Something like to check reading:
head -c 10 /dev/urandom > rnd.bin

and then to check writing:
cat rnd.bin > /dev/urandom
or
strace -o strace.txt sh -c 'cat rnd.bin > /dev/urandom'

It looked like from the strace that reading worked, but writing didn't.

Regards,
Frodak

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Mike Boone]

On Dec 20, 2007 4:29 PM, Frodak Baksik <frodak17@gmail.com> wrote:
> On Dec 20, 2007 10:28 AM, Mike Boone  wrote:
> Can you access /dev/urandom outside of ruby?  It might be a simpler
> check to ensure that /dev/urandom is working.

I ran all the commands you suggested with no problems.

The problem function is this in the Rails code:
def generate_secret_with_urandom
  return File.read("/dev/urandom", 64).unpack("H*")[0]
end

If I put that function directly in its own Ruby code file and run it
from Cygwin, there is no problem, so I guess it has something to do
with how it's called.

Also, for some reason you can change the function to this in the Rails
code and everything works.

def generate_secret_with_urandom
  puts "meaningless output"
  return File.read("/dev/urandom", 64).unpack("H*")[0]
end

I'm going to re-strace things with the modified function and see what
the difference might be.

Mike Boone
http://boonedocks.net/mike/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Mike Boone]

OK, I modified the Rails source code to add the puts statement just
above the statement that reads from /dev/urandom. I then ran strace
again and looked for the relevant section of the output. I trimmed out
the leading numeric stuff so I could run a diff on them.

Here is the output of the failing strace:

stat_worker: 0 = (/dev/urandom, 0x2402BD0)
sig_send: sendsig 0x6F8, pid 2152, signal -34, its_me 1
sig_send: wakeup 0x6E0
sig_send: Waiting for pack.wakeup 0x6E0
wait_sig: signalling pack.wakeup 0x6E0
sig_send: returning 0x0 from sending signal -34

Here is the output of the working strace (with the extra puts statement):

stat_worker: 0 = (/dev/urandom, 0x2402BD0)
get_file_attribute: file: C:\cygwin\home\mb\some disk file
cygpsid::debug_print: get_sids_info: owner SID =
S-1-5-21-1614895754-1935655697-1343024091-1003
cygpsid::debug_print: get_sids_info: group SID =
S-1-5-21-1614895754-1935655697-1343024091-513
get_info_from_sd: ACL 1A4, uid 1003, gid 513
fhandler_base::fstat_helper: 0 = fstat (, 0x2404810) st_atime=476B0E59
st_size=56814726, st_mode=0x81A4, st_ino=19703248369749074, sizeof=96
fstat64: 0 = fstat (1, 0x2404810)
sig_send: sendsig 0x710, pid 3160, signal -34, its_me 1
sig_send: wakeup 0x6F4
sig_send: Waiting for pack.wakeup 0x6F4
wait_sig: signalling pack.wakeup 0x6F4
sig_send: returning 0x0 from sending signal -34

There are some other diffs since the modified code continues on to
successfully complete, but this is the section that references
/dev/urandom and has diffs.

Can anyone make sense of it?

Thanks,
Mike Boone
http://boonedocks.net/mike/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Mike Boone]

I came up with a simple Ruby snippet that reproduces the problem. I'd
appreciate it if other Cygwin users would try it. You can change the
"cause_failure = true" line to say false and it will run.

--urandom_test.rb------
# Test failure of reading /dev/urandom after failed require

# this method is from the Rails' source secret_key_generator.rb
def generate_secret_with_urandom
  return File.read("/dev/urandom", 64).unpack("H*")[0]
end

cause_failure = true

begin
  require 'nonexistent_file'
rescue LoadError
  puts '' unless cause_failure
end

puts generate_secret_with_urandom
-----------------------

So far, the strace output of this is too low level for me to follow.
Perhaps it will be clear to a regular Cygwin hacker.

BTW, I'm running "ruby 1.8.6 (2007-03-13 patchlevel 0) [i386-cygwin]".

Mike Boone
http://boonedocks.net/mike/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Corinna Vinschen]

On Dec 21 22:13, Mike Boone wrote:
> I came up with a simple Ruby snippet that reproduces the problem. I'd
> appreciate it if other Cygwin users would try it. You can change the
> "cause_failure = true" line to say false and it will run.
> 
> --urandom_test.rb------
> # Test failure of reading /dev/urandom after failed require
> 
> # this method is from the Rails' source secret_key_generator.rb
> def generate_secret_with_urandom
>   return File.read("/dev/urandom", 64).unpack("H*")[0]
> end
> 
> cause_failure = true
> 
> begin
>   require 'nonexistent_file'
> rescue LoadError
>   puts '' unless cause_failure
> end
> 
> puts generate_secret_with_urandom
> -----------------------

Looks like a bug in ruby.  First of all, this works fine under Cygwin:

  $ dd if=/dev/urandom bs=64 count=1 | od -c
  1+0 records in
  1+0 records out
  64 bytes (64 B) copied, 0.39 s, 0.2 kB/s
  0000000 9d78 007a 8e7c 0cad 5d1c 6778 1298 88a9
  0000020 4256 8e69 be6f c3b5 be0a 695e 9446 5689
  0000040 db44 8a04 0159 175c b84e 79cf 0ebc 9ee7
  0000060 7e3d 8338 31bd b8e7 15df ab91 ea55 90b6
  0000100

And this, too:

  $ dd if=/dev/urandom of=/dev/urandom bs=64 count=1 
  1+0 records in
  1+0 records out
  64 bytes (64 B) copied, 0.015 s, 4.3 kB/s

Running the above ruby script under strace shows:

 1345 2541402 [main] ruby 3012 open: open (/dev/urandom, 0x0)
 [...]
  255 2543760 [main] ruby 3012 open: 3 = open (/dev/urandom, 0x0)

It now has a valid file descriptor.

  606 2544366 [main] ruby 3012 _cygwin_istext_for_stdio: fd 3: opened as binary
 2272 2546638 [main] ruby 3012 fhandler_base::fstat: here
  349 2546987 [main] ruby 3012 fstat64: 0 = fstat (3, 0x2409720)

fstat returned successfully.

  687 2547674 [main] ruby 3012 isatty: 0 = isatty (3)

/dev/urandom is obviously no tty.

  358 2548032 [main] ruby 3012 readv: readv (3, 0x2409740, 1) blocking, sigcatchers 10
  216 2548248 [main] ruby 3012 readv: no need to call ready_for_read
35355 2583603 [main] ruby 3012 readv: 65536 = readv (3, 0x2409740, 1), errno 2

read returned successfully with 64K random bytes in the buffer.

51453 2635056 [main] ruby 3012 lseek64: 0 = lseek (3, 0, 1)
  660 2635716 [main] ruby 3012 lseek64: 0 = lseek (3, -65472, 0)

Two lseek's of dubious purpose.  Why did it read 64K and then tries to
seek back?  Why does it rely on the return code of the first lseek?
Doing the same on Linux returns also with 0 from the first lseek, but
with -1 from the second call and errno set to EINVAL.  However,
according to the strace, it never *tries* to call lseek on Linux.

 9782 2645498 [main] ruby 3012 close: close (3)
  304 2645802 [main] ruby 3012 close: 0 = close (3)

The file descriptor gets closed.

The above is all about /dev/urandom in the strace output.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Mike Boone]

On Dec 22, 2007 7:51 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote:
> Looks like a bug in ruby.  First of all, this works fine under Cygwin:

Thanks for analyzing this. I'll take it up at comp.lang.ruby and see
what happens.
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/284492

Mike Boone
http://boonedocks.net/mike/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Mike Boone]

Over on the comp.ruby.lang group, I got a reply from Matz, the lead
Ruby developer:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/284495

He states: "The bug must lie between Cygwin and Ruby.  We don't call
lseek explicitly.  You can clearly see this by doing strace on Linux
(or other platforms), which don't call lseek at all.  So, someone (in
the library, I guess) must call lseek weirdly on Cygwin."

I know that Corinna noted that Linux was doing lseeks:

On Dec 22, 2007 7:51 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote:
> Two lseek's of dubious purpose.  Why did it read 64K and then tries to
> seek back?  Why does it rely on the return code of the first lseek?
> Doing the same on Linux returns also with 0 from the first lseek, but
> with -1 from the second call and errno set to EINVAL.  However,
> according to the strace, it never *tries* to call lseek on Linux.

However, when I tried strace ruby urandom_test.rb on a few flavors of
Linux, I did not see lseeks in the output. I ran the code on Ubuntu
7.04, RHEL 2.1AS, and whatever brand of Linux is running on Dreamhost.
Which version did you try?

My Cygwin:
   29  467324 [main] ruby 2944 open: 3 = open (/dev/urandom, 0x0)
   51  467375 [main] ruby 2944 _cygwin_istext_for_stdio: fd 3: opened as binary
 2461  469836 [main] ruby 2944 fhandler_base::fstat: here
   45  469881 [main] ruby 2944 fstat64: 0 = fstat (3, 0x24096E0)
  385  470266 [main] ruby 2944 isatty: 0 = isatty (3)
   35  470301 [main] ruby 2944 sig_send: sendsig 0x6F8, pid 2944,
signal -34, its_me 1
  414  470715 [main] ruby 2944 sig_send: wakeup 0x6FC
   60  470775 [main] ruby 2944 sig_send: Waiting for pack.wakeup 0x6FC
  178  470953 [sig] ruby 2944 wait_sig: signalling pack.wakeup 0x6FC
  146  471099 [main] ruby 2944 sig_send: returning 0x0 from sending signal -34
   36  471135 [main] ruby 2944 readv: readv (3, 0x2409700, 1)
blocking, sigcatchers 10
  133  471268 [main] ruby 2944 readv: no need to call ready_for_read
23838  495106 [main] ruby 2944 readv: 65536 = readv (3, 0x2409700, 1), errno 2
  940  496046 [main] ruby 2944 lseek64: 0 = lseek (3, 0, 1)
  524  496570 [main] ruby 2944 lseek64: 0 = lseek (3, -65472, 0)
   73  496643 [main] ruby 2944 close: close (3)


RHEL 2.1AS (Ubuntu 7.04 output was similar):
open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFCHR|0644, st_rdev=makedev(1, 9), ...}) = 0
ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbffe9c30) = -1 EINVAL
(Invalid argument)
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x40018000
read(3, "\20gI\263\322j5E\345\24[\245\200,\375\23UT}\210(<\365\310"...,
4096) = 4096
close(3)                                = 0

Thanks,
Mike Boone
http://boonedocks.net/mike/

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Igor Peshansky]

On Fri, 21 Dec 2007, Mike Boone wrote:

> I came up with a simple Ruby snippet that reproduces the problem. I'd
> appreciate it if other Cygwin users would try it. You can change the
> "cause_failure = true" line to say false and it will run.
>
> --urandom_test.rb------
> # Test failure of reading /dev/urandom after failed require
>
> # this method is from the Rails' source secret_key_generator.rb
> def generate_secret_with_urandom
>   return File.read("/dev/urandom", 64).unpack("H*")[0]
> end
>
> cause_failure = true
>
> begin
>   require 'nonexistent_file'
> rescue LoadError
>   puts '' unless cause_failure
> end
>
> puts generate_secret_with_urandom
> -----------------------
>
> So far, the strace output of this is too low level for me to follow.
> Perhaps it will be clear to a regular Cygwin hacker.
>
> BTW, I'm running "ruby 1.8.6 (2007-03-13 patchlevel 0) [i386-cygwin]".

As I mentioned before, I don't use Ruby too often.  But I just realized
something: to reproduce the error, you needed a "require
'nonexistent_file'", which will obviously set errno to ENOENT (which is
exactly the error you're seeing).  It's possible that File.read checks
errno and exits right away if it's non-zero, and that nothing resets errno
before File.read.  Inserting the puts call will likely reset errno, thus
avoiding the bug.  So, the bug is probably in Ruby code for exception
handling ("rescue") -- it should reset errno so that subsequent code isn't
affected (and it's also suspicious that the File.read code checks errno
without a valid reason, e.g., a non-zero return value).
	Igor
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_	    pechtcha@cs.nyu.edu | igor@watson.ibm.com
ZZZzz /,`.-'`'    -.  ;-;;,_		Igor Peshansky, Ph.D. (name changed!)
     |,4-  ) )-,_. ,\ (  `'-'		old name: Igor Pechtchanski
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

"That which is hateful to you, do not do to your neighbor.  That is the whole
Torah; the rest is commentary.  Go and study it." -- Rabbi Hillel

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: Ruby on Rails 2.0.2/Cygwin Bug

[Mike Boone]

On Dec 22, 2007 3:46 PM, Igor Peshansky <pechtcha@cs.nyu.edu> wrote:
> As I mentioned before, I don't use Ruby too often.  But I just realized
> something: to reproduce the error, you needed a "require
> 'nonexistent_file'", which will obviously set errno to ENOENT (which is
> exactly the error you're seeing).  It's possible that File.read checks
> errno and exits right away if it's non-zero, and that nothing resets errno
> before File.read.  Inserting the puts call will likely reset errno, thus
> avoiding the bug.  So, the bug is probably in Ruby code for exception
> handling ("rescue") -- it should reset errno so that subsequent code isn't
> affected (and it's also suspicious that the File.read code checks errno
> without a valid reason, e.g., a non-zero return value).

Looks like you got it right. They just fixed the Ruby source:
http://svn.ruby-lang.org/cgi-bin/viewvc.cgi/branches/ruby_1_8/io.c?r1=13774&r2=14858

I made the change to my io.c in the stable-snapshot and it compiles OK
and runs my test script without the not-found error.

Not sure how long this will take to find its way into Cygwin's ruby
package, but I'm glad it was finally solved.

Mike Boone.

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/