Strange bug with /dev/console in mintty

[Erik Bray <erik.m.bray@gmail.com>]

Hi folks,

I noticed a strange discrepancy when running Cygwin through mintty, vs
through a normal cmd.exe console.  This is on a build from the latest
git master, but also on Cygwin 2.8.0.  When run from a cmd.exe
console, we can see the following output from fhandler_dev::readdir:

$ ls -l /dev/con*
crw-rw-rw- 4 Erik M. Bray Erik M. Bray 3, 0 Aug 22 18:29 /dev/conin
crw-rw-rw- 4 Erik M. Bray Erik M. Bray 3, 0 Aug 22 18:29 /dev/conout
crw-rw-rw- 4 Erik M. Bray Erik M. Bray 3, 0 Aug 22 18:29 /dev/cons0
crw-rw-rw- 4 Erik M. Bray Erik M. Bray 3, 0 Aug 22 18:29 /dev/console

The same command when run in mintty returns:

crw-rw-rw- 1 Erik M. Bray Erik M. Bray 5, 255 Aug 22 18:29 /dev/conin
crw-rw-rw- 1 Erik M. Bray Erik M. Bray 5, 254 Aug 22 18:29 /dev/conout
crw-rw-rw- 1 Erik M. Bray Erik M. Bray 5,   1 Aug 22 18:29 /dev/console

In the latter case, the device numbers are the default device numbers
for /dev/console, etc.  In the former case, the inodes for
/dev/console, /dev/conin, and /dev/conout are set to the same as
/dev/cons0.  This logic can be found in fhandler_dev::readdir:

203       if (cdev->get_major () == DEV_TTY_MAJOR
204       && (cdev->is_device (FH_CONIN)
205           || cdev->is_device (FH_CONOUT)
206           || cdev->is_device (FH_CONSOLE)))
207     {
208       /* Make sure conin, conout, and console have the same inode number
209          as the current consX. */
210       de->d_ino = myself->ctty;
211     }

where myself->ctty seems to depend largely on what file types the
stdio handles are attached to.

When running from cmd.exe, GetStdHandle(...) returns handles to a
character stream--i.e. the console itself.  However, when running from
mintty it returns pipes (specifically, to a pty, probably related to
the call to forkpty in mintty).  In Cygwin (particularly, in
dtable::init_std_file_from_handle) the result ends up being that when
the stdio handles are pipes, the /dev/cons0 device never gets created.
And thus /dev/console and friends never get rerouted to a real device.
This results in errors when trying to access /dev/console directly:

$ ls -l /dev/console
ls: cannot access '/dev/console': No such device or address

which is the error message for a ENXIO.  This is coming specifically
from the function build_fh_pc in dtable.cc.  The reason is that trying
to access /dev/console results in trying to create an fhandler_console
for a non-existent console.

I'm not really sure what the correct behavior should be here though,
and if it should be fixed on the mintty side or the Cygwin side.

Thanks,
Erik

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple