Cannot change console mode during usage of expect

Cannot change console mode during usage of expect

[Keith Proctor]

Hello, my name is Keith Proctor.  I test FileMaker Server and I have written 
a test suite on Macintosh. The tests work beautifully on  Macintosh. I’m trying
to Use Cygwin on Windows to reduce the amount of rewriting that I must do 
on the Windows side.    The test includes a dynamically created interactive call
that uses expect to spawn fmsadmin and then interact with fmsadmin by 
providing a user name or a password.  An example is below:

expect -c "spawn fmsadmin open -y -u userName
expect -r .+:
send myPassword\r
interact"

The issue is that the following phrases appear and won’t allow my interactive 
expect command to continue.

GetConsoleMode	// prints once
SetConsoleMode	// prints once
ReadConsoleInput	// spews while collecting input from the command line.

Perform the actions of the script above by hand in the Cygwin CLI does NOT 
cause an issue.  The issue only happens when using expect.  I have tried 3
shells so far (mintty, bash, rxvt).  Are there more?  Mintty provides access to
the terminal types.  So far I have tried xterm-256, vt100, vt120.

An example of the code looks like:

   fdwMode = ENABLE_WINDOW_INPUT | ENABLE_MOUSE_INPUT
          | ENABLE_PROCESSED_INPUT; 
   if (! SetConsoleMode(hStdin, fdwMode) )
		// print error to console

Since FileMaker Server runs on Macintosh or Windows it is doubtful that
I can get a change made in the fmsadmin application. I’m new to CygWin
so I’m hoping that I’ve missed a setting somewhere or maybe I need to
install another module.  

Help me Obi-Wan your my only hope.  ;)

Keith

Re: Cannot change console mode during usage of expect

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1492 bytes --]

On Apr 15 01:17, Keith Proctor wrote:
> Hello, my name is Keith Proctor.  I test FileMaker Server and I have written 
> a test suite on Macintosh. The tests work beautifully on  Macintosh. I’m trying
> to Use Cygwin on Windows to reduce the amount of rewriting that I must do 
> on the Windows side.    The test includes a dynamically created interactive call
> that uses expect to spawn fmsadmin and then interact with fmsadmin by 
> providing a user name or a password.  An example is below:
> 
> expect -c "spawn fmsadmin open -y -u userName
> expect -r .+:
> send myPassword\r
> interact"
> 
> The issue is that the following phrases appear and won’t allow my interactive 
> expect command to continue.
> 
> GetConsoleMode	// prints once
> SetConsoleMode	// prints once
> ReadConsoleInput	// spews while collecting input from the command line.

I don't understand this.  What does Cygwin expect have to do with these
underlying OS calls?!?  They only work in a console window and they are
not to be suppsoed to be used by Cygwin executables since all the
underlying OS details are (more or less) hidden behind the TTY code.
Also, how are they supposed to work in a PTY which is constituted by
Named Pipes, not a console?  And expect certainly uses PTYs to perform
its pseudo-interactive stuff.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Cannot change console mode during usage of expect

[Keith Proctor]

Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:

> 
> On Apr 15 01:17, Keith Proctor wrote:
> > Hello, my name is Keith Proctor.  I test FileMaker Server and I have
> > written a test suite on Macintosh. The tests work beautifully on  
> > Macintosh. I’m trying to Use Cygwin on Windows to reduce the 
> > amount of rewriting that I must do on the Windows side.    The
> > test includes a dynamically created interactive call that uses 
> > expect to spawn fmsadmin and then interact with fmsadmin by 
> > providing a user name or a password.  An example is below:
> > 
> > expect -c "spawn fmsadmin open -y -u userName
> > expect -r .+:
> > send myPassword\r
> > interact"
> > 
> > The issue is that the following phrases appear and won’t allow
> > my interactive expect command to continue.
> > 
> > GetConsoleMode	// prints once
> > SetConsoleMode	// prints once
> > ReadConsoleInput	// spews while collecting input from the command line.
> 
> I don't understand this.  What does Cygwin expect have to do
>  with these underlying OS calls?!?  They only work in a console 
> window and they are not to be suppsoed to be used by Cygwin
>  executables since all the underlying OS details are (more or less)
>  hidden behind the TTY code. Also, how are they supposed to work
>  in a PTY which is constituted by  Named Pipes, not a console?  
> And expect certainly uses PTYs to perform its pseudo-interactive stuff.
> 
> Corinna
> 
Corinna,  Let me make this a little clearer fmsadmin application is making
the 3 calls.  In this case fmsadmin is being called from expect and expect
is the cause of the issue.  If I don't use expect fmsadmin works without the
spam of the ReadConsoleInput strings.  IOW the calls to the Getter and
Setter methods work correctly.  The output looks something like this 
when I do it by hand.

>fmsadmin open -y -u userName
>password:myPassword
Opening file: someFile.fmp12
Opening file: someOtherFile.fmp12
Opened file: someFile.fmp12
Opened file: someOtherFile.fmp12

So this works as expected in the Cygwin Window when doing the 
command by hand.  It is somehow an interaction between Cygwin and
the expect command not allowing the Setter/Getter methods for Console
Mode to be accepted.  I have NO understanding WHY this would be the
case.  All I did was localize down the behavior and the failure to work
correctly.

I am still hoping that the default settings are somehow wrong and 
I need to make some small change to Cygwin to allow 
GetConsoleMode and SetConsoleMode calls to correctly
work.

On a very related note I see this same question being asked for years
and the Cygwin answer has been change your code to work with
Cygwin.  Unfortunately, I don't have that luxury.  fmsadmin is a
DOS application.  I may never be able to get a fix for this issue in
fmsadmin as our target is NOT Cygwin.  

BTW, I expect GetConsoleMode and SetConsoleMode is standard
usage by any console application.  These calls work on Macintosh,
Windows and Cygwin window without expect.

HTH and many thanks,
Keith

Re: Cannot change console mode during usage of expect

[Mark Geisert]

Keith Proctor writes:
> BTW, I expect GetConsoleMode and SetConsoleMode is standard
> usage by any console application.  These calls work on Macintosh,
> Windows and Cygwin window without expect.

Are you sure you're using the Cygwin expect, and not a Windows-native expect
that just happens to be on your path?

..mark


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cannot change console mode during usage of expect

[Keith Proctor]

Mark Geisert <mark <at> maxrnd.com> writes:

> 
> Keith Proctor writes:
> > BTW, I expect GetConsoleMode and SetConsoleMode is standard
> > usage by any console application.  These calls work on Macintosh,
> > Windows and Cygwin window without expect.
> 
> Are you sure you're using the Cygwin expect, and not a Windows-native expect
> that just happens to be on your path?
> 
> ..mark
> 
> 
Typing which expect comes back with /usr/bin/expect.  This appears 
to be the standard Cygwin location. 

I did a search for expect on the system before installing Cygwin and
did not find an executable.  This is a fresh 2k8 R2 VM.  I did not install
another expect command from another source on the system and 
DOS does not include an expect functionality that I am aware of.  Web
shows that I would need to install another package such as HPs
expect or ActiveState.




--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cannot change console mode during usage of expect

[Mark Geisert]

Keith Proctor writes:
> Mark Geisert <XXXXXXXXXXXXXX> writes:

Please don't quote email addresses in replies.

> > Keith Proctor writes:
> > > BTW, I expect GetConsoleMode and SetConsoleMode is standard
> > > usage by any console application.  These calls work on Macintosh,
> > > Windows and Cygwin window without expect.
> > 
> > Are you sure you're using the Cygwin expect, and not a Windows-native expect
> > that just happens to be on your path?
> > 
> > ..mark
> > 
> > 
> Typing which expect comes back with /usr/bin/expect.  This appears 
> to be the standard Cygwin location. 
> 
> I did a search for expect on the system before installing Cygwin and
> did not find an executable.  This is a fresh 2k8 R2 VM.  I did not install
> another expect command from another source on the system and 
> DOS does not include an expect functionality that I am aware of.  Web
> shows that I would need to install another package such as HPs
> expect or ActiveState.

Fair enough.  Expect communicates with spawned processes via ptys.  On
Cygwin ptys are implemented as Windows "named pipes".  Perhaps the spawned
process is trying to use those console-related Windows calls on a named pipe
descriptor that it wants to believe is a console's file descriptor.

In your original post you had a code snippet that showed on GetConsoleMode()
failure, I think, an error message was generated.  Is that error message
what's being output, i.e., the name of the Windows call without any other
error information?

I know this subject has come up on the list before.  There is a little shim
one can insert "in front of" a console-oriented application that makes it
think it's talking to a console instead of a pipe.  I can't recall the name
of that shim, though.  Can I get a little help from the list please?

..mark




--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cannot change console mode during usage of expect

[Keith Proctor]

Mark Geisert <mark <at> maxrnd.com> writes:
<snip>

Interesting your system does the quoting.  I just left it in place.

Your last shows some hope for this issue.  It would be great if the 
solution can be a configuration change so I don't have to pre-pend
something.  Again trying to make the original code duel platform.

In response to your question:  Unfortunately, the code prints
those strings directly to console and provides no additional
information.  If this happens, in DOS, I'm assuming that we
would just want to fix anything that spews those 3 strings.
The Console Mode should simply be settable.

Keith


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cannot change console mode during usage of expect

[Keith Proctor]

Interesting...I found the expect executable on disk, double clicked it
and it opened a window so I typed in my original expect program
without expect -c and surrounding quotes.  I then ran it and found
the same errors.  The GetConsoleMode() and SetConsoleMode()
calls are not being allowed to exit the expect environment.  

I'm not sure that the shim will work or not but I'll give it a shot.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cannot change console mode during usage of expect

[Marco Atzeri]

On 4/16/2015 2:29 AM, Keith Proctor wrote:
> Interesting...I found the expect executable on disk, double clicked it
> and it opened a window so I typed in my original expect program
> without expect -c and surrounding quotes.  I then ran it and found
> the same errors.  The GetConsoleMode() and SetConsoleMode()
> calls are not being allowed to exit the expect environment.
>
> I'm not sure that the shim will work or not but I'll give it a shot.
>

Hi Keith,
as you are using a not-cygwin program with cygwin expect,
this reading can be useful to understand the issue
and eventually catch a solution

https://code.google.com/p/mintty/issues/detail?id=56

this one seems the most promising tool
https://github.com/rprichard/winpty

Regards
Marco

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cannot change console mode during usage of expect

[Keith Proctor]

Marco Atzeri <marco.atzeri <at> gmail.com> writes:
<snip>

Thanks Marco but I'm not sure which direction your asking me
to go.  Should I:

1) Install Console2 ( I didn't find it in the Cygwin installer)
2) Build the tool mentioned in the link and prepend
build/console.exe to all my DOS commands.  

The second seams like a bit of a hack and I would have to do
the equivalent of #define for Windows versus Mac in my code.

Maybe I missed missed another option?

Inquiring minds Inquire, :)
Keith


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cannot change console mode during usage of expect

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3754 bytes --]

On Apr 15 17:57, Keith Proctor wrote:
> Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:
> > On Apr 15 01:17, Keith Proctor wrote:
> > > Hello, my name is Keith Proctor.  I test FileMaker Server and I have
> > > written a test suite on Macintosh.
> > > [...]
> > > The issue is that the following phrases appear and won’t allow
> > > my interactive expect command to continue.
> > > 
> > > GetConsoleMode	// prints once
> > > SetConsoleMode	// prints once
> > > ReadConsoleInput	// spews while collecting input from the command line.
> > 
> > I don't understand this.  What does Cygwin expect have to do
> >  with these underlying OS calls?!?  They only work in a console 
> > window and they are not to be suppsoed to be used by Cygwin
> >  executables since all the underlying OS details are (more or less)
> >  hidden behind the TTY code. Also, how are they supposed to work
> >  in a PTY which is constituted by  Named Pipes, not a console?  
> > And expect certainly uses PTYs to perform its pseudo-interactive stuff.
> > 
> Corinna,  Let me make this a little clearer fmsadmin application is making
> the 3 calls.  In this case fmsadmin is being called from expect and expect
> is the cause of the issue.

Well, not really.  The cause of the issue is that fmsadmin uses these
calls.  Fmsadmin apparently expects to run in a normal Windows console.
Expect on the other hand is a Cygwin POSIX application, so it's using
Cygwin PTYs and offers those as stdio descriptors to the inferior
application.

The problem here seems to be that you're expecting Cygwin to do
something it doesn't, and fmsadmin not being a Cygwin application
not being able to work seamlessly with a Cygwin POSIX appication
in this regard.

> On a very related note I see this same question being asked for years
> and the Cygwin answer has been change your code to work with
> Cygwin.  Unfortunately, I don't have that luxury.  fmsadmin is a
> DOS application.  I may never be able to get a fix for this issue in
> fmsadmin as our target is NOT Cygwin.  

Which makes me wonder if using Cygwin is the right thing to do here.

> BTW, I expect GetConsoleMode and SetConsoleMode is standard
> usage by any console application.  These calls work on Macintosh,
> Windows and Cygwin window without expect.

I have no idea what Mac you're talking about.  IIUC OS X is a BSD-based
system and it does not provide Windows-only calls.  And GetConsoleMode,
SetConsoleMode, and ReadConsoleInput are Windows calls.  I have a hard
time to believe that an OS X console uses the weird Windows console API.
Unless, of course, you're running a Windows fmsadmin tool under some OS
X Windows emulator.  Does wine for OS X exist?

As for Cygwin I'd like to stress that Cygwin tries to emulate POSIX in
the first place.  An application like expect will be compiled under
Cygwin as a POSIX application.  It will quite certainly not use or
understand Windows calls.  As a POSIX application, it will have no
notion of Windows consoles and it will use PTYs as communication
channels to the inferior process.  PTYs are Cygwin-specific, implemented
using named pipes under the hood.  Named Pipes very certainly don't
support the console API.

The bottom line is, the fmsadmin tool, as a native Windows application,
expects to perform I/O via the Windows Console.  Thus it uses the
Windows console API.  Cygwin's expect on the other hand, is implemented
as POSIX application and as such, uses POSIX means, PTYs.

I don't see how the two of them can come together.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Cannot change console mode during usage of expect

[Marco Atzeri]

On 4/16/2015 8:20 AM, Keith Proctor wrote:
> Marco Atzeri <marco.atzeri <at> gmail.com> writes:
> <snip>
>
> Thanks Marco but I'm not sure which direction your asking me
> to go.  Should I:

No directions.
I was giving just a link on similar discussed ones that could
give you an hint and explanation why your trial does not work.

> 1) Install Console2 ( I didn't find it in the Cygwin installer)

It is not a cygwin program.

> 2) Build the tool mentioned in the link and prepend
> build/console.exe to all my DOS commands.

It is a possibility if you want to try the interaction
between your not-cygwin aware program and the cygwin version
of expect. I never used it.

> The second seams like a bit of a hack and I would have to do
> the equivalent of #define for Windows versus Mac in my code.

It is also a hack to use cygwin expect with a
not cygwin program.

> Maybe I missed missed another option?
>
> Inquiring minds Inquire, :)
> Keith
>

Regards
Marco




--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Cannot change console mode during usage of expect

[Keith Proctor]

Marco Atzeri <marco.atzeri <at> gmail.com> writes:
<snip>

OK, I've done both this morning and neither work.  I've figured out
a bit more, In general, expect is working with non password tests.  
So I have turned off anything that would test a password and I'm
 running my automation now.

Again to be clear... fmsadmin works in the Cygwin window when 
typing in commands by hand.  It is the interaction with expect that
is the issue.  Now we know it's specifically password handling.

Keith

P.S.

Last comment is just in passing and for my understanding.  Why 
install Cygwin on Windows if you can't interact with the local 
Windows executables.  If your not suppose to run any of the local
Windows applications then why not just install Linux or some other
flavor of unix.  I believe the ONLY interesting part of Cygwin is 
allowing the interaction between Windows and Unix and the effort
fails if integration is not allowed and extended. It's one thing to be
a purest but I believe the over 2000 people who downloaded winpty
would agree with me.  BTW it's OK that you disagree.  This is just my
opinion and I'm trying to use Cygwin to save a lot of effort and help
your effort along the way.  So far, except for this one thing it has
helped me.  :)


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple