* Tweaking default java.awt.Robot settings @ 2007-09-24 20:31 Steve McKay☄ 2007-09-24 20:50 ` David Herron 2007-09-25 18:10 ` Thomas Fitzsimmons 0 siblings, 2 replies; 19+ messages in thread From: Steve McKay☄ @ 2007-09-24 20:31 UTC (permalink / raw) To: mauve-discuss Hi All, I've noticed that at least some of the tests using java.awt.Robot are non-deterministic due to lags is the underlying window system. The java.awt.Component.keyPressTest, for example, fails some of the time (on linux, windows, linux+wine, ...). It looks like enabling autoWaitForIdle (waits for the awt EventQueue to be empty before adding new events to the queue), and setting autoDelay (pauses for an arbitrary period of time) to some magic number of millis well above zero (I use 100) significantly reduces failures. Would anyone object to configuring the Robot with settings like this by default? If no, should the config mechanism be updated to allow tweaking these settings? -- Steve McKay <smckay@google.com> ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-24 20:31 Tweaking default java.awt.Robot settings Steve McKay☄ @ 2007-09-24 20:50 ` David Herron 2007-09-24 21:22 ` Steve McKay☄ 2007-09-25 18:33 ` Thomas Fitzsimmons 2007-09-25 18:10 ` Thomas Fitzsimmons 1 sibling, 2 replies; 19+ messages in thread From: David Herron @ 2007-09-24 20:50 UTC (permalink / raw) To: Steve McKay☄; +Cc: mauve-discuss Steve McKayâ wrote: > Hi All, > > I've noticed that at least some of the tests using java.awt.Robot are > non-deterministic due to lags is the underlying window system. The > java.awt.Component.keyPressTest, for example, fails some of the time > (on linux, windows, linux+wine, ...). It looks like enabling > autoWaitForIdle (waits for the awt EventQueue to be empty before > adding new events to the queue), and setting autoDelay (pauses for an > arbitrary period of time) to some magic number of millis well above > zero (I use 100) significantly reduces failures. Would anyone object > to configuring the Robot with settings like this by default? If no, > should the config mechanism be updated to allow tweaking these > settings? > > I don't know what the classpath implementation of Robot looks like, but I do know what Sun's Linux/Unix implementation looks like (having written the original version). Generally Robot has to request the OS or X11 to synthesize the event. On Windows there's a direct API call, while on Unix/Linux there is a child process which ends up calling XTEST extension methods. In both cases it means there is a nondeterministic delay due to the current process scheduling characteristics of the given system. In other words it depends on an external entity, who Robot cannot coerce into performing the request within a bounded set of time. I think that means depending on Robot doing it's thing within a given period of time is an invalid test. Robot does not add events to EventQueue but it requests the OS to synthesize an OS-level event. - David Herron ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-24 20:50 ` David Herron @ 2007-09-24 21:22 ` Steve McKay☄ 2007-09-24 21:41 ` David Herron 2007-09-25 18:33 ` Thomas Fitzsimmons 1 sibling, 1 reply; 19+ messages in thread From: Steve McKay☄ @ 2007-09-24 21:22 UTC (permalink / raw) To: David Herron; +Cc: mauve-discuss So would you recommend I ignore the test, delete it, add a comment, ...? --steve On 9/24/07, David Herron <David.Herron@sun.com> wrote: > Steve McKay☄ wrote: > > Hi All, > > > > I've noticed that at least some of the tests using java.awt.Robot are > > non-deterministic due to lags is the underlying window system. The > > java.awt.Component.keyPressTest, for example, fails some of the time > > (on linux, windows, linux+wine, ...). It looks like enabling > > autoWaitForIdle (waits for the awt EventQueue to be empty before > > adding new events to the queue), and setting autoDelay (pauses for an > > arbitrary period of time) to some magic number of millis well above > > zero (I use 100) significantly reduces failures. Would anyone object > > to configuring the Robot with settings like this by default? If no, > > should the config mechanism be updated to allow tweaking these > > settings? > > > > > > I don't know what the classpath implementation of Robot looks like, but > I do know what Sun's Linux/Unix implementation looks like (having > written the original version). > > Generally Robot has to request the OS or X11 to synthesize the event. > On Windows there's a direct API call, while on Unix/Linux there is a > child process which ends up calling XTEST extension methods. In both > cases it means there is a nondeterministic delay due to the current > process scheduling characteristics of the given system. In other words > it depends on an external entity, who Robot cannot coerce into > performing the request within a bounded set of time. > > I think that means depending on Robot doing it's thing within a given > period of time is an invalid test. > > Robot does not add events to EventQueue but it requests the OS to > synthesize an OS-level event. > > - David Herron > > -- Steve McKay <smckay@google.com> ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-24 21:22 ` Steve McKay☄ @ 2007-09-24 21:41 ` David Herron 2007-09-25 18:33 ` Lillian Angel 0 siblings, 1 reply; 19+ messages in thread From: David Herron @ 2007-09-24 21:41 UTC (permalink / raw) To: Steve McKay☄; +Cc: mauve-discuss [resending because the mailing list thingy told me I had to use only plain text messages...] Hmmm.. here's the meat of the test public void runTest(int code, char chr) { KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr, KeyEvent.KEY_LOCATION_STANDARD); f.dispatchEvent(e); f.setSize(200,200); f.show(); r.mouseMove(60, 60); r.keyPress(code); r.keyRelease(code); h.check(key, (int) chr); } I don't understand this. If you're going to create a Java event why use Robot, or vice versa...? It appears h.check is in gnu.testlet.TestHarness and that it simply does an immediate check with no waiting. The dispatchEvent call is going to cause the listener to fire regardless of what's happening using Robot. This looks like an incorrect test, and what I'd recommend is:- a) ditch the two lines saying KeyEvent / dispatchEvent ... they are completely subverting the intent of the test b) insert some code so the runTest method waits for the listener to be triggered. Such as a wait and notify type of semaphor. c) I don't know how the test guarantees runTest executes on the event dispatch thread. Is the EDT as important to classpath as it is to Sun's Swing? - David Herron Steve McKayâ wrote: > So would you recommend I ignore the test, delete it, add a comment, ...? > > --steve > > On 9/24/07, David Herron <David.Herron@sun.com> wrote: > >> Steve McKayâ wrote: >> >>> Hi All, >>> >>> I've noticed that at least some of the tests using java.awt.Robot are >>> non-deterministic due to lags is the underlying window system. The >>> java.awt.Component.keyPressTest, for example, fails some of the time >>> (on linux, windows, linux+wine, ...). It looks like enabling >>> autoWaitForIdle (waits for the awt EventQueue to be empty before >>> adding new events to the queue), and setting autoDelay (pauses for an >>> arbitrary period of time) to some magic number of millis well above >>> zero (I use 100) significantly reduces failures. Would anyone object >>> to configuring the Robot with settings like this by default? If no, >>> should the config mechanism be updated to allow tweaking these >>> settings? >>> >>> >>> >> I don't know what the classpath implementation of Robot looks like, but >> I do know what Sun's Linux/Unix implementation looks like (having >> written the original version). >> >> Generally Robot has to request the OS or X11 to synthesize the event. >> On Windows there's a direct API call, while on Unix/Linux there is a >> child process which ends up calling XTEST extension methods. In both >> cases it means there is a nondeterministic delay due to the current >> process scheduling characteristics of the given system. In other words >> it depends on an external entity, who Robot cannot coerce into >> performing the request within a bounded set of time. >> >> I think that means depending on Robot doing it's thing within a given >> period of time is an invalid test. >> >> Robot does not add events to EventQueue but it requests the OS to >> synthesize an OS-level event. >> >> - David Herron >> >> >> > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-24 21:41 ` David Herron @ 2007-09-25 18:33 ` Lillian Angel 2007-09-25 19:13 ` Steve McKay☄ 2007-09-25 19:27 ` David Herron 0 siblings, 2 replies; 19+ messages in thread From: Lillian Angel @ 2007-09-25 18:33 UTC (permalink / raw) To: David Herron; +Cc: Steve McKay☄, mauve-discuss David Herron wrote: > [resending because the mailing list thingy told me I had to use only > plain text messages...] > > Hmmm.. here's the meat of the test > > public void runTest(int code, char chr) > { > KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr, > KeyEvent.KEY_LOCATION_STANDARD); > f.dispatchEvent(e); > f.setSize(200,200); > f.show(); > r.mouseMove(60, 60); > > r.keyPress(code); > r.keyRelease(code); > h.check(key, (int) chr); > } > > I don't understand this. If you're going to create a Java event why > use Robot, or vice versa...? The test was done this way to determine if the right key was pressed. See the inner class (myFrame) which has 1 function (keyDown) which sets a variable (key) to the key that was pressed. While it might be incorrect, I am unsure how else we can determine that the correct key was pressed without dispatching an event. Lillian > > It appears h.check is in gnu.testlet.TestHarness and that it simply > does an immediate check with no waiting. The dispatchEvent call is > going to cause the listener to fire regardless of what's happening > using Robot. > > This looks like an incorrect test, and what I'd recommend is:- > > a) ditch the two lines saying KeyEvent / dispatchEvent ... they are > completely subverting the intent of the test > > b) insert some code so the runTest method waits for the listener to be > triggered. Such as a wait and notify type of semaphor. > > c) I don't know how the test guarantees runTest executes on the event > dispatch thread. Is the EDT as important to classpath as it is to > Sun's Swing? > > - David Herron > > > Steve McKayâ wrote: >> So would you recommend I ignore the test, delete it, add a comment, ...? >> >> --steve >> >> On 9/24/07, David Herron <David.Herron@sun.com> wrote: >> >>> Steve McKayâ wrote: >>> >>>> Hi All, >>>> >>>> I've noticed that at least some of the tests using java.awt.Robot are >>>> non-deterministic due to lags is the underlying window system. The >>>> java.awt.Component.keyPressTest, for example, fails some of the time >>>> (on linux, windows, linux+wine, ...). It looks like enabling >>>> autoWaitForIdle (waits for the awt EventQueue to be empty before >>>> adding new events to the queue), and setting autoDelay (pauses for an >>>> arbitrary period of time) to some magic number of millis well above >>>> zero (I use 100) significantly reduces failures. Would anyone object >>>> to configuring the Robot with settings like this by default? If no, >>>> should the config mechanism be updated to allow tweaking these >>>> settings? >>>> >>>> >>>> >>> I don't know what the classpath implementation of Robot looks like, but >>> I do know what Sun's Linux/Unix implementation looks like (having >>> written the original version). >>> >>> Generally Robot has to request the OS or X11 to synthesize the event. >>> On Windows there's a direct API call, while on Unix/Linux there is a >>> child process which ends up calling XTEST extension methods. In both >>> cases it means there is a nondeterministic delay due to the current >>> process scheduling characteristics of the given system. In other words >>> it depends on an external entity, who Robot cannot coerce into >>> performing the request within a bounded set of time. >>> >>> I think that means depending on Robot doing it's thing within a given >>> period of time is an invalid test. >>> >>> Robot does not add events to EventQueue but it requests the OS to >>> synthesize an OS-level event. >>> >>> - David Herron >>> >>> >>> >> >> >> > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 18:33 ` Lillian Angel @ 2007-09-25 19:13 ` Steve McKay☄ 2007-09-25 19:27 ` Lillian Angel 2007-09-25 19:27 ` David Herron 1 sibling, 1 reply; 19+ messages in thread From: Steve McKay☄ @ 2007-09-25 19:13 UTC (permalink / raw) To: Lillian Angel; +Cc: David Herron, mauve-discuss > While it might be incorrect, I am unsure how else we can determine that > the correct key was pressed without dispatching an event. As I know understand it, when the Robot's keyPress is called, it should make a keypress happen on your desktop. So if the myFrame window has focus, a keyPress event would be generated, otherwise, if your editor is focused, it'll get the key press. Does anyone know if Selenium uses Robot to do its poking and prodding? --steve > Lillian > > > > > It appears h.check is in gnu.testlet.TestHarness and that it simply > > does an immediate check with no waiting. The dispatchEvent call is > > going to cause the listener to fire regardless of what's happening > > using Robot. > > > > This looks like an incorrect test, and what I'd recommend is:- > > > > a) ditch the two lines saying KeyEvent / dispatchEvent ... they are > > completely subverting the intent of the test > > > > b) insert some code so the runTest method waits for the listener to be > > triggered. Such as a wait and notify type of semaphor. > > > > c) I don't know how the test guarantees runTest executes on the event > > dispatch thread. Is the EDT as important to classpath as it is to > > Sun's Swing? > > > > - David Herron > > > > > > Steve McKay☄ wrote: > >> So would you recommend I ignore the test, delete it, add a comment, ...? > >> > >> --steve > >> > >> On 9/24/07, David Herron <David.Herron@sun.com> wrote: > >> > >>> Steve McKay☄ wrote: > >>> > >>>> Hi All, > >>>> > >>>> I've noticed that at least some of the tests using java.awt.Robot are > >>>> non-deterministic due to lags is the underlying window system. The > >>>> java.awt.Component.keyPressTest, for example, fails some of the time > >>>> (on linux, windows, linux+wine, ...). It looks like enabling > >>>> autoWaitForIdle (waits for the awt EventQueue to be empty before > >>>> adding new events to the queue), and setting autoDelay (pauses for an > >>>> arbitrary period of time) to some magic number of millis well above > >>>> zero (I use 100) significantly reduces failures. Would anyone object > >>>> to configuring the Robot with settings like this by default? If no, > >>>> should the config mechanism be updated to allow tweaking these > >>>> settings? > >>>> > >>>> > >>>> > >>> I don't know what the classpath implementation of Robot looks like, but > >>> I do know what Sun's Linux/Unix implementation looks like (having > >>> written the original version). > >>> > >>> Generally Robot has to request the OS or X11 to synthesize the event. > >>> On Windows there's a direct API call, while on Unix/Linux there is a > >>> child process which ends up calling XTEST extension methods. In both > >>> cases it means there is a nondeterministic delay due to the current > >>> process scheduling characteristics of the given system. In other words > >>> it depends on an external entity, who Robot cannot coerce into > >>> performing the request within a bounded set of time. > >>> > >>> I think that means depending on Robot doing it's thing within a given > >>> period of time is an invalid test. > >>> > >>> Robot does not add events to EventQueue but it requests the OS to > >>> synthesize an OS-level event. > >>> > >>> - David Herron > >>> > >>> > >>> > >> > >> > >> > > > > -- Steve McKay <smckay@google.com> ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 19:13 ` Steve McKay☄ @ 2007-09-25 19:27 ` Lillian Angel 2007-09-25 19:37 ` Steve McKay☄ 0 siblings, 1 reply; 19+ messages in thread From: Lillian Angel @ 2007-09-25 19:27 UTC (permalink / raw) To: Steve McKay☄; +Cc: David Herron, mauve-discuss Steve McKayâ wrote: >> While it might be incorrect, I am unsure how else we can determine that >> the correct key was pressed without dispatching an event. >> > > As I know understand it, when the Robot's keyPress is called, it > should make a keypress happen on your desktop. So if the myFrame > window has focus, a keyPress event would be generated, otherwise, if > your editor is focused, it'll get the key press. > That's right. If these two lines... KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr, KeyEvent.KEY_LOCATION_STANDARD); f.dispatchEvent(e); ...are removed, myFrame.keyDown is not called and therefore, this.key is not set. As it seems, the Robot function keyPress is not dispatching the event to the Component. Lillian > Does anyone know if Selenium uses Robot to do its poking and prodding? > > --steve > > >> Lillian >> >> >>> It appears h.check is in gnu.testlet.TestHarness and that it simply >>> does an immediate check with no waiting. The dispatchEvent call is >>> going to cause the listener to fire regardless of what's happening >>> using Robot. >>> >>> This looks like an incorrect test, and what I'd recommend is:- >>> >>> a) ditch the two lines saying KeyEvent / dispatchEvent ... they are >>> completely subverting the intent of the test >>> >>> b) insert some code so the runTest method waits for the listener to be >>> triggered. Such as a wait and notify type of semaphor. >>> >>> c) I don't know how the test guarantees runTest executes on the event >>> dispatch thread. Is the EDT as important to classpath as it is to >>> Sun's Swing? >>> >>> - David Herron >>> >>> >>> Steve McKayâ wrote: >>> >>>> So would you recommend I ignore the test, delete it, add a comment, ...? >>>> >>>> --steve >>>> >>>> On 9/24/07, David Herron <David.Herron@sun.com> wrote: >>>> >>>> >>>>> Steve McKayâ wrote: >>>>> >>>>> >>>>>> Hi All, >>>>>> >>>>>> I've noticed that at least some of the tests using java.awt.Robot are >>>>>> non-deterministic due to lags is the underlying window system. The >>>>>> java.awt.Component.keyPressTest, for example, fails some of the time >>>>>> (on linux, windows, linux+wine, ...). It looks like enabling >>>>>> autoWaitForIdle (waits for the awt EventQueue to be empty before >>>>>> adding new events to the queue), and setting autoDelay (pauses for an >>>>>> arbitrary period of time) to some magic number of millis well above >>>>>> zero (I use 100) significantly reduces failures. Would anyone object >>>>>> to configuring the Robot with settings like this by default? If no, >>>>>> should the config mechanism be updated to allow tweaking these >>>>>> settings? >>>>>> >>>>>> >>>>>> >>>>>> >>>>> I don't know what the classpath implementation of Robot looks like, but >>>>> I do know what Sun's Linux/Unix implementation looks like (having >>>>> written the original version). >>>>> >>>>> Generally Robot has to request the OS or X11 to synthesize the event. >>>>> On Windows there's a direct API call, while on Unix/Linux there is a >>>>> child process which ends up calling XTEST extension methods. In both >>>>> cases it means there is a nondeterministic delay due to the current >>>>> process scheduling characteristics of the given system. In other words >>>>> it depends on an external entity, who Robot cannot coerce into >>>>> performing the request within a bounded set of time. >>>>> >>>>> I think that means depending on Robot doing it's thing within a given >>>>> period of time is an invalid test. >>>>> >>>>> Robot does not add events to EventQueue but it requests the OS to >>>>> synthesize an OS-level event. >>>>> >>>>> - David Herron >>>>> >>>>> >>>>> >>>>> >>>> >>>> >> > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 19:27 ` Lillian Angel @ 2007-09-25 19:37 ` Steve McKay☄ 2007-09-25 19:45 ` Lillian Angel 0 siblings, 1 reply; 19+ messages in thread From: Steve McKay☄ @ 2007-09-25 19:37 UTC (permalink / raw) To: Lillian Angel; +Cc: David Herron, mauve-discuss > That's right. If these two lines... > KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr, > KeyEvent.KEY_LOCATION_STANDARD); > f.dispatchEvent(e); > > ...are removed, myFrame.keyDown is not called and therefore, this.key is > not set. As it seems, the Robot function keyPress is not dispatching the > event to the Component. That's not my experience. I removed the explicit dispatch lines, then added code to make sure the window is created and focused before the test begins. After that, the test basically works. There appear to be occasional deadlocks, so my thread management is wonky, but the basics work as expected. > > Lillian > > > Does anyone know if Selenium uses Robot to do its poking and prodding? > > > > --steve > > > > > >> Lillian > >> > >> > >>> It appears h.check is in gnu.testlet.TestHarness and that it simply > >>> does an immediate check with no waiting. The dispatchEvent call is > >>> going to cause the listener to fire regardless of what's happening > >>> using Robot. > >>> > >>> This looks like an incorrect test, and what I'd recommend is:- > >>> > >>> a) ditch the two lines saying KeyEvent / dispatchEvent ... they are > >>> completely subverting the intent of the test > >>> > >>> b) insert some code so the runTest method waits for the listener to be > >>> triggered. Such as a wait and notify type of semaphor. > >>> > >>> c) I don't know how the test guarantees runTest executes on the event > >>> dispatch thread. Is the EDT as important to classpath as it is to > >>> Sun's Swing? > >>> > >>> - David Herron > >>> > >>> > >>> Steve McKay☄ wrote: > >>> > >>>> So would you recommend I ignore the test, delete it, add a comment, ...? > >>>> > >>>> --steve > >>>> > >>>> On 9/24/07, David Herron <David.Herron@sun.com> wrote: > >>>> > >>>> > >>>>> Steve McKay☄ wrote: > >>>>> > >>>>> > >>>>>> Hi All, > >>>>>> > >>>>>> I've noticed that at least some of the tests using java.awt.Robot are > >>>>>> non-deterministic due to lags is the underlying window system. The > >>>>>> java.awt.Component.keyPressTest, for example, fails some of the time > >>>>>> (on linux, windows, linux+wine, ...). It looks like enabling > >>>>>> autoWaitForIdle (waits for the awt EventQueue to be empty before > >>>>>> adding new events to the queue), and setting autoDelay (pauses for an > >>>>>> arbitrary period of time) to some magic number of millis well above > >>>>>> zero (I use 100) significantly reduces failures. Would anyone object > >>>>>> to configuring the Robot with settings like this by default? If no, > >>>>>> should the config mechanism be updated to allow tweaking these > >>>>>> settings? > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> I don't know what the classpath implementation of Robot looks like, but > >>>>> I do know what Sun's Linux/Unix implementation looks like (having > >>>>> written the original version). > >>>>> > >>>>> Generally Robot has to request the OS or X11 to synthesize the event. > >>>>> On Windows there's a direct API call, while on Unix/Linux there is a > >>>>> child process which ends up calling XTEST extension methods. In both > >>>>> cases it means there is a nondeterministic delay due to the current > >>>>> process scheduling characteristics of the given system. In other words > >>>>> it depends on an external entity, who Robot cannot coerce into > >>>>> performing the request within a bounded set of time. > >>>>> > >>>>> I think that means depending on Robot doing it's thing within a given > >>>>> period of time is an invalid test. > >>>>> > >>>>> Robot does not add events to EventQueue but it requests the OS to > >>>>> synthesize an OS-level event. > >>>>> > >>>>> - David Herron > >>>>> > >>>>> > >>>>> > >>>>> > >>>> > >>>> > >> > > > > > > > > -- Steve McKay <smckay@google.com> ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 19:37 ` Steve McKay☄ @ 2007-09-25 19:45 ` Lillian Angel 0 siblings, 0 replies; 19+ messages in thread From: Lillian Angel @ 2007-09-25 19:45 UTC (permalink / raw) To: Steve McKay☄; +Cc: David Herron, mauve-discuss Steve McKayâ wrote: >> That's right. If these two lines... >> KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr, >> KeyEvent.KEY_LOCATION_STANDARD); >> f.dispatchEvent(e); >> >> ...are removed, myFrame.keyDown is not called and therefore, this.key is >> not set. As it seems, the Robot function keyPress is not dispatching the >> event to the Component. >> > > That's not my experience. I removed the explicit dispatch lines, then > added code to make sure the window is created and focused before the > test begins. After that, the test basically works. There appear to be > occasional deadlocks, so my thread management is wonky, but the basics > work as expected. > Great! If it works, feel free to change the test. I am obviously doing something incorrectly on my end. Lillian > >> Lillian >> >> >>> Does anyone know if Selenium uses Robot to do its poking and prodding? >>> >>> --steve >>> >>> >>> >>>> Lillian >>>> >>>> >>>> >>>>> It appears h.check is in gnu.testlet.TestHarness and that it simply >>>>> does an immediate check with no waiting. The dispatchEvent call is >>>>> going to cause the listener to fire regardless of what's happening >>>>> using Robot. >>>>> >>>>> This looks like an incorrect test, and what I'd recommend is:- >>>>> >>>>> a) ditch the two lines saying KeyEvent / dispatchEvent ... they are >>>>> completely subverting the intent of the test >>>>> >>>>> b) insert some code so the runTest method waits for the listener to be >>>>> triggered. Such as a wait and notify type of semaphor. >>>>> >>>>> c) I don't know how the test guarantees runTest executes on the event >>>>> dispatch thread. Is the EDT as important to classpath as it is to >>>>> Sun's Swing? >>>>> >>>>> - David Herron >>>>> >>>>> >>>>> Steve McKayâ wrote: >>>>> >>>>> >>>>>> So would you recommend I ignore the test, delete it, add a comment, ...? >>>>>> >>>>>> --steve >>>>>> >>>>>> On 9/24/07, David Herron <David.Herron@sun.com> wrote: >>>>>> >>>>>> >>>>>> >>>>>>> Steve McKayâ wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> I've noticed that at least some of the tests using java.awt.Robot are >>>>>>>> non-deterministic due to lags is the underlying window system. The >>>>>>>> java.awt.Component.keyPressTest, for example, fails some of the time >>>>>>>> (on linux, windows, linux+wine, ...). It looks like enabling >>>>>>>> autoWaitForIdle (waits for the awt EventQueue to be empty before >>>>>>>> adding new events to the queue), and setting autoDelay (pauses for an >>>>>>>> arbitrary period of time) to some magic number of millis well above >>>>>>>> zero (I use 100) significantly reduces failures. Would anyone object >>>>>>>> to configuring the Robot with settings like this by default? If no, >>>>>>>> should the config mechanism be updated to allow tweaking these >>>>>>>> settings? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> I don't know what the classpath implementation of Robot looks like, but >>>>>>> I do know what Sun's Linux/Unix implementation looks like (having >>>>>>> written the original version). >>>>>>> >>>>>>> Generally Robot has to request the OS or X11 to synthesize the event. >>>>>>> On Windows there's a direct API call, while on Unix/Linux there is a >>>>>>> child process which ends up calling XTEST extension methods. In both >>>>>>> cases it means there is a nondeterministic delay due to the current >>>>>>> process scheduling characteristics of the given system. In other words >>>>>>> it depends on an external entity, who Robot cannot coerce into >>>>>>> performing the request within a bounded set of time. >>>>>>> >>>>>>> I think that means depending on Robot doing it's thing within a given >>>>>>> period of time is an invalid test. >>>>>>> >>>>>>> Robot does not add events to EventQueue but it requests the OS to >>>>>>> synthesize an OS-level event. >>>>>>> >>>>>>> - David Herron >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>> >>> >> > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 18:33 ` Lillian Angel 2007-09-25 19:13 ` Steve McKay☄ @ 2007-09-25 19:27 ` David Herron 1 sibling, 0 replies; 19+ messages in thread From: David Herron @ 2007-09-25 19:27 UTC (permalink / raw) To: Lillian Angel; +Cc: Steve McKay☄, mauve-discuss Lillian Angel wrote: > David Herron wrote: >> [resending because the mailing list thingy told me I had to use only >> plain text messages...] >> >> Hmmm.. here's the meat of the test >> >> public void runTest(int code, char chr) >> { >> KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, >> chr, KeyEvent.KEY_LOCATION_STANDARD); >> f.dispatchEvent(e); >> f.setSize(200,200); >> f.show(); >> r.mouseMove(60, 60); >> >> r.keyPress(code); >> r.keyRelease(code); >> h.check(key, (int) chr); >> } >> >> I don't understand this. If you're going to create a Java event why >> use Robot, or vice versa...? > > The test was done this way to determine if the right key was pressed. > See the inner class (myFrame) which has 1 function (keyDown) which > sets a variable (key) to the key that was pressed. > > While it might be incorrect, I am unsure how else we can determine > that the correct key was pressed without dispatching an event. > > Lillian > Maybe I misread the code in my brief look. The side effect of using Robot to keyPress and keyRelease should be the dispatch of two KeyEvent's. That's why I said those two lines should go, because the way I read it they're masking whether Robot successfully delivered the KeyEvent's. - David Herron ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-24 20:50 ` David Herron 2007-09-24 21:22 ` Steve McKay☄ @ 2007-09-25 18:33 ` Thomas Fitzsimmons 2007-09-25 18:57 ` Steve McKay☄ 2007-09-25 19:24 ` David Herron 1 sibling, 2 replies; 19+ messages in thread From: Thomas Fitzsimmons @ 2007-09-25 18:33 UTC (permalink / raw) To: David Herron; +Cc: Steve McKay☄, mauve-discuss David Herron wrote: > Steve McKayâ wrote: >> Hi All, >> >> I've noticed that at least some of the tests using java.awt.Robot are >> non-deterministic due to lags is the underlying window system. The >> java.awt.Component.keyPressTest, for example, fails some of the time >> (on linux, windows, linux+wine, ...). It looks like enabling >> autoWaitForIdle (waits for the awt EventQueue to be empty before >> adding new events to the queue), and setting autoDelay (pauses for an >> arbitrary period of time) to some magic number of millis well above >> zero (I use 100) significantly reduces failures. Would anyone object >> to configuring the Robot with settings like this by default? If no, >> should the config mechanism be updated to allow tweaking these >> settings? >> >> > > I don't know what the classpath implementation of Robot looks like, but > I do know what Sun's Linux/Unix implementation looks like (having > written the original version). It uses the XTEST extension. > > Generally Robot has to request the OS or X11 to synthesize the event. > On Windows there's a direct API call, while on Unix/Linux there is a > child process which ends up calling XTEST extension methods. In both > cases it means there is a nondeterministic delay due to the current > process scheduling characteristics of the given system. In other words > it depends on an external entity, who Robot cannot coerce into > performing the request within a bounded set of time. > > I think that means depending on Robot doing it's thing within a given > period of time is an invalid test. Interesting... > Robot does not add events to EventQueue but it requests the OS to > synthesize an OS-level event. How has Sun implemented GUI testing? When I was considering how to do GUI testing in Mauve, I considered the EventQueue-posting approach, but decided on a Robot-based approach instead. I thought Robot tests would be more realistic, testing things like window manager interactions and the native peers' event processing code. I knew Robot tests would be more fragile, but I assumed that we could compensate for the fragility: e.g. fix timing problems by introducing delays, as Steve has proposed. Did Sun experiment with Robot tests, then abandon them? If Robot can't be counted on to do something within some time delay, is it also useless in non-test applications? I've always wondered how the TCK certified AWT and Swing functionality. Does it use EventQueue-posting? Tom ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 18:33 ` Thomas Fitzsimmons @ 2007-09-25 18:57 ` Steve McKay☄ 2007-09-25 19:58 ` Thomas Fitzsimmons 2007-09-25 19:24 ` David Herron 1 sibling, 1 reply; 19+ messages in thread From: Steve McKay☄ @ 2007-09-25 18:57 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: David Herron, mauve-discuss Hi Thomas, > If Robot can't be counted on to do something within some time > delay, is it also useless in non-test applications? David's suggestion to use multi-threaded paradigms for dealing with the issue seems pretty good to me. For example, we can block until a window is ready to process events with code like this (only half baked at this time): // before running any tests // show the window f.setSize(200, 200); f.show(); waitForWindow(); // blocks until the frame has responded to a keypress events .... class myFrame extends Frame { public boolean keyDown(Event e, int i) { synchronized(lock) { // lock is some explicit lock object key = new Integer(e.key); lock.notify(); } return super.keyDown(e, i); } } /** * Blocks until the frame has responded to a keypress event. */ private void waitForWindow() { synchronized(lock) { while (key == null) { r.keyPress(32); // send a key press event r.keyRelease(32); // don't press the key forever try { // wait for a notify from the frame, or timeout in case the // window missed the keypress event entirely lock.wait(100); } catch (InterruptedException ie) { // interrupted, if key is still null, we'll try again } } } } The obvious downside is you have to deal with all the pitfalls of multi-threaded code everytime you write the code. --steve > I've always wondered how the TCK certified AWT and Swing functionality. Does it > use EventQueue-posting? > > Tom > > -- Steve McKay <smckay@google.com> ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 18:57 ` Steve McKay☄ @ 2007-09-25 19:58 ` Thomas Fitzsimmons 2007-09-25 20:28 ` Steve McKay☄ 0 siblings, 1 reply; 19+ messages in thread From: Thomas Fitzsimmons @ 2007-09-25 19:58 UTC (permalink / raw) To: Steve McKay☄; +Cc: David Herron, mauve-discuss Steve McKayâ wrote: > Hi Thomas, > >> If Robot can't be counted on to do something within some time >> delay, is it also useless in non-test applications? > > David's suggestion to use multi-threaded paradigms for dealing with > the issue seems pretty good to me. Yes, I shouldn't have said useless.. maybe "hard to use" :-) In Mauve we traded timing-independent reproducibility for test case simplicity. I did try writing synchronized Robot-using tests but it was a) difficult, and b) invasive, in that it required overriding and modifying the behaviour of methods that were being tested. > For example, we can block until a > window is ready to process events with code like this (only half baked > at this time): Even after you've ensured that the frame has responded, subsequent checks will still need to be synchronized, since Robot calls are not synchronous. Or is this the specific problem that was causing test failures for you before -- that frames take a long time to be ready to receive events on other runtimes (e.g. Wine)? If that's the main problem then maybe a hybrid approach of synchronizing frame readiness, then assuming small delays for Robot method calls, is a better approach. Otherwise the proposal is to rewrite every Robot-using test to be perfectly synchronous (fine by me, but a big job). Tom ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 19:58 ` Thomas Fitzsimmons @ 2007-09-25 20:28 ` Steve McKay☄ 2007-10-04 0:43 ` Steve McKay☄ 0 siblings, 1 reply; 19+ messages in thread From: Steve McKay☄ @ 2007-09-25 20:28 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: David Herron, mauve-discuss Thomas, > Even after you've ensured that the frame has responded, subsequent checks will > still need to be synchronized, since Robot calls are not synchronous. That was just a code snippet to demonstrate how to solve one part of the problem. The code that does the actual testing is synchronized on the lock as well. But like I said, there were liveness issues. I'll try to work those out. Still, I can't honestly say that I'd want to go through the same amount of effort for every test involving Robot, so this change is sorta speculative rather than practical. -- Steve McKay <smckay@google.com> ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 20:28 ` Steve McKay☄ @ 2007-10-04 0:43 ` Steve McKay☄ 2007-10-04 13:04 ` Lillian Angel 0 siblings, 1 reply; 19+ messages in thread From: Steve McKay☄ @ 2007-10-04 0:43 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: David Herron, mauve-discuss, Dan Kegel [-- Attachment #1: Type: text/plain, Size: 875 bytes --] Here's a patch for the test. Everything seems to be working well. I haven't looked through the other tests yet to see how this might be shared. --steve On 9/25/07, Steve McKay☄ <smckay@google.com> wrote: > Thomas, > > > Even after you've ensured that the frame has responded, subsequent checks will > > still need to be synchronized, since Robot calls are not synchronous. > > That was just a code snippet to demonstrate how to solve one part of > the problem. The code that does the actual testing is synchronized on > the lock as well. But like I said, there were liveness issues. I'll > try to work those out. Still, I can't honestly say that I'd want to go > through the same amount of effort for every test involving Robot, so > this change is sorta speculative rather than practical. > > -- > Steve McKay <smckay@google.com> > -- Steve McKay <smckay@google.com> [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: KeyPressTest.patch --] [-- Type: text/x-patch; name="KeyPressTest.patch", Size: 3663 bytes --] Index: keyPressTest.java =================================================================== RCS file: /cvs/mauve/mauve/gnu/testlet/java/awt/Component/keyPressTest.java,v retrieving revision 1.2 diff -u -r1.2 keyPressTest.java --- keyPressTest.java 31 Mar 2006 23:08:06 -0000 1.2 +++ keyPressTest.java 4 Oct 2007 00:36:00 -0000 @@ -31,17 +31,26 @@ public class keyPressTest implements Testlet { - int key = 0; + //arbitrary lock for use in synchronizing test and awt threads + final Object lock = new Object(); + + volatile Integer key = null; + Robot r; myFrame f; TestHarness h; - + public void test (TestHarness h) { f = new myFrame(); r = h.createRobot (); this.h = h; - + + f.setSize(200,200); // assumes that the window is positioned at 0,0. + f.show(); + + waitForWindow(); + runTest(KeyEvent.VK_A, 'a'); runTest(KeyEvent.VK_B, 'b'); runTest(KeyEvent.VK_C, 'c'); @@ -68,31 +77,84 @@ runTest(KeyEvent.VK_X, 'x'); runTest(KeyEvent.VK_Y, 'y'); runTest(KeyEvent.VK_Z, 'z'); + + f.dispose(); } - + public void runTest(int code, char chr) { - KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr, KeyEvent.KEY_LOCATION_STANDARD); - f.dispatchEvent(e); - - - f.setSize(200,200); - f.show(); + int k; // assigned in the synchronized block r.mouseMove(60, 60); - r.keyPress(code); - r.keyRelease(code); - h.check(key, (int) chr); - } + synchronized(lock) { + key = null; // reset the key + + // queue the events + r.keyPress(code); + r.keyRelease(code); // don't press they key forever - class myFrame - extends Frame - { - - public boolean keyDown(Event e, int i) - { - key = e.key; + try { + + // release the lock so that the frame can handle the keypress event + // once it has handled the event, it will notify on the lock. + // at this point the key should be non-null. We test the result + // and return + lock.wait(); + } catch (InterruptedException e) { + // ignore, we want to get started again + } + + k = key.intValue(); + } + + h.check(k, chr); + } + + class myFrame extends Frame { + + public boolean keyDown(Event e, int i) { + synchronized(lock) { + key = new Integer(e.key); + lock.notifyAll(); + } return super.keyDown(e, i); } } + + /** + * Blocks until the frame has able to respond to keypress events + */ + private void waitForWindow() { + + // wait until the window starts process events + synchronized(lock) { + while (key == null) { + r.keyPress(KeyEvent.VK_EQUALS); // send a key press event + r.keyRelease(KeyEvent.VK_EQUALS); // don't press the key forever + try { + + // wait for a notify from the frame, or timeout in case the + // window missed the key press event entirely + lock.wait(100); + } + catch (InterruptedException ie) { + // interrupted, if key is still null, we'll try again + } + } + + // send one more "magic" key press as a marker that we've processed + // all of our probing key strokes + r.keyPress(KeyEvent.VK_SEMICOLON); // send a key press event + r.keyRelease(KeyEvent.VK_SEMICOLON); // don't press the key forever + } + + // Eat up any straggler key strokes, wait for the final magic key press + while(key != KeyEvent.VK_SEMICOLON) { + try { + Thread.sleep(100); + } catch (InterruptedException e) { + // shouldn't happen + } + } + } } ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-10-04 0:43 ` Steve McKay☄ @ 2007-10-04 13:04 ` Lillian Angel 0 siblings, 0 replies; 19+ messages in thread From: Lillian Angel @ 2007-10-04 13:04 UTC (permalink / raw) To: Steve McKay☄ Cc: Thomas Fitzsimmons, David Herron, mauve-discuss, Dan Kegel Looks good, I tested it out and committed it for you: 2007-10-03 Steve McKay <smckay@google.com> * gnu/testlet/java/awt/Component/keyPressTest.java: Fixed test so event is not created and dispatched. Completely waits for the frame to receive and respond keypress events. Lillian Steve McKayâ wrote: > Here's a patch for the test. Everything seems to be working well. I > haven't looked through the other tests yet to see how this might be > shared. > > --steve > > On 9/25/07, Steve McKayâ <smckay@google.com> wrote: > >> Thomas, >> >> >>> Even after you've ensured that the frame has responded, subsequent checks will >>> still need to be synchronized, since Robot calls are not synchronous. >>> >> That was just a code snippet to demonstrate how to solve one part of >> the problem. The code that does the actual testing is synchronized on >> the lock as well. But like I said, there were liveness issues. I'll >> try to work those out. Still, I can't honestly say that I'd want to go >> through the same amount of effort for every test involving Robot, so >> this change is sorta speculative rather than practical. >> >> -- >> Steve McKay <smckay@google.com> >> >> > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 18:33 ` Thomas Fitzsimmons 2007-09-25 18:57 ` Steve McKay☄ @ 2007-09-25 19:24 ` David Herron 1 sibling, 0 replies; 19+ messages in thread From: David Herron @ 2007-09-25 19:24 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: Steve McKay☄, mauve-discuss Thomas Fitzsimmons wrote: > David Herron wrote: >> Steve McKayâ wrote: > >> Robot does not add events to EventQueue but it requests the OS to >> synthesize an OS-level event. > > How has Sun implemented GUI testing? When I was considering how to do > GUI testing in Mauve, I considered the EventQueue-posting approach, > but decided on a Robot-based approach instead. I thought Robot tests > would be more realistic, testing things like window manager > interactions and the native peers' event processing code. I knew > Robot tests would be more fragile, but I assumed that we could > compensate for the fragility: e.g. fix timing problems by introducing > delays, as Steve has proposed. Did Sun experiment with Robot tests, > then abandon them? If Robot can't be counted on to do something > within some time delay, is it also useless in non-test applications? > > I've always wondered how the TCK certified AWT and Swing > functionality. Does it use EventQueue-posting? > > Tom > We make heavy use of Robot. I haven't been writing tests for a long time so I don't know for sure the techniques used by the team. Sometimes I worry about whether they are using sleep's and the like to synchronize validation that a desired event has occurred. If you click on something, move a mouse pointer, type a key.. there's a listener fired in 99% of the cases you would be testing. So I think it'd be more reliable to synchronize the validation based on when the listener fires. Unfortunately that can turn into difficult coding to ensure the test doesn't deadlock or otherwise go off into the weeds because cross-thread notification of the listener firing didn't happen in the right order. I'm sure it's much simpler to code up a simple sleep and 99% of the time the system will be fast enough responding to post the listener callback within a couple milliseconds. I believe that the TCK, where it has automated tests, uses Robot as well. I don't know that for sure since I've not been involved with writing the TCK's, and that's a different team than mine. The TCK also has manual tests. And the reason you give for using Robot is exactly the reasoning we have. Since we're testing AWT/Swing a we have to make sure to test AWT and the peers. Stuffing events in the EventQueue doesn't test the native parts of AWT including the peers. That was the rationale for putting Robot into the platform in the first place (that and to support writing freestanding demo apps), and it's true today. - David Herron P.S. there's a summary here: http://wiki.java.net/bin/view/Javapedia/TestingGUIApplications of GUI testing methods.. it's been added to over several years and has some good pointers. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-24 20:31 Tweaking default java.awt.Robot settings Steve McKay☄ 2007-09-24 20:50 ` David Herron @ 2007-09-25 18:10 ` Thomas Fitzsimmons 2007-09-25 18:14 ` Steve McKay☄ 1 sibling, 1 reply; 19+ messages in thread From: Thomas Fitzsimmons @ 2007-09-25 18:10 UTC (permalink / raw) To: Steve McKay☄; +Cc: mauve-discuss Steve McKayâ wrote: > Hi All, > > I've noticed that at least some of the tests using java.awt.Robot are > non-deterministic due to lags is the underlying window system. The > java.awt.Component.keyPressTest, for example, fails some of the time > (on linux, windows, linux+wine, ...). It looks like enabling > autoWaitForIdle (waits for the awt EventQueue to be empty before > adding new events to the queue), and setting autoDelay (pauses for an > arbitrary period of time) to some magic number of millis well above > zero (I use 100) significantly reduces failures. Would anyone object > to configuring the Robot with settings like this by default? No objections here, that sounds like a good idea. When you say "significantly reduces failures", does that mean some non-deterministic failures persist with the new auto* settings? Tom ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Tweaking default java.awt.Robot settings 2007-09-25 18:10 ` Thomas Fitzsimmons @ 2007-09-25 18:14 ` Steve McKay☄ 0 siblings, 0 replies; 19+ messages in thread From: Steve McKay☄ @ 2007-09-25 18:14 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: mauve-discuss Thomas, I think David's suggestion was better, so I'll work that up instead of the autoDelay changes I proposed. --steve On 9/25/07, Thomas Fitzsimmons <fitzsim@redhat.com> wrote: > Steve McKay☄ wrote: > > Hi All, > > > > I've noticed that at least some of the tests using java.awt.Robot are > > non-deterministic due to lags is the underlying window system. The > > java.awt.Component.keyPressTest, for example, fails some of the time > > (on linux, windows, linux+wine, ...). It looks like enabling > > autoWaitForIdle (waits for the awt EventQueue to be empty before > > adding new events to the queue), and setting autoDelay (pauses for an > > arbitrary period of time) to some magic number of millis well above > > zero (I use 100) significantly reduces failures. Would anyone object > > to configuring the Robot with settings like this by default? > > No objections here, that sounds like a good idea. When you say "significantly > reduces failures", does that mean some non-deterministic failures persist with > the new auto* settings? > > Tom > -- Steve McKay <smckay@google.com> ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2007-10-04 13:04 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2007-09-24 20:31 Tweaking default java.awt.Robot settings Steve McKay☄ 2007-09-24 20:50 ` David Herron 2007-09-24 21:22 ` Steve McKay☄ 2007-09-24 21:41 ` David Herron 2007-09-25 18:33 ` Lillian Angel 2007-09-25 19:13 ` Steve McKay☄ 2007-09-25 19:27 ` Lillian Angel 2007-09-25 19:37 ` Steve McKay☄ 2007-09-25 19:45 ` Lillian Angel 2007-09-25 19:27 ` David Herron 2007-09-25 18:33 ` Thomas Fitzsimmons 2007-09-25 18:57 ` Steve McKay☄ 2007-09-25 19:58 ` Thomas Fitzsimmons 2007-09-25 20:28 ` Steve McKay☄ 2007-10-04 0:43 ` Steve McKay☄ 2007-10-04 13:04 ` Lillian Angel 2007-09-25 19:24 ` David Herron 2007-09-25 18:10 ` Thomas Fitzsimmons 2007-09-25 18:14 ` Steve McKay☄
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).