Created
December 20, 2012 20:18
-
-
Save jaredhirsch/4348207 to your computer and use it in GitHub Desktop.
Stability: are tests with implicit waits more or less stable than tests with only explicit waits?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
**QUESTION** | |
Some folks assert that a broad class of bugs is solved by favoring | |
explicit over implicit waits. We had previously been using implicit | |
waits by default in our code. | |
**ANSWER** | |
No substantial stability differences were observed. | |
Failures across chrome, ff, ie9, ie8: | |
* explicit waits only: 1, 0, 2, 10 | |
* implicit waits only: 1, 1, 3, 10 | |
**DATA** | |
We conducted an experiment: | |
* run all tests 10x, across 5 platforms (vista/ff, vista/chrome, vista/IE9, xp/IE8, mac/safari 5) | |
* analyze errors on each platform | |
* implicit waits enabled: https://gist.github.com/4347710 | |
* implicit waits disabled: https://gist.github.com/4333301 | |
win/chrome: | |
implicit waits: 9/10 | |
1 mysterious health-check failure | |
explicit waits: 9/10 success | |
1 missed click | |
win/ff: | |
implicit waits: 9/10 success (+ 1 infrastructure fail) | |
1 stale element ref error | |
explicit waits: 10/10 success | |
win/IE9: | |
implicit waits: 7/10 success (+ 1 infrastructure fail) | |
2 missed click | |
2 mysterious health-check failure | |
explicit waits: 8/10 success | |
2 missed clicks | |
win/IE8: | |
implicit waits: 0/10 success | |
looked basically identical to explicit waits case--didn't look past first 2 builds | |
explicit waits: 0/10 success | |
27 'we are very sorry there has been an error' | |
17 eyedeeme hangs after creating account |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment