Created
August 31, 2012 20:24
-
-
Save lloyd/3558478 to your computer and use it in GitHub Desktop.
automated test stability
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <stability%> (<fail>/<success>) platform | |
| Report of all errors with analysis by platform | |
| 100% (0/30) identity.browserid.dev.mac.firefox.14 | |
| 100% (0/30) identity.browserid.dev.windows.7.firefox.13 | |
| 97% (1/30) identity.browserid.dev.windows.7.chrome | |
| * GIT CLONE ERROR: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.windows.7.chrome/1082/console | |
| 97% (1/30) identity.browserid.dev.windows.xp.ie.8 | |
| * INTERNAL SERVER ERROR (sauce labs selenium): https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.windows.xp.ie.8/1067/ | |
| 93% (2/30) identity.browserid.dev.linux.firefox.13 | |
| * GIT CLONE ERROR: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.firefox.13/1089/console | |
| * GIT CLONE ERROR: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.firefox.13/1086/console | |
| 83% (5/30) identity.browserid.dev.windows.7.ie.9 | |
| * GIT CLONE ERROR: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.windows.7.ie.9/1092/ | |
| * NEXT CLICK INEFFECTUAL: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.windows.7.ie.9/1095/ | |
| * INITIAL LOAD OF INCLUDE.JS FAILS: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.windows.7.ie.9/1072/ | |
| * GIT CLONE ERROR: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.windows.7.ie.9/1085/ | |
| * GIT CLONE ERROR: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.windows.7.ie.9/1088/ | |
| 73% (8/30) identity.browserid.dev.linux.opera.12 | |
| * (NETWORK) LOAD OF INCLUDE.JS FAILS: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.opera.12/1121/ | |
| * TITLE EXTRACTION IN POLLING FAILS: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.opera.12/1119/ | |
| * GIT CLONE ERROR: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.opera.12/1111/ | |
| * (NETWORK) load of 12done fails: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.opera.12/1106/ | |
| * (NETWORK) load of 12done fails: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.opera.12/1108/ | |
| * (NETWORK) load of 12done fails: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.opera.12/1103/ | |
| * (NETWORK) load of 12done fails: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.opera.12/1102/ | |
| * (NETWORK) load of 12done fails: https://ci.mozilla.org/view/Persona/job/identity.browserid.dev.linux.opera.12/1098/ | |
| Summary: | |
| 17/210 test groups fail - 80% success rate | |
| Broken down by failure categories: | |
| (7) 41% are ci.mozilla.org unable to clone from github | |
| (6) 35% are network connectivity on linux/opera | |
| (2) 12% are uninvestigated issues in IE9 | |
| (1) 6% are uninvestigated errors in linux/opera | |
| (1) 6% are internal saucelabs errors | |
| NOTE: each of these 210 "tests" are actually four distinct sub-tests (4 different saucelabs VMs spun up), |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm investigating the github errors, though it may just have been a github internal burp (all the affected jobs ran between 6:40 and 9:10 am this morning).
Opera isn't a focus leading up to beta 1, I'd suggest we defer troubleshooting those tests for the moment. Cosimo at Opera pinged me offering to help, and I owe him an explanation of the avoid-proxy stuff.
That leaves 4 tests out of 210, which puts us in 98% green range. I'm good to let the matter drop for now.