gistfile1.txt

tldr; make a db connection           `sis <-> sis automation suite <-> canvas`
we can test end to end and every combination of steps in between.
************
if we use beta or test or something for canvas and sis app, we can get a head start
************
the alternative i can see is fine but it has several downsides.  maybe i'm not seeing something even more easy, though.

exp test case:

- cause error in grade passback w/ pinnacle by leaving sis id off of a user
- expect appropriate error message to appear in canvas, via message from sis app, in the grade sync tab```


possible implementation for automation (with out doing something new):
* i'm not sure how to have all these orchestrated to start with jenkins job unless i finish the docker stuff or we wait for the new frame work or we do the test instances, like above
```
- create assignment(s) and grade them (valid sis data and settings)
- push buttons in canvas, inside sis app lti.  pause before submitting
- delete sis id from student via api call (if possible)
- do a grade passback through the lti tool # happy path is fine
- visit canvas' accounts/:account_id/settings/grade/external_tools/:external_tools_id
- check page with selenium
```
all that relies on 3 instances and is what we need currently.

downsides:
- that should be many separate specs, not 1
- frail
- slow


possible implementation with modification to the instances.  

--- modify the instances to have a database/ActiveRecord connection ---
- seed canvas db with accurate data # * see notes about "accurate data"
- create assignment(s) and grade them (valid sis data and settings) in ActiveRecord # setup steps, not testing
- grade students # ActiveRecord again.  We aren't testing canvas.  we only care about GPB right now
- push buttons in the gradebook and lti tool in canvas
- delete student sis id from database
- push the submit button
- visit canvas' accounts/:account_id/settings/grade/external_tools/:external_tools_id
- check page with selenium

downsides:
- make sure we think when we're building the framework
- get ops to open up ports on instances and give us test or beta or something + 1 # i'll sacrifice my beta instance

upsides:
- faster
- we can check every aspect of the error message, even parts we don't expect to see in the UI.  that allows us to mutate agents and do lots of other edge cases and assure that NOTHING is happening.
- we can avoid a lot of un-needed selenium, that won't be a part of what we're testing.
- it opens up a lot of edge cases.  we can let good data get inserted and change it, then expect correct handling.  that's not a thing right now

* Accurate data can be gathered in a scheduled job.  the job could run on the test app's instance. 
it gets valid data from all the agents, whenever we think it should (daily?, weekly?) # setup steps, not testing.