DVR conversation

gistfile1.txt

Oki, I'll call it the InsaneHypermediaApiProfileTester.
dn: Stay for me for a second :-)
Jared Ning: :)
dn: let's say we already got the HC client
dn: we now build the IHAPT Gem
dn: hc = HC.connect('localhost:9393')
dn: IHAPT.build_profile hc
dn: def build_profile
   # visit '/'
   # visit all forms & record inputs/required etc
end
dn: (that's the simple version, in theory you also want to call the actions once)
dn: IHAPT.check_profile 'ss_profile', hc
dn: done
dn: check_profile just verifies that all previous forms exists and all inputs etc. still exists
dn: it might show a warning if something is new and explodes if something is missing
Jared Ning: So you have a gem that is external to any service. It makes requests and verifies the responses.
dn: Yes, and we just add the gem to group 'test' of e.g. SS
dn: then SS itself!!! can test if it works/changes
Jared Ning: Like CI for APIs
dn: YES
dn: We still have to find a good way to also check the responses, not just the forms/inputs.
dn: But I can see us doing two things:
dn: wait
dn: It's not a problem anymore!
dn: since we are testing it WITHIN the app itself, we have FIXTURES!
dn: woha!
dn: :)
Jared Ning: So you could stub responses with the fixtures in the testing gem
dn: Not sure if I get that right (language fun).
dn: I would say then SS has a fixture for a Search and it's segments.
dn: The test gem would just call /recieve-results/#{fixture.id}
dn: and can apply the same logic
dn: this thingie should always return the data in the same way (structure) it might add something, but it never removes something
Jared Ning: Not sure I totally understand either, but I think the important thing is that we have something that is capable of verifying responses
dn: Shall I try again?
Jared Ning: Sure.
dn: Does the build_profile thingie make sense for you for the forms and their possible fields?
dn: (that's verifying the input the api expects)
Jared Ning: hold on a sec
Jared Ning: Yes. Forms/fields are easy to verify.
dn: *idle mode until you say*
Jared Ning: Go on.
dn: Imho the response is just the same. I think it's even easier.
dn: Given we are within the SS test code and have included the HypermediaTestGem
dn: I can easily do the following
dn: search = Factory.create(:search_with_segments)
dn: profile = IHMTG.make_profile_of_response recieve_results_path(search.id)
dn: (it will only create a profile if it does not exist yet)
dn: IHMTG.verify_profile profile, recieve-results_path(search.id)
dn: done
dn: make sense? if not please complain. I'm sometimes very optimistic :)
Jared Ning: So are you comparing the response from making the request from IHMTG and the response you get from within the service doing rr_path(search.id)?
dn: yes, first time it will always pass obviously, but any change that is a breaking change will not work.
Jared Ning: So it's like a hypermedia way version of VCR
dn: Yes, but a way smarter. Because it's comparing the bodies and not just by line - it understands 'json' and it understands what is important and what not.
Jared Ning: I get it.
dn: So the question is, does it make sense and maybe I'm missing something. But if we have such a system the 'burden' of making a lot of small service imho goes down A LOT
Jared Ning: But this is only good for public API responses that aren't allowed to change much.
dn: Why? The API itself can change a lot - it doesn't matter.
dn: It's just preventing breaking! changes - not changes at all.
dn: adding a new field, won't make it break
Jared Ning: Oh, ok, I see what you're saying.
Jared Ning: I'll bet we run into problems with dates and stuff.
dn: Adding a new endpoint is also easy - build it, record it, publish it (intern or external) - and you are save.
dn: I don't think so. We wouldn't do that
dn: I would test it like the following:
dn: a lot of kind_of?
dn: e.g. Is the root element Array or Hash
dn: (test that for each response object
dn: there aren't many objects, because we are talking about JSON)
dn: then we can still define custom rules like
Jared Ning: That makes sense. Test the structure of the response, not the details
dn: if it's marked es date field id should be in the stftime format %Y%M%DT%H%M%Z
dn: yeap
dn: and if we have to verify a type to be in a certain way it's more like a rails validation
dn: we don't test the exact date (we could, since we use fixtures)
dn: but we don't have to
dn: because we don't care
dn: and because we tested (before) that SS stores the dates in the right way
dn: no need to verify that again