This doc contains some initial dev feedback as a first-time SDK app developer. Some of the feedback might be in the form of confused questions, which could give some ideas about usability of the SDK.
My feedback generally fits into a few categories:
- Faster testing turnaround
- Code mixing/stubbing and package management
- Config consolidation
- Docs on SDK apps
Things I liked:
- Defining a spec for the app with KIDL
- kb-sdk validate
- Having kb-sdk as a cli tool
- Dockerizing everything so all is reproducible
- Composition of other SDK apps
Testing the app is relatively slow -- it would be nice to support TDD with faster turnaround.
- Most tests upload and download files from your local disk. Could we stub workspace downloads and uploads so they don't actually do any network requests? Could we cache downloaded files?
- Instead of having to re-build the docker container on every test run, could we run tests within an existing running instance of the app's docker container?
Way I've found to support TDD is to create separate unittest tests that bypass kb-sdk and any docker containers and run python modules that don't depend on workspace downloads/uploads.
It's difficult to tweak the input UI of your app (especially while figuring out parameters), and it's difficult to test output reports.
- Could we have a report preview utility that opens a browser tab with the report view (eg. the
webbrowserpackage) - It would be ideal to actually run the narrative server locally with your SDK app available in a local catalog while you tweak your app, but I realize this is far off.
It's confusing to have a lot of auto-generated code mixed into the same packages and modules as your own app-specific code. What happens if I modify this code and there is an update with kb_sdk that requires me to regenerate this code? Is this code versioned? It's initially confusing to know which pieces of what files the app developer is meant to edit.
A clearer division between these parts of the app would be helpful:
- Auto-generated build code (not committed, separate dir, not meant to be edited)
- Actual app-specific code (and test code) -- separate python package dir -- app author writes and edits everything in this pacakge.
- Dependencies and configuration
For example, there is a lot of repeated, autogenerated *Client code in /lib for app dependencies that's mixed into your own module. Instead of auto-generating and committing python source code and mixing it into your packages, could you convert that code into discrete pip packages and install stuff like client classes, test config, etc using pip packages?
Regarding the SDK dependencies (ie. kb-sdk install x):
- Why do these dependencies inject code into the app author's packages and tests? Why is this injected code committed to the repo?
- Do SDK app versions get locked? Are they signed or get checksummed when they're installed? What happens to my repo when I upgrade an SDK app? Do they use semantic versioning? What happens when an SDK app gets a backwards-incompatible update?
In general it might be friendlier and more scaleable to use more package management with pip to import python functionality, rather than autogenerating a lot of code into the code base.
Updating a single parameter of the app requires updating several config files, all with different formats. It might be worth consolidating data found in spec.json, display.yml, kbase.yml, and dependencies.json into fewer files.
Almost all SDK apps seem to have blank READMEs. It would be useful to have API references for every SDK app, especially ones that are re-used a lot when making another app. Could we use something like Sphinx to auto-generate doc pages for every SDK app?
Misc. docs I would have liked:
- Simple overview of how the SDK apps work
- Extend the existing tutorial to be more have more instructions on how to build an app, rather than just being a description of pre-existing code
- A HOWTO collection on stuff like: using scratch, using the workspace, generating html reports, attaching files, etc.
- It seems like KIDL could be used to auto-generate simple runtime python type checks
- The default python version 2.7.6 is very old -- had a hard time using certain pip packages
- When an app is in dev mode on appdev, can it pull the latest commit automatically rather than reregistering on every push?