oam-roadmap.md

These are some opiniated ideas for improvements to OAM/OIN. They are first of all straw men: merely ideas to stimulate conversation. My guiding principles here are the general software engineering best practices I've come to appreciate over the years. Therefore there is little here in terms of new features and little that an end-user would explicitly notice. They concern the health of the code: reduction of regression bugs, ease and confidence of refactoring, reduction of the learning curve for newcomers and contributors, scalability and browser performance.

These are of course good things in any circumstances. Though I have also borne in mind HOT's plans to promote and expand OAM beyond its current audience and vision.

Github Improvements

Contribution policies

• Always rebase, rather than merge, by default on github. This makes the Git logs more readable.
• PRs must include new/amended test cases.
• master as the default branch, with staging and production reflecting increasing stability.

Consolidate Repos

As acknowledged by Development Seed (the original OAM devs), OAM suffers from repo fragmentation. I propose that OAM merge itself into essentially 2 main repos for the front and back end:

Frontend

• hotosm/oam-frontend will contain:
  • OpenAerialMap
  • openaerialmap.org
  • oam-browser
  • oam-uploader
  • oam-docs
  • oam-uploader-admin
• Consolidation of subdomains to /docs, /upload, etc.

Backend

• hotosm/oam-api will contain:
  • oam-catalog
  • oam-uploader-api
  • oam-dynamic-tiler
• hotosm/oam-deployer (new)

Misc

• Remain the same:
  • hotosm/oam-qgis-plugin
  • hotosm/oam-design-system
• deprecate all server-*repos* into a single hotosm/archives repo, including their commit histories.

Code Improvements

Frontend And Backend

• Linting with prettier: https://github.com/prettier/prettier.
• Cyclic complexity and method length lint rules.
• Test coverage over 75%, no Github merge if branch goes below that.
• No 'magic numbers' or config commited to git, only use ENV files using dotenv.

Frontend

• Switch to Webpack and Yarn.
• Webpack has good code splitting support, so only portions of JS are downloaded as needed.
• Single package.json for all repos will make it easier to upgrade packages.
• Single React dependencies, eg; only Redux for state management.
• Single XHR lib, eg; window.fetch.
• Use modern browser features by default. Serve an extra polyfill bundle for any browser that misses a feature, see: https://philipwalton.com/articles/loading-polyfills-only-when-needed/
• No jquery.
• Use browserHistory URLs instead of hashHistory hash URLs. This will mean moving off Github for hosting.
• JSX in seperate files.
• Lightweight, DOM-independent tests (stubbing out window/document features), most likely using the enzyme framework.
• End to end integration tests with the entire, development-state stack across multiple browsers using Sauce Labs/Browserstack.

Backend

• Only one server-side language: node 6.x (a bit of BASH of course). No python, etc.
• Plaform-specific (AWS/Heroku) code can only appear in hotosm/oam-deployer.
• Entire stack should be capable of running locally without Docker.
• Follow https://12factor.net principles as closely as possible allowing deployment to be as platform agnostic as possible.
• Use https://flynn.io or https://kubernetes.io

Open Imagery Network Suggestions

I don't feel like I know OIN well enough to describe these as any more than suggestions. Nevertheless, in the name of stimulating conversation, here are some thoughts. It occurs to me that the consolidation of the repos into front and backend, resonates with the distinction between OAM and OIN: OAM is more the client and OIN is more the server. As I currently see it there are some scalability hurdles associated with defining OIN as just a metadata format for imagery. Firstly it creates a single point of responsibilty in the Catalog API: it is a master node, rather than a peer. In the extreme case, if the Catalog API goes down, so does all of OIN. Secondly it makes imagery updates an O(N) rather than an O(1) process: the Catalog API constantly needs to query s3-compatible bucket endpoints to get the latest list of available imagery and changes to imagery metadata. Thirdly it reinforces implementation-specific architectural decisions: in our case, namely AWS and particularly AWS lambda.

I propose that OIN instead be defined by a minimal set of RESTful API methods: GET /image, GET /search (by metadata, geo reference) and GET /transaction_log (log of updates). This way the current OAM backend becomes the reference implementation of OIN. This would not negate the need for the current OIN metadata format, but rather suplement it. The OIN could then become a more diverse and robust ecosystem, with a mixture of buckets, independent sources and full-on mirrors. Contributors I imagine would mostly use our OIN reference implementation, but in theory they would be free to implement their own infrastructure that conformed to the OIN API protocol.