To-Do List
Major Problems:
- PROB-1: Performance of main generate task
- The main generate task is barely keeping up with the incoming reports. A cursory examination reveals nothing that can be easily improved without parallelization.
- Solution: Refactor into a Minion task that can be parallelized using Beam::Minion
- Critical chain: BACKEND-1 API-3 BACKEND-2
- PROB-2: AmazonDB SimpleDB Metabase is expensive (money wise)
- Solution: Make new API to write to local Metabase cache