For our upcoming SSO features, we need a way for a user to retrieve new episodes for their Favorite shows through the KPCC app. To sum up our requirements:
- Users should be able to retrieve up to n most recent episodes from their favorite shows per download.
- Episodes downloaded to a user's queue should sync up across devices
- Finished or deleted episodes in the queue should not be re-downloaded.
Through proofing out some approaches, I came to the conclusion that the simplest approach is the right one for what we want to achieve.
My proposal is to lazily load episodes on our devices, as opposed to having background processes(replicators, crons, etc.) that we would need to maintain and could pose as a single point of failure. I believe that this is the correct approach no matter what technology we choose.
- The user opens the app for the first time and is asked to select some favorite shows. If they already have an existing account(say the user just switched to a new phone), the app would use the mobile dataset SDK to retrieve their favorites, queue, history, and all their existing settings. This shouldn't be more complicated than calling the synchronize function on the SDK.
- The app then contacts the SCPRv4 API and asks for a list of the last n episodes from the user's favorite shows(by providing an array of object_ids and the value for n). This is up to n episodes per-show. n is determined by a setting in the app. A
last_fetched_attimestamp would also be provided so that the API would only return episodes that have been published since the last time the app had downloaded new episodes; this way, the same episodes should never be re-downloaded. - The API returns a structure containing an array of episodes. To make things as simple as possible, each "episode" can be a tuple; the first item in the tuple would be the object_id and the last item would be a JSON for the actual episode data. The first item of the tuple would be the key for the new record in the dataset, and the last item of the tuple would be the value. We could include lots of episode information, but for our purposes we only need the title, url, and the content_type(this might be implicit in our case). NOTE: This is making a big assumption about how we'd want our API to return data. We may not decide to have the API return data in that structure, but the process should be very similar no matter what we decide to have the API return.
- The next time a user interaction(such as onStart) happens, the synchronize function in the SDK could be called to sync with their remote dataset. This way, changes to their queue get pushed to our end so the user can access this data through other apps(such as SCPRv4).
There are a few ideas currently in mind to account for cases where we need to edit episode metadata or delete an episode on all devices.
- Tie Cognito with Redshift through Kinesis so that we can store all synched data in a way that we can parse it to know what users to target for an update(users that have favorited the show of an episode we need to update.
- Go through each of the datasets for all users that have obtained that episode and delete the records for the episode. The next time that a user syncs, the records should be removed from their devices.
- Optionally, use Lambda to send a silent notification to a device in order to trigger a synchronization.
- When we change information about an episode, this sends a message to an Amazon SQS message queue. This could contain the exact changes, including the method(say DELETE).
- On start, the app contacts the message queue and asks for new messages. It receives recent messages telling it what to do, including the message that says to delete a specific episode. The app carries out the delete on the records pertaining to the episode.
- The message in the SQS queue is given a TTL so that it will expire after a while and messages don't pile up.
What happens if an episode title gets changed in SCPRv4, or an episode gets unpublished? - ewr