[napkin sketch]
-
Create 1-N test bot accounts that follow different subsets of each other. Have them post predictable content on a regular schedule.
-
Monitor the bot posting software. Have it report errors and latency, and store the ID of each post when it's generated.
-
Write a client that, authenticated as each bot, fetches various permutations of the bot's home timeline with/without min_id/max_id/limit set. Because each bot's posts are predictable you can predict the timeline contents for each query, using the status IDs stored earlier. Have the client report good/bad results and latency to the monitoring system.
-
Alert on errors, and if latency exceeds some threshold.