experimentation.md

1. Remove right rail scrollbar. This enables the subsequent experiments and gives us right-rail content that will work on any page, regardless of experiment outcomes. This also gives us what we need for air-gapped environments.

   • Implement new version of "in this article". Shows first N sections, and remaining count.

     N should be a little flexible so that we never end up in a state where it shows less than "and 2 more".

     In this article:
     - Lorem
     - Ipsum
     - Sumit
     - Dolar
     - And 5 more...
     

   • Introduce Generic Q&A link (except in air gapped envs)

2. Experiment: Recommendations

   Question: Do recommendations perform better than "in this article"?

   Outcome: Determines whether we should retire "in this article".

   Description: On a subset of high-traffic pages which have recommendations, use target to decide whether to display the new "in this article" experience from step 1, or recommendations.

3. Implement presumed winner of the recommendations experiment... if an article supports recommendations, show them, otherwise use the new "in this article" experience.

4. Experiment: Specific Q&A

   Question: Does a link to an on-topic Q&A perform better than a generic Q&A link?

   Outcome: Determines whether investment in Q&A recommendations makes sense.

5. Experiment: Learn recos

   Question: Does a link to an on-topic learning resources perform better vs ??????

   Outcome: Determines whether investment in learn recommendations makes sense.

6. Move tier 2 events to rail

7. Experiment: Alternate tier 2 event design

   Question: Which card design for events works best.

   Outcome: Docs event referral performance does not suffer.

8. Implement event design winner.

9. Experiment: right rail card design