Skip to content

Instantly share code, notes, and snippets.

@dmargala

dmargala/how-many-critics-does-it-take-to-rank-all-the-movies.ipynb

Created July 9, 2017 07:28
Show Gist options
  • Save dmargala/d74215ca61a6268ca95dd431a3de4b54 to your computer and use it in GitHub Desktop.
Save dmargala/d74215ca61a6268ca95dd431a3de4b54 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
how-many-critics-does-it-take-to-rank-all-the-movies.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"https://fivethirtyeight.com/features/how-many-critics-does-it-take-to-rank-all-the-movies/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You run a film magazine, Groovy Movies, and you have been invited to attend a new film festival. The festival organizers will screen 30 films evenly distributed across three different screens. Each film will premiere at this festival, and you want to get the scoop on which one was the best. The problem is, though, that because there are three screens, you don’t know which screen will show the best film. You could watch only Screen A, see the best movie there and report on it, but it may not be as good as one of the movies on Screen B or C.\n",
"\n",
"Some more details you know from your many years of experience in the magazine biz:\n",
"\n",
" * Whenever a film is playing on one screen, the other two screens also have a film playing. But there is enough time between each movie that one person can always watch the nth round on one screen and the n+1th round on another screen.\n",
" \n",
" * The best movie on one screen will never play at the same time as the best movie on another screen. However, you don’t know what time slots they will occupy for each theater.\n",
" \n",
" * All of your reviewers are good rankers — they won’t have any disagreement about which movies are better than others that they’ve seen. (They’re ordinal reviewers, in other words.)\n",
" \n",
" * That said, all of your reviewers are terrible raters, so they cannot give an objective measure of how good a movie was (a 9 out of 10, say) and compare it to another reviewer’s measure of how good a different movie was. (To put it another way: They aren’t cardinal reviewers.)\n",
" \n",
" * With all that in mind, if you want to know for sure what the best film at the festival was, what is the minimum number of reviewers you would need to send to the festival?\n",
"\n",
"Extra credit: What if there were more movies shown per screen? What if there were more screens?"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"class Festival(object):\n",
" def __init__(self, num_screens, num_films_per_screen):\n",
" self.num_screens = num_screens\n",
" self.num_films_per_screen = num_films_per_screen\n",
" self.num_films = num_screens * num_films_per_screen\n",
" \n",
" self.time_slots = range(num_films_per_screen)\n",
" \n",
" self.screens = None\n",
" \n",
" def _get_random_films(self):\n",
" return np.random.permutation(self.num_films).reshape(self.num_films_per_screen, -1)\n",
" \n",
" def _are_time_slots_valid(self):\n",
" best_per_screen = np.argmin(self.screens, axis=0)\n",
" return len(set(best_per_screen)) == self.num_screens\n",
" \n",
" def randomize_films(self, seed=None):\n",
" if seed:\n",
" np.random.seed(seed)\n",
" self.screens = self._get_random_films()\n",
" while not self._are_time_slots_valid():\n",
" self.screens = self._get_random_films()\n",
" \n",
" def lookup_films(self, schedule):\n",
" return self.screens[self.time_slots, schedule]\n",
" \n",
" def rank_films(self, films):\n",
" # returns the rank of each film\n",
" return np.argsort(np.argsort(films))\n",
" \n",
" def get_argmin(self, screens_like):\n",
" return np.unravel_index(screens_like.argmin(), self.screens.shape)\n",
" \n",
" def evaluate_schedules(self, schedules, debug=False):\n",
" \n",
" rankings = np.ma.masked_all((\n",
" len(schedules), \n",
" self.num_films_per_screen, \n",
" self.num_screens\n",
" ), dtype=int)\n",
"\n",
" for i, schedule in enumerate(schedules):\n",
" films = self.lookup_films(schedule)\n",
" ranking = self.rank_films(films)\n",
" \n",
" if debug:\n",
" print('sched : {}'.format(schedule))\n",
" print('films : {}'.format(films.tolist()))\n",
" print('rank : {}'.format(ranking.tolist()))\n",
" print('')\n",
"\n",
" rankings[i, self.time_slots, schedule] = ranking\n",
" \n",
" return rankings"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Simple two screen test"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"test_fest = Festival(2, 6)\n",
"test_fest.randomize_films(seed=1234)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 7, 11],\n",
" [ 2, 3],\n",
" [ 1, 9],\n",
" [ 5, 4],\n",
" [ 0, 8],\n",
" [ 6, 10]])"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Each value represents the true rank of a film playing on a screen at given time slot\n",
"# The index of the value corresponds to (time slot, screen)\n",
"test_fest.screens"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(4, 0)"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# The best film is playing at time slot 4 on screen 0\n",
"test_fest.get_argmin(test_fest.screens)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# a screening schedule for a reviewer is specified as a sequence \n",
"# of length `num_films_per_screen` with values specifying the screen index\n",
"\n",
"# a simple schedule\n",
"test_schedule = (0, 0, 0, 1, 1, 1)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([ 7, 2, 1, 4, 8, 10])"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# lookup films seen by a reviewer following a schedule\n",
"test_fest.lookup_films(test_schedule)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([3, 1, 0, 2, 4, 5])"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# get the reviewers ranking of the films they saw\n",
"test_fest.rank_films(test_fest.lookup_films(test_schedule))"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"test_schedules1 = [\n",
" (0, 0, 0, 0, 0, 0),\n",
" (1, 1, 1, 1, 1, 1),\n",
"]\n",
"\n",
"test_schedules2 = [\n",
" (0, 0, 0, 0, 0, 0),\n",
" (1, 0, 0, 0, 0, 0),\n",
" (1, 1, 0, 0, 0, 0),\n",
" (1, 1, 1, 0, 0, 0),\n",
" (1, 1, 1, 1, 0, 0),\n",
" (1, 1, 1, 1, 1, 0),\n",
" (1, 1, 1, 1, 1, 1),\n",
"]\n",
"\n",
"test_schedules3 = [\n",
" (0, 0, 0, 0, 0, 0),\n",
" (1, 0, 0, 0, 0, 0),\n",
" (0, 1, 0, 0, 0, 0),\n",
" (0, 0, 1, 0, 0, 0),\n",
" (0, 0, 0, 1, 0, 0),\n",
" (0, 0, 0, 0, 1, 0),\n",
" (0, 0, 0, 0, 0, 1),\n",
" (1, 1, 1, 1, 1, 1),\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"sched : (0, 0, 0, 0, 0, 0)\n",
"films : [7, 2, 1, 5, 0, 6]\n",
"rank : [5, 2, 1, 3, 0, 4]\n",
"\n",
"sched : (1, 0, 0, 0, 0, 0)\n",
"films : [11, 2, 1, 5, 0, 6]\n",
"rank : [5, 2, 1, 3, 0, 4]\n",
"\n",
"sched : (0, 1, 0, 0, 0, 0)\n",
"films : [7, 3, 1, 5, 0, 6]\n",
"rank : [5, 2, 1, 3, 0, 4]\n",
"\n",
"sched : (0, 0, 1, 0, 0, 0)\n",
"films : [7, 2, 9, 5, 0, 6]\n",
"rank : [4, 1, 5, 2, 0, 3]\n",
"\n",
"sched : (0, 0, 0, 1, 0, 0)\n",
"films : [7, 2, 1, 4, 0, 6]\n",
"rank : [5, 2, 1, 3, 0, 4]\n",
"\n",
"sched : (0, 0, 0, 0, 1, 0)\n",
"films : [7, 2, 1, 5, 8, 6]\n",
"rank : [4, 1, 0, 2, 5, 3]\n",
"\n",
"sched : (0, 0, 0, 0, 0, 1)\n",
"films : [7, 2, 1, 5, 0, 10]\n",
"rank : [4, 2, 1, 3, 0, 5]\n",
"\n",
"sched : (1, 1, 1, 1, 1, 1)\n",
"films : [11, 3, 9, 4, 8, 10]\n",
"rank : [5, 0, 3, 1, 2, 4]\n",
"\n"
]
}
],
"source": [
"test_rankings = test_fest.evaluate_schedules(test_schedules3, debug=True)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"masked_array(data =\n",
" [[4.5 5.0]\n",
" [1.6666666666666667 1.0]\n",
" [0.8333333333333334 4.0]\n",
" [2.6666666666666665 2.0]\n",
" [0.0 3.5]\n",
" [3.6666666666666665 4.5]],\n",
" mask =\n",
" [[False False]\n",
" [False False]\n",
" [False False]\n",
" [False False]\n",
" [False False]\n",
" [False False]],\n",
" fill_value = 1e+20)"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# one way to merge the rankings to take the average ranking at each (time slot, screen)\n",
"np.ma.average(test_rankings, axis=0)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(4, 0)"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# pluck the minumum from that\n",
"test_fest.get_argmin(np.ma.average(test_rankings, axis=0))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Test two screen strategy"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def two_screen_strategy(num_films_per_screen):\n",
" schedules = []\n",
"\n",
" schedules.append(num_films_per_screen * (0, ))\n",
"\n",
" for i in range(num_films_per_screen):\n",
" schedules.append(i*(0, ) + (1, ) + (num_films_per_screen-i-1)*(0,))\n",
"\n",
" schedules.append(num_films_per_screen * (1, ))\n",
"\n",
" return schedules"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[(0, 0, 0, 0, 0, 0),\n",
" (1, 0, 0, 0, 0, 0),\n",
" (0, 1, 0, 0, 0, 0),\n",
" (0, 0, 1, 0, 0, 0),\n",
" (0, 0, 0, 1, 0, 0),\n",
" (0, 0, 0, 0, 1, 0),\n",
" (0, 0, 0, 0, 0, 1),\n",
" (1, 1, 1, 1, 1, 1)]"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"two_screen_strategy(6)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)\n",
"(1, 0, 0, 0, 0, 0, 0, 0, 0, 0)\n",
"(0, 1, 0, 0, 0, 0, 0, 0, 0, 0)\n",
"(0, 0, 1, 0, 0, 0, 0, 0, 0, 0)\n",
"(0, 0, 0, 1, 0, 0, 0, 0, 0, 0)\n",
"(0, 0, 0, 0, 1, 0, 0, 0, 0, 0)\n",
"(0, 0, 0, 0, 0, 1, 0, 0, 0, 0)\n",
"(0, 0, 0, 0, 0, 0, 1, 0, 0, 0)\n",
"(0, 0, 0, 0, 0, 0, 0, 1, 0, 0)\n",
"(0, 0, 0, 0, 0, 0, 0, 0, 1, 0)\n",
"(0, 0, 0, 0, 0, 0, 0, 0, 0, 1)\n",
"(1, 1, 1, 1, 1, 1, 1, 1, 1, 1)\n",
"\n",
"All good! Passed 1000 random trials. \n",
"Num reviewers used: 12\n"
]
}
],
"source": [
"num_films_per_screen = 10\n",
"\n",
"# create schedules\n",
"schedules = two_screen_strategy(num_films_per_screen)\n",
"\n",
"for schedule in schedules:\n",
" print(schedule)\n",
"print('')\n",
"\n",
"# run random trials\n",
"fest = Festival(2, num_films_per_screen)\n",
"\n",
"num_trials = 1000\n",
"all_good = True\n",
"\n",
"for i in range(num_trials):\n",
" fest.randomize_films(seed=i)\n",
" truth = fest.get_argmin(fest.screens)\n",
" \n",
" rankings = fest.evaluate_schedules(schedules)\n",
" \n",
" merged_rankings = np.ma.average(rankings, axis=0)\n",
" \n",
" guess = fest.get_argmin(merged_rankings)\n",
" \n",
" # check if minimum is unique\n",
" count = np.count_nonzero(merged_rankings == merged_rankings[guess])\n",
" \n",
" if count > 1 or truth != guess:\n",
" all_good = False\n",
" print(count)\n",
" print('Screens:')\n",
" print(fest.screens.T)\n",
" print('')\n",
" print('Rankings:')\n",
" print(rankings.transpose((0, 2, 1)))\n",
" print('')\n",
" print('Merged Rankings:')\n",
" print(merged_rankings.T)\n",
" print(truth, guess)\n",
" print('')\n",
" break\n",
" \n",
"if all_good:\n",
" print(\"All good! Passed {:d} random trials. \".format(num_trials))\n",
" print(\"Num reviewers used: {:d}\".format(len(schedules)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Extend to three screens"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"test3_fest = Festival(3, 6)\n",
"test3_fest.randomize_films(seed=512)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 5, 14, 10],\n",
" [ 4, 0, 16],\n",
" [ 6, 9, 1],\n",
" [ 2, 15, 3],\n",
" [17, 12, 8],\n",
" [ 7, 11, 13]])"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"test3_fest.screens"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1, 1)"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"test3_fest.get_argmin(test3_fest.screens)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"test3_schedules1 = [\n",
" (0, 0, 0, 0, 0, 0),\n",
" (1, 1, 1, 1, 1, 1),\n",
" (2, 2, 2, 2, 2, 2),\n",
"]\n",
"\n",
"test3_schedules2 = [\n",
" (0, 0, 0, 0, 0, 0),\n",
" (1, 0, 0, 0, 0, 0),\n",
" (0, 1, 0, 0, 0, 0),\n",
" (0, 0, 1, 0, 0, 0),\n",
" (0, 0, 0, 1, 0, 0),\n",
" (0, 0, 0, 0, 1, 0),\n",
" (0, 0, 0, 0, 0, 1),\n",
" (2, 0, 0, 0, 0, 0),\n",
" (0, 2, 0, 0, 0, 0),\n",
" (0, 0, 2, 0, 0, 0),\n",
" (0, 0, 0, 2, 0, 0),\n",
" (0, 0, 0, 0, 2, 0),\n",
" (0, 0, 0, 0, 0, 2),\n",
" (1, 1, 1, 1, 1, 1),\n",
" (2, 1, 1, 1, 1, 1),\n",
" (1, 2, 1, 1, 1, 1),\n",
" (1, 1, 2, 1, 1, 1),\n",
" (1, 1, 1, 2, 1, 1),\n",
" (1, 1, 1, 1, 2, 1),\n",
" (1, 1, 1, 1, 1, 2),\n",
" (2, 2, 2, 2, 2, 2),\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"sched : (0, 0, 0, 0, 0, 0)\n",
"films : [5, 4, 6, 2, 17, 7]\n",
"rank : [2, 1, 3, 0, 5, 4]\n",
"\n",
"sched : (1, 0, 0, 0, 0, 0)\n",
"films : [14, 4, 6, 2, 17, 7]\n",
"rank : [4, 1, 2, 0, 5, 3]\n",
"\n",
"sched : (0, 1, 0, 0, 0, 0)\n",
"films : [5, 0, 6, 2, 17, 7]\n",
"rank : [2, 0, 3, 1, 5, 4]\n",
"\n",
"sched : (0, 0, 1, 0, 0, 0)\n",
"films : [5, 4, 9, 2, 17, 7]\n",
"rank : [2, 1, 4, 0, 5, 3]\n",
"\n",
"sched : (0, 0, 0, 1, 0, 0)\n",
"films : [5, 4, 6, 15, 17, 7]\n",
"rank : [1, 0, 2, 4, 5, 3]\n",
"\n",
"sched : (0, 0, 0, 0, 1, 0)\n",
"films : [5, 4, 6, 2, 12, 7]\n",
"rank : [2, 1, 3, 0, 5, 4]\n",
"\n",
"sched : (0, 0, 0, 0, 0, 1)\n",
"films : [5, 4, 6, 2, 17, 11]\n",
"rank : [2, 1, 3, 0, 5, 4]\n",
"\n",
"sched : (2, 0, 0, 0, 0, 0)\n",
"films : [10, 4, 6, 2, 17, 7]\n",
"rank : [4, 1, 2, 0, 5, 3]\n",
"\n",
"sched : (0, 2, 0, 0, 0, 0)\n",
"films : [5, 16, 6, 2, 17, 7]\n",
"rank : [1, 4, 2, 0, 5, 3]\n",
"\n",
"sched : (0, 0, 2, 0, 0, 0)\n",
"films : [5, 4, 1, 2, 17, 7]\n",
"rank : [3, 2, 0, 1, 5, 4]\n",
"\n",
"sched : (0, 0, 0, 2, 0, 0)\n",
"films : [5, 4, 6, 3, 17, 7]\n",
"rank : [2, 1, 3, 0, 5, 4]\n",
"\n",
"sched : (0, 0, 0, 0, 2, 0)\n",
"films : [5, 4, 6, 2, 8, 7]\n",
"rank : [2, 1, 3, 0, 5, 4]\n",
"\n",
"sched : (0, 0, 0, 0, 0, 2)\n",
"films : [5, 4, 6, 2, 17, 13]\n",
"rank : [2, 1, 3, 0, 5, 4]\n",
"\n",
"sched : (1, 1, 1, 1, 1, 1)\n",
"films : [14, 0, 9, 15, 12, 11]\n",
"rank : [4, 0, 1, 5, 3, 2]\n",
"\n",
"sched : (2, 1, 1, 1, 1, 1)\n",
"films : [10, 0, 9, 15, 12, 11]\n",
"rank : [2, 0, 1, 5, 4, 3]\n",
"\n",
"sched : (1, 2, 1, 1, 1, 1)\n",
"films : [14, 16, 9, 15, 12, 11]\n",
"rank : [3, 5, 0, 4, 2, 1]\n",
"\n",
"sched : (1, 1, 2, 1, 1, 1)\n",
"films : [14, 0, 1, 15, 12, 11]\n",
"rank : [4, 0, 1, 5, 3, 2]\n",
"\n",
"sched : (1, 1, 1, 2, 1, 1)\n",
"films : [14, 0, 9, 3, 12, 11]\n",
"rank : [5, 0, 2, 1, 4, 3]\n",
"\n",
"sched : (1, 1, 1, 1, 2, 1)\n",
"films : [14, 0, 9, 15, 8, 11]\n",
"rank : [4, 0, 2, 5, 1, 3]\n",
"\n",
"sched : (1, 1, 1, 1, 1, 2)\n",
"films : [14, 0, 9, 15, 12, 13]\n",
"rank : [4, 0, 1, 5, 2, 3]\n",
"\n",
"sched : (2, 2, 2, 2, 2, 2)\n",
"films : [10, 16, 1, 3, 8, 13]\n",
"rank : [3, 5, 0, 1, 2, 4]\n",
"\n"
]
}
],
"source": [
"test3_rankings = test3_fest.evaluate_schedules(test3_schedules2, debug=True)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"masked_array(data =\n",
" [[1.9090909090909092 4.0 3.0]\n",
" [1.0 0.0 4.666666666666667]\n",
" [2.6363636363636362 1.5714285714285714 0.3333333333333333]\n",
" [0.18181818181818182 4.714285714285714 0.6666666666666666]\n",
" [5.0 3.2857142857142856 2.6666666666666665]\n",
" [3.5454545454545454 2.5714285714285716 3.6666666666666665]],\n",
" mask =\n",
" [[False False False]\n",
" [False False False]\n",
" [False False False]\n",
" [False False False]\n",
" [False False False]\n",
" [False False False]],\n",
" fill_value = 1e+20)"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# one way to merge the rankings to take the average ranking at each (time slot, screen)\n",
"np.ma.average(test3_rankings, axis=0)"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1, 1)"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# pluck the minumum from that\n",
"test3_fest.get_argmin(np.ma.average(test3_rankings, axis=0))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Test three screen strategy"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [],
"source": [
"def n_screen_strategy(num_screens, num_films_per_screen):\n",
" schedules = []\n",
" \n",
" for j in range(num_screens):\n",
" schedules.append(num_films_per_screen * (j, ))\n",
" \n",
" for k in range(j+1, num_screens):\n",
" \n",
" for i in range(num_films_per_screen):\n",
" schedules.append(i*(j,) + (k,) + (num_films_per_screen-i-1)*(j,))\n",
" \n",
" return schedules"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[(0, 0, 0, 0, 0, 0),\n",
" (1, 0, 0, 0, 0, 0),\n",
" (0, 1, 0, 0, 0, 0),\n",
" (0, 0, 1, 0, 0, 0),\n",
" (0, 0, 0, 1, 0, 0),\n",
" (0, 0, 0, 0, 1, 0),\n",
" (0, 0, 0, 0, 0, 1),\n",
" (2, 0, 0, 0, 0, 0),\n",
" (0, 2, 0, 0, 0, 0),\n",
" (0, 0, 2, 0, 0, 0),\n",
" (0, 0, 0, 2, 0, 0),\n",
" (0, 0, 0, 0, 2, 0),\n",
" (0, 0, 0, 0, 0, 2),\n",
" (1, 1, 1, 1, 1, 1),\n",
" (2, 1, 1, 1, 1, 1),\n",
" (1, 2, 1, 1, 1, 1),\n",
" (1, 1, 2, 1, 1, 1),\n",
" (1, 1, 1, 2, 1, 1),\n",
" (1, 1, 1, 1, 2, 1),\n",
" (1, 1, 1, 1, 1, 2),\n",
" (2, 2, 2, 2, 2, 2)]"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"n_screen_strategy(3, 6)"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)\n",
"(1, 0, 0, 0, 0, 0, 0, 0, 0, 0)\n",
"(0, 1, 0, 0, 0, 0, 0, 0, 0, 0)\n",
"(0, 0, 1, 0, 0, 0, 0, 0, 0, 0)\n",
"(0, 0, 0, 1, 0, 0, 0, 0, 0, 0)\n",
"(0, 0, 0, 0, 1, 0, 0, 0, 0, 0)\n",
"(0, 0, 0, 0, 0, 1, 0, 0, 0, 0)\n",
"(0, 0, 0, 0, 0, 0, 1, 0, 0, 0)\n",
"(0, 0, 0, 0, 0, 0, 0, 1, 0, 0)\n",
"(0, 0, 0, 0, 0, 0, 0, 0, 1, 0)\n",
"(0, 0, 0, 0, 0, 0, 0, 0, 0, 1)\n",
"(2, 0, 0, 0, 0, 0, 0, 0, 0, 0)\n",
"(0, 2, 0, 0, 0, 0, 0, 0, 0, 0)\n",
"(0, 0, 2, 0, 0, 0, 0, 0, 0, 0)\n",
"(0, 0, 0, 2, 0, 0, 0, 0, 0, 0)\n",
"(0, 0, 0, 0, 2, 0, 0, 0, 0, 0)\n",
"(0, 0, 0, 0, 0, 2, 0, 0, 0, 0)\n",
"(0, 0, 0, 0, 0, 0, 2, 0, 0, 0)\n",
"(0, 0, 0, 0, 0, 0, 0, 2, 0, 0)\n",
"(0, 0, 0, 0, 0, 0, 0, 0, 2, 0)\n",
"(0, 0, 0, 0, 0, 0, 0, 0, 0, 2)\n",
"(1, 1, 1, 1, 1, 1, 1, 1, 1, 1)\n",
"(2, 1, 1, 1, 1, 1, 1, 1, 1, 1)\n",
"(1, 2, 1, 1, 1, 1, 1, 1, 1, 1)\n",
"(1, 1, 2, 1, 1, 1, 1, 1, 1, 1)\n",
"(1, 1, 1, 2, 1, 1, 1, 1, 1, 1)\n",
"(1, 1, 1, 1, 2, 1, 1, 1, 1, 1)\n",
"(1, 1, 1, 1, 1, 2, 1, 1, 1, 1)\n",
"(1, 1, 1, 1, 1, 1, 2, 1, 1, 1)\n",
"(1, 1, 1, 1, 1, 1, 1, 2, 1, 1)\n",
"(1, 1, 1, 1, 1, 1, 1, 1, 2, 1)\n",
"(1, 1, 1, 1, 1, 1, 1, 1, 1, 2)\n",
"(2, 2, 2, 2, 2, 2, 2, 2, 2, 2)\n",
"\n",
"All good! Passed 1000 random trials. \n",
"Num reviewers used: 33\n"
]
}
],
"source": [
"num_screens = 3\n",
"num_films_per_screen = 10\n",
"\n",
"# create schedules\n",
"schedules = n_screen_strategy(num_screens, num_films_per_screen)\n",
"\n",
"for schedule in schedules:\n",
" print(schedule)\n",
"print('')\n",
"\n",
"# run random trials\n",
"fest = Festival(num_screens, num_films_per_screen)\n",
"\n",
"num_trials = 1000\n",
"all_good = True\n",
"num_good_so_far = 0\n",
"\n",
"for i in range(num_trials):\n",
" fest.randomize_films(seed=i)\n",
" truth = fest.get_argmin(fest.screens)\n",
" \n",
" rankings = fest.evaluate_schedules(schedules)\n",
" \n",
" merged_rankings = np.ma.average(rankings, axis=0)\n",
" \n",
" guess = fest.get_argmin(merged_rankings)\n",
" \n",
" # check if minimum is unique\n",
" count = np.count_nonzero(merged_rankings == merged_rankings[guess])\n",
" \n",
" if count > 1 or truth != guess:\n",
" all_good = False\n",
" print(count)\n",
" print('Screens:')\n",
" print(fest.screens.T)\n",
" print('')\n",
" print('Rankings:')\n",
" print(rankings.transpose((0, 2, 1)))\n",
" print('')\n",
" print('Merged Rankings:')\n",
" print(merged_rankings.T)\n",
" print(truth, guess)\n",
" print('')\n",
" break\n",
" else:\n",
" num_good_so_far += 1\n",
" \n",
"if all_good:\n",
" print(\"All good! Passed {:d} random trials. \".format(num_trials))\n",
" print(\"Num reviewers used: {:d}\".format(len(schedules)))\n",
"else:\n",
" print('passed {:d} before failing'.format(num_good_so_far))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.13"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment