election_talk.txt

wycats> soooo
1:15 PM <wycats> back-of-the-envelope
1:16 PM <wycats> what frustrates me about election night coverage is this:
1:16 PM <wycats> we have awesome data
1:16 PM <wycats> but I have to watch TV to let Chuck Todd tell me that some county leans Republican and it's not all in yet
1:16 PM <wycats> and draw statewide projections
1:16 PM <wycats> humans are actually not even very good at that
1:17 PM <wycats> what I want is an election scoreboard that shows clearly how states are trending, based on county-level results from the last N elections
1:17 PM <wycats> I know newsrooms are leery about making projections, but this wouldn't be calling anything
1:17 PM <wycats> it would just be showing visually what people are trying to cram into 30s here and there on TV
1:18 PM <wycats> I would do it myself, but I don't have $50k for AP election results :P
1:18 PM <jashkenas> fortunately, we have an outlet this year for projection-oriented steam, in the form of fivethirtyeight... 
1:18 PM <knowtheory> hunh.  seems to me like everyone's been making projections, but could be biased sampling on my part.
1:18 PM <thejefflarson> that's a really cool idea I've never worked on an election app before, but for 2008 I made an internal scoreboard at the Nation that tracked votes over time so our editors could blog about trend lines
1:18 PM <wycats> 538 sucked last year
1:19 PM <wycats> no offense
1:19 PM <jashkenas> none take.
1:19 PM <wycats> jashkenas: last year, 538 "live blogged"
1:19 PM <wycats> well
1:19 PM <jashkenas> what exactly to you mean by "trending", in the context of county level primary results?
1:19 PM <wycats> 2010
1:19 PM <wycats> and 2008
1:19 PM <wycats> but disappeared for multiple hours
1:19 PM <wycats> probably writing blog posts :P
1:19 PM <thejefflarson> Jake Harris who isn't here right now has some insight into this I think. jashkenas: you know if he is around?
1:20 PM → _jayb_ (40ec803e@gateway/web/freenode/ip.64.236.128.62) joined
1:20 PM <wycats> jashkenas: what I mean is using the trend in fully-in counties, and applying it to counties that are not fully in
1:20 PM <wycats> and using statistics to get a range
1:20 PM <jashkenas> he's not in the office yet.
1:20 PM <wycats> ah primary
1:20 PM <wycats> primaries are harder
1:20 PM <wycats> but you can possibly do something like… look at the county variance over the past elections
1:20 PM <wycats> vs. the entire state
1:21 PM <wycats> and get a range of how outlier counties can be
1:21 PM <bycoffe> one issue is that by the time any counties are fully (or even mostly) reporting, some outlets are already able to call a county or a state winner based on exit polls.
1:21 PM <wycats> bycoffe: outlets have been doing a lot of "too early to call" lately
1:21 PM <wycats> which means "we know the answer but aren't telling you yet" :
1:21 PM <wycats> :P
1:22 PM <wycats> my point is that by exposing the raw data, you can give the information without putting your ass on the line
1:22 PM <wycats> let people draw their own conclusions
1:22 PM <wycats> bloggers are already trying to piece it together
1:23 PM <wycats> 538 didn't even do a really obvious thing last year
1:24 PM <wycats> which was to start with their predictions as a baseline
1:24 PM <wycats> and then +/- based on actual results
1:24 PM <wycats> I had to do it myself in excel :P
1:24 PM <thejefflarson> yeah amanda cox and shan carter did an electorla shifts graphic for the last election: http://www.nytimes.com/interactive/2010/11/03/us/politics/election-results-house-shift.html
1:24 PM <wycats> should be live :p
1:25 PM <thejefflarson> but those guys at harvard have ways to rerun elections to provide statistical baselines... I agree with you on live, would be cool.
1:25 PM <wycats> I've been doing excel spreadsheets every year since 2000 to try to see the trends
1:25 PM <wycats> 2000 was very unsatisfying :p
1:25 PM <wycats> I just want someone to do it for me
1:25 PM <wycats> 538 already exposes a bunch of good predictions; we have baselines for last year
1:25 PM <wycats> last election*
1:25 PM <wycats> I want to see how the real numbers are faring against those on a county-by-county basis
1:26 PM <wycats> and to have someone use those trends to provide a sense of what the buffer looks like for each candidate
1:26 PM <wycats> it shouldn't be that hard :P
1:26 PM <thejefflarson> let me see if I can find the paper on rerunning elections... Also, karin macdonald at berkeley does a lot of this research
1:26 PM <_jayb_> i've been doing live projections for the last 3 primaries -- not sure if it's exactly what you described (joined late), but here's what SC looked like: http://jayboice.com/sc.html
1:26 PM <wycats> once you've paid the AP tax
1:26 PM <_jayb_> the only interesting part is down below 1% reporting
1:26 PM <wycats> _jayb_: nice
1:27 PM <wycats> _jayb_: I think that's probably good enough, although a bit of extra county-level data would be nice for nerds like me
1:27 PM <wycats> but only useful for the general
1:27 PM <wycats> in primaries, there's too much divergence from 2008 to make seeing last election very useful
1:27 PM <_jayb_> sure -- this is just the aggregated more presentable version
1:27 PM <wycats> _jayb_: confirm
1:28 PM <_jayb_> agreed: i'm just using 2008 as a baeline -- would like to get more prior years
1:28 PM <wycats> I want to be able to hover over a map, see a county, see the 2004/2008 numbers, and see the 2012 results
1:28 PM <wycats> and then I want something like what you're doing, which uses that info to project a +/-
1:28 PM <wycats> calling things is way too binary
1:28 PM <wycats> "too early to call" doesn't tell you who's winning :P
1:29 PM <wycats> you can look at the exit polls yourself
1:29 PM <wycats> the entire thing is a charade :p
1:29 PM <bycoffe> wycats: we've been showing 2008 numbers with our 2012 results. e.g. http://www.huffingtonpost.com/2012/01/21/map-south-carolina-primary-2012-results_n_1221160.html
1:29 PM <wycats> I'm tired of having to do a ton of manual work because news outlets are too risk averse
1:29 PM <wycats> bycoffe: exactly like that
1:30 PM <wycats> I swear to god, if the general election comes around and I have to manually aggregate all the available information by hand again, I am going to cry :P
1:32 PM <a_l> wycats: have you seen http://polltracker.talkingpointsmemo.com/contest/fl-pres-12-r .. gives you a pretty good idea of what will happen
1:34 PM <wycats> yeah… I follow TPM's polls closely ;)
1:35 PM <bycoffe> i would put more money on recent polls as a predictor than on four-year-old election results. tough to get polls at the county level, though.
1:35 PM <wycats> what I want is a map visualization that lets me roll over states/counties, gives me baseline numbers (probably 538 predictions, 2004, and 2008 results), makes some basic projections with +/-, and with a sidebar that shows me "events" as data comes in
1:35 PM <wycats> bycoffe: 2008 numbers are useful because they're more comprehensive
1:35 PM <wycats> you have detailed county-by-county results
1:35 PM <wycats> so you can see divergence
1:36 PM <wycats> you don't expect 2012 to look like 2008, but if the first 10 counties are 10% more GOP than last time, that tells you something
1:36 PM <wycats> and you can figure out what exactly it tells you by doing some stats on the previous results
1:36 PM <wycats> some states may have very even distributions… so if 10 counties skew GOP, the other counties will follow
1:36 PM <wycats> other states may not
1:36 PM <wycats> but it should be possible to figure something out
1:37 PM <wycats> and the reason I know that is that they do it on TV
1:37 PM <wycats> I'll stop talking now :p
1:38 PM <knowtheory> having the data would be neat you could cross reference against census/demographic data too
1:39 PM <thejefflarson> Yeah nyt runs exit poll graphics after the election on that: http://elections.nytimes.com/2012/primaries/states/south-carolina/exit-polls
1:39 PM <thejefflarson> I seem to remember a similar graphic before hand too. 
1:40 PM <wycats> I just don't want to have 10 tabs open and have to watch TV to get a faint sense of what's going on
1:41 PM <knowtheory> i'd hope that if you had that many tabs open and a tv running you'd have a better than faint sense of what's going on :P
1:41 PM <wycats> knowtheory: it's actually very hard
1:41 PM <wycats> there are a lot of pieces of data to juggle
1:41 PM <wycats> in different places
1:41 PM <knowtheory> we're still talking about the general election yeh?
1:41 PM <wycats> c
1:45 PM <knowtheory> wycats: i guess there are two parts to this, which is live election data feed, and then having the historical stuff and other info you can pull together to do quick reference analyses
1:45 PM <wycats> people do live elections now
1:45 PM <wycats> but cross-referencing N counties in another tab is too much work to actually do
1:48 PM <knowtheory> wycats: it sounds kind of like you'd just like raw data to be able to play around w/ and project against, rather than an actual concrete set of visualizations ;)
1:49 PM <knowtheory> are there a set of visualizations that would scratch the itch for you?
1:49 PM <knowtheory> (maybe i'm projecting a bit as well heh)
1:50 PM <wycats> knowtheory: I don't care about the raw data
1:50 PM <wycats> I already said what I wanted ;)
1:50 PM <wycats> I want to roll over a county
1:51 PM <wycats> and see 2004/2008 and probably 538 predictions, as well as the current results and % in
1:51 PM <wycats> then, I would like someone to project based on the diff of already-in counties, what that means for not in counties
1:51 PM <wycats> for partially in counties, I want to see a linear projection
1:51 PM <wycats> or whatever makes sense
1:51 PM <wycats> rolled up to a number with a +/-
1:51 PM <wycats> again, people are actually doing this
1:51 PM <wycats> on TV, in newsrooms
1:51 PM <wycats> just expose the answer
1:52 PM <wycats> I also want an event stream on the right that shows when new data has come in for a state
1:52 PM <wycats> so I can know what I should be rolling over
1:52 PM <wycats> instead of polling every county every 5 minutes
1:52 PM <wycats> $(scoreboard).bind("new-data", …) :P
1:52 PM <wycats> #nerdjoke
1:55 PM <thejefflarson> I really like the idea of an event stream. #websockets
1:57 PM <wycats> thejefflarson: :)
1:57 PM <wycats> I'm basically asking for a visualization I can sit at while watching MSNBC that aggregates the other stuff I usually have open
1:57 PM <wycats> which isn't actually much
1:59 PM → ryanpitts ([email protected]) joined
2:00 PM <thejefflarson> wycats: serious question: what do you feel about maps? are the useful for you, or would you rather that space be used for something else?
2:01 PM <kleinmatic> It would be cool if a news site gave a general prediction based on exit polls: Romney 80% likely to win based on exit polls, etc.
2:01 PM <wycats> thejefflarson: I personally find maps useful
2:01 PM <wycats> especially if clicking on an item in the event stream zoomed into the state
2:02 PM <wycats> thejefflarson: I don't find today's maps very useful
2:02 PM <thejefflarson> wycats: yeah tpm did that in 2010, it was really cool
2:02 PM <wycats> but I think there's ways to increase information density that would make them more useful
2:02 PM <wycats> kleinmatic: there's a lot more statistical rigor that can be applied to projections
2:02 PM <wycats> outlets are just very risk-averse, especially after 2000
2:02 PM <wycats> hence "too early to call"
2:02 PM <wycats> what a bunch of crap :P
2:02 PM <wycats> "too early to call" == "we have the answer but don't want to tell you yet"
2:02 PM <kleinmatic> wycats: all the things they do statistically before election day they could do between polls closing and the winner being called.
2:03 PM <wycats> which you can EASILY determine by doing simple math on the public exit polls
2:03 PM <thejefflarson> ha, well dewey defeats truman looms large too.
2:03 PM <wycats> which means more tabs open
2:03 PM <wycats> hehe
2:03 PM <kleinmatic> wycats: I suspect that's the case, but I don't know the details. I'm guessing they wait until they're reasonably certain but not a minute longer.
2:03 PM <bycoffe> wycats: "too early to call != "we have the answer but don't want to tell you yet", at least not always.
2:03 PM <kleinmatic> thejefflarson: or Bush/Gore 2000.
2:04 PM <wycats> bycoffe: what % of viewers do you think understand the difference between "too close to call" and "too early to call"
2:04 PM <wycats> it's totally insider baseball risk management
2:04 PM <bycoffe> wycats: South Carolina was called as soon as the polls closed. Iowa wasn't called until a week later. 
2:04 PM <kleinmatic> wycats: The Times, and I suspect many others, publish exit poll results as soon as the polls close, so it's not like they're hiding the data that would go into that decision.
2:05 PM <wycats> EXACTLY
2:05 PM <wycats> so yuno just tell people what they can figure out by using a calculator and another page on the same site
2:05 PM <wycats> it's not like your blogging competitors haven't figured it out already
2:05 PM <a_l> usually if the exit polls tell a certain story, they will call it that way
2:06 PM <wycats> It's bad that I get more information from secondary sources than the original reporters
2:06 PM <kleinmatic> wycats: Getting some election night nerds to come here.
2:06 PM <wycats> hehe
2:06 PM <wycats> seems good
2:06 PM <wycats> fwiw, I covered elections for my student newspapers in 2000 and 2004 :P
2:07 PM → harrisj ([email protected]) joined
2:07 PM <kleinmatic> harrisj: Welcome!
2:07 PM <wycats> hola
2:07 PM <wycats> holla
2:07 PM <harrisj> hello
2:07 PM <harrisj> I heard there is election talk here