Skip to content

Instantly share code, notes, and snippets.

@chazcheadle

chazcheadle/mpx_data.json

Last active August 29, 2015 14:25
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save chazcheadle/43bb2635f07b76f96a0b to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save chazcheadle/43bb2635f07b76f96a0b to your computer and use it in GitHub Desktop.
Download ZIP
MPX data processing with Python
Raw
mpx_data.json
{
"entries": [
{
"title": "Fiscal cliffhanger: GOP going against public sentiment?",
"description": "MSNBC’s Thomas Roberts talks with Rep. Diane Black, R-Tenn., and the political power panel--Karen Tumulty of the Washington Post, Democratic strategist Karen Finney, and Republican strategist Robert Traynham--about fiscal cliff talks.",
"added": 1354862683000
},
{
"title": "The impact of changing demographics in the south",
"description": "MSNBC’s Thomas Roberts talks with Atlanta Mayor Kasim Reed and the Rev. C.L. Bryant about the changing demographics in the south and how it could impact future elections.",
"added": 1354862684000
},
{
"title": "Fiscal cliff concession?",
"description": "MSNBC’s Thomas Roberts talks with MSNBC’s Ed Schultz about a shift in GOP opposition to Pres. Obama’s fiscal cliff proposal.",
"added": 1354862685000
},
{
"title": "Same-sex couples line up for licenses",
"description": "Gay couples lined up to be the first to be legally married in Washington state's King County which started issuing licenses at midnight. MSNBC’s Chris Jansing reports.",
"added": 1354862686000
},
{
"title": "n_roberts_egypt_121206",
"description": "",
"added": 1354862686000
},
{
"title": "Battle for heart and soul of the GOP",
"description": "MSNBC’s Thomas Roberts talks to Former Republican Congressman J.C. Watts and a political power panel, including MSNBC Contributor and Grio Political Editor Perry Bacon, MSNBC Contributor Ari Melber, and republican strategist Alice Stewart about the in-fighting in the Republican Party.",
"added": 1354862688000
},
{
"title": "Democrats face tough decisions in fiscal cliff negotiations",
"description": "MSNBC’s Thomas Roberts talks to Sen. Ben Cardin, D-Md., about the decisions before Democrats in fiscal cliff negotiations.",
"added": 1354862688000
},
{
"title": "Why it matters: Elizabeth Warren and the Banking Committee",
"description": "MSNBC’s Thomas Roberts talks to MSNBC Contributor Jimmy Williams about the importance of Elizabeth Warren’s potential seat on the banking committee.",
"added": 1354862689000
},
{
"title": "Iran claims to have seized US reconnaissance drone",
"description": "MSNBC’s Thomas Roberts talks to NBC’s Ali Arouzi and MSNBC Analyst Jack Jacobs about claims Iranian forces have seized a US reconnaissance drone.",
"added": 1354862690000
},
{
"title": "White House rejects GOP cliff proposal: so what’s next?",
"description": "MSNBC’s Thomas Roberts talks to Sen. Bob Casey,D-Pa., and a political power panel, including Jackie Kucinich of USA today, democratic strategist Morris Reid, and Alfonso Alguilar, Executive Director of the Latino Partnership for Conservative Principles, about the next step in the fiscal cliff negotiations.",
"added": 1354862690000
},
{
"title": "n_roberts_egypt_121204",
"description": "",
"added": 1354862691000
},
{
"title": "Parties dig in with fiscal cliff deadline looming",
"description": "MSNBC’s Thomas Roberts talks to Congresswoman Allyson Schwartz, D-Pa., and a political power panel including: MSNBC Contributors Susan Del Percio and Joy-Ann Reid, and Democratic Strategist Blake Zeff about the posturing underway on both sides of the aisle with the fiscal cliff deadline looming.",
"added": 1354862692000
},
{
"title": "Should Gitmo detainees be moved to US?",
"description": "Chris Anders, of the ACLU, speaks with NBC’s Craig Melvin about trying Guantanamo Bay detainees in the United States mainland.",
"added": 1354862692000
},
{
"title": "Hate legislation being pushed through Ugandan parliament",
"description": "MSNBC’s Thomas Roberts talks to Civil Rights Activist Kerry Kennedy and Ugandan LGBT Rights Activist Frank Mugisha about Uganda’s so-called “Kill the Gays” bill.",
"added": 1354862693000
},
{
"title": "Why it matters: Obama's nomination for Secretary of State",
"description": "MSNBC’s Thomas Roberts talks to MSNBC Contributor Jimmy Williams about the fight over Susan Rice’s potential nomination as Secretary of State.",
"added": 1354862694000
},
{
"title": "Duchess Kate being treated for morning sickness",
"description": "NBC’s Keir Simmons reports from London, where Prince William’s wife, Kate Middleton, has been hospitalized for acute morning sickness as the couple released a statement announcing that they are expecting their first child, who will become third in line to the throne.",
"added": 1354862695000
},
{
"title": "Scores sickened by carbon monoxide at Ga. school",
"description": "A Georgia elementary school has been evacuated after an incident of carbon monoxide poisoning. MSNBC's Thomas Roberts reports.",
"added": 1354862696000
},
{
"title": "Writer: 'Unfathomable' for Chiefs to play after tragedy",
"description": "Msnbc's Alex Witt talks with The Nation's Dave Zirin about the Chiefs playing a game on Sunday after Jovan Belcher killed his girlfriend before driving to a practice facility and killing himself.",
"added": 1354862697000
},
{
"title": "n_lohan_week_121202",
"description": "",
"added": 1354862698000
},
{
"title": "n_cm_clinton_121202",
"description": "",
"added": 1354862698000
},
{
"title": "n_office_pltcs_121205",
"description": "",
"added": 1354862699000
},
{
"title": "n_wx_forecast_121202",
"description": "",
"added": 1354862699000
},
{
"title": "Celebrating World AIDS Day",
"description": "RED is a popular worldwide campaign designed to raise awareness and money for those fighting AIDS. RED CEO Deborah Dugan joins Craig to discuss the campaign's efforts.",
"added": 1354862700000
},
{
"title": "Speculation over Hillary Clinton's next career move",
"description": "An MSNBC panel takes a look at what the future may hold for U.S. Secretary of State Hillary Clinton.",
"added": 1354862701000
},
{
"title": "West bracing for storms",
"description": "The Weather Channel's Chris Warren reports from California where residents are expecting a string of heavy storms.",
"added": 1354862702000
},
{
"title": "Protests delay Mexican president's inauguration",
"description": "Violence erupts near the site of Enrique Pena Nieto's presidential inauguration in Mexico City. Telemundo's Jose Diaz-Balart reports.",
"added": 1354862704000
},
{
"title": "4 servicewomen file suit against military",
"description": "Major Mary Jennings Hegar is among a group that is suing the U.S. military over a ban on women serving in combat. Hegar explains the lawsuit to MSNBC's Alex Witt.",
"added": 1354862705000
},
{
"title": "n_bren7am_121201",
"description": "",
"added": 1354862706000
},
{
"title": "n_bren_121201",
"description": "",
"added": 1354862706000
},
{
"title": "Thousands of Morsi supporters rally in Egypt",
"description": "Jim Maceda reports from Egypt where groups supporting President Mohammed Morsi marched through the streets of Cairo amid anger by many citizens over Morsi's rewrite of the Constitution.",
"added": 1354862707000
},
{
"title": "NFL player reportedly commits suicide at team facility",
"description": "A member of the Kansas City Chiefs has reportedly died after shooting himself at the team facility early Saturday.",
"added": 1354862708000
},
{
"title": "Geist reflects on TODAY role",
"description": "NBC's Willie Geist joins MSNBC's Alex Witt to discuss his new spot on the TODAY Show and his love of Vanderbilt University.",
"added": 1354862709000
},
{
"title": "SCOTUS to announce plan for same-sex marriage cases",
"description": "Patricia Ann Millett of Akin Gump Partners joins MSNBC to discuss the next steps that the U.S. Supreme Court may take regarding same-sex marriage cases.",
"added": 1354862709000
},
{
"title": "Behind the scenes with Willie Geist",
"description": "NBC's Willie Geist sits down with MSNBC's Alex Witt to talk about his career and family.",
"added": 1354862710000
},
{
"title": "Israeli expansion sparks new controversy in Mideast",
"description": "Israel approves plans to build more than 3,000 homes in East Jerusalem. ITN's John Ray reports from Tel Aviv.",
"added": 1354862711000
},
{
"title": "n_lui_carter_121201",
"description": "",
"added": 1354862713000
},
{
"title": "Forbes ranks the world's 71 most powerful people",
"description": "Forbes Magazine's Randall Lane joins Morning Joe to discuss why President Obama, Germany's Angela Merkel and 69 others are the world's most powerful people. The National Journal's Joe Conason also joins the discussion.",
"added": 1354862714000
},
{
"title": "Investors offer their opinions on the best stocks for 2013",
"description": "Fortune Magazine's Andy Serwer joins Morning Joe to reveal the magazine's 2013 Investor's Guide. Mika Brzezinski also reads from a Joe Scarborough column on the fiscal cliff.",
"added": 1354862714000
},
{
"title": "10 marriage equality cases go before Supreme Court",
"description": "Msnbc’s Thomas Roberts talks to NBC News Justice Correspondent Pete Williams and Buzzfeed Senior Political Reporter Chris Geidner about the slate of marriage equality cases before the U.S. Supreme Court.",
"added": 1354862714000
},
{
"title": "Los Angeles riots take center stage in new Michael Connelly book",
"description": "New York Times best-selling author Michael Connelly joins Morning Joe to discuss \"The Black Box,\" his newest novel. The book happens to be his 25th, and the 18th involving character Harry Bosch. Connelly, a former L.A. Times reporter, writes about a murder that took place during the 1992 riots.",
"added": 1354862715000
},
{
"title": "Time interviews NFL commissioner Goodell in latest issue",
"description": "Time's Rick Stengel joins Morning Joe to reveal the magazine's latest issue, which features NFL commissioner Roger Goodell.",
"added": 1354862716000
},
{
"title": "Obama's new Romney ad, Jeb Bush and future of the GOP",
"description": "Former White House Chief of Staff Bill Daley joins Morning Joe to discuss President Obama's re-election campaign, the president's return to message in a new ad hitting Romney over his job creation record as Mass. governor, and Jeb Bush's recent criticisms of the contemporary Republican Party.",
"added": 1354862717000
},
{
"title": "Rubio clarifies Earth comments; discusses abortion, homosexuality with Politico",
"description": "Politico Playbook: In a recent GQ interview, Sen. Marco Rubio of Florida discussed the Earth's age from both a scientific and religious view. Politico's Mike Allen sat down with Rubio Wednesday, and Rubio clarified his comments on this, and he discussed his views on abortion and homosexuality.",
"added": 1354862718000
},
{
"title": "Where does House Speaker Boehner stand in House now?",
"description": "The New York Times has reported that House Speaker John Boehner is currently enjoying \"...the broadest support of his tumultuous two-year speakership from House Republicans.\" The National Review's Robert Costa and NBC News' Chuck Todd join a discussion about Boehner and the fiscal cliff proceedings.",
"added": 1354862718000
},
{
"title": "Scarborough: Obama could stiff-arm GOP and win politically, but he shouldn't",
"description": "Top Talkers: Is the White House ready to go over the fiscal cliff if the GOP won't agree to certain terms? Is the fiscal cliff THE defining issue of the president's second term? The Morning Joe panel – including economic analyst Steve Rattner, Time's Mark Halperin, and former DLC Chair Harold Ford Jr. – discusses the breakdown in cliff communication.",
"added": 1354862719000
},
{
"title": "House GOP presses White House over Fast and Furious program",
"description": "Politico Playbook: House Republicans want the White House and Justice Department to hand over documents related to the Fast and Furious program, and Politico's Mike Allen joins Morning Joe to discuss the latest as well as talk about the president's Chicago campaign office.",
"added": 1354862720000
},
{
"title": "Obama shows flexibility on taxes",
"description": "Sam Stein, Dr. Zbigniew Brzezinski and Steve Rattner join Morning Joe to talk about the fiscal cliff stalemate and compare both President Barack Obama and House Speaker John Boehner’s proposals.",
"added": 1354862721000
},
{
"title": "McCaskill: The alternative is worse for Republicans",
"description": "Sen. Claire McCaskill says that if Republicans don’t settle on the fiscal cliff, Democrats are going to come back and pass tax cuts on those who make under $250,000. ABC’s Cokie Roberts joins the conversation to discuss why the parties are at odds over the fiscal cliff.",
"added": 1354862721000
},
{
"title": "Sen. Coburn: Would rather see tax rates go up",
"description": "Sen. Tom Coburn, R-Okla., and \"Meet the Press\" moderator David Gregory join Morning Joe to discuss the fiscal cliff talks and where they stand in early December. Coburn says he understands revenue must be on the table, and he signaled a willingness to discuss raising rates on the country's top earners.",
"added": 1354862722000
},
{
"title": "Reps. Van Hollen, Cole debate fiscal cliff negotiations",
"description": "Rep. Chris Van Hollen, D-Md., and Rep. Tom Cole, R-Okla., join Morning Joe to discuss the fiscal cliff negotiations and the Senate rejection of a U.N. disabilities treaty.",
"added": 1354862723000
},
{
"title": "House Speaker Boehner a 'feckless herder of cats'?",
"description": "Must-Read Op-Eds: Mika Brzezinski reads from a Matt Lewis column from The Week on House Speaker John Boehner and why the House Speaker job has become largely thankless. Matt Lewis joins the conversation.",
"added": 1354862724000
},
{
"title": "UN disability treaty fails in Senate; Dems push for Obama data",
"description": "Politico Playbook: A U.N. disability treaty fails in the Senate despite impassioned pleas from Sens. John McCain and John Kerry. And Politico's Mike Allen joins Morning Joe to discuss how Democrats are pushing for the Obama campaign to share their data.",
"added": 1354862725000
},
{
"title": "Scarborough: It's not about what Boehner wants; it's about what he can deliver",
"description": "Top Talkers: The Morning Joe panel – including NBC News' Andrea Mitchell, the Huffington Post's Sam Stein, Mike Barnicle and Morning Joe economic analyst Steve Rattner – continues its discussion on the looming fiscal cliff and where the negotiations stand now. On Tuesday, December 4, 2012, President Obama rejected a GOP offer on taxes from the country's wealthiest.",
"added": 1354862725000
},
{
"title": "‘Morning’ Joe reflects on raising son with Asperger’s",
"description": "During a panel discussion with author Ron Fournier about raising children with Asperger’s syndrome, Joe Scarborough shares anecdotes and observations about his own parenting of a child with Asperger’s.",
"added": 1354862726000
},
{
"title": "n_mj_mustread_120412",
"description": "",
"added": 1354862727000
},
{
"title": "How both sides can walk away feeling like winners in fiscal talks",
"description": "Must-Read Op-Eds: Mika Brzezinski reads from David Brooks' latest NYT column on a deal to avoid going over the fiscal cliff. Joe Scarborough explains a scenario in which both sides of the aisle can walk away feeling as if they've won.",
"added": 1354862727000
},
{
"title": "n_mj_blair_121204",
"description": "",
"added": 1354862728000
},
{
"title": "n_mj_learned_121204",
"description": "",
"added": 1354862729000
},
{
"title": "n_mj_giunta_121204",
"description": "",
"added": 1354862729000
},
{
"title": "n_mj_7amhour_120412",
"description": "",
"added": 1354862730000
},
{
"title": "Ron Fournier on raising a son with Asperger's: He's got 30 IQ points on me",
"description": "The National Journal's Ron Fournier recently wrote about his son Tyler's Asperger syndrome, and he joins Morning Joe to discuss how he learned his son had Asperger's and what his son has taught him over the years. Joe Scarborough, who also has a child with Asperger's, shares experiences he's had with his child.",
"added": 1354862731000
},
{
"title": "n_mj_avrich_120412",
"description": "",
"added": 1354862731000
},
{
"title": "Why President Obama has to do something big on fiscal deal",
"description": "Top Talkers: With less than a month to go before the new year, Democrats and Republicans seem no closer to reaching a deal on averting the fiscal cliff. The Morning Joe panel – including Time's Mark Halperin and Random House's Jon Meacham – discusses where we are just 27 days before January.",
"added": 1354862732000
},
{
"title": "n_mj_politico_120412",
"description": "",
"added": 1354862733000
},
{
"title": "The joys of the holidays for an Irish family",
"description": "Actor and filmmaker Ed Burns shares details from his new movie, “The Fitzgerald Family Christmas” which is about the challenges of family during the holidays.",
"added": 1354862734000
},
{
"title": "What did the Morning Joe crew learn on Monday?",
"description": "What Did We Learn: Jon Meacham, Michael Steele, Mike Barnicle and Leigh Gallagher tell us what they learned today.",
"added": 1354862734000
},
{
"title": "Thomas Jefferson and his relationship with slavery",
"description": "Must-Read Op-Eds: Willie Geist reads from a Paul Finkelman NYT column on Thomas Jefferson and his relationship with slavery. Jefferson biographer and Morning Joe regular, Jon Meacham, offers some context.",
"added": 1354862735000
},
{
"title": "New appointments, no talks behind the scenes and Lincoln",
"description": "Will new Obama appointments come this week? Is there really more than meets the eye right now when it comes to fiscal cliff discussions? How would Lincoln deal with the fiscal cliff, and is there a way to get both John Kerry and Susan Rice into the Obama cabinet? NBC News' Chuck Todd and Time's Joe Klein join the discussion.",
"added": 1354862736000
},
{
"title": "Remaking a classic – the new face of Lincoln",
"description": "President and CEO of Ford Alan Mulally and Jim Farley, the executive vice president of global marketing for Ford and Lincoln, talk about the launch of the new Lincoln MKZ.",
"added": 1354862736000
},
{
"title": "Will fiscal talks conclude before Christmas? Before NYE?",
"description": "Will Washington wrap up a deal on avoiding the fiscal cliff before Christmas? Or is it more likely to come around New Year's Eve? New York Magazine's John Heilemann and NBC News' Kelly O'Donnell join a conversation on the issue.",
"added": 1354862737000
},
{
"title": "How Washington could reach a deal on fiscal cliff in 24 hours",
"description": "Top Talkers: Will Washington be able to come together to avert going over the fiscal cliff? Treasury Secretary Tim Geithner, the Americans for Tax Reform's Grover Norquist and John Boehner all went on the Sunday news shows sharing their opinions about the latest standoff. The Morning Joe panel – including Mike Barnicle, Random House's Jon Meacham, Fortune Magazine's Leigh Gallagher and former RNC Chair Michael Steele – discusses.",
"added": 1354862738000
}]
}
Raw
mpx_data_process.py
import json
import matplotlib.pyplot as plt
my_data = json.loads(open('mpx_data.json').read())
videos = my_data['entries']
# Create list of values from a list of dicts
my_list = []
for i, val in enumerate(videos):
my_list.append([len(videos[i]['description']), videos[i]['added']])
# Calculate lengths of list items:
# Total list items
total_list_items = len(my_list)
# List items with a description
list_items = len([pair[0] for pair in my_list if pair[0] > 0])
print("Number of videos: " + str('{0:,}'.format(total_list_items)))
print("Number of videos with no description: " + str('{0:,}'.format(total_list_items - list_items)))
# Average length of list items
total_characters = sum([pair[0] for pair in my_list if pair[0] > 0])
average_length = total_characters / list_items
print("Average length*: " + str('{0:,}'.format(int(average_length))))
print("* Calculation exclude videos with no description.")
# Number of list items greater than 255
length255 = len([pair[0] for pair in my_list if pair[0] > 255])
print("Descriptions over 255 characters: " + str('{0:,}'.format(length255)))
# Longest and shortest (non-empty) description
print("Longest: " + str(max([pair[0] for pair in my_list])))
print("Shortest: " + str(min([pair[0] for pair in my_list if pair[0] > 0])))
# Plot histogram of description length counts
plt.hist([pair[0] for pair in my_list if pair[0] > 0], bins=80)
plt.title('Distribution of Lengths of Descriptions')
plt.xlabel('Length')
plt.ylabel('Count')
plt.show()
# Plot of Description length over time
x_axis = [pair[1] for pair in my_list if pair[1] > 0]
y_axis = [pair[0] for pair in my_list]
plt.plot(x_axis, y_axis)
plt.title('Description Length Over Time')
plt.xlabel('Date')
plt.ylabel('Length')
plt.show()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment