Last active
December 18, 2015 03:48
-
-
Save pstaender/5720519 to your computer and use it in GitHub Desktop.
Summarizes plain text by taking the first sentence of each paragraph and additionally the last sentence of the last paragraph.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### | |
The MIT License (MIT) | |
Copyright © 20130 Philipp Staender <[email protected]> | |
Permission is hereby granted, free of charge, to any person obtaining a copy | |
of this software and associated documentation files (the "Software"), to deal | |
in the Software without restriction, including without limitation the rights | |
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |
copies of the Software, and to permit persons to whom the Software is | |
furnished to do so, subject to the following conditions: | |
The above copyright notice and this permission notice shall be included in | |
all copies or substantial portions of the Software. | |
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN | |
THE SOFTWARE. | |
### | |
class Summary | |
constructor: (@text) -> | |
throw Error('You need to past as first argument a text to summarize') if typeof @text isnt 'string' | |
throw Error('HTML is not supported, only plain text') if /\<[a-z].*\>/i.test(@text) | |
extractParagraphs: -> | |
ps = @text.split(/\n+/) | |
paragraphs = [] | |
for p in ps | |
p = p.trim() | |
paragraphs.push(p) if /^[^\W\d_]+/.test(p) | |
paragraphs | |
first_sentence: (s) -> | |
s.match(/^(.+?)[^0-9][\.\?\;\!]\s/)?[0]?.trim() or s | |
last_sentence: (s, fallback = '') -> | |
s = s.trim() | |
sentences = s.split(/([^0-9][\.\?\;\!])/) | |
sentences[sentences.length-3]?.trim()+sentences[sentences.length-2] or fallback | |
summarize: (join = '\n\n') -> | |
paragraphs = @extractParagraphs() | |
summary = for paragraph, i in paragraphs | |
firstSentence = Summary::first_sentence(paragraph) | |
if i is paragraphs.length-1 | |
lastSentence = Summary::last_sentence(paragraph) | |
if lastSentence isnt firstSentence | |
firstSentence + ' ' + lastSentence | |
else | |
firstSentence | |
else | |
firstSentence | |
if join then summary.join(join) else summary | |
example = """ | |
### Welcome to Echelon 2.0 | |
The NSA isn't spying on you. They don't have to. They just outsource it to businesses. | |
Back 20 years ago, there was a conspiracy theory going around called "ECHELON" that claimed the NSA was monitoring the content of everyone's phone calls and emails, everywhere in the world, including the United States. Echelon (probably) existed, but it's wasn't technically feasible to be as extensive as claimed. It was also against the constitution: the NSA is a bunch of bastards, but they would never cross that line. | |
But they'll move the line. That's what today's revelation by Glen Greenwald (@ggreenwald) at the Guardian has shown us. While the NSA cannot intercept signals within the United States, then can get a court order for most of the information from businesses, with help from the FBI and the FISA court. The court order demanded everyone's call records from Verizon. It demanded not just records of calls made to foreign countries, but those wholly within the United States as well. | |
According to the "third party doctrine", only your personal effects are protected by the Fourth Amendment. Once you give a thing to somebody else, that thing is no longer private. | |
For that reason, your telephone records aren't private. Neither are your credit card receipts, banking records, hotel visits, automobile registrations, Internet logs, and so on. The government doesn't need Fourth Amendment "probable cause" to grab this information. A simple administrative subpoena is sufficient. To get that, they just need to tell a judge that they are interested in the information as part of an investigation. | |
Not only can they easily get the records, they are passing more and more laws requiring "data retention", forcing businesses to keep the records in the first place. Even without specific laws requiring data retention, the government can pressure businesses into it. Hotel's require your ID not because of a law, but because if illegal stuff happens too often in the hotel, the government will just confiscate it. | |
It's getting harder and harder to buy anything without a credit card and identification. The maximum printed currency has been $100 for over a century, despite inflation. That's because the government wants you to stop using cash and use electronic means that they can track. The government's attacks on Liberty Reserve and BitCoin are so that they can stop anonymous money transactions. | |
Recently, I went to T-Mobile in order to get a prepaid SIM for my Android phone. Right now in the United States, T-Mobile's prepaid plan doesn't require you to show identification. I paid cash, and told them my name was "John Smith" (telling the cashier it wasn't my real name). I asked the cashier about this, that it seems this could be abused for drug dealers. He pointed to the security cameras surrounding the store: apparently T-Mobile regularly gives police the camera feeds. (That's the picture up above in the blog post -- the picture I took of the cameras in the store). | |
Even if you are doing legitimate things, it's illegal to hide them from the government. Banks are required to notify the government of any transaction over $10,000. So, you might do two transactions, one for $7,000 and one for $3,000. This "structuring" of payments is illegal in of itself, no matter how innocent the intended use of the money. | |
The contents of your emails are still private, but only if you pull them off the servers immediately. If you leave emails in your GMail account for too long, the third party doctrine applies, and the FBI can grab them. Also, emails in things like draft folders aren't private. That's what happened in the Paula Broadwell case: they couldn't intercept the "live" emails, just the "non-live" ones. | |
The gadgets we buy no longer belong to us -- we just borrow them from the government. It's against the law to "unlock" your phone. By law, cars now include tracking systems, little "black boxes" that will tell investigators everything you did prior to a crash, and what your emissions look like. That a person's devices would spy on them for the government seemed like crazy science-fiction in Orwell's book 1984, but such things are now taken for granted in 2013. | |
Last year, Andy Greenberg (@a_greenberg) had a story on Fortune.com about the NSA chief Keith Alexander denying domestic surveillance. If you read the transcript, you'll see that everything Alexander says is true. The NSA itself is not spying on us -- they are just leaning on the FBI to subpoena the information from businesses. | |
Echelon 1.0 was a theory that couldn't really work in practice. Welcome to Echelon 2.0: Outsourced Edition. | |
""" | |
summary = new Summary(example) | |
console.log summary.summarize() | |
### output: | |
Today, the Guardian newspaper confirmed what EFF (and many others) have long claimed: the NSA is conducting widespread, untargeted, domestic surveillance on millions of Americans. | |
In a report by Glenn Greenwald, the paper published an order from the Foreign Intelligence Surveillance Court (or FISC) that directs Verizon to provide “on an ongoing daily basis” all call records for any call “wholly within the United States, including local telephone calls” and any call made “between the United States and abroad.” | |
In plain language: the order gave the NSA a record of every Verizon customer’s call history -- every call made, the location of the phone, the time of the call, the duration of the call, and other “identifying information” for the phone and call -- from April 25, 2013 (the date the order was issued) to July 19, 2013. | |
There is no indication that this order to Verizon was unique or novel. | |
This type of untargeted, wholly domestic surveillance is exactly what EFF, and others have been suing about for years. | |
But at each step of the way, the government has tried to hide the truth from the American public: in Hepting, behind telecom immunity; | |
I want to deliver a warning this afternoon: when the American people find out how their government has secretly interpreted the Patriot Act, they will be stunned and they will be angry. | |
Today is that day. And it’s time to end the NSA’s unconstitutional domestic surveillance program. | |
### |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment