Skip to content

Instantly share code, notes, and snippets.

@jdkato
jdkato / golden-rules.csv
Last active July 13, 2018 17:37
Segment Evaluations
Name Language License GRS (English) GRS (Other) Speed
Pragmatic Segmenter Ruby MIT 98.08% (51/52) 100.00% 3.84 s
prose Go MIT 75.00% (39/52) N/A 0.96 s (different hardware)
TactfulTokenizer Ruby GNU GPLv3 65.38% (34/52) 48.57% 46.32 s
OpenNLP Java APLv2 59.62% (31/52) 45.71% 1.27 s
Standford CoreNLP Java GNU GPLv3 59.62% (31/52) 31.43% 0.92 s
Splitta Python APLv2 55.77% (29/52) 37.14% N/A
Punkt Python APLv2 46.15% (24/52) 48.57% 1.79 s
SRX English Ruby GNU GPLv3 30.77% (16/52) 28.57% 6.19 s
Scapel Ruby GNU GPLv3 28.85% (15/52) 20.00% 0.13 s
Type Example
Email addresses Jane.Doe@example.com
Hashtags #trending
Mentions @jdkato
URLs https://github.com/jdkato/prose
Emoticons :-), >:(, o_0, etc.
@jdkato
jdkato / tokenize.go
Last active July 13, 2018 17:13
`prose` v2.0.0 example
package main
import (
"gopkg.in/jdkato/prose.v2"
)
func main() {
doc, _ := prose.NewDocument("This is a sentence.")
for _, token := range doc.Tokens() {
fmt.Println(token.Text)
@jdkato
jdkato / api.go
Created July 13, 2018 17:08
`prose` v2.0.0 API
package main
import (
"gopkg.in/jdkato/prose.v2"
)
func main() {
// You can pass "functional options" to control the document-creation
// pipeline -- e.g., `prose.WithExtraction(false)` disables named-entity
// extraction.
@jdkato
jdkato / analysis.md
Last active January 30, 2018 18:41
Linode Analysis
@jdkato
jdkato / Demo.c
Created August 9, 2017 06:07 — forked from mikeando/Demo.c
Example of using C++ from C.
#include "HMyClass.h"
#include <stdio.h>
void my_eh( const char * error_message, void * unused)
{
printf("my_eh: %s\n", error_message);
}
int main()
{
#!/bin/sed -f
# Sed script to produce Penn Treebank tokenization on arbitrary raw text.
# Yeah, sure.
# expected input: raw text with ONE SENTENCE TOKEN PER LINE
# by Robert MacIntyre, University of Pennsylvania, late 1995.
# If this wasn't such a trivial program, I'd include all that stuff about
@jdkato
jdkato / recover_source_code.md
Created May 28, 2017 16:40 — forked from simonw/recover_source_code.md
How to recover lost Python source code if it's still resident in-memory

How to recover lost Python source code if it's still resident in-memory

I screwed up using git ("git checkout --" on the wrong file) and managed to delete the code I had just written... but it was still running in a process in a docker container. Here's how I got it back, using https://pypi.python.org/pypi/pyrasite/ and https://pypi.python.org/pypi/uncompyle6

Attach a shell to the docker container

Install GDB (needed by pyrasite)

apt-get update && apt-get install gdb

Keybase proof

I hereby claim:

  • I am jdkato on github.
  • I am jdkato (https://keybase.io/jdkato) on keybase.
  • I have a public key whose fingerprint is E374 305F 11CF 8879 591C 8083 72B6 1C8B C681 C52E

To claim this, I am signing this object:

This is intended as a quick reference and showcase. For more complete info, see [John Gruber's original spec](http://daringfireball.net/projects/markdown/) and the [Github-flavored Markdown info page](http://github.github.com/github-flavored-markdown/).
^ Paragraph start
^ Block end
^ Linting paragraph
This cheatsheet is specifically *Markdown Here's* version of Github-flavored Markdown. This differs slightly in styling and syntax from what Github uses, so what you see below might vary a little from what you get in a *Markdown Here* email, but it should be pretty close.
^ Paragraph start
^ Block end
^ Linting paragraph