Skip to content

Instantly share code, notes, and snippets.

View tanthml's full-sized avatar
💭
I may be slow to respond.

Tan TRAN tanthml

💭
I may be slow to respond.
  • Internet
  • Ho Chi Minh
  • 15:57 (UTC +07:00)
View GitHub Profile
@tanthml
tanthml / kullback_leibler.py
Created April 26, 2018 08:52 — forked from ognis1205/kullback_leibler.py
Calculate Kullback-Leibler Divergence of Given Corpus
#!/usr/bin/env python
# -*- coding: utf-8 -*-
""" Caluculates symmetric Kullback-Leibler divergence.
@author Shingo OKAWA
"""
import numpy
import sys
import scipy.stats as stats
import matplotlib.pyplot as plotter
from gensim import corpora, models, similarities, matutils

Hello all. We must choose an extraction library for our new Goliath system to provide default values when no rules have been set. The choice has been narrowed down to Boilerpipe and Goose. They both have sub-par documenation (Boilerpipe, Goose), so I've dug around in the code to find the exact process by which they pull out data. Here I will compare them so we can choose one.

#Boilerpipe

This bad mamba jamba was developed by a Ph.D-having guy who, along with some other folks, wrote a big fat academic paper around the algorithm it uses, which you can find in our Dropbox if you really want to read it. Basically, they use link density, text density, and number of words on a block-by-block basis to distinguish boilerplate blocks from content blocks. A block is simply a contiguous piece of text terminated by the start

@tanthml
tanthml / 1-elementary-os-apps.md
Created April 3, 2018 09:04 — forked from ankurk91/1-elementary-os-apps.md
elementary OS 0.4.1 Loki

elementaryOS Apps and Configs

Enbale PPA support

sudo apt-get update
sudo apt-get -y install software-properties-common

Install original plank dock

@tanthml
tanthml / botos3upload.py
Created August 29, 2016 02:24 — forked from SavvyGuard/botos3upload.py
Use boto to upload directory into s3
import boto
import boto.s3
import os.path
import sys
# Fill these in - you get them when you sign up for S3
AWS_ACCESS_KEY_ID = ''
AWS_ACCESS_KEY_SECRET = ''
# Fill in info on data to upload