Skip to content

Instantly share code, notes, and snippets.

View ikegami-yukino's full-sized avatar

IKEGAMI Yukino ikegami-yukino

377 followers · 27 following
View GitHub Profile
@ikegami-yukino
ikegami-yukino / wikipedia_anob.py
Created December 19, 2013 16:48
Wikipediaの不要見出し語をカットするためのsuffix一覧
#! /usr/bin/env python
# -*- coding: utf-8 -*-
from collections import Counter
import re
suffixes = Counter()
re_anob = re.compile(u'(?P<A>.+[^の])の(?P<B>[^の].+)')
re_hiragana = re.compile(u'[ぁ-ゖ]+')
def extract_anob(text):
@ikegami-yukino
ikegami-yukino / gist:6928667
Created October 11, 2013 02:20
Mac OS Xにmadokaインストール
git clone https://github.com/s-yata/madoka.git
cd madoka
glibtoolize --copy; aclocal ; autoheader; automake -a -c --add-missing; autoconf
./configure
make
sudo make install
@ikegami-yukino
ikegami-yukino / mahout_ubuntu_install.sh
Created October 10, 2013 10:56
buntu 12.10にMahout 0.8を入れるスクリプト
#
# Ubuntu 12.10にMahout 0.8を入れるスクリプト
#
# Installing Maven
#sudo apt-get install maven
sudo apt-get install subversion
# Installing Mahout
@ikegami-yukino
ikegami-yukino / hadoop_ubuntu_install.sh
Last active December 25, 2015 03:59
Ubuntu 12.10にHadoopを入れるスクリプト
#
# Ubuntu 12.10にHadoopを入れるスクリプト
#
# Java setting
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer
sudo update-java-alternatives -s java-7-oracle
echo 'export JAVA_HOME=/usr/lib/jvm/java-7-oracle' >> ~/.bashrc
@ikegami-yukino
ikegami-yukino / oll_client.py
Last active September 13, 2017 08:51
OLL client is a client for using OLL on Python. OLL is a library supporting several for online-learning algorithms.
from subprocess import Popen, PIPE
import time, sys, os
"""
oll_client 0.0.3 - Online Machine Learning Module
OLL client is a client for using OLL, which is a machine learning library have implemented several online-learning algorithms, on Python.
Currently, Oll supports following algorithms:
@ikegami-yukino
ikegami-yukino / naistjdic2ipadic.py
Created June 8, 2013 06:04
NAIST-JDicをIPA-Dicの文脈IDに変換
#coding:utf-8
"""
NAIST-JDicをIPA-Dicの文脈IDに変換
以下の品詞細分類の単語は無視される
五段・タ行/五段・ナ行/五段・バ行/五段・ワ行ウ音便
"""
import sys, codecs, optparse
@ikegami-yukino
ikegami-yukino / palindromejp.py
Last active May 15, 2018 16:11
MeCabで日本語の回文判定
# -*- coding: utf-8 -*-
import MeCab
mecab_y = MeCab.Tagger("-Oyomi")
def toYomi(text):
return mecab_y.parse(text).rstrip().replace(' ', '')
@ikegami-yukino
ikegami-yukino / reverse.c
Created March 13, 2013 02:47
インプレイス処理で文字列を反転する
#include<stdio.h>
#define NULL_LENGTH 4
char* reverse(char* text){
int i, length = sizeof text - NULL_LENGTH;
for(i = length;i > (length / 2);i--){
*(text+(i)) = *(text+(i)) ^ *(text+(length-i));
*(text+(length-i)) = *(text+(i)) ^ *(text+(length-i));
*(text+(i)) = *(text+(i)) ^ *(text+(length-i));
}
@ikegami-yukino
ikegami-yukino / crossvalidation.sh
Created March 13, 2013 01:52
交差検定検定用スクリプト (ファイルを行ごとにランダムに分割してトレーニング用コマンドと検定用コマンドを実行する) For N-fold cross validation, execute given commands, for training and testing, after randomize and split data.
#!/bin/bash
# Check the number of parameters
if [ $# -ne 4 ]; then
echo "usage: ./cv.sh [FILE] [division number] [train command] [test command]" 1>&2
echo "The file for training is named as trainfile" 1>&2
echo "The file for testing is named as testfile" 1>&2
echo "for example:" 1>&2
echo './cv.sh data 5 "opal trainfile train.model -" "opal - train.model testfile"' 1>&2
exit 1
@ikegami-yukino
ikegami-yukino / randomsplit.sh
Last active July 31, 2018 03:01
ファイルを行ごとにランダムにソートして、指定した数のファイルに分割する。randomly sort and split given file in given number.
#!/bin/bash
# usage:
# ./randomsplit.sh [FILE] [division number]
#
# Check the number of parameters
if [ $# -ne 2 ]; then
echo "usage: ./randomsplit.sh [FILE] [division number]" 1>&2
exit 1