Skip to content

Instantly share code, notes, and snippets.

View serihiro's full-sized avatar

Kazuhiro Serizawa serihiro

View GitHub Profile
@serihiro
serihiro / reading_list.md
Created December 22, 2018 06:31
2018年に大学の図書館で借りた本リスト
  • スーパーコンピュータ. 小柳義夫 [ほか] 著. 岩波書店, 2012 (岩波講座計算科学:別巻).
  • コンパイラの構成と最適化, 第2版. 中田育男著. 朝倉書店, 2009.
  • 宇宙ロケット工学入門. 宮澤政文著. 朝倉書店, 2016.
  • ネットワーク分析, 第2版. 鈴木努著. 共立出版, 2017 (Rで学ぶデータサイエンス:8).
  • きつねさんでもわかるLLVM : コンパイラを自作するためのガイドブック. 柏木餅子, 風薬著/矢上栄一イラスト. インプレスジャパン/インプレスコミュニケーションズ, 2013.
  • Chainer v2による実践深層学習. 新納浩幸著. オーム社, 2017.
  • 異常検知と変化検知. 井手剛, 杉山将著. 講談社, 2015 (MLP機械学習プロフェッショナルシリーズ).
  • 基礎からわかる時系列分析 : Rで実践するカルマンフィルタ・MCMC・粒子フィルタ. 萩原淳一郎, 瓜生真也, 牧山幸史著. 技術評論社, 2018 (Data science library).
  • データ分析プロセス. 福島真太朗著. 共立出版, 2015 (シリーズUseful R:2).
  • Rで学ぶクラスタ解析. 新納浩幸著. オーム社, 2007.
@serihiro
serihiro / Csv4.c
Last active November 28, 2018 06:12
wgrib2 csv plugin
/******************************************************************************************
Copyright (C) 2008 Niklas Sondell, Storm Weather Center
This file is part of wgrib2 and could be distributed under terms of the GNU General Public License
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "grb2.h"
@serihiro
serihiro / Csv3.c
Created November 22, 2018 03:17
wgrib2 extended csv output plugin
/******************************************************************************************
Copyright (C) 2008 Niklas Sondell, Storm Weather Center
This file is part of wgrib2 and could be distributed under terms of the GNU General Public License
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "grb2.h"
@serihiro
serihiro / generate_gray_scale_image_list.py
Last active November 14, 2018 04:00
Find gray scale image and generate a list of them
from PIL import Image
import numpy as np
import argparse
import glob
import sys
import concurrent.futures
def is_gray_scale(path):
try:
import pynvml
pynvml.nvmlInit()
handle0 = pynvml.nvmlDeviceGetHandleByIndex(0)
handle1 = pynvml.nvmlDeviceGetHandleByIndex(1)
util_rate0 = pynvml.nvmlDeviceGetUtilizationRates(handle0)
util_rate1 = pynvml.nvmlDeviceGetUtilizationRates(handle1)
@serihiro
serihiro / summarize_chainer_log.py
Last active October 31, 2018 03:56
A just script to summarize chainer log and output as csv
import argparse
import csv
import json
import glob
import numpy as np
parser = argparse.ArgumentParser()
parser.add_argument('--input', type=str, required=True)
parser.add_argument('--output', default='result.csv')
args = parser.parse_args()
@serihiro
serihiro / nvml_sample.c
Last active October 12, 2018 06:46
nvml sample(written in c), nvml API reference is here: https://docs.nvidia.com/deploy/nvml-api/index.html
#include <stdio.h>
#include <nvml.h>
int main(int argc, char **argv)
{
nvmlReturn_t result;
result = nvmlInit();
if (NVML_SUCCESS != result)
{
@serihiro
serihiro / not_easy_to_remember_and_google.md
Created October 9, 2018 13:56
たまに名前を忘れるがググり方も難しいのでメモっておくリスト

pkg-config

  • Provide full path of cflags and libs
@serihiro
serihiro / num_records.patch
Created September 6, 2018 20:52
temporary patch
diff --git a/digdag-standards/src/main/java/io/digdag/standards/operator/td/BaseTdJobOperator.java b/digdag-standards/src/main/java/io/digdag/standards/operator/td/BaseTdJobOperator.java
index 386a1499e..15c22ecd4 100644
--- a/digdag-standards/src/main/java/io/digdag/standards/operator/td/BaseTdJobOperator.java
+++ b/digdag-standards/src/main/java/io/digdag/standards/operator/td/BaseTdJobOperator.java
@@ -61,7 +61,10 @@ abstract class BaseTdJobOperator
// Set last_job_id param
taskResult.getStoreParams()
.getNestedOrSetEmpty("td")
- .set("last_job_id", job.getJobId());
+ .set("last_job_id", job.getJobId()) // for compatibility with old style
@serihiro
serihiro / __init__.py
Last active September 6, 2018 19:54
digdag task parallelization experiments
import os
import time
import random
import digdag
class ParamSet(object):
def set_param(self):
time.sleep(int(os.environ['sleep']))
digdag.env.store({'value': os.environ['value']})
print(f"{os.environ['value']} finished")