Skip to content

Instantly share code, notes, and snippets.

View YuukiToriyama's full-sized avatar
🚲
On Cycling

ToriChan YuukiToriyama

🚲
On Cycling
  • Kyoto, Japan
View GitHub Profile
@mak00s
mak00s / process-large-csv-with-dask.py
Last active November 14, 2024 00:41
PythonのDASKを使ってpandasでは処理できない巨大CSVを前処理する方法
import dask.dataframe as dd
# 分割されたCSVでも一つの巨大CSVでも同じように読み込める
file1 = 'DW-exported-*.csv'
df = dd.read_csv(file1, header=0, names=('Pages','VisitNum','HitDepth','Date','CV','Customer ID','Referrer','PV'), dtype={'Pages':'object','VisitNum':'int16','HitDepth':'int16','Date':'object','CV':'object','eVar13':'object','Referrer':'object','PV':'int16'})
# カラム名や型を指定しない場合
#df = dd.read_csv(file1, header=0)
# この結果、複数のパーティションに分割されたpandas dataframeが生成される
ply
format ascii 1.0
comment Created by Blender 2.79 (sub 0) - www.blender.org, source file: ''
comment # Copyright 2019 The TensorFlow Authors. All Rights Reserved.
comment #
comment # Licensed under the Apache License, Version 2.0 (the "License");
comment # you may not use this file except in compliance with the License.
comment # You may obtain a copy of the License at
comment #
comment # http://www.apache.org/licenses/LICENSE-2.0
@0rbianta
0rbianta / jobject_examples.h
Last active December 21, 2024 13:36
JNI Jobject examples
using namespace std;
//----------------------Convert a Java data------------------------\\
jstring create_jstring(JNIEnv *env, const char *data) {
return env->NewStringUTF(data);
}
jint create_jint(JNIEnv *env, int data){