Skip to content

Instantly share code, notes, and snippets.

@fish8
fish8 / read_timematrix.scala
Created November 12, 2021 07:30
read time matrix
case class TimeMatrixCase(
userId: Long,
windowDateList: List[String],
sourceCellIdPlaceId: Map[String, Int],
timeMatrix: List[List[Int]],
time_map: Map[String, Int],
weekdayDaySeconds: Map[String, Int],
weekdayEveningSeconds: Map[String, Int],
weekendDaySeconds: Map[String, Int],
weekendEveningSeconds: Map[String, Int],
@fish8
fish8 / networkx-graphviz.py
Created October 6, 2021 17:23
draw networkx graph with graphviz
from networkx.drawing.nx_agraph import graphviz_layout, to_agraph
from PIL import Image
from io import BytesIO
A = to_agraph(g2)
A.layout(prog='dot')
# graphviz_layout(prog=neato)
img = A.draw(format='png')
image = Image.open(BytesIO(img))
image.show(title="Graph")
@fish8
fish8 / name services
Last active April 2, 2022 05:37
cubd/ss name services
大集群: hdfs://10.244.12.215:8020
小集群: hdfs://10.244.12.75:8020
花园集群: hdfs://10.244.13.106:8020
@fish8
fish8 / sq usage.sh
Created March 8, 2021 11:35
sq usage
#bash command completion
source <(sq completion bash)
#add source
sq add ./CBDB_20200528.db -h @cbdb_20200528
#list source
sq ls
#inspecting, list table names
@fish8
fish8 / location2utm.scala
Created February 27, 2021 07:58
location2utm scala
import scala.math.Integral.Implicits._
def location2utm(location_id:Long):(Long,Long,Long) = {
// convert a location id to utm coordination
// THIS IS THE UTM VALUE AS IS IN SS OUTPUTS. TO GET CORRECTED UTM/LAT-LON USE sstool.ssutm.SSUtm
// return a tuple of (x, y, size) as left-bottom corner of a rectange
// if location_id < 100000000000:
// return None
@fish8
fish8 / jupyter table showing.py
Created February 25, 2021 06:25
itables qgrid datatables
#itables
from itables import init_notebook_mode
init_notebook_mode(all_interactive=True)
import world_bank_data as wb
df = wb.get_countries()
df
#qgrid
qgrid.show...
@fish8
fish8 / jupyter-code.sh
Last active February 5, 2021 01:21
jupyter notebook code search and print
#pynb-code, show all codes in a jupyter notebook
jq -j '.cells | map( select(.cell_type == "code") | .source +["\n\n"]) | .[][] ' $@
#beautify code in notebook and copy to clipboard as rich text
pynb-code xxx.ipynb |pygmentize -f rtf | xclip -t text/rtf -selection clipboard
#find print
find . -name "*.ipynb" -exec pynb-code {} +
#find grep
@fish8
fish8 / kde_ridge_plot.py
Created January 7, 2021 05:29
ridge plot, joyplot
"""
Overlapping densities ('ridge plot')
====================================
"""
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
@fish8
fish8 / command line excel.bash
Last active September 14, 2020 07:15
command line excel, csv
#to csv
#via csvkit
in2csv --sheet 区县级 基站工参统计.xlsx #to stdout
# via Gnumeric
ssconvert -T Gnumeric_stf:stf_csv 基站工参统计.xlsx
#select columns
in2csv --sheet 区县级 基站工参统计.xlsx |csvcut -c "城市名称","人数"
@fish8
fish8 / pdf ucr.bash
Last active February 4, 2021 03:51
pdf ocr
# via Tesseract
pdfsandwich -lang chi_sim zgsy.pdf
#merging pages
pdftk A=zgsy.pdf B=zgsy_ocr.pdf cat A1 B2-27 A28 B29-57 A58 B59-140 A141 B142-end output zzz.pdf