Herminio Vazquez canimus

canimus / testcase.ipynb

Last active October 28, 2020 00:11

Test Case - Rules

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

canimus / iso.sh

Created October 18, 2020 17:58

Write ISO to USB

sudo dd bs=4M if=/path/to/manjaro.iso of=/dev/sd[drive letter] status=progress oflag=sync

canimus / clean-docker-log.sh

Created July 17, 2020 11:26

Clean docker logs

truncate -s 0 $(docker inspect --format='{{.LogPath}}' t1)

canimus / qname.py

Created April 18, 2020 19:02

Etree remove namespace

	from lxml import etree
	from functional import seq
	p = etree.load(open('a.xml'))
	root = p.getroot()
	seq(root.getchildren()).map(lambda x: etree.QName(x).localname)

canimus / sparker.py

Created March 20, 2020 18:55

Spark custom separator reader

sc.binaryFiles(new_file).values().flatMap(lambda x: x.decode("iso-8859-1").split(chr(172))).map(lambda x: x.split(chr(171)))

canimus / datastore_reader.py

Created February 4, 2020 23:35

Custom delimiter file reader

	def DataStoreFileReader(file, chunk_size=512, lineterminator=172, delimiter=171):
	chunk = ""
	while True:
	curr = file.read(chunk_size)
	chunk += curr
	if not curr:
	break
	if chr(lineterminator) in chunk:
	lines = chunk.split(chr(lineterminator))
	for line in lines[0:-1]:

canimus / asyncio.py

Created January 22, 2020 20:46

Asyncio Example with Function and Subprocess

	import asyncio
	import glob
	import aiofiles
	import subprocess


	async def read_file(f):
	print("start: " + f)
	async with aiofiles.open(f, 'r') as fd:
	lines = await fd.read()

canimus / graph.html

Created January 8, 2020 23:40

JsGraphNetworkX

	<!DOCTYPE html>
	<html lang="en">
	<head>
	<meta charset="UTF-8">
	<meta name="viewport" content="width=device-width, initial-scale=1">
	<meta http-equiv="X-UA-Compatible" content="ie=edge">
	<script src="/static/d3.min.js"></script>
	<script src="/static/vue.js"></script>
	<script src="/static/jsnetworkx.js"></script>

canimus / msgpack.js

Created December 20, 2019 16:47

msgpack javascript example

	function readFile(stream) {
	return new Promise((resolve, reject) => {
	var fr = new FileReader();
	fr.onload = () => {
	resolve(msgpack.decode(new Uint8Array(fr.result)))
	};
	fr.readAsArrayBuffer(stream);
	});
	}

canimus / extract_data.py

Created December 15, 2019 20:57

DaskDataFrame Collector Parquet

	def extract_data(query_var, file_prefix):
	idx = pd.date_range(start="2019-01-01", periods=13, freq="MS").strftime("%Y-%m-%d").values
	dt = []
	for i in range(len(idx)-1):
	name = str(i+1).zfill(2)
	df = pd.read_sql(query_var.format(idx[i], idx[i+1]), conn)
	dt.append(df.dtypes)
	df.to_parquet(f'parquet/{file_prefix}_{name}.parquet')

	# Unique dataframe with all types