Tarang Shah t27

Setting up TOCO on macOS

Steps

Install bazel

brew tap bazelbuild/tap
brew tap-pin bazelbuild/tap
brew install bazel

Dealing with the Linux OOM Killer at the program level

Do this in cases when you dont want to change the os-level settings, but only want to disable the OOM killer for a single process. This is useful when youre on a shared machine/server.

The OOM killer uses the process level metric called oom_score_adj to decide if/when to kill a process. This file is present in /proc/$pid/oom_score_adj. The oom_score_adj can vary from -1000 to 1000, by default it is 0.

You can add a large negative score to this file to reduce the probability of your process getting picked and terminated by OOM killer. When you set it to -1000, it can use 100% memory and still avoid getting terminated by OOM killer.

	#List files by extension
	# from https://unix.stackexchange.com/a/18508/
	#find <folder> -type f \| sed 's/.*\.//' \| sort \| uniq -c
	find DEU/ -type f \| sed 's/.*\.//' \| sort \| uniq -c

	from tensorboard.backend.event_processing import event_accumulator
	import json

	eval_log_file_path = "path/to/events.out.tfevents.1555555555.0.0.0.0"

	def modelEvalPathParser(path_to_eval_log):
	ea = event_accumulator.EventAccumulator(path_to_eval_log)
	ea.Reload()
	jsonresult = {}
	csvresult = "classname,ap\n"

	#
	# Sets Prezto options.
	#
	# Authors:
	# Sorin Ionescu <[email protected]>
	#

	#
	# General
	#

	--Postgres
	--just find duplicates
	select col1,col2,col3,col4,col5,count() from my_table group by 1,2,3,4,5,6 HAVING count()>1

	-- find duplicates and aggregate Ids, so that you can save one and delete the rest ('id' is a column in the table)
	select col1,col2,col3,col4,col5,count(), string_agg(id::character varying, ';' order by sign_code) from my_table group by 1,2,3,4,5,6 HAVING count()>1

	# need to install `rename`
	find /path/to/directory -depth -exec rename 's/[<>:"\\\|?*]/_/g' {} +

	document.querySelectorAll('#center > div > tf-category-pane > button').forEach(e=>e.click())
	// just copy the above and run in browser console to open all the image previews in a Tensorboard.
	// Especially useful if youre logging image visualisations(for eg. detections in an object detector model)

	# https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%20every%20X%20seconds%20of%20the%20video
	ffmpeg -i input.flv -vf fps=1 out%d.png

	# fps is number of images per second