himaprasoon’s gists

himaprasoon / convert_to_300kb.py

Created November 14, 2021 09:08

convert_to_300kb

	import os
	from flask import Flask, flash, request, redirect, url_for
	from werkzeug.utils import secure_filename

	import tempfile

	tempdir = tempfile.mkdtemp(prefix="temp_image_upload")

	UPLOAD_FOLDER = tempdir
	ALLOWED_EXTENSIONS = {'png', 'jpg', 'jpeg'}

himaprasoon / zip_folder_python.py

Created February 15, 2021 13:48

Python code snippet to zip a folder

	import os
	import zipfile

	def zip_dir(folder_path, zip_name):
	"""

	:param folder_path: Folder path which has to be zipped
	:param zip_name: the detsinatio zip name
	:return:
	"""

himaprasoon / horovod_setup_mac.sh

Created February 14, 2021 05:52

How to setup horovod with tensorflow on mac os catalina 10.15.5

	HOMEBREW_NO_AUTO_UPDATE=1 brew install openmpi

	HOMEBREW_NO_AUTO_UPDATE=1 brew install cmake

	HOMEBREW_NO_AUTO_UPDATE=1 brew install libuv

	HOROVOD_WITH_MPI=1 pip3 install horovod

himaprasoon / dynamic_init.py

Created July 18, 2020 05:08

Create dynamic init function for base class based on annotations

	from types import FunctionType


	class BaseClass:

	def __new__(cls, args, *kwargs):
	# Creating a custom __init__ function
	params_string = ",".join(cls.params) # For init function arguments
	params_kwargs = ','.join([f"{i}={i}" for i in cls.params]) # Creating
	k=f"""def __init__(self,{params_string}):\n\tBaseClass.__init__(self, {params_kwargs})

himaprasoon / horovod_tf_2.1_docker.MD

Last active February 14, 2021 05:54

Creating a horovod image for tensorflow 2.1

Get the lates tf gpu image

Get Tensorflow GPU Image

docker pull tensorflow/tensorflow:latest-gpu-py3

# apt-get wget
wget https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.3.tar.gz
gunzip -c openmpi-4.0.3.tar.gz | tar xf -

himaprasoon / tensorboard_without_fit_or_summary.py

Last active June 23, 2023 16:36

Easy way to launch tensorboard without calling fit or running model with data in tensorflow 2.0 using callbacks api

	"""
	This example doesn't use summary writer . This is the easiest way to launch tensorboard without calling fit or running model with data
	"""
	from tensorflow.python import keras
	from tensorflow.python.keras import layers
	import tensorflow as tf
	import shutil
	from tensorboard import program

	logdir = "/tmp/my_tensorboard/"

himaprasoon / keras_tf_2.0_rnn_outputs.py

Created July 22, 2019 06:42

Understand return_sequences and return_state in Tensorflow 2.0 Keras RNN layer.

	from tensorflow.python import keras

	output_dim = 3
	time_steps = 10
	input_dim = 4
	cells = [
	keras.layers.LSTMCell(10, name="l1"),
	keras.layers.SimpleRNNCell(11, name="l2"),
	keras.layers.LSTMCell(12, name="l3"),
	]

himaprasoon / tf_get_dependencies.py

Created July 11, 2019 11:23

Gets all placeholders required to run a tensor op

	# source : https://stackoverflow.com/a/47802308/3534616
	def get_tensor_dependencies(tensor):

	# If a tensor is passed in, get its op
	try:
	tensor_op = tensor.op
	except:
	tensor_op = tensor

	# Recursively analyze inputs

himaprasoon / horovod_test.py

Created March 22, 2019 10:52

Test if Horovod is running in multiple machines

	import horovod.tensorflow as hvd
	import tensorflow as tf
	hvd.init()
	hvd_r=int(hvd.rank())
	#each process compute a small part of something and then compute the average etc
	#compute a small part
	x= tf.random_uniform(shape=())
	#compute the average for all processes
	y=hvd.allreduce(x)
	tf.set_random_seed(hvd.rank())

himaprasoon / block_import_decorator.py

Created February 26, 2019 07:44

Blocks system import inside function