Skip to content

Instantly share code, notes, and snippets.

View DmitryBe's full-sized avatar

Dmitry B DmitryBe

View GitHub Profile
@younesbelkada
younesbelkada / bnb-serialization.py
Created December 25, 2023 18:10
push bnb 4 bit models on the hub
# pip install -U bitsandbytes
# pip install -U git+https://github.com/huggingface/transformers.git
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "mistralai/Mixtral-8x7B-Instruct-v0.1"
path_to_hub = XXX
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True)
model.push_to_hub(path_to_hub)
@afranzi
afranzi / SparkWithHadoop.md
Last active February 18, 2024 18:45
Spark 2.4.0 with Hadoop 2.8.5

Setup Environmnet variables for Hadoop.

export HADOOP_VERSION=2.8.5
export HADOOP_HOME=${HOME}/hadoop-$HADOOP_VERSION
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export PATH=${HADOOP_HOME}/bin:$PATH

Download Hadoop files.

@iandanforth
iandanforth / continuous_cartpole.py
Last active January 9, 2025 19:51
Continuous Cartpole for OpenAI Gym
"""
Classic cart-pole system implemented by Rich Sutton et al.
Copied from http://incompleteideas.net/sutton/book/code/pole.c
permalink: https://perma.cc/C9ZM-652R
Continuous version by Ian Danforth
"""
import math
import gym
@zparnold
zparnold / one_liner.sh
Last active January 13, 2025 17:40
A simply script to delete all failed pods from Kubernetes
kubectl get pods --all-namespaces | grep Evicted | awk '{print $2 " --namespace=" $1}' | xargs kubectl delete pod
@EvenOldridge
EvenOldridge / ExportKerasToTensorflowServing.py
Last active January 11, 2019 21:42
Export a Keras Model for Tensorflow Serving
import numpy as np
import tensorflow as tf
import keras as k
from keras.applications.resnet50 import ResNet50
from keras import backend as K
from keras.layers.core import Flatten, Dense, Dropout, Lambda
from keras.models import Model
from keras.preprocessing import image
@jubel-han
jubel-han / pipeline_venv_workarounds.groovy
Created April 13, 2017 08:14
Jenkins pipeline python virtualenv workarounds
node {
stage 'Checkout and Build'
createVirtualEnv 'env'
executeIn 'env', 'pip install -r requirements.txt'
executeIn 'env', './manage.py test'
executeIn 'env', './manage.py integration-test'
virtualEnv('true')
runCmd('pip install -r requirements.txt')
@ruzickap
ruzickap / aws_create_site.yml
Created February 16, 2017 12:35
Ansible playbook which creates instances and tag volumes
---
- name: Create Instance in AWS
hosts: localhost
connection: local
gather_facts: false
vars:
aws_access_key: "xxxxxx"
aws_secret_key: "xxxxxx"
security_token: "xxxxxx"
@alexislucena
alexislucena / install-scala-sbt-and-java-on-ubuntu.md
Created December 5, 2016 12:35
Ubuntu: Install Scala, SBT and Java on Ubuntu 16.04

Install Scala 2.11.8

$ sudo apt-get remove scala-library scala
$ sudo wget www.scala-lang.org/files/archive/scala-2.11.8.deb
$ sudo dpkg -i scala-2.11.8.deb

Check Scala version

$ scala -version
@karpathy
karpathy / pg-pong.py
Created May 30, 2016 22:50
Training a Neural Network ATARI Pong agent with Policy Gradients from raw pixels
""" Trains an agent with (stochastic) Policy Gradients on Pong. Uses OpenAI Gym. """
import numpy as np
import cPickle as pickle
import gym
# hyperparameters
H = 200 # number of hidden layer neurons
batch_size = 10 # every how many episodes to do a param update?
learning_rate = 1e-4
gamma = 0.99 # discount factor for reward
@fancellu
fancellu / MergeSort.scala
Created November 25, 2015 12:12
MergeSort in Scala with recursive merge
object MergeSort {
// recursive merge of 2 sorted lists
def merge(left: List[Int], right: List[Int]): List[Int] =
(left, right) match {
case(left, Nil) => left
case(Nil, right) => right
case(leftHead :: leftTail, rightHead :: rightTail) =>
if (leftHead < rightHead) leftHead::merge(leftTail, right)
else rightHead :: merge(left, rightTail)