Skip to content

Instantly share code, notes, and snippets.

@thomwolf
thomwolf / parallel.py
Last active August 8, 2023 15:50
Data Parallelism in PyTorch for modules and losses
##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
## Created by: Hang Zhang, Rutgers University, Email: [email protected]
## Modified by Thomas Wolf, HuggingFace Inc., Email: [email protected]
## Copyright (c) 2017-2018
##
## This source code is licensed under the MIT-style license found in the
## LICENSE file in the root directory of this source tree
##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
"""Encoding Data Parallel"""

Quick Tips for Fast Code on the JVM

I was talking to a coworker recently about general techniques that almost always form the core of any effort to write very fast, down-to-the-metal hot path code on the JVM, and they pointed out that there really isn't a particularly good place to go for this information. It occurred to me that, really, I had more or less picked up all of it by word of mouth and experience, and there just aren't any good reference sources on the topic. So… here's my word of mouth.

This is by no means a comprehensive gist. It's also important to understand that the techniques that I outline in here are not 100% absolute either. Performance on the JVM is an incredibly complicated subject, and while there are rules that almost always hold true, the "almost" remains very salient. Also, for many or even most applications, there will be other techniques that I'm not mentioning which will have a greater impact. JMH, Java Flight Recorder, and a good profiler are your very best friend! Mea

@kmclaugh
kmclaugh / Keras_Linear_Model.py
Last active February 4, 2021 15:13
Keras Model for a Simple Linear Function (ie Keras modeling a linear regression)
import pandas as pd
import numpy as np
import seaborn as sns
from keras.layers import Dense
from keras.models import Model, Sequential
from keras import initializers
## ---------- Create our linear dataset ---------------
## Set the mean, standard deviation, and size of the dataset, respectively

Développeur Backend Scala - H/F (ARTIK Cloud - SAMSUNG SSIC - PARIS Centre)

Le Projet

ARTIK Cloud est le futur Cloud IoT de Samsung. C'est une plateforme distribuée, dans le “cloud”, qui collecte héberge et sécurise les données venant de n’importe quel objet connecté. ARTIK Cloud est aussi ouvert aux développeurs d'applications souhaitant, avec l'accord de l'utilisateur, fournir des services basés sur ses données IoT. L'API est publique et disponible pour tous: https://developer.artik.cloud/.

Ce projet est développé au sein d'un département innovation chez Samsung: SSIC (Samsung Strategy and Innovation Center) basé à San José, San Franciso et Paris.

Le Poste

@awni
awni / ctc_decoder.py
Last active September 18, 2024 11:44
Example CTC Decoder in Python
"""
Author: Awni Hannun
This is an example CTC decoder written in Python. The code is
intended to be a simple example and is not designed to be
especially efficient.
The algorithm is a prefix beam search for a model trained
with the CTC loss function.
@timvw
timvw / ByteStringSerializer.scala
Created December 20, 2016 19:03
Kafka serializer for Akka ByteString
val byteStringSerializer = new org.apache.kafka.common.serialization.Serializer[ByteString]() {
override def configure(configs: util.Map[String, _], isKey: Boolean): Unit = {}
override def serialize(topic: String, data: ByteString): Array[Byte] = data.toArray
override def close(): Unit = {}
}
@milessabin
milessabin / gist:25b7b669b5a9ac051a71
Created June 5, 2015 14:32
A safe ADT+shapeless drop-in replacement for the unsafe standard Scala Enumeration ...
// An ADT+shapeless as a drop-in replacement for a standard Scala Enumeration.
//
// First the unsafe standard Scala Enumeration ...
//
object ScalaEnumDemo extends App {
// Example from scala.Enumeration scaladoc. Terse ...
object WeekDay extends Enumeration {
type WeekDay = Value
val Mon, Tue, Wed, Thu, Fri, Sat, Sun = Value
}
@atamborrino
atamborrino / prolog.scala
Last active August 29, 2015 14:13
Playing with type level programming
// We want to use the Scala type system as a logic programming language by resolving at compile
// the simple "ancestor" Prolog exercise.
import shapeless._
import syntax.singleton._
// Prolog version
// parent(stacy, bob).
// parent(bob, bill).
@jbenet
jbenet / simple-git-branching-model.md
Last active November 9, 2024 04:55
a simple git branching model

a simple git branching model (written in 2013)

This is a very simple git workflow. It (and variants) is in use by many people. I settled on it after using it very effectively at Athena. GitHub does something similar; Zach Holman mentioned it in this talk.

Update: Woah, thanks for all the attention. Didn't expect this simple rant to get popular.

@gre
gre / deploy.sh
Last active October 8, 2021 00:33
Super-small scripts for easy PlayFramework deployment
#!/bin/bash
REMOTE=play@SERVER_IP
REMOTE_APP=/home/play/PROJECT_NAME/
sbt stage || exit 1;
rsync -va target/ $REMOTE:$REMOTE_APP/target;
ssh $REMOTE "cd $REMOTE_APP; ./stop.sh";
ssh $REMOTE "cd $REMOTE_APP; ./start.sh";