hsm207’s gists

hsm207 / subreddit_latest.py

Created March 11, 2017 01:33 — forked from dangayle/subreddit_latest.py

Get all available submissions within a subreddit newer than x

	import sys
	from datetime import datetime, timedelta
	import praw

	user_agent = "hot test 1.0 by /u/dangayle"
	r = praw.Reddit(user_agent=user_agent)

	class SubredditLatest(object):
	"""Get all available submissions within a subreddit newer than x."""

hsm207 / endtoend_estimator_keras_repro.py

Created January 25, 2018 09:42 — forked from zmjjmz/endtoend_estimator_keras_repro.py

keras estimator weirdness.

	from __future__ import division

	import urlparse
	import os

	import numpy
	import boto3
	import tensorflow
	from tensorflow.python.keras._impl import keras
	from tensorflow.python.estimator.export.export_output import PredictOutput

hsm207 / conv2d_as_matmul.ipynb

Last active January 17, 2024 17:12

Code to accompany my blog post at https://bit.ly/2KfmQ76

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

hsm207 / einsum_with_pytorch.ipynb

Last active May 11, 2022 13:57

Code to accompany my blog post at https://bit.ly/2PmRjiC

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

hsm207 / rpr_with_einsum_gpu.ipynb

Last active April 21, 2019 11:08

Code to accompany my blog post at https://bit.ly/2PmRjiC

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

hsm207 / getAnalytics.scala

Last active September 16, 2019 13:27

implementation of getAnalytics

	def getAnalytics(bucketName: String, brand: String, bucketYear: String, bucketMonth: String, bucketDay: String, candidate_field: String=candidateField, groups: String=Groups): DataFrame = {
	var rankingOrderedIds = Window.partitionBy("c12").orderBy("id")
	val s3PathAnalytics = getS3Path(bucketName, brand, bucketFolder, year=bucketYear, month=bucketMonth, day=bucketDay)
	readJSON(s3PathAnalytics)
	.distinct
	.withColumn("x", explode($"payload"))
	// a few more calls to withColumn to create columns
	.withColumn("c10", explode(when(size(col("x1")) > 0, col("x1")).otherwise(array(lit(null).cast("string")))))
	// a few more calls to withColumn to create columns
	.withColumn("id", monotonically_increasing_id)

hsm207 / bert_pretraining_share.ipynb

Created October 1, 2019 23:50

BERT_pretraining_share.ipynb

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

hsm207 / keras_estimator_feature_column_bug.ipynb

Created October 22, 2019 15:56

keras_estimator_feature_column_bug.ipynb

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

hsm207 / working_with_random_samples.html

Last active December 24, 2019 12:43

Working with random samples on Spark

	<!DOCTYPE html>
	<html>
	<head>
	<meta name="databricks-html-version" content="1">
	<title>working_with_random_samples - Databricks</title>

	<meta charset="utf-8">
	<meta name="google" content="notranslate">
	<meta name="robots" content="nofollow">
	<meta http-equiv="Content-Language" content="en">

hsm207 / copy_redis.sh

Last active February 5, 2020 17:33

Script to migrate all keys inside a redis instance into another redis instance

	# source: https://stackoverflow.com/questions/37166947/copying-all-keys-in-redis-database-using-migrate

	# make sure the redis in SOURCE_HOST and DESTINATION_HOST are identical!

	SOURCE_HOST=127.0.0.1
	SOURCE_PORT=6381

	DESTINATION_HOST=foo.bar
	DESTINATION_PORT=6379

hsm207 hsm207