Skip to content

Instantly share code, notes, and snippets.

View onesuper's full-sized avatar

Dreamsome onesuper

View GitHub Profile
@younesbelkada
younesbelkada / finetune_llama_v2.py
Last active April 7, 2025 18:27
Fine tune Llama v2 models on Guanaco Dataset
# coding=utf-8
# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
@abelsonlive
abelsonlive / dbt_backfill.py
Created April 11, 2019 02:14
Script for backfilling DBT models which use partitions
#!/usr/bin/env python
import json
import copy
import sys
import argparse
from datetime import datetime, timedelta
from subprocess import Popen, PIPE
# dictionary of partiton names to variable generating functions
@darkarnium
darkarnium / SpotFleets.md
Last active September 26, 2022 08:16
A quick and dirty AWS EC2 Spot Fleet requestor

SpotFleets

The following code will request an AWS SpotFleet with the specified parameters.

Credentials

By default credentials will be located using boto3's built-in enumeration mechanism. The easiest way to ensure that credentials are available is to either use environment variables, or ensure there is a ~/.aws/credentials file for the user running this script.

Required Code Changes

@stebennett
stebennett / feeds.opml
Created August 12, 2016 15:24
The feeds I currently subscribe to.
<?xml version="1.0" encoding="UTF-8"?>
<opml version="1.0">
<head>
<title>Stephen subscriptions in feedly Cloud</title>
</head>
<body>
<outline text="Companies" title="Companies">
<outline type="rss" text="Trello" title="Trello" xmlUrl="https://trello.engineering/feed.xml" htmlUrl="http://tech.trello.com/"/>
<outline type="rss" text="IMVU" title="IMVU" xmlUrl="http://engineering.imvu.com/feed/" htmlUrl="https://engineering.imvu.com"/>
@JonCole
JonCole / Redis-BestPractices-General.md
Last active March 13, 2025 14:30
Redis Best Practices

Some of the Redis best practices content has moved

This content from this markdown file has moved a new, happier home where it can serve more people. Please check it out : https://docs.microsoft.com/azure/azure-cache-for-redis/cache-best-practices.

NOTE: Client specific guidance listed below is still valid and should still be considered. I will update this document once all content has been moved.

@arturmkrtchyan
arturmkrtchyan / get_job_status.sh
Last active October 22, 2024 05:45
Apache Spark Hidden REST API
curl http://spark-cluster-ip:6066/v1/submissions/status/driver-20151008145126-0000
@MarioAriasC
MarioAriasC / spark.kt
Last active December 21, 2019 11:22
Word Count with Apache Spark and Kotlin
package org.cakesolutions.spark
import org.apache.spark.SparkConf
import org.apache.spark.api.java.JavaSparkContext
import scala.Tuple2
fun main(args: Array<String>) {
val inputFile = args[0]
val outputFile = args[1]
@jseabold
jseabold / spot_pricing.py
Created August 11, 2015 14:33
Plot EC2 spot pricing with boto3 and pandas
import pandas as pd
from boto3 import client
client = client(service_name='ec2')
prices = client.describe_spot_price_history(InstanceTypes=["m3.medium"],
AvailabilityZone="us-east-1a")
df = pd.DataFrame(prices['SpotPriceHistory'])
df.set_index("Timestamp", inplace=True)
df["SpotPrice"] = df.SpotPrice.astype(float)
@ololobus
ololobus / Spark+ipython_on_MacOS.md
Last active September 26, 2024 08:50
Apache Spark installation + ipython/jupyter notebook integration guide for macOS

Apache Spark installation + ipython/jupyter notebook integration guide for macOS

Tested with Apache Spark 2.1.0, Python 2.7.13 and Java 1.8.0_112

For older versions of Spark and ipython, please, see also previous version of text.

Install Java Development Kit

@alexvictoor
alexvictoor / pb-avro-test_pom.xml
Last active November 10, 2024 10:52
Demo of Protobuff integration within Avro
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.avro.is.great</groupId>
<artifactId>protobuff-avro-demo</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>Demo of protobuff integration with Avro</name>
<build>
<plugins>