Skip to content

Instantly share code, notes, and snippets.

View malomarrec's full-sized avatar

Malo Marrec malomarrec

View GitHub Profile
// Setup: pip3 install pandas matplotlib wordcloud
import pandas as pd
import matplotlib.pyplot as plt
from wordcloud import WordCloud
import re
def replace_substring(test_str, s1, s2):
# Replacing all occurrences of substring s1 with s2
timeline
title Roadmap
section Q2 FY2024
New features:
Autocomplete :
Plugins :
Guardrails / Filters MVP :
Cody in Sourcegraph UI :
And more!
Context fetching:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:*",
"eks:*",
"autoscaling:*"
],
h3 {
text-align: center;
}
@malomarrec
malomarrec / cloudskiff-aws-iam-permissions.json
Last active February 14, 2020 16:06
A basic set of premissions Cloudskiff needs to manage your EKS clusters
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:*",
"eks:*",
"autoscaling:*"
],
@malomarrec
malomarrec / tensorport_minimal_distributed_tensorflow.py
Last active March 2, 2018 16:36
The minimal template to use distributed TensorFlow on TensorPort
# Notes:
# You need to have the clusterone package installed (pip install tensorport)
# Export logs and outputs to /logs, your data is in /data.
import tensorflow as tf
from clusterone import get_data_path, get_logs_path
# Get the environment parameters for distributed TensorFlow
try:
@malomarrec
malomarrec / slide_df.R
Last active June 8, 2017 03:23
"Slide" dataframe (merge two columns that are identical but are NAs in a partition of rows)
#This function "slides" dataframe blocks based on NA values
#Takes a dataframe and two column names
#On each row, one of these columns have to be NA
#The function returns a dataframe with a new column called <newname> with the combined non NA values of the first columns
#Different factor levels in both columns are dealt with by casting to character, then back to factor after the merge
#If newname is not specified, the first columns is overriden by the new column
#Optionnal parameters allows to drop the old columns
slide_df <- function(df,name1,name2,newname = NULL, drop = FALSE){
require(dplyr)
read.spss_wrapper <- function (path, verbose = FALSE){
#In R 3.4.0, reading SPSS with foreign::read.spss raises errors when converting int factor levels to their labels with use.value.labels=TRUE
#This is a workaround.
list_data <- read.spss(path, to.data.frame=FALSE, use.value.labels=FALSE)
#list_data <- read.sav(path, to.data.frame=FALSE, use.value.labels=TRUE)
l1 = length(list_data)
indx <- sapply(list_data, is.factor)
list_data[indx] <- lapply(list_data[indx], function(x) {
levels(x) <- make.unique(levels(x))
@malomarrec
malomarrec / benjamini_hochberg.R
Last active June 2, 2017 21:35
benjamini_hochberg
benjamini_hochberg <- function(df, fdr){
#input:
#df = dataframe where the rows are the tests and one of the column named "pval"" contains the p-values
#fdr = false discovery rate threshold
# output
# "threshold_BH" is the benjamini_hochberg threshold
# "reject" indicates whether we reject the null or not in the output
n = nrow(df)
bh = df %>% mutate(rank = dense_rank(pval)) %>% mutate(threshold_BH = (rank/n)*fdr, reject = pval < threshold_BH) %>% arrange(pval)
return(bh)