Skip to content

Instantly share code, notes, and snippets.

View farrajota's full-sized avatar

M. Farrajota farrajota

View GitHub Profile
@farrajota
farrajota / convert_module.lua
Last active April 15, 2017 00:10
Convert a cudnn batchnorm module to nn backend (cycles all modules of a network).
local function ConvertBNcudnn2nn(net)
local function ConvertModule(net)
return net:replace(function(x)
if torch.type(x) == 'cudnn.BatchNormalization' then
return cudnn.convert(x, nn)
else
return x
end
end)
end
@farrajota
farrajota / svhn_convert_json.m
Last active September 29, 2016 21:19
Convert SVHN metadata .mat file format to .json
function svhn_convert_json(path)
% convert .mat to json files
%% Create .json file
if isempty(path)
error('Must specify a path to the dataset')
end
@farrajota
farrajota / padarray.lua
Created November 8, 2016 21:25
Pad arrays in torch
local function dimnarrow(x,sz,pad,dim)
local xn = x
for i=1,x:dim() do
if i > dim then
xn = xn:narrow(i,pad[i]+1,sz[i])
end
end
return xn
end
@farrajota
farrajota / parabola.lua
Last active December 3, 2016 20:00
Sub-pixel precision using 1D parabola fitting
-- Use three (consecutive) points to fit a 1D parabola
-- and return its maximum value. To achieve sub-pixel
-- precision for a 2D maxima (x,y), just fit the parabola over the
-- x and y coordinates separately with two neighboring points.
local function fitParabola(x1,x2,x3,y1,y2,y3)
local x1_sqr = x1*x1
local x2_sqr = x2*x2
local x3_sqr = x3*x3
local div = (x1_sqr-x1*(x2+x3)+x2*x3)*(x2-x3)
@farrajota
farrajota / mysplittable.lua
Created January 16, 2017 11:32
Splits a tensor into a table with `N` tensors (`N` is specified by the user).
local SplitTable, parent = torch.class('nn.MySplitTable', 'nn.Module')
function SplitTable:__init(dimension, nTensors)
parent.__init(self)
self.dimension = dimension
self.nTensors = nTensors
self.joinTable = nn.JoinTable(dimension)
end
function SplitTable:getSize(input)
@farrajota
farrajota / bobp-python.md
Created May 31, 2017 14:10 — forked from sloria/bobp-python.md
A "Best of the Best Practices" (BOBP) guide to developing in Python.

The Best of the Best Practices (BOBP) Guide for Python

A "Best of the Best Practices" (BOBP) guide to developing in Python.

In General

Values

  • "Build tools for others that you want to be built for you." - Kenneth Reitz
  • "Simplicity is alway better than functionality." - Pieter Hintjens
@farrajota
farrajota / guideline.md
Last active July 3, 2018 15:49
Data cleaning guidelines for multivariate data exploration (using Python's scipy stack).

Missing data

A Simple Example of a Missing Data Analysis Understanding the Reasons Leading to Missing Data Ignorable Missing Data Other Types of Missing Data Processes Examining the Patterns of Missing Data Diagnosing the Randomness of the Missing Data Process

Rules of Thumb

@farrajota
farrajota / checklist.md
Last active June 28, 2023 20:41
Data science process with checklists

data science checklist


Step 1


Data loading# data science checklist


Step 1

@farrajota
farrajota / imputation_rules.md
Last active July 23, 2024 16:17
Rules of thumb for when imputation of missing values should not be used.

When imputation should not be used

  • If data are MCAR, imputation may not be not needed.
  • If missingness is due to unmeasured variables related to the dependent variable, data are MNAR and should not be imputed.
  • Imputation assumes data are MAR and should not be used with sparse data. Sparse data occur when missingness is non-random, such as a shopping cart survey of items purchased (coded 1) or not purchased (coded 0), because the null response (0) is non-random, due to unmeasured factors possibly not even known to the shopper.
  • Imputation should not be used to impute all the data for a subject
  • Imputation should not be used for a missing value for a given observation if that observation is also missing values on predictively critical variables in the imputation model. While this is difficult to check for each value to be imputed, a table of missing value patterns will show how many cases missing on a given variable also have missing values on other variables. In some cases this may lead a researcher