This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using Test | |
using Convex1D | |
using BenchmarkTools | |
function minimize_scaled_L1_diff(xs, ys) | |
# find t::Number that minimizers f(t) = sum(abs, t*xs - ys) | |
# f is convex with piecewise constant derivative given by | |
# f'(t) = Σ xi * sign(t*xi - yi) | |
# One of the points ti := yi/xi must be a minimizer (for some xi !=0. If all xi==0 then f == const anyway) | |
# Based on remarks of Mathieu Tanneau in slack |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using PyCall | |
using Flux | |
function doit_keras(cfg) | |
keras = pyimport("tensorflow.keras") | |
inp = keras.layers.Input((nothing, 1)) | |
x = inp | |
x = keras.layers.Conv1D(kernel_size=51, filters=50)(x) | |
x = keras.layers.Conv1D(kernel_size=1, filters=1)(x) | |
out = x |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# This reproduces a pytorch_lightning issue | |
# where gradient checkpointing + ddp results in nan loss | |
# | |
# * Run with gpus=1 and it works fine. | |
# * Run with gpus=4 and it loss becomes nan quickly | |
# | |
# See also https://forums.pytorchlightning.ai/t/gradient-checkpointing-ddp-nan/398 | |
import torch | |
from torch import nn | |
from torch.nn import functional as F |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Hello World! |
OlderNewer