Albert Buchard albertbuchard
albertbuchard
/ ddp_debug.py
Created
September 11, 2023 13:35
Distributed Data Parallel Debugging - Pytorch / PDB - ddp_pdb()
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class DistributedIdentity: | |
| """ | |
| Singleton class to hold distributed identity information. | |
| Handles SLURM, torchrun, and local runs. | |
| Looks for the following environment variables: | |
| - RANK | |
| - WORLD_SIZE | |
| - LOCAL_RANK |
albertbuchard
/ sparsemax_torch.py
Last active
April 18, 2023 18:31
This code defines a PyTorch implementation of the Sparsemax activation function. Sparsemax is an alternative to the softmax activation function that produces sparse probability distributions (euclidian projection to the simplex). The implementation is provided as a PyTorch nn.Module, making it easy to integrate into any architecture.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import torch | |
| import torch.nn as nn | |
| class Sparsemax(nn.Module): | |
| def __init__(self, dim=-1): | |
| super(Sparsemax, self).__init__() | |
| self.dim = dim | |
| def forward(self, x): | |
| # Move the dimension to apply Sparsemax to the front |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| setup=''' | |
| import numpy as np | |
| import torch | |
| V_nat = [[1, 2], [3, 4]] | |
| U_nat = [[2, -1, 0, 0, 0, 0], | |
| [5, 2, 8, -1, 0, 0]] | |
| def compute_using_fleuret_1(): |
albertbuchard
/ init.coffee
Last active
September 4, 2017 11:29
automatic update by http://atom.io/packages/sync-settings
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Your init script | |
| # | |
| # Atom will evaluate this file each time a new window is opened. It is run | |
| # after packages are loaded/activated and after the previous editor state | |
| # has been restored. | |
| # | |
| # An example hack to log to the console when each text editor is saved. | |
| # | |
| # atom.workspace.observeTextEditors (editor) -> | |
| # editor.onDidSave -> |
NewerOlder