PetrochukM
/ hyperband.py
Last active
April 11, 2023 06:39
Here we implement hyperband and successive halving adaptions. We found that the original hyperband implementation was messy and not tested. We also wanted to adapt it to include model reuse.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| We implement additional hyperparameter optimization methods not present in | |
| https://scikit-optimize.github.io/. | |
| Gist: https://gist.github.com/Deepblue129/2c5fae9daf0529ed589018c6353c9f7b | |
| """ | |
| import math | |
| import logging | |
| import random |
PetrochukM
/ top_k_viterbi.py
Last active
June 17, 2022 22:20
Implemented a Top K Viterbi Decoder algorithm in PyTorch. Useful for Conditional Random Fields (CRFs)-based probabilistic graphical modelling. Learn more here: https://nlp.stanford.edu/joberant/esslli_2016/kbest-ict.pdf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import torch | |
| # Credits to AllenNLP for the base implementation and base tests: | |
| # https://github.com/allenai/allennlp/blob/master/allennlp/nn/util.py#L174 | |
| # Modified AllenNLP `viterbi_decode` to support `top_k` sequences efficiently. | |
| def viterbi_decode(tag_sequence: torch.Tensor, transition_matrix: torch.Tensor, top_k: int=5): | |
| """ | |
| Perform Viterbi decoding in log space over a sequence given a transition matrix | |
| specifying pairwise (transition) potentials between tags and a matrix of shape |
PetrochukM
/ optimizer_1.py
Last active
July 26, 2019 22:41
PyTorch Optimizer_1 from `Neural Optimizer Search with Reinforcement Learning`
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import torch | |
| from torch.optim import Optimizer | |
| class Optimizer_1(Optimizer): | |
| """Implements Optimizer_1 algorithm. | |
| It was been proposed in `http://proceedings.mlr.press/v70/bello17a/bello17a.pdf`. |
PetrochukM
/ subword_text_tokenizer.py
Last active
July 8, 2021 23:53
Tensor2Tensor Subword Text Tokenizer.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # coding=utf-8 | |
| # Copyright 2017 The Tensor2Tensor Authors. | |
| # | |
| # Licensed under the Apache License, Version 2.0 (the "License"); | |
| # you may not use this file except in compliance with the License. | |
| # You may obtain a copy of the License at | |
| # | |
| # http://www.apache.org/licenses/LICENSE-2.0 | |
| # | |
| # Unless required by applicable law or agreed to in writing, software |