Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""This script should be run on a machine with at least 2 GPUs and an MPS server running. You can launch an MPS daemon with | |
``` | |
nvidia-cuda-mps-control -d | |
``` | |
The script first uses `test_cuda` to verify a CUDA context can be created on each GPU. It then spawns two workers; a | |
'good' worker and a 'bad' worker. The workers collaborate through Pytorch's DataDistributedParallel module to calculate | |
the gradient for a trivial computation. The 'good' worker carries out both the forward and backward pass, while the | |
bad worker carries out the forward pass and then exits. This seems to lock up the MPS server, and any subsequent | |
attempts to create CUDA contexts fail by hanging eternally. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""Hack to add per-session state to Streamlit. | |
Usage | |
----- | |
>>> import SessionState | |
>>> | |
>>> session_state = SessionState.get(user_name='', favorite_color='black') | |
>>> session_state.user_name | |
'' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def top_k_top_p_filtering(logits, top_k=0, top_p=0.0, filter_value=-float('Inf')): | |
""" Filter a distribution of logits using top-k and/or nucleus (top-p) filtering | |
Args: | |
logits: logits distribution shape (vocabulary size) | |
top_k >0: keep only top k tokens with highest probability (top-k filtering). | |
top_p >0.0: keep the top tokens with cumulative probability >= top_p (nucleus filtering). | |
Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751) | |
""" | |
assert logits.dim() == 1 # batch size 1 for now - could be updated for more but the code would be less clear | |
top_k = min(top_k, logits.size(-1)) # Safety check |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def argmax1(array): | |
return array.index(max(array)) | |
def argmax2(array): | |
return max(range(len(array)), key=lambda x: array[x]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Download Google Drive files with WGET | |
Example Google Drive download link: | |
https://docs.google.com/open?id=[ID] | |
To download the file with WGET you need to use this link: | |
https://googledrive.com/host/[ID] | |
Example WGET command: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding: utf-8 -*- | |
"""Parse Python source code and get or print docstrings.""" | |
__all__ = ('get_docstrings', 'print_docstrings') | |
import ast | |
from itertools import groupby | |
from os.path import basename, splitext |