aluo-x

import torch
import torch.distributed as dist
import torch.nn as nn
import torch.multiprocessing as mp

from torch.nn.parallel import DistributedDataParallel as DDP
from fairscale.nn.data_parallel import ShardedDataParallel as ShardedDDP
from fairscale.optim.oss import OSS
from fairscale.nn.data_parallel import FullyShardedDataParallel as FSDP
import os

rwightman

shawwn

Hello,

When you look at a repo, be sure to use git log --all and look at the latest commits across all branches. You would see that my work is on the "dec6" branch, not the main branch.

In general, it's more reliable to DM me on twitter than to email me. I'll usually respond within a day. Emails are hit-or-miss.

Here is the critical difference:

Eoin-ONeill-Yokai

Document Revision 1.2 [07/15/22]

Author Notes: Thanks to everyone who has been testing or using this installation process. I've refined the instructions to try to make this installation as simple and cross platform as humanly possible. I've also updated the Steam Deck instructions now that I have had mine for a while and thoroughly tested the installation process.

Basic Game Installation

Final Fantasy IX should be installed like any standard steam game through the steam client. Ragarding compatibility layers: it should work with a stable release of Proton 7 (7.0.X recommended) through the steam client. This also includes controller support if you are using Steam's native controller configurations. If you have any problems with a given Proton release, I would also recommend trying the latest GloriousEggroll proton builds to se

jph00

#!/usr/bin/env bash
###
# NB: You probably don't want this gist any more.
#     Instead, use this version from `fastsetup`:
#     https://github.com/fastai/fastsetup/blob/master/setup-conda.sh
###

set -e

cd

yk

set nocompatible              " be iMproved, required
let g:python3_host_prog = '/usr/local/opt/[email protected]/bin/python3.8'

if empty(glob('~/.vim/autoload/plug.vim'))
  silent !curl -fLo ~/.vim/autoload/plug.vim --create-dirs
    \ https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim
  autocmd VimEnter * PlugInstall --sync | source $MYVIMRC
endif

call plug#begin('~/.vim/plugged')

TengdaHan

Multi-node-training on slurm with PyTorch

What's this?

• A simple note for how to start multi-node-training on slurm scheduler with PyTorch.
• Useful especially when scheduler is too busy that you cannot get multiple GPUs allocated, or you need more than 4 GPUs for a single job.
• Requirement: Have to use PyTorch DistributedDataParallel(DDP) for this purpose.
• Warning: might need to re-factor your own code.
• Warning: might be secretly condemned by your colleagues because using too many GPUs.

mingfeima

(Internal Tranining Material)

Usually the first step in performance optimization is to do profiling, e.g. to identify performance hotspots of a workload. This gist tells basic knowledge of performance profiling on PyTorch, you will get:

• How to find the bottleneck operator?
• How to trace source file of a particular operator?
• How do I indentify threading issues? (oversubscription)
• How do I tell a specific operator is running efficiently or not?

This tutorial takes one of my recent projects - pssp-transformer as an example to guide you through path of PyTorch CPU peformance optimization. Focus will be on Part 1 & Part 2.

alexjc

Specific Papers

• https://arxiv.org/abs/1505.07376

  Texture Synthesis Using Convolutional Neural Networks
• https://arxiv.org/abs/1508.06576

  A Neural Algorithm of Artistic Style
• https://arxiv.org/abs/1603.03417

  Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
• https://arxiv.org/abs/1701.02096

  Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Texture Synthesis

amir-saniyan

Ubuntu 22.04 for Deep Learning

In the name of God

This gist contains steps to setup Ubuntu 22.04 for deep learning.

---

Install Ubuntu 22.04