rwightman

import math
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn.modules.utils import _pair, _quadruple


class MedianPool2d(nn.Module):
    """ Median pool (usable as median filter when stride=1) module.
    

rwightman

''' Two sample RNN (1d CNN + LSTM) networks used in the Kaggle 
QuickDraw Challenge (https://www.kaggle.com/c/quickdraw-doodle-recognition)

Both of these networks expect a tuple input with first element being the sequences
and second being the sequence lengths (typical sorted packed format). The sequence tensor
should adhere to the following shape: (batch_size, channels, seq_len).
Where channels consists of stroke [x, y, t, end]. End indicates whether the 
stroke is the last in a segment (pen up). It could easily be changed to start
(pen down) or combo of both.

rwightman

import torch
from torch import nn
import torch.nn.functional as F
from collections import OrderedDict
import math

def pdist(v):
    dist = torch.norm(v[:, None] - v, dim=2, p=2)
    return dist

rwightman

rwightman

import torch.utils.data as data
import os
import re
import torch
import tarfile
from PIL import Image


IMG_EXTENSIONS = ['.png', '.jpg', '.jpeg']

rwightman

rwightman

rwightman

""" A simple U-Net w/ timm backbone encoder

Based off an old version of Unet in https://github.com/qubvel/segmentation_models.pytorch

Hacked together by Ross Wightman
"""

from typing import Optional, List

import torch

rwightman

aa: rand-m6-n5-inc1-mstd1.0
amp: false
apex_amp: false
aug_splits: 0
batch_size: 256
bn_eps: null
bn_momentum: null
bn_tf: false
channels_last: false
checkpoint_hist: 10

rwightman

Using TPU VM instance w/ pre-alpha timm bits setup as per: https://github.com/rwightman/pytorch-image-models/tree/bits_and_tpu/timm/bits#readme

python3 launch_xla.py --num-devices 8 train.py gs://my-imagenet --config hparams.yaml

Note the config yaml files do have args that are not used or active based on other overriding code or the state of the current training code. The bits code is under heavy development so these configs will likely need specific revision (currently https://github.com/rwightman/pytorch-image-models/commit/5e95ced5a7763541f7219f35fd155e3fbfe66e8b)

The gMlp hparams are the last (latest) in the series and likely will produce better results than the earlier gmixer / resmlp variants...

Note, for adapting the LR to differenrt batch size. AdamW is being used here and I use a sqrt scaling for the learning rate wrt to (global) batch size. I typicall use linear LR scaling w/ SGD or RMSProp for most from-scratch training.