| r"""PyTorch implementation of SOAP | |
| Adapted from https://github.com/nikhilvyas/SOAP | |
| References: | |
| | SOAP: Improving and Stabilizing Shampoo using Adam (Vyas et al., 2024) | |
| | https://arxiv.org/abs/2409.11321 | |
| """ | |
| import torch |
This doc servers as a quick reference for the _scaled_mm API and how it has changed overtime for each major version of PyTorch.
NOTE The leading underscore is intended here and we make no current FC/BC guarantees on this API. That being said it is currently the only OP that has native support for FP8 matmuls within the PyTorch Libary. We are planning to make an official Public api for this. Until then this is subject to change but you can use this doc as a reference.
| import math | |
| import torch | |
| def hadamard(n: int, dtype=torch.int8): | |
| """This function is a port of the one in scipy.linalg""" | |
| if n < 1: | |
| lg2 = 0 | |
| else: | |
| lg2 = int(math.log(n, 2)) |
| # Holds repositories directory | |
| sudo chown -R -v git:root /var/opt/gitlab/git-data | |
| sudo chmod -R -v 0700 /var/opt/gitlab/git-data | |
| # Holds git repositories | |
| sudo chown -R -v git:git /var/opt/gitlab/git-data/repositories | |
| sudo chmod -R -v 2770 /var/opt/gitlab/git-data/repositories | |
| # Holds large object directories | |
| sudo chown -R -v git:gitlab-www /var/opt/gitlab/gitlab-rails/shared |