This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import tensorflow as tf | |
import re | |
from model import modeling | |
from model import optimization | |
from pretrain import pretrain_data | |
from pretrain import pretrain_helpers | |
from util import training_utils | |
from util import utils |
- Creating binary "input values" file for TFLite benchmarking tool:
import numpy as np
inp_feat_array = ....
...
inp_feat_array.astype(TYPE_OF_TFLITE_ARRAY_DTYPE).write(FNAME)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import tensorflow as tf | |
import tensorflow_text as text | |
tf.enable_eager_execution() | |
CLS_TOKEN, SEP_TOKEN, MASK_TOKEN = 101, 102, 103 | |
def merge_dims(rt, axis=0): | |
to_expand = rt.nested_row_lengths()[axis] | |
to_elim = rt.nested_row_lengths()[axis + 1] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def mask_all_out(self, x, lengths): | |
""" | |
Returns tensor with all tokens masked out one at a time, with respect for seq lengths | |
Inputs: | |
x : (slen, bsz) LongTensor | |
lengths : (bsz) LongTensor | |
Returns: |
- Filter CSV based on number of words in a field using
awk
:
awk -F ',' 'split($2,a," ") == 2 {print $2}' fname
- Convert Kubernetes events to local timezone (for comparison with other logs):
kubectl get events -o json | jq '.items[] | .lastTimestamp + "," + .message' | tr -d '"' | awk -F',' '{cmd="date +\"%d/%b/%Y:%T\" --date=" $1; cmd | getline conv_date; print "[" conv_date "] (kubectl events) " $2}'
- Remove EOS/BOS from a text data file using
sed
- Setting up docker to use a different image directory: https://stackoverflow.com/a/34731550
- Jellyfin docker image:
docker run --name "jellyfin-server" --restart=unless-stopped -d -v $(pwd)/config/:/config -v $(pwd)/cache/:/cache -v /home/nemty/media:/media --net=host jellyfin/jellyfin
- Install qemu-user-static (
yay -S qemu-user-static
).- This might need you to install
pcre-static
and update PGP keys (follow the tips in the comments here).
- This might need you to install
- If not already present, install
systemd-binfmt
, the revamped version ofbinfmt-support
tools- Your system has to support transparent Qemu user emulation, but fortunately, that is mostly enabled once the steps here have been followed.
- Check the status of the
systemd-binfmt
unit (systemctl status systemd-binfmt
) and (re)start if needed (sudo systemctl restart systemd-binfmt
)- This unit has ARM support by default, but check the current documentation to make install it if needed
- Mount the root partition of the ARM system into a folder (for e.g.,
sudo mount /dev/sdb2 arm_mountpoint
) - Copy the QEMU ARM static binary (
/usr/bin/qemu-arm-static
on my distro) to the mounted root directory'susr/bin
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/env python3 | |
import argparse, sys | |
import os, subprocess | |
import xml.etree.ElementTree as ET | |
parser = argparse.ArgumentParser() | |
parser.add_argument("--rssfile", help="Location of RSS feed file downloaded from Panopto", | |
required=True, type=argparse.FileType('r')) | |
args = parser.parse_args() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Kubernetes Server: 1.3 | |
# If this is the first time that apply is being called | |
curl -X POST -H "Content-Type: application/json" "${KUBERNETES_API_SERVER}"/apis/extensions/v1beta1/namespaces/my-namespace/deployments/ --data @my-deployment-spec.json | |
# If this is the first time that apply is being called | |
curl -X PATCH -H "Content-Type: application/json-patch+json" "${KUBERNETES_API_SERVER}"/apis/extensions/v1beta1/namespaces/my-namespace/deployments/my-deployment --data @my-deployment-spec.json |
NewerOlder