Table of Contents:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Docker compose to set up containers for all services you need: | |
# VPN | |
# Sonarr, Radarr, Lidarr, Qbittorrent | |
# Non-VPN | |
# Plex, get_iplayer | |
# Before running docker-compose, you should pre-create all of the following folders. | |
# Folders for Docker State: | |
# /volume1/dockerdata. - root where this docker-compose.yml should live | |
# /volume1/dockerdata/plex - Plex config and DB | |
# /volume1/dockerdata/sonarr - Sonarr config and DB |
- A simple note for how to start multi-node-training on slurm scheduler with PyTorch.
- Useful especially when scheduler is too busy that you cannot get multiple GPUs allocated, or you need more than 4 GPUs for a single job.
- Requirement: Have to use PyTorch DistributedDataParallel(DDP) for this purpose.
- Warning: might need to re-factor your own code.
- Warning: might be secretly condemned by your colleagues because using too many GPUs.
(Internal Tranining Material)
Usually the first step in performance optimization is to do profiling, e.g. to identify performance hotspots of a workload. This gist tells basic knowledge of performance profiling on PyTorch, you will get:
- How to find the bottleneck operator?
- How to trace source file of a particular operator?
- How do I indentify threading issues? (oversubscription)
- How do I tell a specific operator is running efficiently or not?
This tutorial takes one of my recent projects - pssp-transformer as an example to guide you through path of PyTorch CPU peformance optimization. Focus will be on Part 1 & Part 2.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
g++ -I/usr/local/include/opencv4/ benchmark.cpp -lopencv_core -lopencv_imgproc -lopencv_dnn -lopencv_imgcodecs -O3 -std=c++17 |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# https://stackoverflow.com/questions/33650974/opencv-python-read-specific-frame-using-videocapture | |
import numpy as np | |
import cv2 | |
cap = cv2.VideoCapture(video_name) #video_name is the video being called | |
cap.set(1,frame_no); # Where frame_no is the frame you want | |
ret, frame = cap.read() # Read the frame | |
cv2.imshow('window_name', frame) # show frame on window | |
#If you want to hold the window, until you press exit: | |
while True: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
name: release | |
on: | |
push: | |
branches: | |
- master | |
jobs: | |
release: | |
name: Build | |
runs-on: ubuntu-latest |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.