MlFlow is awesome. We use it all the time to track our ML models and their artifacts. Logging model training runs are super easy, and if you need any custom logic, the API is also pretty easy to use.
The one small caveat to this is that the model should already be supported by MlFlow. See the following list for all the models that MlFlow natively supports. However, what happens when you have to log a model that isn't natively supported by MlFlow?
Typically installing K3s is a one-liner. However, because we need to enable Nvidia GPU support, we'll need a few extra steps.
First, run the following the download and install K3s. INSTALL_K3S_SKIP_START skips automatically starting K3s because we'll need to modify containerd to use the nvidia-container-runtime (more on that later). INSTALL_K3S_EXEC="server --no-deploy traefik" is to skip Traefik because KubeFlow uses Istio. If this were used, the Istio wouldn't initialise properly:
curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_START=true INSTALL_K3S_EXEC="server --no-deploy traefik" sh
| ; Ender 3 Custom Start G-code | |
| M117 Getting the bed up to temp! | |
| M140 S{material_bed_temperature_layer_0} ; Set Heat Bed temperature | |
| M190 S{material_bed_temperature_layer_0} ; Wait for Heat Bed temperature | |
| M117 Pre-heating the extruder! | |
| M104 S160; start warming extruder to 160 | |
| G28 ; Home all axes | |
| M117 Auto bed-level GO! | |
| G29 ; Auto bed-level (BL-Touch) | |
| G92 E0 ; Reset Extruder |
| #include "FastLED.h" | |
| #include "limits.h" | |
| #define NUM_LEDS 60 // How many leds in your strip? | |
| #define DATA_PIN 5 // led data transfer | |
| #define MICROPHONE_PIN A0 | |
| long sensorMax = 0; | |
| long sensorMin = LONG_MAX; |
| ; Ender 3 Custom Start G-code | |
| M117 Getting the bed up to temp! | |
| M140 S{material_bed_temperature_layer_0} ; Set Heat Bed temperature | |
| M190 S{material_bed_temperature_layer_0} ; Wait for Heat Bed temperature | |
| M117 Pre-heating the extruder! | |
| M104 S160; start warming extruder to 160 | |
| G28 ; Home all axes | |
| M117 Auto bed-level GO! | |
| G29 ; Auto bed-level (BL-Touch) | |
| G92 E0 ; Reset Extruder |
| import cv2 | |
| def equalize_histogram(path): | |
| # Color | |
| print('Processing (Color): {}'.format(path)) | |
| img = cv2.imread(path) | |
| lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB) | |
| lab_planes = cv2.split(lab) |
| # RenderDoc Python scripts, powered by IronPython 2.7.4.1000 | |
| # The 'renderdoc' object is the Core class instance. | |
| config = {} | |
| config['py_lib_dir'] = 'C:\\Python27\\Lib\\' | |
| config['save_dir'] = 'C:\\workspace\\playing-for-data\\data\\processed\\' # where we store extraction results | |
| # creates a prefixes for files and directories from logfilename | |
| config['dir_prefix'] = lambda logFilename: '' | |
| config['file_prefix'] = lambda logFilename: basename(logFilename)[:-4] + '_' |
| import numpy as np | |
| import matplotlib.pyplot as plt | |
| filename = 'C:\\Users\\Benjamin Tan\\Desktop\\00016351_object_id.raw' | |
| vals = np.linspace(0,1,256) | |
| np.random.shuffle(vals) | |
| cmap = plt.cm.colors.ListedColormap(plt.cm.jet(vals)) | |
| # Load the object_id.raw file and skip the first 4 bytes |
| import tensorflow as tf | |
| import os | |
| from tensorflow.python.keras import Input, Model | |
| from tensorflow.python.keras.models import load_model | |
| from darknet import darknet_base | |
| from definitions import ROOT_DIR | |
| from yolo_layer import YOLOLayer |
| name: kaggle | |
| channels: | |
| - defaults | |
| dependencies: | |
| - backcall=0.1.0=py36_0 | |
| - bleach=2.1.3=py36_0 | |
| - ca-certificates=2018.03.07=0 | |
| - certifi=2018.4.16=py36_0 | |
| - dbus=1.13.2=h714fa37_1 | |
| - decorator=4.3.0=py36_0 |