This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
parameter | data_type | explanation | |
---|---|---|---|
experiment_name | string | Name of the experiment to place the current training run under (new experiment is created if no previous experiment with the same name is found) | |
compute_target_name | string | Name of VM where training will be done (new compute name is created if no previous compute of the same name is found) | |
vm_size | string | Ability to choose from various available VM configurations (e.g. Standard_D5_v2) | |
data_folder | string | Name of the folder in the Azure storage is store your training data. | |
local_directory | string | Path to the training data's local directory | |
conda_packages | list | List of conda packages required to train your model | |
script | string | Name of the training script (placed in the current working directory) | |
model_name | string | Name of trained model to be registered under in the workspace. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
parameters | data_type | explanation | |
---|---|---|---|
model_version | int | Version of the trained model (can be located via the VSC IDE) | |
pip_packages | list | List of pip packages to be installed | |
conda_packages | list | List of conda packages to be installed | |
conda_env_file | string | Name of the conda env file | |
path_scoring_script | string | Path to prediction script [score.py] | |
docker_image_name | string | Name of the docker image |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
parameters | data_type | explanation | |
---|---|---|---|
docker_image_version | int | Version of the docker image created in the previous step (can be found via the VSC IDE) | |
cpu_cores | int | Number of CPU cores for the Docker container | |
memory | int | Memory for the Docker container | |
service_name | string | Deployment service name |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Create an experiment | |
from azureml.core import Experiment | |
# NOTE: New experiment is created if one with the following name is not found | |
experiment_name = config['train']['experiment_name'] | |
exp = Experiment(workspace=ws, name=experiment_name) | |
# Create a target compute - VM | |
from azureml.core.compute import DsvmCompute | |
from azureml.core.compute_target import ComputeTargetException |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# get the default datastore and upload data from local folder to VM | |
ds = ws.get_default_datastore() | |
print(ds.name, ds.datastore_type, ds.account_name, ds.container_name) | |
# Upload data to default data storage | |
data_folder = config['train']['data_folder'] | |
ds.upload(config['train']['local_directory'],target_path=data_folder,overwrite=True) | |
print ('Finished Uploading Data.') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
src = ScriptRunConfig(source_directory='./', | |
script= config['train']['script'], | |
run_config=conda_run_config, | |
# pass the datastore reference as a parameter to the training script | |
arguments=['--data-folder', str(ds.as_download())] | |
) | |
run = exp.submit(config=src) | |
run.wait_for_completion(show_output=True) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Register the model | |
print('Registering model...') | |
model = run.register_model(model_name=config['train']['model_name'], model_path='./outputs/ridge_1.pkl') | |
print('Done registering model.') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Retrive registered model by version and name | |
from azureml.core.model import Model | |
regression_models = Model.list(workspace=ws,name=config['train']['model_name']) | |
for m in regression_models: | |
if m.version == int(config['docker']['model_version']): | |
model = m |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Create conda environment for docker image | |
from azureml.core.conda_dependencies import CondaDependencies | |
# DEFINE CONDA DEPENDENCIES | |
myenv = CondaDependencies.create(pip_packages=ast.literal_eval(config['docker']['pip_packages']),conda_packages=ast.literal_eval(config['train']['conda_packages'])) | |
myenv.add_pip_package("pynacl==1.2.1") | |
# CREATE CONDA ENVIRONMENT FILE | |
with open(config['docker']['conda_env_file'],"w") as f: | |
f.write(myenv.serialize_to_string()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Create docker image | |
from azureml.core.image import Image, ContainerImage | |
image_config = ContainerImage.image_configuration(runtime= "python", | |
execution_script=config['docker']['path_scoring_script'], | |
conda_file=config['docker']['conda_env_file'], | |
tags = {'area': "diabetes", 'type': "regression"}, | |
description = "test") | |
image = Image.create(name = config['docker']['docker_image_name'], |