- Local Install Requirements
Python 3.7
MXNet 1.8
Pandas >= 1.2.4
AutoGluon 0.2.0
- 👉 create sagemaker base environment
conda create -n sagemaker_py310 python=3.10
conda activate sagemaker_py310
pip install boto3 awscli sagemaker
conda install -c conda-forge jupyterlab
python -m ipykernel install --user --name sagemaker_py310
aws configure
jupyter lab
- 👉 create an environment cloned from the base environment
conda create --name awsmle_py310 --clone sagemaker_py310
python -m ipykernel install --user --name awsmle_py310
conda install -c conda-forge mamba (having unicode file names issue)
mamba install -c conda-forge autogluon (failed)
conda install -c conda-forge mamba==1.4.7
conda update conda
mamba install -c conda-forge autogluon
mamba install -c conda-forge "ray-tune >=2.6.3,<2.7" "ray-default >=2.6.3,<2.7"
pip install bokeh==2.0.1
pip install kaggle
pip install ipywidgets
pip install ydata-profiling
pip install smdebug
pip install jinja2==2.11.3
pip install markupsafe==2.0.1
pip install 'sagemaker[local]'
pip install sagemaker_inference
- To run SageMaker in the local environment, add the following 3 keys to
~/.aws/credentials.
notepad C:\Users\*\.aws\credentials(awsmle_py310) PS D:\github\udacity-nd009t-C2-Developing-ML-Workflow> cat ~/.aws/credentials
[default]
aws_access_key_id = <?>
aws_secret_access_key = <?>
aws_session_token = *vQ== <?>## reset the session after updating credentials
import boto3
boto3.DEFAULT_SESSION = None- 👉 create an env with
cudaenabled,torchandtransformersinstalled.
conda create --name conda_py310 python=3.10
conda activate conda_py310
nvidia-smi
conda install cuda --channel "nvidia/label/cuda-12.1.0"
nvcc --version
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
pip install transformers
pip install pandas
pip install scikit-learn
conda install -c conda-forge ipywidgets
pip install wandb- docker commands
docker images
docker ps
- Windows Powershell commands
conda deactivate
conda env remove --name awsmle_py37
- install wsl-ubuntu
wsl --list --online
wsl --install -d Ubuntu-22.04
install ubuntu, restart Windows OS
install Docker Desktop, restart Windows
- Tips:
- Choose the torch version according to the cuda version at https://pytorch.org/
- Choose the MXNet version according to the cuda version at mxnet.apache.org.
MXNet 1.8 does not officially support CUDA 12.x, as the latest version (MXNet 1.8) was designed for compatibility with CUDA versions up to 11.x. However, you can try using a previous version of CUDA (such as CUDA 11.0 or 10.2) to install MXNet 1.8 with GPU support.
- Choose the torch version according to the cuda version at https://pytorch.org/
- Some output in the terminal
(awsmle_py37) PS D:\github\udacity-cd0385-exercise-files> nvidia-smi
Tue Nov 19 14:19:10 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.99 Driver Version: 555.99 CUDA Version: 12.5 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Driver-Model | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce GTX 1650 Ti WDDM | 00000000:01:00.0 Off | N/A |
| N/A 46C P0 14W / 50W | 0MiB / 4096MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
(awsmle_py37) PS D:\github\udacity-cd0385-exercise-files> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Wed_Feb__8_05:53:42_Coordinated_Universal_Time_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0
(awsmle_py37) PS D:\github\udacity-cd0385-exercise-files> python -m ipykernel install --user --name=awsmle_py37
Installed kernelspec awsmle_py37 in C:\Users\*\AppData\Roaming\jupyter\kernels\awsmle_py37
(awsmle_py310) PS D:\github\udacity-cd0385-exercise-files> aws configure
AWS Access Key ID [****************HX33]:
AWS Secret Access Key [****************HD5x]:
Default region name [us-east-1]: us-east-1
Default output format [None]: json
Support building source against CUDA 12.1 #21190
Open
kevnzhao opened this issue on Mar 27, 2023 · 5 comments
apache/mxnet#21190