20250205_VSCdoe remove Jupyter kernels.md

• Uninstall all VS Code extensions
  Delete C:\Users\*\.vscode\extensions folder
  Reinstall extensions

• Remove Jupyter kernels

(base) PS D:\github\udacity-nd009t-capstone-starter> jupyter kernelspec list
Available kernels:
  awsmle_py37        C:\Users\guido\AppData\Roaming\jupyter\kernels\awsmle_py37
  drlnd_py310        C:\Users\guido\AppData\Roaming\jupyter\kernels\drlnd_py310
  drlnd_py311        C:\Users\guido\AppData\Roaming\jupyter\kernels\drlnd_py311
  sagemaker_py310    C:\Users\guido\AppData\Roaming\jupyter\kernels\sagemaker_py310
  python3            D:\Users\guido\miniconda3\share\jupyter\kernels\python3
(base) PS D:\github\udacity-nd009t-capstone-starter> jupyter kernelspec uninstall awsmle_py37
Kernel specs to remove:
  awsmle_py37           C:\Users\guido\AppData\Roaming\jupyter\kernels\awsmle_py37
Remove 1 kernel specs [y/N]: y
Removed C:\Users\guido\AppData\Roaming\jupyter\kernels\awsmle_py37
(base) PS D:\github\udacity-nd009t-capstone-starter> jupyter kernelspec uninstall awsmle_py37
Couldn't find kernel spec(s): awsmle_py37
(base) PS D:\github\udacity-nd009t-capstone-starter> jupyter kernelspec list
Available kernels:
  drlnd_py310        C:\Users\guido\AppData\Roaming\jupyter\kernels\drlnd_py310
  drlnd_py311        C:\Users\guido\AppData\Roaming\jupyter\kernels\drlnd_py311
  sagemaker_py310    C:\Users\guido\AppData\Roaming\jupyter\kernels\sagemaker_py310
  python3            D:\Users\guido\miniconda3\share\jupyter\kernels\python3
(base) PS D:\github\udacity-nd009t-capstone-starter> jupyter kernelspec uninstall sagemaker_py310                                                    
Kernel specs to remove:                                                                                                                              
  sagemaker_py310       C:\Users\guido\AppData\Roaming\jupyter\kernels\sagemaker_py310
Remove 1 kernel specs [y/N]: y
Removed C:\Users\guido\AppData\Roaming\jupyter\kernels\sagemaker_py310
(base) PS D:\github\udacity-nd009t-capstone-starter> jupyter kernelspec list
Available kernels:
  drlnd_py310    C:\Users\guido\AppData\Roaming\jupyter\kernels\drlnd_py310
  drlnd_py311    C:\Users\guido\AppData\Roaming\jupyter\kernels\drlnd_py311
  python3        D:\Users\guido\miniconda3\share\jupyter\kernels\python3

• Remove conda envs

(base) PS D:\github\udacity-nd009t-capstone-starter> conda env list
# conda environments:
#
base                  *  D:\Users\guido\miniconda3
awsmle_py310             D:\Users\guido\miniconda3\envs\awsmle_py310
cuda_py310               D:\Users\guido\miniconda3\envs\cuda_py310
drlnd_py310              D:\Users\guido\miniconda3\envs\drlnd_py310
drlnd_py310_backup       D:\Users\guido\miniconda3\envs\drlnd_py310_backup
drlnd_py311              D:\Users\guido\miniconda3\envs\drlnd_py311
huggingface_py311        D:\Users\guido\miniconda3\envs\huggingface_py311
sagemaker_py310          D:\Users\guido\miniconda3\envs\sagemaker_py310
sagemaker_py313          D:\Users\guido\miniconda3\envs\sagemaker_py313
cuda_py310               d:\Users\guido\miniconda3\envs\cuda_py310
huggingface_py311        d:\Users\guido\miniconda3\envs\huggingface_py311

(base) PS D:\github\udacity-nd009t-capstone-starter> conda env remove -n cuda_py310

Remove all packages in environment D:\Users\guido\miniconda3\envs\cuda_py310:

(base) PS D:\github\udacity-nd009t-capstone-starter> conda env remove -n huggingface_py311 

Remove all packages in environment D:\Users\guido\miniconda3\envs\huggingface_py311:

(base) PS D:\github\udacity-nd009t-capstone-starter> conda env remove -n sagemaker_py310  

Remove all packages in environment D:\Users\guido\miniconda3\envs\sagemaker_py310:

(base) PS D:\github\udacity-nd009t-capstone-starter> conda env remove -n sagemaker_py313

Remove all packages in environment D:\Users\guido\miniconda3\envs\sagemaker_py313:

(base) PS D:\github\udacity-nd009t-capstone-starter> conda env list
# conda environments:
#
base                  *  D:\Users\guido\miniconda3
awsmle_py310             D:\Users\guido\miniconda3\envs\awsmle_py310
drlnd_py310              D:\Users\guido\miniconda3\envs\drlnd_py310
drlnd_py310_backup       D:\Users\guido\miniconda3\envs\drlnd_py310_backup
drlnd_py311              D:\Users\guido\miniconda3\envs\drlnd_py311

Locate where your Jupyter kernelspec files are stored:

• Linux/macOS: ~/.local/share/jupyter/kernels/
• Windows: C:\Users<YourUserName>\AppData\Roaming\jupyter\kernels\

Clear Cached Kernels (Optional): If the above steps don't work, you can clear the Jupyter extension cache manually:
Look for the cache in:

• Linux/macOS: ~/.vscode/extensions/ms-toolsai.jupyter-*/
• Windows: C:\Users<YourUserName>.vscode\extensions\ms-toolsai.jupyter-*

Inside C:\Users\*\AppData\Roaming\jupyter\kernels\drlnd_py310\kernel.json, you’ll see a structure like this:

{
 "argv": [
  "D:\\Users\\guido\\miniconda3\\envs\\drlnd_py310\\python.exe",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "drlnd_py310",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}

D:\Users\guido\miniconda3\share\jupyter\kernels\python3\kernel.json

{
 "argv": [
  "python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3 (ipykernel)",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}

ChatGPT suggestion (might be wrong)

The issue you're facing with libiomp5md.dll causing a kernel crash typically occurs due to conflicts with OpenMP runtime libraries being loaded multiple times, particularly in environments where different libraries are linked to OpenMP (such as Intel’s OpenMP and another library like MKL or other threading libraries).

Here are a few steps you can take to resolve this issue:

1. Set the KMP_DUPLICATE_LIB_OK Environment Variable

As a workaround, you can allow multiple OpenMP libraries to be loaded by setting the KMP_DUPLICATE_LIB_OK environment variable to TRUE. This can help bypass the conflict.

Steps:

1. Open Command Prompt or PowerShell.

2. Set the environment variable:

   set KMP_DUPLICATE_LIB_OK=TRUE

3. If you're using VSCode, you can add this environment variable directly in your VSCode configuration:

   • Open Command Palette (Ctrl+Shift+P) in VSCode.
   • Type Preferences: Open User Settings (JSON).
   • Add this line to set the environment variable globally for VSCode:

     "terminal.integrated.env.windows": {
         "KMP_DUPLICATE_LIB_OK": "TRUE"
     }

4. Restart your VSCode or terminal session.

Note: This is an unsafe workaround, as it allows multiple OpenMP runtimes to be loaded, which can lead to crashes or incorrect results in some cases. Use with caution.

2. Reinstall or Update the OpenMP Libraries

The issue may stem from an outdated or incompatible version of the libiomp5md.dll file. You can try reinstalling or updating the library to the latest version:

1. If you’re using Intel MKL or MKL-related libraries, make sure they’re up to date. You can update Intel's MKL package:

   conda update mkl

2. If using libomp via Conda:

   conda install -c conda-forge libomp

3. If using an older version of Intel’s compiler or other conflicting libraries, make sure those are updated as well.

3. Ensure Only One OpenMP Runtime is Linked

The issue arises from having multiple OpenMP runtimes linked, which is dangerous and can lead to crashes. To ensure that only one OpenMP runtime is being used:

1. Check your environment for multiple instances of libiomp5md.dll. This DLL should only be loaded once.

2. If you’re using both Intel’s OpenMP runtime and another OpenMP library, ensure that only one is being linked at runtime.

   • You can check if any other libraries are loading multiple OpenMP runtimes by running the following command (for debugging purposes):

     tasklist /m libiomp5md.dll

   • If there are multiple versions of libiomp5md.dll being loaded, you may need to adjust the PATH environment variable to prioritize the correct version.

4. Update or Change Your Compiler/Build Settings

If you're building your own libraries or using custom extensions, ensure you're not mixing compilers that use different OpenMP implementations.

1. If you’re compiling code using MSVC (Microsoft Visual C++) and Intel compilers together, you can set DISABLE_OPENMP or configure to avoid mixing OpenMP runtimes.

2. If you're using Intel's compiler with icc, ensure that it’s properly linking to libiomp5md.dll.

5. Use the --no-binary Flag (If Using pip Installations)

If the issue occurs while installing dependencies, especially related to numpy or other dependencies compiled with OpenMP, you can try installing these libraries from source, which might help avoid runtime conflicts:

pip install --no-binary :all: <library_name>

6. Check for Conflicts in Python Packages

Sometimes Python packages (e.g., scipy, numpy, etc.) can cause OpenMP conflicts. It’s worth ensuring that your libraries are compatible and that there aren’t any old, conflicting packages.

You can run the following to check for outdated or problematic packages:

conda list

Summary

1. Set the KMP_DUPLICATE_LIB_OK=TRUE environment variable (as a workaround).
2. Update the OpenMP libraries and related packages (e.g., Intel MKL, libomp).
3. Ensure only one OpenMP runtime is being linked by checking your environment and build settings.
4. Reinstall dependencies using --no-binary if necessary to avoid binary conflicts.

After applying these fixes, try restarting your Jupyter kernel or VSCode and see if the issue persists.

Let me know if any of these steps help or if you'd like further clarification!

🟢 Create conda env awsmle_py310

conda create --name awsmle_py310 python=3.10
conda activate awsmle_py310
conda install ipykernel
pip install boto3 sagemaker matplotlib pillow
pip install webdataset  
pip install wandb
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126