fully configuring embedded Python on Windows 10

python-embedded-for-Win10.md

Update: use PowerShell script PythonEmbed4Win.ps1.

The instructions in this gist have some subtle problems and this gist will not be updated.

About

Embedded Python (Windows embeddable package zip file) will not find installed packages and modules . It requires a few extra steps to fully configure.

These instructions will install, configure, and then create a new virtual python environment based on the embedded distribution.
This Python distribution and any derived virtual environments will run in a portable manner (no need for environment variable hacks, no special registry settings, etc.).

Tested with python-3.9.6-embed-amd64.zip on Windows 10 using powershell.

---

1. Install and configure embedded Python
2. Create a new virtual environment based on the embedded distribution

---

Install and configure embedded Python

It is recommended to run these steps as an Administrator.

Download and unzip

1. Download the .zip file

2. Unzip to some path (this will use example path C:\python-3.9.6-embed-amd64\).

   PS > Expand-Archive .\python-3.9.6-embed-amd64.zip -DestinationPath C:\python-3.9.6-embed-amd64
   

   Unzipped contents will look similar to

PS > dir

  Directory: C:\python-3.9.6-embed-amd64

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         8/27/2021  12:12 PM        3406016 libcrypto-1_1.dll
-a----         8/27/2021  12:12 PM          32792 libffi-7.dll
-a----         8/27/2021  12:12 PM         690368 libssl-1_1.dll
-a----         8/27/2021  12:12 PM          32628 LICENSE.txt
-a----         8/27/2021  12:12 PM         192744 pyexpat.pyd
-a----         8/27/2021  12:12 PM         561429 python.cat
-a----         8/27/2021  12:12 PM         101608 python.exe
-a----         8/27/2021  12:12 PM          59624 python3.dll
-a----         8/27/2021  12:12 PM        4485864 python39.dll
-a----         8/27/2021  12:12 PM        2502717 python39.zip
-a----         8/27/2021  12:12 PM             79 python39._pth
-a----         8/27/2021  12:12 PM         100072 pythonw.exe
-a----         8/27/2021  12:12 PM          28904 select.pyd
-a----         8/27/2021  12:12 PM        1538792 sqlite3.dll
-a----         8/27/2021  12:12 PM        1121512 unicodedata.pyd
-a----         8/27/2021  12:12 PM          97160 vcruntime140.dll
-a----         8/27/2021  12:12 PM          37256 vcruntime140_1.dll
-a----         8/27/2021  12:12 PM          29928 winsound.pyd
-a----         8/27/2021  12:12 PM          65256 _asyncio.pyd
-a----         8/27/2021  12:12 PM          86760 _bz2.pyd
-a----         8/27/2021  12:12 PM         127208 _ctypes.pyd
-a----         8/27/2021  12:12 PM         270568 _decimal.pyd
-a----         8/27/2021  12:12 PM         179944 _elementtree.pyd
-a----         8/27/2021  12:12 PM          66280 _hashlib.pyd
-a----         8/27/2021  12:12 PM         163048 _lzma.pyd
-a----         8/27/2021  12:12 PM          40168 _msi.pyd
-a----         8/27/2021  12:12 PM          30440 _multiprocessing.pyd
-a----         8/27/2021  12:12 PM          46312 _overlapped.pyd
-a----         8/27/2021  12:12 PM          29416 _queue.pyd
-a----         8/27/2021  12:12 PM          80616 _socket.pyd
-a----         8/27/2021  12:12 PM          89832 _sqlite3.pyd
-a----         8/27/2021  12:12 PM         155368 _ssl.pyd
-a----         8/27/2021  12:12 PM          23784 _uuid.pyd
-a----         8/27/2021  12:12 PM          47336 _zoneinfo.pyd

3. Unzip C:\python-3.9.6-embed-amd64\python39.zip to C:\python-3.9.6-embed-amd64\python39

   PS> Expand-Archive C:\python-3.9.6-embed-amd64\python39.zip -DestinationPath C:\python-3.9.6-embed-amd64\python39
   

   Without these changes, you may see an error regarding pickle.

   error: [Errno 0] Error: 'lib2to3\\Grammar3.6.5.final.0.pickle'
   

   The issue is pickle cannot read files within a .zip file.

   3a. Optional: delete python39.zip

   PS> Remove-Item "C:\python-3.9.6-embed-amd64\python39.zip"
   

adjust paths

Running C:\python-3.9.6-embed-amd64\python -m pip will fail with No module named pip.

Path settings must be adjusted.

4. Change C:\python-3.9.6-embed-amd64\python39._pth to

   .\python39
   .\Scripts
   .
   # importing site will run sitecustomize.py
   import site
   

   Without these changes, you may see error

   Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
   Python runtime state: core initialized
   ModuleNotFoundError: No module named 'encodings'
   

5. Create C:\python-3.9.6-embed-amd64\sitecustomize.py with contents

   import sys
   

   This will override default built-in sitecustomize changes that would add a global site path to sys.path.
   i.e. sys.path[-1] by default has value "C:\\Users\\user1\\AppData\\Roaming\\Python\\Python39\\site-packages". But that is not wanted in this case.

6. Create empty directory DLLs

   PS> New-Item -type directory "C:\python-3.9.6-embed-amd64\DLLs"
   

   Adding directory DLLs will workaround the module-loading error:

   FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\python-3.9.6-embed-amd64\\DLLs'
   

7. Check paths.
   From a different working directory run

   PS> C:\python-3.9.6-embed-amd64\python.exe -c "import sys; print(sys.path)"
   ['C:\\python-3.9.6-embed-amd64\\python39', 'C:\\python-3.9.6-embed-amd64', 'C:\\python-3.9.6-embed-amd64\\Scripts', 'C:\\python-3.9.6-embed-amd64\\lib\\site-packages']
   

Install pip using get-pip.py

PS> Invoke-WebRequest -OutFile "C:\python-3.9.6-embed-amd64\get-pip.py" "https://bootstrap.pypa.io/get-pip.py"
PS> C:\python-3.9.6-embed-amd64\python.exe "C:\python-3.9.6-embed-amd64\get-pip.py"

Install virtualenv package

PS> C:\python-3.9.6-embed-amd64\python.exe -m pip install virtualenv

Create a new virtual environment based on the embedded distribution

This may be run as a normal user.

From some other directory run

PS> C:\python-3.9.6-embed-amd64\python.exe -m virtualenv --copies ".venv-3.9-embedded"

This creates a new fully functioning and fully independent virtual python environment, i.e. does not use symbolic links or Windows Registry entries or $env environment variables.

Activate the virtual environment

PS> .\.venv-3.9-embedded\Scripts\activate.ps1

Verify the expected python.exe is used

PS> Get-Command python.exe | Format-Table -Autosize

CommandType Name       Version       Source
----------- ----       -------       ------
Application python.exe 3.9.6150.1013 C:\Users\user1\My-Project\.venv-3.9-embedded/Scripts\python.exe

---

Helpful References:

• Setting up python's Windows embeddable distribution (properly)
• Python 3.9.6
• Using Python on Windows - Finding Modules
• stackoverflow question: Python Windows embeddable package fails to run "No module named pip" "The system cannot find the path specified: 'C:\python-3.9.6-embed-amd64\DLLs'"

Great writeup! It was very helpful when I set up an embedded Python package for distribution.

Tip: To include the current working directory in sys.path, add sys.path.append('.') to the sitecustomize.py

Really helpful. But for my case for python 3.12 my ._pth file is the following:

python312.zip
.
.\Lib
.\Lib\site-packages
.\Scripts

# Uncomment to run site.main() automatically
import site

When i tried to use your example (but with .\python312) it was throwing an error whenever i tried to run the python command

Really helpful. But for my case for python 3.12 my ._pth file is the following:

python312.zip
.
.\Lib
.\Lib\site-packages
.\Scripts

# Uncomment to run site.main() automatically
import site

When i tried to use your example (but with .\python312) it was throwing an error whenever i tried to run the python command

Use PowerShell script PythonEmbed4Win.ps1.

Tip: To include the current working directory in sys.path, add sys.path.append('.') to the sitecustomize.py

This is done in the PowerShell script PythonEmbed4Win.ps1.

Really helpful! Thank you very much!!!!!!

@jtmoon79 I tried to follow the steps that the powershell script https://github.com/jtmoon79/PythonEmbed4Win does manually to understand how it works. I can almost get it to work by repeating the steps manually except for one small issue: after using pip to install what I need I start python.exe and then try to run "import sklearn" and I get this error.

Any thoughts? This issue doesn't occur with any other module from my vast requirements.txt file, and when I use the powershell script to automatically set things up, it all works perfectly...