Dev Environment on Windows (without WSL)

These are some notes for setting up a development system on Windows without using Windows Subsystem Linux (WSL). Setting up WSL is also sometimes valuable, but in some cases it can be slow or otherwise undesirable, so having a setup that only uses Windows is handy. In fact, I rarely use WSL at all anymore.

This currently focuses on setting up a proper shell built on top of Windows Powershell and Windows Terminal (or Cmder) and then setting up Python and friends in Windows (poetry, etc.).

Many of the shell setup ideas here came directly from this article with some help from this gist. Most of the rest comes from my own gist discussing setting up a dev system using WSL (or Linux directly or macOS).

Package Manager

Install scoop. This works like Homebrew or Chocolatey. On macOS, Homebrew is indispensible. I've tried Chocolatey for Windows, but never got into it. Scoop seems to work well.

Open a Powershell terminal (non-admin) and run:

Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')

If you get an execution policy error, run:

Set-ExecutionPolicy RemoteSigned -scope CurrentUser

and retry.

Shell and Prompt Setup

Install the Windows Terminal app from the Windows Store, and then set up fonts and colors in its profiles.json file. This app supports color emoji, custom color profiles, and other niceties.

If, for some reason (e.g. your version of Win10 is too old), you cannot use the relatively new Windows Terminal app, you can install Cmder instead and get most of the way there. Cmder can be installed via scoop:

scoop install cmder

Now install Silver Searcher, vim, posh-git, powerline, and oh-my-posh. Silver Searcher is an improved ack-like command-line file searcher. Vim should be obvious. Posh-git improves git integration with Powershell. Powerline provides prompt customization for Powershell (and zsh, bash, etc.) Oh-my-posh provides prompt themes.

The first two get installed via scoop:

scoop install ag vim

Launch Cmder and do some basic setup to your liking. Be sure to save your settings to a ConEmu.xml file and back this up (better, check into version control). Some basic tips:

Set the font to one of the Powerline fonts that include additional glyphs for customizing your prompt. I like Source Code Pro for Powerline.
Choose Powershell (as opposed to cmd) as the default startup task.
Move the tab bar to the top
Play around with the colors a bit until you find something you like.
Turn off Window transparency (why do you need to see through it?)

Now add PowerLine, posh-git, and oh-my-posh to Powershell:

Install-Module PowerLine -Scope CurrentUser
Install-Module posh-git -Scope CurrentUser
Install-Module oh-my-posh -Scope CurrentUser
Set-Prompt

Powershell Profile

To complete the powershell prompt setup and add some other useful features (especially if you are accustomed to macOS or Linux prompts), add a Powershell profile. To find where Powershell looks for this file, run echo $PROFILE from a Powershell window. This should reveal something like:

C:\Users\<username>\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1

First, I like ls to give me a brief file listing and ll to give me a long listing. In Linux/macOS, you can just alias ll to be ls -la. In PowerShell it is a bit trickier because ls and dir both already give a long listing. Add this to your Powershell Profile to get there:

New-Alias ll dir
function l {
  Get-ChildItem $args -Exclude .* | Format-Wide Name -AutoSize
}
Set-Alias ls l -Option AllScope

Next, I like the which command in Linux-like OSes that tells you the path to an executable. This exists in PowerShell, but it is called get-command which is not only longer but includes a hyphen. Get around this with:

New-Alias which get-command

Then, for speed in version control, I like git status to be simply g:

function g {
  &"$env:Programfiles\Git\cmd\git.exe" "status"
}

Recursive deleting of files and folders is also a pain in PowerShell. In Linux, this is simply rm -rf, so add a function to your Profile:

function rmrf {
  Param(
    [Parameter(Mandatory=$true)]
    [string]$Target
  )
  Remove-Item -Recurse -Force $Target
}

Next, cURL is a handy tool for transferring data with URLs. Unfortunately, PowerShell adds an alias for curl to its own Invoke-WebRequest, so if you're used to using curl on Linux, do two things: Install curl from scoop with scoop install curl; and add the line rm alias:curl to your Profile to eliminate the default alias.

Finally, set up the prompt customization as described below.

Of course, once this has stabilized, check it into version control.

Prompt Theme

You can customize your prompt by creating a theme file. The oh-my-posh repository has several you can start from.

Create a file called e.g. MyTheme.psm1 and put it in C:\Users\<username>\Documents\WindowsPowerShell\PoshThemes. Now you can customize seeing git branches, virtual environment status, the current path (obviously), etc.

Then add the following lines to the bottom of your Powershell Profile so the prompt gets set up properly each time you open a new window:

Import-Module PowerLine
Import-Module posh-git
Import-Module oh-my-posh
Set-Theme MyTheme

And again, of course, once this has stabilized check it into version control.

Dotfiles

On Linux/macOS I have fairly extensive .dotfiles for setting up terminals and vim and git and so on. This is a work in progress for Windows, but so far I at least include a .gitconfig. This adds some aliases (e.g. st for status, ci for commit (a.k.a. check in), etc.) and sets up gitmoji as well (see below).

Version Control

Basically I use git for all of my version control needs. A few things use SVN especially if there are a lot of binary files or I need exclusive locking (this is more important for schematics, PCB layout, 3D modeling). Otherwise, it's git.

Gitmoji

I also like the idea of gitmoji. I don't know why, but I just like it. This is a system of conventions for prefixing git commit messages with an icon indicating what sort of change was made (e.g. new feature, bug fix, performance improvement).

It runs via Node.js (i.e. it is written in Javascript), so set that up first by following the provided instructions for Windows. Once complete, you should be able to open a powershell window, type node, and get a Node.js REPL.

With Node.js and its package manager, npm, installed, you can now install the Gitmoji CLI globally:

npm i -g gitmoji-cli
npm i -g gitmoji-log

If you get a message about missing peer dependencies, install those too. For example:

npm i -g inquirer@^6.0.0

Also ensure that globally installed npm packages appear in your Windows path. See the Node.js section below.

Finally, add the gitmoji definition file to your user folder:

gitmoji --update

Node.js

If you installed Gitmoji above, you already have Node.js. If not you can install it by going to the Node.js site and following the instructions there. Now you have all the power of Javascript and things like Electron at your fingertips.

Also ensure that globally installed npm packages appear in your Windows path. From a PowerShell prompt, run which gitmoji-log which should show a location like C:\Users\<username>\AppData\Roaming\npm\gitmoji-log.ps1. If it gives an error, edit your Path environment variable and add an entry for C:\Users\<username>\AppData\Roaming\npm.

Python Scientific (and Whatever Else) Programming

Python is useful for countless things, but one thing I like using it for is scientific programming and data visualization through numpy, scipy, and plotly. This way it becomes a viable, open alternative to things like Matlab.

Setup Python and Friends

To ensure that the same versions of libraries (and Python itself via pyenv) get used regardless of the development machine, OS, etc., use poetry to manage package versions and a virtual environment for running code. (If you are familiar with pip and virtualenv, using poetry means you no longer have to manage these separately. (pipenv is another alternative that I no longer use.))

Also, note that for many users, it is sufficient to skip installing pyenv and simply install the latest version of python 3.x from the Windows Store. Your mileage may vary. If you do not install pyenv, skip down to installing poetry.

Install `pyenv`

First, install pyenv to manage the Python version in your virtual environment.

pip install pyenv-win --target $HOME/.pyenv

(This assumes you already have at least one version of python and pip installed which you can do via the Windows Store.)

Then, edit your Windows path to include the following paths:

C:\Users\<username>\.pyenv\pyenv-win\bin
C:\Users\<username>\.pyenv\pyenv-win\shims

Ensure these are included before any other Python related paths.

Install `python` Version

Now install an appropriate python version, e.g.:

$ pyenv install --list
$ pyenv install 3.8.0-amd64

This will also install the compatible version of pip. To make this version the system default run:

$ pyenv global 3.8.0-amd64

Also be sure to set up the shims after any version installation:

$ pyenv rehash

Now, running which python should point to something like C:\Users\<username>\.pyenv\pyenv-win\shims\python.bat, and running python --version should show the correct version.

Note that if you have an old version of Python (e.g. v2.7) installed, you may need to remove it or adjust your Windows path so that the pyenv shimmed version is found first.

Install `poetry` Globally

To install poetry, the package and environment manager, follow the docs, which currently have you run the following in PowerShell:

$ (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python -

If you get path warnings and you are using pyenv, be sure to update your pyenv shims with pyenv rehash. Then you should be able to run poetry --version and see the correct version.

Test Python/Scipy

To do a quick test of the installation, install numpy and matplotlib and create a plot.

$ mkdir plottest
$ cd plottest
$ poetry init  # Accept all defaults here
$ poetry add numpy matplotlib

The last command creates new virtual environment and installs the numerical and plotting libraries.

Now create a simple test program as test.py using your editor of choice:

$ vi test.py

import numpy as np
import matplotlib.pyplot as plt

t = np.arange(0, 10, 1/100)
plt.plot(t, np.sin(1.7 * 2 * np.pi * t) + np.sin(1.9 * 2 * np.pi * t))
plt.show()

Now run the script:

$ poetry run python test.py

The graph should appear in a window. Close the plot window to allow the script to finish.

Note that this command runs python test.py from inside the virtual environment that poetry set up. Alternatively, you can make every command you type run from inside the environment. Try:

$ poetry shell
plottest-kWdR7Cds-py3.9 $ cd plottest     # You might need to change directories again
plottest-kWdR7Cds-py3.9 $ python test.py
plottest-kWdR7Cds-py3.9 $ exit
$

At this point, you can remove the generated virtualenv and test directory. From within plottest/:

$ rmrf (poetry env info -p)
$ cd ..
$ rmrf plottest

Note that the plottest-kWdR7Cds-py3.9 $ above demonstrates a customized prompt that indicates the name of the virtual environment that is active. You may run into issues where poetry tries modifying your prompt and overrides your customizations. If this happens, define VIRTUAL_ENV_DISABLE_PROMPT=1 in your Windows environment variables.

Editor Setup

The editor of choice is, of course, Visual Studio Code with, of course, the vim plugin, and vim from within a terminal window.

Because setting these up is largely platform independent, see more details here.

More vim details

Installing vim on Windows, as noted above, is just:

scoop install vim

and then bringing in any .vimrc files you may have for customization. Note it does seem that on Windows, vim wants it to be _vimrc and not .vimrc in your User folder.

If you are using dotfiles as in the gist linked above, you can make symbolic links to ~\.dotfiles\vimrc as _vimrc and other files. and to ~\.dotfiles\vim (the folder) as vimfiles.

For example, running PowerShell as admin:

New-Item -Path C:\Users\<username>\_vimrc -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\vimrc
New-Item -Path C:\Users\<username>\.vimrc.local -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\local\vimrc.local
New-Item -Path C:\Users\<username>\.vimrc.bundles -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\vimrc.bundles

Then make a copy of ~\.dotfiles\vim (the folder) as ~\vimfiles. When you run vim the first time, it will update plugins, and you may need to manually run :PlugInstall inside vim once.

Because vim comes with git (and is the default editor), you may need to set a Windows environment variable to make sure that when you commit with git, the correct version of vim is used. (Without this, you could run into problems loading vim plugins because has('win32) may resolve to false even on Windows with the version that comes with git.) Edit your Windows environment and define GIT_EDITOR as "C:\Users\<username>\scoop\shims\vim.exe" (with the surrounding quotes; adjust the path as necessary).

More .dotfiles details

If using cross-platform .dotfiles as in this gist. set up symbolic links to your clone of the dotfiles repo (assumed to be in ~\.dotfiles) for things like your vimrc and gitconfig. Running PowerShell as admin:

New-Item -Path C:\Users\<username>\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1 -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\WindowsTerminal\Microsoft.PowerShell_profile.ps1
New-Item -Path C:\Users\<username>\Documents\WindowsPowerShell\PoshThemes\MyTheme.psm1 -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\WindowsTerminal\MyTheme.psm1
New-Item -Path C:\Users\<username>\.git_template -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\git_template
New-Item -Path C:\Users\<username>\.gitconfig -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\gitconfig
New-Item -Path C:\Users\<username>\.gitconfig.local -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\local\gitconfig.local
New-Item -Path C:\Users\<username>\.gitignore -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\gitignore
New-Item -Path C:\Users\<username>\.gitmessage -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\gitmessage

gbingersoll/WindowsDevSetup.md