Dev Environment on Windows (without WSL)

These are some notes for setting up a development system on Windows without using Windows Subsystem Linux (WSL). Setting up WSL is also valuable, but in some cases it can be slow or otherwise undesirable, so having a setup that only uses Windows is handy.

This currently focuses on setting up a proper shell built on top of Windows Powershell and Windows Terminal (or Cmder) and then setting up Python and friends in Windows (pyenv, pipenv, etc.).

Many of the shell setup ideas here came directly from this article with some help from this gist. Most of the rest comes from my own gist discussing setting up a dev system using WSL (or Linux directly or macOS).

Package Manager

Install scoop. This works like Homebrew or Chocolatey. On macOS, Homebrew is indispensible. I've tried Chocolatey for Windows, but never got into it. Scoop seems to work well.

Open a Powershell terminal (non-admin) and run:

Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')

If you get an execution policy error, run:

Set-ExecutionPolicy RemoteSigned -scope CurrentUser

and retry.

Shell and Prompt Setup

Install Cmder, Silver Searcher, vim, posh-git, powerline, and oh-my-posh. Cmder is an application that serves as an improved terminal window (has tabs, you can change the font, etc.) Silver Searcher is an improved ack-like command line file searcher. Vim should be obvious. Posh-git improves git integration with Powershell. Powerline provides prompt customization for Powershell (and zsh, bash, etc.) Oh-my-posh provides prompt themes.

The first three get installed via scoop:

scoop install cmder ag vim

Launch Cmder and do some basic setup to your liking. Be sure to save your settings to a ConEmu.xml file and back this up (better, check into version control). Some basic tips:

Set the font to one of the Powerline fonts that include additional glyphs for customizing your prompt. I like Source Code Pro for Powerline.
Choose Powershell (as opposed to cmd) as the default startup task.
Move the tab bar to the top
Play around with the colors a bit until you find something you like.
Turn off Window transparency (why do you need to see through it?)

Now add PowerLine, posh-git, and oh-my-posh to Powershell:

Install-Module PowerLine -Scope CurrentUser
Install-Module posh-git -Scope CurrentUser
Install-Module oh-my-posh -Scope CurrentUser
Set-Prompt

(It is currently (2020-03-24) in preview, but if you're on a recent version of Windows 10, you can install the Windows Terminal app from the Windows Store instead of Cmder, and then set up fonts and colors in its profiles.json file. This has the added advantage of supporting color emoji, custom color profiles, and other niceties.)

Powershell Profile

To complete the powershell prompt setup and add some other useful features (especially if you are accustomed to macOS or Linux prompts), add a Powershell profile. To find where Powershell looks for this file, run echo $PROFILE from a Powershell window. This should reveal something like:

C:\Users\<username>\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1

First, I like ls to give me a brief file listing and ll to give me a long listing. In Linux/macOS, you can just alias ll to be ls -la. In PowerShell it is a bit trickier because ls and dir both already give a long listing. Add this to your Powershell Profile to get there:

New-Alias ll dir
function l {
  Get-ChildItem $args -Exclude .* | Format-Wide Name -AutoSize
}
Set-Alias ls l -Option AllScope

Next, I like the which command in Linux-like OSes that tells you the path to an executable. This exists in PowerShell, but it is called get-command which is not only longer but includes a hyphen. Get around this with:

New-Alias which get-command

Then, for speed in version control, I like git status to be simply g:

function g {
  &"$env:Programfiles\Git\cmd\git.exe" "status"
}

Recursive deleting of files and folders is also a pain in PowerShell. In Linux, this is simply rm -rf, so add a function to your Profile:

function rmrf {
  Param(
    [Parameter(Mandatory=$true)]
    [string]$Target
  )
  Remove-Item -Recurse -Force $Target
}

Next, cURL is a handy tool for transferring data with URLs. Unfortunately, PowerShell adds an alias for curl to its own Invoke-WebRequest, so if you're used to using curl on Linux, do two things: Install curl from scoop with scoop install curl; and add the line rm alias:curl to your Profile to eliminate the default alias.

Finally, set up the prompt customization as described below.

Of course, once this has stabilized, check it into version control.

Prompt Theme

You can customize your prompt by creating a theme file. The oh-my-posh repository has several you can start from.

Create a file called e.g. MyTheme.psm1 and put it in C:\Users\<username>\Documents\WindowsPowerShell\PoshThemes. Now you can customize seeing git branches, virtual environment status, the current path (obviously), etc.

Then add the following lines to the bottom of your Powershell Profile so the prompt gets set up properly each time you open a new window:

Import-Module PowerLine
Import-Module posh-git
Import-Module oh-my-posh
Set-Theme MyTheme

And again, of course, once this has stabilized check it into version control.

Dotfiles

On Linux/macOS I have fairly extensive .dotfiles for setting up terminals and vim and git and so on. This is a work in progress for Windows, but so far I at least include a .gitconfig. This adds some aliases (e.g. st for status, ci for commit (a.k.a. check in), etc.) and sets up gitmoji as well (see below).

Version Control

Basically I use git for all of my version control needs. A few things use SVN especially if there are a lot of binary files or I need exclusive locking (this is more important for schematics, PCB layout, 3D modeling). Otherwise, it's git.

Gitmoji

I also like the idea of gitmoji. I don't know why, but I just like it. This is a system of conventions for prefixing git commit messages with an icon indicating what sort of change was made (e.g. new feature, bug fix, performance improvement).

It runs via Node.js (i.e. it is written in Javascript), so set that up first by following the provided instructions for Windows. Once complete, you should be able to open a powershell window, type node, and get a Node.js REPL.

With Node.js and its package manager, npm, installed, you can now install the Gitmoji CLI globally:

npm i -g gitmoji-cli
npm i -g gitmoji-log

If you get a message about missing peer dependencies, install those too. For example:

npm i -g inquirer@^6.0.0

Also ensure that globally installed npm packages appear in your Windows path. See the Node.js section below.

Finally, add the gitmoji definition file to your user folder:

gitmoji --update

Node.js

If you installed Gitmoji above, you already have Node.js. If not you can install it by going to the Node.js site and following the instructions there. Now you have all the power of Javascript and things like Electron at your fingertips.

Also ensure that globally installed npm packages appear in your Windows path. From a PowerShell prompt, run which gitmoji-log which should show a location like C:\Users\<username>\AppData\Roaming\npm\gitmoji-log.ps1. If it gives an error, edit your Path environment variable and add an entry for C:\Users\<username>\AppData\Roaming\npm.

Python Scientific (and Whatever Else) Programming

Python is useful for countless things, but one thing I like using it for is scientific programming and data visualization through numpy, scipy, and matplotlib. This way it becomes a viable, open alternative to things like Matlab.

Setup Python and Friends

To ensure that the same versions of libraries (and Python itself via pyenv) get used regardless of the development machine, OS, etc., use pipenv to manage package versions and a virtual environment for running code. *If you are familiar with pip and virtualenv, using pipenv means you no longer have to manage these separately.)

Install `pyenv`

First, install pyenv to manage the Python version in your virtual environment.

pip install pyenv-win --target $HOME/.pyenv

(This assumes you already have at least one version of python and pip installed which you can do via the Windows Store.)

Then, edit your Windows path to include the following paths:

C:\Users\<username>\.pyenv\pyenv-win\bin
C:\Users\<username>\.pyenv\pyenv-win\shims

Ensure these are included before any other Python related paths.

Install `python` Version

Now install an appropriate python version, e.g.:

$ pyenv install --list
$ pyenv install 3.8.0-amd64

This will also install the compatible version of pip. To make this version the system default run:

$ pyenv global 3.8.0-amd64

Also be sure to set up the shims after any version installation:

$ pyenv rehash

Now, running which python should point to something like C:\Users\<username>\.pyenv\pyenv-win\shims\python.bat, and running python --version should show the correct version.

Note that if you have an old version of Python (e.g. v2.7) installed, you may need to remove it or adjust your Windows path so that the pyenv shimmed version is found first.

Install `pipenv` Globally

To install pipenv, the package and environment manager, run:

$ pip install pipenv

If you get path warnings, be sure to update your pyenv shims with pyenv rehash. Then you should be able to run pipenv --version and see the correct version.

Test Python/Scipy

To do a quick test of the installation, install numpy and matplotlib and create a plot. (On Windows/WSL, ensure your XServer is running so the plot window can be displayed.)

$ mkdir plottest
$ cd plottest
$ pipenv install numpy matplotlib

The last command creates new virtual environment and installs the numerical and plotting libraries.

Now create a simple test program as test.py using your editor of choice:

$ vi test.py

import numpy as np
import matplotlib.pyplot as plt

t = np.arange(0, 10, 1/100)
plt.plot(t, np.sin(1.7 * 2 * np.pi * t) + np.sin(1.9 * 2 * np.pi * t))
plt.show()

Now run the script. (Again, if you're on Windows/WSL, make sure your XServer is active at this point.)

$ pipenv run python test.py

The graph should appear in a window. Close the plot window to allow the script to finish.

Note that this command runs python test.py from inside the virtual environment that pipenv set up. Alternatively, you can make every command you type run from inside the environment. Try:

$ pipenv shell
plottest-uHQw95fx $ cd plottest     # You might need to change directories again
plottest-uHQw95fx $ python test.py
plottest-uHQw95fx $ exit
$

At this point, you can remove the generated virtualenv and test directory. From within plottest/:

$ pipenv --rm
$ cd ..
$ rmrf plottest

One thing I've noticed is that running pipenv shell actually creates the subshell using cmd.exe instead of powershell.exe. There are some closed issue threads in pipenv, but I'm still seeing a problem. A workaround is to instead run pipenv run powershell.

Also, if you have included the virtual environment as part of your customized PowerShell prompt, make sure that pipenv/virtualenv does not change it further by defining VIRTUAL_ENV_DISABLE_PROMPT=1 in your Windows environment variables.

Editor Setup

The editor of choice is, of course, Visual Studio Code with, of course, the vim plugin, and vim from within a terminal window.

Because setting these up is largely platform independent, see more details here.

More vim details

Installing vim on Windows, as noted above, is just:

scoop install vim

and then bringing in any .vimrc files you may have for customization. Note it does seem that on Windows, vim wants it to be _vimrc and not .vimrc in your User folder.

If you are using dotfiles as in the gist linked above, you can make symbolic links to ~\.dotfiles\vimrc as _vimrc and other files. and to ~\.dotfiles\vim (the folder) as vimfiles.

For example, running PowerShell as admin:

New-Item -Path C:\Users\<username>\_vimrc -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\vimrc
New-Item -Path C:\Users\<username>\.vimrc.local -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\local\vimrc.local
New-Item -Path C:\Users\<username>\.vimrc.bundles -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\vimrc.bundles

Then make a copy of ~\.dotfiles\vim (the folder) as ~\vimfiles. When you run vim the first time, it will update plugins, and you may need to manually run :PlugInstall inside vim once.

Because vim comes with git (and is the default editor), you may need to set a Windows environment variable to make sure that when you commit with git, the correct version of vim is used. (Without this, you could run into problems loading vim plugins because has('win32) may resolve to false even on Windows with the version that comes with git.) Edit your Windows environment and define GIT_EDITOR as "C:\Users\<username>\scoop\shims\vim.exe" (with the surrounding quotes; adjust the path as necessary).

More .dotfiles details

If using cross-platform .dotfiles as in this gist. set up symbolic links to your clone of the dotfiles repo (assumed to be in ~\.dotfiles) for things like your vimrc and gitconfig. Running PowerShell as admin:

New-Item -Path C:\Users\<username>\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1 -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\WindowsTerminal\Microsoft.PowerShell_profile.ps1
New-Item -Path C:\Users\<username>\Documents\WindowsPowerShell\PoshThemes\MyTheme.psm1 -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\WindowsTerminal\MyTheme.psm1
New-Item -Path C:\Users\<username>\.git_template -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\git_template
New-Item -Path C:\Users\<username>\.gitconfig -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\gitconfig
New-Item -Path C:\Users\<username>\.gitconfig.local -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\local\gitconfig.local
New-Item -Path C:\Users\<username>\.gitignore -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\gitignore
New-Item -Path C:\Users\<username>\.gitmessage -ItemType SymbolicLink -Value C:\Users\<username>\.dotfiles\gitmessage

jimbrig/WindowsDevSetup.md