Download MSVC compiler/linker & Windows SDK without installing full Visual Studio

!README.md

This downloads standalone MSVC compiler, linker & other tools, also headers/libraries from Windows SDK into portable folder, without installing Visual Studio. Has bare minimum components - no UWP/Store/WindowsRT stuff, just files & tools for native desktop app development.

Run py.exe portable-msvc.py and it will download output into msvc folder. By default it will download latest available MSVC & Windows SDK from newest Visual Studio.

You can list available versions with py.exe portable-msvc.py --show-versions and then pass versions you want with --msvc-version and --sdk-version arguments.

To use cl.exe/link.exe first run setup_TARGET.bat - after that PATH/INCLUDE/LIB env variables will be updated to use all the tools as usual. You can also use clang-cl.exe with these includes & libraries.

To use clang-cl.exe without running setup.bat, pass extra /winsysroot msvc argument (msvc is folder name where output is stored).

portable-msvc.py

#!/usr/bin/env python3

import io
import os
import sys
import stat
import json
import shutil
import hashlib
import zipfile
import tempfile
import argparse
import subprocess
import urllib.error
import urllib.request
from pathlib import Path

OUTPUT = Path("msvc")         # output folder
DOWNLOADS = Path("downloads") # temporary download files

# NOTE: not all host & target architecture combinations are supported

DEFAULT_HOST = "x64"
ALL_HOSTS    = "x64 x86 arm64".split()

DEFAULT_TARGET = "x64"
ALL_TARGETS    = "x64 x86 arm arm64".split()

DEFAULT_VERSION = "latest"
ALL_VERSIONS    = "2019 2022 2026 latest".split()

MANIFEST_URLS = {
  "latest": ["https://aka.ms/vs/stable/channel",     "https://aka.ms/vs/insiders/channel"   ],
  "2026":   ["https://aka.ms/vs/18/stable/channel",  "https://aka.ms/vs/18/insiders/channel"],
  "2022":   ["https://aka.ms/vs/17/release/channel", "https://aka.ms/vs/17/pre/channel"     ],
  "2019":   ["https://aka.ms/vs/16/release/channel", "https://aka.ms/vs/16/pre/channel"     ],
}

ssl_context = None

def download(url):
  with urllib.request.urlopen(url, context=ssl_context) as res:
    return res.read()

total_download = 0

def download_progress(url, check, filename):
  fpath = DOWNLOADS / filename
  if fpath.exists():
    data = fpath.read_bytes()
    if hashlib.sha256(data).hexdigest() == check.lower():
      print(f"\r{filename} ... OK")
      return data

  global total_download
  with fpath.open("wb") as f:
    data = io.BytesIO()
    with urllib.request.urlopen(url, context=ssl_context) as res:
      total = int(res.headers["Content-Length"])
      size = 0
      while True:
        block = res.read(1<<20)
        if not block:
          break
        f.write(block)
        data.write(block)
        size += len(block)
        perc = size * 100 // total
        print(f"\r{filename} ... {perc}%", end="")
    print()
    data = data.getvalue()
    digest = hashlib.sha256(data).hexdigest()
    if check.lower() != digest:
      sys.exit(f"Hash mismatch for f{pkg}")
    total_download += len(data)
    return data

# super crappy msi format parser just to find required .cab files
def get_msi_cabs(msi):
  index = 0
  while True:
    index = msi.find(b".cab", index+4)
    if index < 0:
      return
    yield msi[index-32:index+4].decode("ascii")

def first(items, cond = lambda x: True):
  return next((item for item in items if cond(item)), None)


### parse command-line arguments

ap = argparse.ArgumentParser()
ap.add_argument("--show-versions", action="store_true", help="Show available MSVC and Windows SDK versions")
ap.add_argument("--accept-license", action="store_true", help="Automatically accept license")
ap.add_argument("--msvc-version", help="Get specific MSVC version")
ap.add_argument("--sdk-version", help="Get specific Windows SDK version")
ap.add_argument("--vs", default=DEFAULT_VERSION, help=f"Visual Studio version to use for installation", choices=ALL_VERSIONS)
ap.add_argument("--insiders", action="store_true", help="Use insiders channnel")
ap.add_argument("--preview", action="store_true", help="Allow preview / release candidate MSVC versions")
ap.add_argument("--target", default=DEFAULT_TARGET, help=f"Target architectures, comma separated ({','.join(ALL_TARGETS)})")
ap.add_argument("--host", default=DEFAULT_HOST, help=f"Host architecture", choices=ALL_HOSTS)
args = ap.parse_args()

host = args.host
targets = args.target.split(',')
for target in targets:
  if target not in ALL_TARGETS:
    sys.exit(f"Unknown {target} target architecture!")


### get main manifest

URL = MANIFEST_URLS[args.vs][args.insiders]

try:
  manifest = json.loads(download(URL))
except urllib.error.URLError as err:
  import ssl
  if isinstance(err.args[0], ssl.SSLCertVerificationError):
    # for more info about Python & issues with Windows certificates see https://stackoverflow.com/a/52074591
    print("ERROR: ssl certificate verification error")
    try:
      import certifi
    except ModuleNotFoundError:
      print("ERROR: please install 'certifi' package to use Mozilla certificates")
      print("ERROR: or update your Windows certs, see instructions here: https://woshub.com/updating-trusted-root-certificates-in-windows-10/#h2_3")
      sys.exit()
    print("NOTE: retrying with certifi certificates")
    ssl_context = ssl.create_default_context(cafile=certifi.where())
    manifest = json.loads(download(URL))
  else:
    raise

### download VS manifest

ITEM_NAME = "Microsoft.VisualStudio.Manifests.VisualStudioPreview" if args.insiders else "Microsoft.VisualStudio.Manifests.VisualStudio"

vs = first(manifest["channelItems"], lambda x: x["id"] == ITEM_NAME)
payload = vs["payloads"][0]["url"]

vsmanifest = json.loads(download(payload))


### find MSVC & WinSDK versions

packages = {}
for p in vsmanifest["packages"]:
  packages.setdefault(p["id"].lower(), []).append(p)

msvc = {}
sdk = {}

for pid,p in packages.items():
  if pid.startswith("Microsoft.VC.".lower()) and pid.endswith(".Tools.HostX64.TargetX64.base".lower()) and "premium" not in pid.lower():
    pver = ".".join(pid.split(".")[2:4])
    if pver[0].isnumeric():
      msvc[pver] = pid
  elif pid.startswith("Microsoft.VisualStudio.Component.Windows10SDK.".lower()) or \
       pid.startswith("Microsoft.VisualStudio.Component.Windows11SDK.".lower()):
    pver = pid.split(".")[-1]
    if pver.isnumeric():
      sdk[pver] = pid

if not args.preview:
  # remove preview version from non-preview package list
  p = packages.get("Microsoft.VC.Preview.Tools.HostX64.TargetX64".lower())
  if p:
    pver = ".".join(p[0]["version"].split(".")[:2])
    msvc.pop(pver)

if args.show_versions:
  print("MSVC versions:", " ".join(sorted(msvc.keys())))
  print("Windows SDK versions:", " ".join(sorted(sdk.keys())))
  sys.exit(0)

msvc_ver = args.msvc_version or max(sorted(msvc.keys()))
sdk_ver = args.sdk_version or max(sorted(sdk.keys()))

if msvc_ver in msvc:
  msvc_pid = msvc[msvc_ver]
  msvc_ver = msvc_pid.removeprefix("microsoft.vc.").removesuffix(".tools.hostx64.targetx64.base")
else:
  sys.exit(f"Unknown MSVC version: f{args.msvc_version}")

if sdk_ver in sdk:
  sdk_pid = sdk[sdk_ver]
else:
  sys.exit(f"Unknown Windows SDK version: f{args.sdk_version}")

### non-windows host requires msiextract tool

if sys.platform != "win32":
  try:
    subprocess.check_output(["msiextract", "--version"])
  except FileNotFoundError:
    sys.exit("ERROR: msiextract tool not available, please install msitools for your distro")


print(f"Downloading MSVC v{msvc_ver} and Windows SDK v{sdk_ver}")


### agree to license

tools = first(manifest["channelItems"], lambda x: x["id"] == "Microsoft.VisualStudio.Product.BuildTools")
resource = first(tools["localizedResources"], lambda x: x["language"] == "en-us")
license = resource["license"]

if not args.accept_license:
  accept = input(f"Do you accept Visual Studio license at {license} [Y/N] ? ")
  if not accept or accept[0].lower() != "y":
    sys.exit(0)

OUTPUT.mkdir(exist_ok=True)
DOWNLOADS.mkdir(exist_ok=True)


### download MSVC

msvc_packages = [
  f"microsoft.visualcpp.dia.sdk",
  f"microsoft.vc.{msvc_ver}.crt.headers.base",
  f"microsoft.vc.{msvc_ver}.crt.source.base",
  f"microsoft.vc.{msvc_ver}.asan.headers.base",
  f"microsoft.vc.{msvc_ver}.pgo.headers.base",
]

for target in targets:
  msvc_packages += [
    f"microsoft.vc.{msvc_ver}.tools.host{host}.target{target}.base",
    f"microsoft.vc.{msvc_ver}.tools.host{host}.target{target}.res.base",
    f"microsoft.vc.{msvc_ver}.crt.{target}.desktop.base",
    f"microsoft.vc.{msvc_ver}.crt.{target}.store.base",
    f"microsoft.vc.{msvc_ver}.premium.tools.host{host}.target{target}.base",
    f"microsoft.vc.{msvc_ver}.pgo.{target}.base",
  ]
  if target in ["x86", "x64"]:
    msvc_packages += [f"microsoft.vc.{msvc_ver}.asan.{target}.base"]

  redist_suffix = ".onecore.desktop" if target == "arm" else ""
  redist_pkg = f"microsoft.vc.{msvc_ver}.crt.redist.{target}{redist_suffix}.base"
  if redist_pkg not in packages:
    redist_name = f"microsoft.visualcpp.crt.redist.{target}{redist_suffix}"
    redist = first(packages[redist_name])
    redist_pkg = first(redist["dependencies"], lambda dep: dep.endswith(".base")).lower()
  msvc_packages += [redist_pkg]

for pkg in sorted(msvc_packages):
  if pkg not in packages:
    print(f"\r{pkg} ... !!! MISSING !!!")
    continue
  p = first(packages[pkg], lambda p: p.get("language") in (None, "en-US"))
  for payload in p["payloads"]:
    filename = payload["fileName"]
    download_progress(payload["url"], payload["sha256"], filename)
    with zipfile.ZipFile(DOWNLOADS / filename) as z:
      for name in z.namelist():
        if name.startswith("Contents/"):
          out = OUTPUT / Path(name).relative_to("Contents")
          out.parent.mkdir(parents=True, exist_ok=True)
          out.write_bytes(z.read(name))


### download Windows SDK

sdk_packages = [
  f"Windows SDK for Windows Store Apps Tools-x86_en-us.msi",
  f"Windows SDK for Windows Store Apps Headers-x86_en-us.msi",
  f"Windows SDK for Windows Store Apps Headers OnecoreUap-x86_en-us.msi",
  f"Windows SDK for Windows Store Apps Libs-x86_en-us.msi",
  f"Universal CRT Headers Libraries and Sources-x86_en-us.msi",
]

for target in ALL_TARGETS:
  sdk_packages += [
    f"Windows SDK Desktop Headers {target}-x86_en-us.msi",
    f"Windows SDK OnecoreUap Headers {target}-x86_en-us.msi",
  ]

for target in targets:
  sdk_packages += [f"Windows SDK Desktop Libs {target}-x86_en-us.msi"]

with tempfile.TemporaryDirectory(dir=DOWNLOADS) as d:
  dst = Path(d)

  sdk_pkg = packages[sdk_pid][0]
  sdk_pkg = packages[first(sdk_pkg["dependencies"]).lower()][0]

  msi = []
  cabs = []

  # download msi files
  for pkg in sorted(sdk_packages):
    payload = first(sdk_pkg["payloads"], lambda p: p["fileName"].replace("\\", "/") == f"Installers/{pkg}")
    if payload is None:
      continue
    msi.append(DOWNLOADS / pkg)
    data = download_progress(payload["url"], payload["sha256"], pkg)
    cabs += list(get_msi_cabs(data))

  if len(cabs) == 0:
    sys.exit("No cab files found for Windows SDK, something is wrong!")

  # download .cab files
  for pkg in cabs:
    payload = first(sdk_pkg["payloads"], lambda p: p["fileName"].replace("\\", "/") == f"Installers/{pkg}")
    download_progress(payload["url"], payload["sha256"], pkg)

  print("Unpacking msi files...")

  # run/unpack msi files
  if sys.platform == "win32":
    for m in msi:
      subprocess.check_call(f'msiexec /a "{m}" /quiet /qn TARGETDIR="{OUTPUT.resolve()}"')
      (OUTPUT / m.name).unlink()
  else:
    subprocess.check_output(["msiextract", "-C", OUTPUT.resolve()] + msi)

    pf = OUTPUT / "Program Files"
    for src, dirs, files in (pf / "Windows Kits").walk():
      dst = OUTPUT / src.relative_to(pf)
      dst.mkdir(exist_ok = True)
      for f in files:
        (src / f).replace(dst / f)
    shutil.rmtree(pf, ignore_errors = True)

### versions

msvcv = first((OUTPUT / "VC/Tools/MSVC").glob("*")).name
sdkv = first((OUTPUT / "Windows Kits/10/bin").glob("*")).name


# place debug CRT runtime files into MSVC bin folder (not what real Visual Studio installer does... but is reasonable)
# NOTE: these are Target architecture, not Host architecture binaries

redist = OUTPUT / "VC/Redist"

if redist.exists():
  redistv = first((redist / "MSVC").glob("*")).name
  src = redist / "MSVC" / redistv / "debug_nonredist"
  for target in targets:
    for f in (src / target).glob("**/*.dll"):
      dst = OUTPUT / "VC/Tools/MSVC" / msvcv / f"bin/Host{host}" / target
      f.replace(dst / f.name)

  shutil.rmtree(redist)


# copy msdia140.dll file into MSVC bin folder
# NOTE: this is meant only for development - always Host architecture, even when placed into all Target architecture folders

msdia140dll = {
  "x86": "msdia140.dll",
  "x64": "amd64/msdia140.dll",
  "arm": "arm/msdia140.dll",
  "arm64": "arm64/msdia140.dll",
}

dst = OUTPUT / "VC/Tools/MSVC" / msvcv / f"bin/Host{host}"
src = OUTPUT / "DIA%20SDK/bin" / msdia140dll[host]
for target in targets:
  shutil.copyfile(src, dst / target / src.name)

shutil.rmtree(OUTPUT / "DIA%20SDK")


### cleanup

shutil.rmtree(OUTPUT / "Common7", ignore_errors=True)
shutil.rmtree(OUTPUT / "VC/Tools/MSVC" / msvcv / "Auxiliary")
for target in targets:
  for f in [f"store", "uwp", "enclave", "onecore"]:
    shutil.rmtree(OUTPUT / "VC/Tools/MSVC" / msvcv / "lib" / target / f, ignore_errors=True)
  shutil.rmtree(OUTPUT / "VC/Tools/MSVC" / msvcv / f"bin/Host{host}" / target / "onecore", ignore_errors=True)
for f in ["Catalogs", "DesignTime", f"bin/{sdkv}/chpe", f"Lib/{sdkv}/ucrt_enclave"]:
  shutil.rmtree(OUTPUT / "Windows Kits/10" / f, ignore_errors=True)
for arch in ["x86", "x64", "arm", "arm64"]:
  if arch not in targets:
    shutil.rmtree(OUTPUT / "Windows Kits/10/Lib" / sdkv / "ucrt" / arch, ignore_errors=True)
    shutil.rmtree(OUTPUT / "Windows Kits/10/Lib" / sdkv / "um" / arch, ignore_errors=True)
  if arch != host:
    shutil.rmtree(OUTPUT / "VC/Tools/MSVC" / msvcv / f"bin/Host{arch}", ignore_errors=True)
    shutil.rmtree(OUTPUT / "Windows Kits/10/bin" / sdkv / arch, ignore_errors=True)

# executable that is collecting & sending telemetry every time cl/link runs
for target in targets:
  (OUTPUT / "VC/Tools/MSVC" / msvcv / f"bin/Host{host}/{target}/vctip.exe").unlink(missing_ok=True)


# extra files for nvcc
build = OUTPUT / "VC/Auxiliary/Build"
build.mkdir(parents=True, exist_ok=True)
(build / "vcvarsall.bat").write_text("rem both bat files are here only for nvcc, do not call them manually")
(build / "vcvars64.bat").touch()

### setup.bat

for target in targets:

  SETUP = fr"""@echo off

set VSCMD_ARG_HOST_ARCH={host}
set VSCMD_ARG_TGT_ARCH={target}

set VCToolsVersion={msvcv}
set WindowsSDKVersion={sdkv}\

set VCToolsInstallDir=%~dp0VC\Tools\MSVC\{msvcv}\
set WindowsSdkBinPath=%~dp0Windows Kits\10\bin\

set PATH=%~dp0VC\Tools\MSVC\{msvcv}\bin\Host{host}\{target};%~dp0Windows Kits\10\bin\{sdkv}\{host};%~dp0Windows Kits\10\bin\{sdkv}\{host}\ucrt;%PATH%
set INCLUDE=%~dp0VC\Tools\MSVC\{msvcv}\include;%~dp0Windows Kits\10\Include\{sdkv}\ucrt;%~dp0Windows Kits\10\Include\{sdkv}\shared;%~dp0Windows Kits\10\Include\{sdkv}\um;%~dp0Windows Kits\10\Include\{sdkv}\winrt;%~dp0Windows Kits\10\Include\{sdkv}\cppwinrt
set LIB=%~dp0VC\Tools\MSVC\{msvcv}\lib\{target};%~dp0Windows Kits\10\Lib\{sdkv}\ucrt\{target};%~dp0Windows Kits\10\Lib\{sdkv}\um\{target}
"""
  (OUTPUT / f"setup_{target}.bat").write_text(SETUP)

print(f"Total downloaded: {total_download>>20} MB")
print("Done!")

Well, it's weird.

• Running the command as-is does nothing.
• When omitting the /qn parameter (oops, I just notice you didn't mention it, I somehow imagined it to be there or I copied it incorrectly from the code) I receive an error (especially 1620 in the logfile) that indicates an invalid installer file.

So something is not okay with my system. Repairing/reinstalling msiexec (resp. "DesktopAppInstaller") didn't help though, and I don't see any issues otherwise, the installer seems to be working in general. The downloaded packages look fine as well (with size > 0) and multiple attempts produce the same files, so the download is also not the problem… I'm confused.

Summary:

Seems to be an issue with my installer, not with the portable-msvc code itself. Sorry for the noise!

The one thing I wonder about is: manually extracting the .vsix packages with un-zip works and produces the same content as present in the msvc folder (i.e. the DIA%20SDK and VC sub-directories). Still no Windows Kits sub-directory. Where is it supposed to come from, is it created by msiexec when performing an actual installation of the .vsix files (instead of only extracting them)?

Update

Looking at the code it seems not only some .vsix files should be downloaded (for MSVC), but also some .msi and .cab packages (for Windows SDK)!

The latter didn't happen, there are only 13 .vsix files in the downloads directory). But there was also no warning or error message that the download was not complete fort some reason, after the .vsix downloads immediately the output quoted above (Unpacking msi files… followed by the failed sdvk = … assignment) was shown.

So I guess it's a donwload/retrieval problem instead? 🤔

Update 2

Just tested it:

• The download works when specifying SDK version 26100 (I get many more files then and a successful "setup").
• When using SDK version 28000 on the other hand (which is the default, being the latest), nothing is downloaded at all. There is also no hint that the download could not be performed, maybe a warning should be provided in that case? Maybe a hiccup on MS servers, with no files available currently.

Well, I receive the following error now:
sdkv = first((OUTPUT / "Windows Kits/10/bin").glob("*")).name
AttributeError: 'NoneType' object has no attribute 'name'

Getting the same error here.

The download works when specifying SDK version 26100 (I get many more files then and a successful "setup").

Worked for me too. I get no .cab files in SDK 28000.

Pushed updated that fixes sdk 28000 installation. MS changed file name formatting in json manifest, so my script needed update. Without this fix it was simply not downloading msi+cab files for 28000 sdk version.

The one thing I wonder about is: manually extracting the .vsix packages with un-zip works and produces the same content as present in the msvc folder (i.e. the DIA%20SDK and VC sub-directories). Still no Windows Kits sub-directory. Where is it supposed to come from, is it created by msiexec when performing an actual installation of the .vsix files (instead of only extracting them)?

msvc (compiler/linker/runtime) is distributed as vsix files which are just zip files to extract.
winsdk (windows headers/libs/crt/tools) are distributed as msi & cab files that need to be unpacked with msiexec. It does not really install them, just asks to unpack them into destination folder.

I see. Thanks for the quick fix 👏, works perfectly now (and identification of the "no cabs found/downloaded" error was added as well 👍 ).

thanks! now the question is if the build tools installer and the VS updater will keep it all up todate ?

thanks! now the question is if the build tools installer and the VS updater will keep it all up todate ?

Er, no? It's a portable setup. It is not registered within the system, so not discovered and thus managed by any existing VS installation. If you need to update it, you simple have to run the Python script again.

I have to admit though it would be fine if the current "installation" (i.e., the version numbers of both MSVC and the SDK) were stored somehow (in kind of a local config file?) and checked against when attempting a new download – this way the script either could report "Nothing to do" (or ask for an explicit "run anyway?" confirmation) or otherwise continue and perform an actual update.

But this kind of storing the user options/selection and/or the actually downloaded version and evaluating them later on re-runs would make the script way more complex I guess and might go beyond its originally intended purpose…

Well, I wouldn't say it is way more complex. but as it turns out there is interest in your code. if this was me - next step would be - pip package and facilitating at least security updates for the existing installation.
thanks for all the work so far.