Steam Catalog Research: AppID, Name, Launch Executable, Engine

Source: Open-source tooling survey with live verification of contested facts
Audience: Human-facing
Author: Claude Fable 5, for William Thompson / ShruggieTech
Date: 2026-08-12
Status: Research complete, no code written

Summary

The requirement covers four fields per title: Steam AppID, game name, the name of the executable Steam calls to launch the game, and the engine the game runs on. Two of those four are solved off the shelf and should not be built. The launch executable is not exposed by any public Steam Web API and lives only in PICS appinfo, reachable through the Steam client protocol. Engine attribution has no API at all: the method SteamDB uses is gated behind depot access that requires owning a license per title, so at catalog scale the field must come from a substitute source.

No single existing project covers all four fields. The build is a composition of three data sources plus one novel collection stage.

Field	Availability	Action
AppID	Public Web API, daily mirrors exist	Consume
Name	Public Web API, daily mirrors exist	Consume
Launch executable	PICS appinfo only, no bulk source	Build
Engine	No API, depot-gated	Substitute source

Defining the Coverage Target

The stated target is "the top 75% of games." That phrase needs a population and a metric attached before it can be implemented, because neither is currently defined.

The population problem is the larger of the two. The full application list returns on the order of a quarter million entries, and the majority of those are DLC, soundtracks, videos, demos, server binaries, and developer tools rather than games. Taking 75% of that set produces a list dominated by non-games. The population must be filtered to type == "game" first, which the category-split app list endpoint already does.

The metric problem follows. Steam's catalog has an enormous long tail of abandoned releases, asset flips, and titles with near-zero lifetime activity. Counting each title equally gives an obscure release with a handful of players the same weight as a game with millions of owners. A literal count-based 75% cut therefore measures nothing useful, and it also silently expands the crawl every quarter as the catalog grows.

The correct reinterpretation of the target is coverage of actual usage: the smallest set of games accounting for the overwhelming majority of real ownership and play activity. Steam popularity metrics follow a severe power law, so that set is dramatically smaller than 75% of the catalog by count.

The primary selection metric is total review count with a fixed threshold (for example, 500 or more total reviews). Review counts are free, unauthenticated, available per-appid, strongly correlated with real adoption, and stable over time. A fixed threshold is deterministic, degrades gracefully as the catalog grows, and gives better coverage per unit of collection work than any percentile cut.

SteamSpy owner estimates, peak concurrent users, and playtime figures are collected as secondary enrichment, not as the selection gate. SteamSpy's ownership numbers have been estimates of degraded quality since Valve's 2018 profile-privacy changes, and its API rate limits aggressively; the fields are still worth storing because they allow the corpus cut to be re-evaluated later without re-collecting anything. For deterministic ranking within the selected corpus, a log-scaled composite works:

popularity_score =
    log10(review_count + 1)             * 0.50
  + log10(estimated_owners_mid + 1)     * 0.35
  + log10(peak_ccu + 1)                 * 0.15

This is a ranking heuristic, not a statistical model. It prevents the largest titles from dominating linearly, rewards review volume, and produces a deterministic ordering from data the pipeline already holds.

AppID and Name

Fully solved. Two endpoints exist:

ISteamApps/GetAppList/v2 returns names and appids for every application regardless of category, unauthenticated, no key required.
IStoreService/GetAppList/v1 is the modern replacement. It requires a Steamworks Web API key (free with any Steam account), supports large paginated responses, and filters by application type at the source, which separates games from DLC, soundtracks, and tools before anything is downloaded. This is the primary enumeration endpoint for the pipeline.

Two repositories publish daily-refreshed mirrors via GitHub Actions, so even the enumeration crawl is optional. dgibbs64/SteamCMD-AppID-List stores every AppID and name as JSON, CSV, XML, and a Markdown table. jsnli/steamappidlist does the same against the category-split endpoint.

For the richer store metadata layer (price, genres, review counts, release dates, descriptions, SteamSpy fields), FronkonGames/Steam-Games-Scraper is the strongest existing base. It is MIT licensed and already handles application enumeration, non-game filtering, store metadata retrieval, SteamSpy integration, retry and backoff, incremental scraping, and JSON persistence. It collects the popularity fields the corpus cut needs: estimated_owners, positive, negative, peak_ccu, and the average and median playtime fields. It does not collect launch executables or engines; those are the enrichment stages this project adds. vintagedon/steam-dataset-2025 is the reference alternative, an explicitly modernized rebuild of the 2019 Kaggle Steam dataset using current APIs.

One operational note: the store appdetails endpoint rate-limits at roughly 200 successful requests per five-minute window. Full-catalog enrichment through that endpoint is an overnight job, not a five-minute one. Every stage that touches it needs resumability and a persistent cursor.

Launch Executable

Not available from any Steam Web API. The data lives in PICS (Product Information Cache Server) appinfo, under the config.launch section of an application's product info. It does not need to be inferred from installed files; Steam publishes the exact configuration its client executes.

The hierarchy per launch entry is:

config
  launch
    <index>
      executable
      arguments
      description
      type
      config
        oslist
        osarch
        betakey

Team Fortress 2 (appid 440) illustrates the shape:

config.launch.0.executable = "hl2.exe"
config.launch.0.arguments  = "-steam -game tf"
config.launch.0.config.oslist = "windows"

Access paths

Three access paths exist, and one works at catalog scale.

The correct path is the Steam client protocol with an anonymous login, calling PICS product-info requests against batches of appids. Three mature client implementations expose this:

ValvePython/steam (Python). The most mature Python implementation of the client protocol; get_product_info() under anonymous_login() is the recommended stage 3 mechanism because it keeps the pipeline in one language.
DoctorMcKay/node-steam-user (Node). Documents getProductInfo and getProductAccessTokens as working under anonymous login. The most actively maintained of the three; the fallback if the Python library's maintenance state ever becomes a blocker.
SteamRE/SteamKit (C#). The reference implementation the other two follow.

One access caveat applies to all three. Most apps return full appinfo to an anonymous session, but a subset flags missingToken (exposed as _missing_token in the Python library), and the per-app access token required to unlock full appinfo is only issued to accounts holding a license for that app. Access tokens are global rather than per-user and do not expire, which is exactly why SteamDB crowdsources them. For this project the consequence is high but not total launch-config coverage from an anonymous account; titles that come back token-gated are recorded with a token_required flag rather than treated as errors.

For inspecting the data shape before writing any code, steamctl (the CLI built on ValvePython/steam) exposes it directly:

steamctl apps product_info 440

DoctorMcKay/steam-pics-api wraps the same capability behind a small HTTP service if a language-agnostic boundary is ever preferred to embedding a Steam client in the collector.

The local-file alternative parses appinfo.vdf out of the Steam client cache. ValveResourceFormat/SteamAppInfo locates the Steam install, reads appinfo.vdf and packageinfo.vdf, dumps appids and tokens, and documents the binary format in its README. steamtinkerlaunch does the same thing in production for per-game metadata extraction on Linux. Both are format references, not catalog solutions, because the local cache only covers applications the installed client has actually seen.

The schema warning

The requirement as stated ("the name of the executable") assumes a scalar. It is not one. config.launch is an array of launch configurations, each carrying its own operating system filter, architecture, config type, and beta branch selector. A single Windows title routinely has separate entries for the 64-bit build, the 32-bit build, a DX11 versus DX12 renderer choice, a VR mode, a safe mode, and a legacy branch.

The second problem is worse. For launcher-mediated titles, the executable Steam invokes is the publisher launcher, not the game binary. The Elder Scrolls Online and The Division 2 are both in this category: Steam starts a ZeniMax or Ubisoft Connect launcher, which then starts the actual client. The generic pattern is:

Steam -> Launcher.exe -> Game-Win64-Shipping.exe

Steam correctly reports Launcher.exe; the process of interest after startup is the shipping binary. Any schema that flattens the launch config to one column records the launcher and silently loses the game.

Three schema rules follow:

Persist the full launch array with its filters intact. Reduction to a single "most likely game binary" is a separate resolution layer built on top, with its own heuristics and failure modes, never a transformation applied during collection.
Name the column launch_executable, never process_name. The two are not semantically equivalent, and process attribution work later will need an observed_game_processes field populated by runtime observation, not by this collector.
Carry a per-title flag marking known launcher-mediated titles so the downstream process-tree layer knows where indirection is expected.

Engine

Not available from any API, and materially harder than the other three fields.

SteamDB derives engine attribution by running regular expressions against depot filename lists. The ruleset is open source and MIT licensed at SteamDatabase/FileDetectionRuleSets, which defines patterns in rules.ini across Engine, Evidence, Container, Emulator, AntiCheat, and SDK sections. Detected values include Engine.Unity, Engine.Unreal, Engine.Godot, Engine.GameMaker, Engine.Source, Engine.XNA, and Engine.FNA, plus AntiCheat.*, SDK.*, Emulator.*, and Container.* categories that come along at no extra cost. Detection operates on filenames and paths only, never on file contents, so a depot file manifest is sufficient and no payload download is required. Their own README is explicit that the output is a set of educated guesses and that an application matching multiple engines simply means files matching all of those signatures were found somewhere in its depots (a bundled level editor built in a different engine is the common cause).

The depot gate

The blocker is data access, not the rules. Reproducing the detection requires depot file manifests, manifest requests require per-depot request codes, filename decryption requires depot keys, and both are issued only to accounts holding a license for the application. This is precisely why SteamDB crowdsources the inputs through its SteamTokenDumper program, which dumps app tokens, package tokens, and depot keys from contributors' licensed accounts. An anonymous account cannot replicate the SteamDB technology table, and a normal account can only run the detection across the titles it owns.

SteamRE/DepotDownloader (GPL-2.0) supports a -manifest-only workflow that retrieves file lists without downloading payloads, and it works for exactly the depots the authenticated account is licensed for. That makes depot-based detection an optional enrichment stage over an owned library, not the catalog mechanism. The GPL licensing also means it is invoked as an external executable, never vendored; if manifest retrieval ever needs to live in-process, it is reimplemented on SteamKit instead.

Scraping steamdb.info directly is not a workaround. It violates their terms of service, the site sits behind Cloudflare, and SteamDB's own FAQ directs third parties to retrieve Steam data from Steam interfaces. SteamDB's roles in this project are exactly three: open-source detection rules, research, and manual validation.

The substitute source

The catalog-scale engine source is PCGamingWiki, which stores structured game metadata in Cargo tables queryable by Steam AppID through the standard MediaWiki API. The Infobox_game table carries both Steam_AppID and Engines as list-type columns, populated from the engines infobox parameter on every game page, and list columns are queried with the HOLDS operator:

https://www.pcgamingwiki.com/w/api.php?action=cargoquery
  &tables=Infobox_game
  &fields=Infobox_game._pageName=Page,Infobox_game.Steam_AppID,Infobox_game.Engines
  &where=Infobox_game.Steam_AppID%20HOLDS%20%221245620%22
  &format=json

It is free, unauthenticated, returns JSON, and is a sanctioned API rather than a scrape. Underscore-prefixed columns such as _pageName must be aliased in the fields parameter. Engine values are wiki page names (Unity, Unreal Engine 4, and so on) and need a normalization map into the same vocabulary the SteamDB rules use. Coverage is strong for titles with a meaningful playerbase and thin in the long tail, which aligns exactly with the review-threshold corpus: the games the corpus keeps are the games the wiki documents. Cloudflare fronts the site, so the collector sends a descriptive User-Agent and throttles politely.

Heuristic cross-check

A secondary signal worth capturing at no extra cost: the launch executable data itself is frequently diagnostic. A GameAssembly.dll sibling implies Unity IL2CPP, a UnityPlayer.dll implies Unity, a *-Win64-Shipping.exe naming pattern implies Unreal, a .pck payload beside the binary implies Godot. This does not replace PCGamingWiki, but it provides a cheap confidence cross-check and partial coverage where the wiki has none.

Every engine value is stored with provenance and confidence:

{
  "engine": "Unreal",
  "engine_source": "pcgamingwiki",
  "engine_confidence": "high"
}

Confidence values are confirmed, high, medium, low, and unknown. Sources are pcgamingwiki, exe_heuristic, and depot_filename_rules. A failed engine lookup never invalidates the rest of a game's record; the row keeps engine = NULL with engine_confidence = 'unknown'.

Recommended Pipeline

One tool, staged enrichment, SQLite as the boundary between every stage so any stage can be re-run independently without re-collecting upstream data. Each stage is idempotent and resumable, because Steam requests fail transiently, SteamSpy rate-limits, PICS lookups need batching, and engine detection is unavailable for some titles even when every other field succeeds.

flowchart TD
    A["Stage 1: catalog enumeration (type = game)"] --> B["Stage 2: popularity enrichment (reviews, SteamSpy)"]
    B --> C["Stage 3: corpus selection (review threshold)"]
    C --> D["Stage 4: PICS product info, full launch array"]
    D --> E["Stage 5: PCGamingWiki Cargo engine backfill"]
    D --> F["Executable-name engine heuristics"]
    D --> G["Stage 6 (optional): depot filename detection, licensed titles only"]
    E --> H[("SQLite catalog")]
    F --> H
    G --> H

Stage 4 is the only stage that requires novel code. Everything else is consuming a documented endpoint, a maintained mirror, or an existing MIT ruleset. Stage 6 runs only against titles the collecting account is licensed for and exists to validate and upgrade the confidence of stages 5 and the heuristics, not to provide primary coverage.

Validation (a permanent stage, not a one-off) spot-checks one representative title per category: Unity, Unreal, Source, Godot, custom engine, launcher-mediated, multi-executable, Windows plus Linux, and VR.

On language: the collector is I/O-bound end to end, so runtime performance is not the deciding factor. Python is the implementation language for the whole pipeline. FronkonGames/Steam-Games-Scraper (the stage 1 and 2 base) is Python, and ValvePython/steam keeps stage 4 in the same runtime instead of bolting a Node worker onto a Python orchestrator. If that library's maintenance state ever blocks an upgrade, the stage 4 boundary is a batch of appids in and normalized JSON out, so swapping in a node-steam-user worker is a contained change.

Database Schema

The schema preserves Steam's multi-entry launch configuration instead of flattening it, and keeps technology attribution in a separate table so one title can carry an engine, an anti-cheat, and several SDKs at once.

CREATE TABLE games (
    appid INTEGER PRIMARY KEY,
    name TEXT NOT NULL,

    estimated_owners_min INTEGER,
    estimated_owners_max INTEGER,
    positive_reviews INTEGER,
    negative_reviews INTEGER,
    peak_ccu INTEGER,

    launcher_mediated INTEGER NOT NULL DEFAULT 0,
    token_required INTEGER NOT NULL DEFAULT 0,

    engine TEXT,
    engine_source TEXT,
    engine_confidence TEXT
);

CREATE TABLE launch_entries (
    appid INTEGER NOT NULL,
    launch_index INTEGER NOT NULL,

    os TEXT,
    osarch TEXT,
    launch_type TEXT,
    beta_branch TEXT,

    executable TEXT NOT NULL,
    arguments TEXT,
    description TEXT,

    PRIMARY KEY (appid, launch_index),
    FOREIGN KEY (appid) REFERENCES games(appid)
);

CREATE TABLE technologies (
    appid INTEGER NOT NULL,
    category TEXT NOT NULL,
    technology TEXT NOT NULL,
    evidence TEXT,

    FOREIGN KEY (appid) REFERENCES games(appid)
);

Technology categories are engine, anti_cheat, sdk, framework, emulator, container, and runtime, mirroring the sections of the SteamDB ruleset.

Repository Index

Repository	Purpose	Role
`dgibbs64/SteamCMD-AppID-List`	Daily AppID and name dump (JSON, CSV, XML, MD)	Consume directly
`jsnli/steamappidlist`	Daily AppID lists split by category	Consume directly
`FronkonGames/Steam-Games-Scraper`	Store metadata and SteamSpy to JSON, MIT	Stage 1-2 base
`vintagedon/steam-dataset-2025`	Modernized Kaggle-style Steam dataset	Reference
`ValvePython/steam`	Steam client protocol, Python, anonymous PICS	Stage 4 implementation
`ValvePython/steamctl`	CLI over the above, `apps product_info`	Exploration
`DoctorMcKay/node-steam-user`	Steam client protocol, Node, anonymous PICS	Stage 4 fallback
`DoctorMcKay/steam-pics-api`	HTTP wrapper around anonymous PICS	Optional boundary
`SteamRE/SteamKit`	Steam client protocol, C#, reference impl	Reference
`SteamRE/DepotDownloader`	Depot manifests, `-manifest-only`, GPL-2.0	Stage 6, external exe
`ValveResourceFormat/SteamAppInfo`	Local `appinfo.vdf` parser and format docs	Format reference
`sonic2kk/steamtinkerlaunch`	Production local appinfo extraction (Linux)	Prior art
`SteamDatabase/FileDetectionRuleSets`	Engine detection regex ruleset, MIT	Stage 6 rules, vendorable
`SteamDatabase/SteamTokenDumper`	Crowdsourced PICS token collection	Explains the depot gate
PCGamingWiki Cargo API	Engine metadata keyed on Steam AppID	Stage 5 source

Expected Coverage

Field	Expected coverage	Notes
AppID	~100%	Catalog endpoints and mirrors
Name	~100%	Same sources
Popularity	High	Reviews primary, SteamSpy secondary
Launch executable	High	Anonymous PICS; token-gated subset flagged
Launch arguments	High	Wherever launch metadata exists
Engine	Partial, best effort	PCGamingWiki head-heavy, heuristics fill
Anti-cheat, SDK	Partial, best effort	Stage 6 only, licensed titles

Open Questions

These need answers before implementation starts.

What review-count threshold defines the corpus? The recommendation is a fixed threshold over type == "game"; the specific number (500 is the working figure) should be picked after stage 2 data exists and the cumulative-coverage curve can actually be plotted.
Does the downstream consumer need the full launch configuration array, or a single resolved "most likely game binary" per title? The former is what the source provides; the latter requires a resolution layer with its own heuristics and failure modes.
Is launcher indirection in scope for this tool, or is it handled downstream by the process-tree detection layer? Keeping it downstream is cleaner; the launcher_mediated flag exists either way.
What refresh cadence does the catalog need? App lists change daily, launch configs change per game update, and engine attribution is effectively static. Those three cadences should not share a schedule.

h8rt3rmin8r/Steam-Catalog-Research.md

Select an option

No results found