Skip to content

Instantly share code, notes, and snippets.

@unnitallman
Created May 6, 2026 07:34
Show Gist options
  • Select an option

  • Save unnitallman/cd5ee9958c213809b8adfab8fbcda151 to your computer and use it in GitHub Desktop.

Select an option

Save unnitallman/cd5ee9958c213809b8adfab8fbcda151 to your computer and use it in GitHub Desktop.
NeetoCI build speed: improvement opportunities

NeetoCI build speed: improvement opportunities

Reference build: neeto-auth-web, wall-clock 274s. Tests take 74s. The other 200s is infrastructure overhead.

Timing breakdown

Step Time Category
Cache restore 53.5s S3 download
bundle exec rake setup 45.1s Unknown — needs investigation
Postgres startup 40.6s Image pull + init + pgvector apt-get
bundle exec rake db:create db:schema:load 22.6s DB setup
bundle install 18.3s Gem install
yarn install 17.4s JS install
neetoci-version (Ruby + Node) 8.2s Runtime install
Redis startup 3.3s Image pull + init

Improvements

1. Pre-warm yarn global cache in CI Docker image (saves ~15–20s)

yarn install hits the network on every build. Yarn's global cache (~/.yarn/cache) can be pre-populated in the Docker image with the ~130 packages shared across all neeto apps (neeto-commons-frontend, React, Rails UJS, etc.). Cache hits skip network entirely.

How: Add a yarn cache import step to the Dockerfile using a lockfile snapshot from neeto-auth-web.
Savings: Reduces yarn install from ~17s toward ~2–5s.
Risk: Cache in image becomes stale when package versions change. Acceptable — it falls back to network, no correctness issue.


2. Bake Ruby + Node into CI image (saves ~5–6s)

Currently neetoci-version ruby 4.0.1 and neetoci-version node 22.13 download tarballs from http://10.100.0.30:80/binaries on every build. The CI image ships rbenv/nvm but no pre-installed versions.

Almost every Neeto product uses Ruby 4.0.1 + Node 22.13. Pre-install these in the Dockerfile.

How: Add RUN neetoci-install ruby 4.0.1 && neetoci-install node 22.13 to the CI Dockerfile.
Savings: ~5–6s per build, plus eliminates a dependency on the internal binary server.


3. Bake pgvector into the postgres:18 image (saves ~4s)

service::start_postgres runs apt-get update && apt-get install postgresql-18-pgvector inside the running container on every build. No check — always runs.

How: Create docker-postgres/Dockerfile extending postgres:18 with pgvector pre-installed, push to 10.100.0.20:5000/postgres:18, drop the apt-get line from neetoci-service.
Savings: ~4s per build, removes a runtime apt-get and its failure surface.


4. Parallelize postgres + redis startup (saves ~3s)

The default CI config starts postgres (40.6s) then redis (3.3s) sequentially. Some playwright configs already use the correct pattern.

How: Change all default.yml files from:

- neetoci-service start postgres 18
- neetoci-service start redis 7.0.5

to:

- neetoci-service start postgres 18 & neetoci-service start redis 7.0.5 & wait

Savings: ~3s per build.
Note: Background & does not survive across command boundaries in the NeetoCI executor (each command is a separate subshell). The & wait must be on a single line.


5. Investigate bundle exec rake setup (potential 30–45s savings)

rake setup takes 45.1s and runs after tests. It's unclear what it does — likely seeds sample data. This is the second largest cost after cache restore.

Action needed: Profile what rake setup does. If it's seeding data not used by CI, remove it. If it must run, parallelize with simplecov_coverage:publish.


6. K8s addon for postgres (saves ~36s — architectural change)

The core problem: the CI pod is ephemeral, so podman image cache is empty on every run. The postgres image is pulled fresh each build.

The K8s addon system (postgres-addon.yml, imagePullPolicy: IfNotPresent) already exists for shared-redis. Nodes cache the image after first use — subsequent builds skip the pull entirely.

How:

  • Add pgvector to postgres addon image
  • Support addon: postgres 18 in neetoci YAML config (parsed server-side, provisioned before commands start)
  • Update database.yml.ci in all apps to connect via service name (e.g., postgres-service-<container_id>) instead of localhost

Savings: ~36s per build once image is cached on node.
Effort: ~1 sprint. Blocks parallelization of postgres startup with cache restore.


Priority order

  1. #3 + #4: pgvector bake-in + parallelize (combined ~7s, low effort, do now)
  2. #1: yarn cache pre-warm (~15–20s, medium effort)
  3. #2: Ruby/Node bake-in (~5–6s, low effort)
  4. #5: investigate rake setup (unknown savings, low-hanging if it can be dropped)
  5. #6: K8s addon for postgres (~36s, high effort, needs platform changes)

Combined quick wins (#1–4): ~30s savings, 274s → ~244s.
With K8s addon (#6): further ~36s, → ~208s.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment