Prompt: configure a secure self-hosted VPS browser with link-based control

Configure a browser running locally on this VPS that the user can control through a password-protected web link.

Do not use paid browser/cloud services such as Browserbase, Browser Use, or Firecrawl. Do the setup end-to-end: install, configure, start, verify, save non-secret memory, then send the final link and credentials.

Architecture

User browser
  -> free HTTPS tunnel
  -> nginx on 127.0.0.1:<NGINX_PORT> with Basic Auth + secret path
  -> noVNC/websockify on 127.0.0.1:<NOVNC_PORT>
  -> x11vnc on 127.0.0.1:<VNC_PORT>
  -> Xvfb display :<DISPLAY_NUM>
  -> local Chrome/Chromium on the VPS

Agent automation upgrade: shared noVNC + CDP control

For the best UX, run one Chrome profile that both the user and the agent can control:

The user connects through noVNC to handle login, CAPTCHA, 2FA, and visual checks.
The agent connects to the same Chrome window through Chrome DevTools Protocol (CDP).

This avoids the bad split-brain setup where the noVNC browser is logged in but the agent's built-in browser is a separate, logged-out headless session.

Chrome launch flags

When starting Chrome in $BASE/start-secure-browser.sh, include a localhost-only CDP port:

"$BROWSER" --display=":$DISPLAY_NUM" --user-data-dir="$BASE/chrome-profile" \
  --remote-debugging-address=127.0.0.1 \
  --remote-debugging-port=9222 \
  --no-first-run --no-default-browser-check --disable-dev-shm-usage --disable-gpu \
  --window-size=1280,800 --new-window https://example.com >"$LOG/chrome.log" 2>&1 &

Security rule: never expose port 9222 publicly and never tunnel it. CDP must stay bound to 127.0.0.1.

Configure Hermes to use the same browser

If the agent is Hermes, point browser tools at that CDP endpoint:

hermes config set browser.cdp_url http://127.0.0.1:9222
hermes config set browser.dialog_policy pause
hermes config set browser.dialog_timeout_s 600

Then restart the gateway or start a fresh session if config/tool discovery is cached:

/restart
# or /reset for a fresh chat context

Alternative for a live CLI/TUI session that supports it:

/browser connect http://127.0.0.1:9222

Verify shared control

Check CDP is reachable locally:

curl -s http://127.0.0.1:9222/json/version

Expected: JSON with webSocketDebuggerUrl.

Ask the browser tool to navigate to https://example.com.
Confirm the noVNC window visibly changes to Example Domain.
Manually navigate in noVNC, then use the browser tool snapshot to confirm the agent sees the same page.

Operating pattern

Agent automates normal browsing through CDP.
If login/CAPTCHA/2FA appears, agent pauses and asks the user to complete it through noVNC.
User completes the manual step in the same Chrome window.
Agent resumes through CDP with the same cookies/session.

Hard requirements

Bind nginx, noVNC, and VNC to 127.0.0.1 only.
Never expose raw VNC publicly.
Use HTTP Basic Auth in nginx.
Use a random secret URL path token.
Use a separate VNC password too.
Use a free tunnel, preferably cloudflared, to expose only nginx.
Verify 404 for wrong paths, 401 without auth, 200 with auth.
Store secrets with chmod 600; directory with chmod 700.
Do not save passwords, tokens, tunnel URLs, or cookies to memory.

1. Inspect system and install tools

uname -a
id
command -v sudo || true
for c in google-chrome chromium chromium-browser Xvfb x11vnc websockify nginx htpasswd cloudflared fluxbox curl openssl ss; do
  printf '%-18s ' "$c"; command -v "$c" || true
done

On Ubuntu/Debian:

sudo apt-get update
sudo apt-get install -y xvfb x11vnc novnc websockify nginx apache2-utils fluxbox curl openssl iproute2

Ensure one browser exists: google-chrome, chromium, or chromium-browser. Install Chromium/Chrome if none exists.

If cloudflared is missing, install the free Cloudflare Tunnel binary/package. If impossible, use another free HTTPS tunnel that forwards to localhost.

2. Pick explicit free ports and display

Do not hardcode ports if they are busy. Choose free values.

Use this pattern:

pick_port() {
  python3 - <<'PY'
import socket
s=socket.socket()
s.bind(('127.0.0.1', 0))
print(s.getsockname()[1])
s.close()
PY
}

NGINX_PORT="${NGINX_PORT:-$(pick_port)}"
NOVNC_PORT="${NOVNC_PORT:-$(pick_port)}"
VNC_PORT="${VNC_PORT:-$(pick_port)}"

pick_display() {
  for n in $(seq 70 99); do
    [ ! -e "/tmp/.X11-unix/X$n" ] && echo "$n" && return 0
  done
  return 1
}
DISPLAY_NUM="${DISPLAY_NUM:-$(pick_display)}"

Validate:

ss -ltn | grep -E ":($NGINX_PORT|$NOVNC_PORT|$VNC_PORT)\b" && exit 1 || true

3. Create private working directory and secrets

BASE="$HOME/secure-browser"
mkdir -p "$BASE/logs" "$BASE/chrome-profile" "$BASE/vnc"
chmod 700 "$BASE" "$BASE/vnc"

BASIC_USER="browseruser"
BASIC_PASS="$(openssl rand -base64 32 | tr -d '\n' | tr -dc 'A-Za-z0-9_@%+=:,.~-' | cut -c1-28)"
VNC_PASS="$(openssl rand -base64 24 | tr -d '\n' | tr -dc 'A-Za-z0-9' | cut -c1-16)"
PATH_TOKEN="$(openssl rand -hex 16)"

printf '%s\n' "$BASIC_USER" > "$BASE/basic-user.txt"
printf '%s\n' "$BASIC_PASS" > "$BASE/basic-password.txt"
printf '%s\n' "$VNC_PASS" > "$BASE/vnc-password.txt"
printf '%s\n' "$PATH_TOKEN" > "$BASE/path-token.txt"
printf '%s\n' "$NGINX_PORT" > "$BASE/nginx-port.txt"
printf '%s\n' "$NOVNC_PORT" > "$BASE/novnc-port.txt"
printf '%s\n' "$VNC_PORT" > "$BASE/vnc-port.txt"
printf '%s\n' "$DISPLAY_NUM" > "$BASE/display-num.txt"
chmod 600 "$BASE"/*.txt

htpasswd -bc "$BASE/.htpasswd" "$BASIC_USER" "$BASIC_PASS"
x11vnc -storepasswd "$VNC_PASS" "$BASE/vnc/passwd"
chmod 600 "$BASE/.htpasswd" "$BASE/vnc/passwd"

Password rules:

Generate passwords with openssl rand or a cryptographically secure equivalent.
Do not use human words, dates, usernames, or reused passwords.
Do not print passwords except in the final answer to the user.
Send credentials only in the final private chat response, not in public logs or comments.

Make nginx able to read the htpasswd file:

sudo cp "$BASE/.htpasswd" /etc/nginx/secure-browser.htpasswd
sudo chmod 640 /etc/nginx/secure-browser.htpasswd
sudo chown root:www-data /etc/nginx/secure-browser.htpasswd

4. Create start script

Create $BASE/start-secure-browser.sh:

#!/usr/bin/env bash
set -euo pipefail
BASE="$HOME/secure-browser"
LOG="$BASE/logs"
NGINX_PORT="$(cat "$BASE/nginx-port.txt")"
NOVNC_PORT="$(cat "$BASE/novnc-port.txt")"
VNC_PORT="$(cat "$BASE/vnc-port.txt")"
DISPLAY_NUM="$(cat "$BASE/display-num.txt")"
export DISPLAY=":$DISPLAY_NUM"
mkdir -p "$LOG" "$BASE/chrome-profile" "$BASE/vnc"

pkill -f "Xvfb :$DISPLAY_NUM" 2>/dev/null || true
pkill -f "x11vnc .*:$DISPLAY_NUM" 2>/dev/null || true
pkill -f "websockify .*127.0.0.1:$NOVNC_PORT" 2>/dev/null || true
pkill -f "fluxbox" 2>/dev/null || true
pkill -f "google-chrome.*secure-browser/chrome-profile" 2>/dev/null || true
pkill -f "chromium.*secure-browser/chrome-profile" 2>/dev/null || true
sleep 1

Xvfb ":$DISPLAY_NUM" -screen 0 1280x800x24 -nolisten tcp >"$LOG/xvfb.log" 2>&1 &
sleep 1
fluxbox >"$LOG/fluxbox.log" 2>&1 &

x11vnc -display ":$DISPLAY_NUM" -localhost -rfbport "$VNC_PORT" -forever -shared \
  -rfbauth "$BASE/vnc/passwd" -noxdamage -quiet >"$LOG/x11vnc.log" 2>&1 &

websockify --web /usr/share/novnc "127.0.0.1:$NOVNC_PORT" "127.0.0.1:$VNC_PORT" \
  >"$LOG/websockify.log" 2>&1 &

sleep 2
BROWSER=""
for c in google-chrome chromium chromium-browser; do
  if command -v "$c" >/dev/null 2>&1; then BROWSER="$c"; break; fi
done
[ -n "$BROWSER" ] || { echo "No Chrome/Chromium browser found" >&2; exit 1; }

"$BROWSER" --display=":$DISPLAY_NUM" --user-data-dir="$BASE/chrome-profile" \
  --remote-debugging-address=127.0.0.1 \
  --remote-debugging-port=9222 \
  --no-first-run --no-default-browser-check --disable-dev-shm-usage --disable-gpu \
  --window-size=1280,800 --new-window https://example.com >"$LOG/chrome.log" 2>&1 &

sleep 3
ss -ltnp | grep -E ":($VNC_PORT|$NOVNC_PORT|$NGINX_PORT)\b" || true

Then:

chmod 700 "$BASE/start-secure-browser.sh"

5. Create stop script

Create $BASE/stop-secure-browser.sh:

#!/usr/bin/env bash
set -euo pipefail
BASE="$HOME/secure-browser"
NGINX_PORT="$(cat "$BASE/nginx-port.txt" 2>/dev/null || echo 18080)"
NOVNC_PORT="$(cat "$BASE/novnc-port.txt" 2>/dev/null || true)"
DISPLAY_NUM="$(cat "$BASE/display-num.txt" 2>/dev/null || true)"
pkill -f "cloudflared tunnel --url http://127.0.0.1:$NGINX_PORT" 2>/dev/null || true
[ -n "$DISPLAY_NUM" ] && pkill -f "Xvfb :$DISPLAY_NUM" 2>/dev/null || true
[ -n "$DISPLAY_NUM" ] && pkill -f "x11vnc .*:$DISPLAY_NUM" 2>/dev/null || true
[ -n "$NOVNC_PORT" ] && pkill -f "websockify .*127.0.0.1:$NOVNC_PORT" 2>/dev/null || true
pkill -f "fluxbox" 2>/dev/null || true
pkill -f "google-chrome.*secure-browser/chrome-profile" 2>/dev/null || true
pkill -f "chromium.*secure-browser/chrome-profile" 2>/dev/null || true
echo "secure browser stopped"

Then:

chmod 700 "$BASE/stop-secure-browser.sh"

6. Configure nginx

PATH_TOKEN="$(cat "$BASE/path-token.txt")"
NGINX_PORT="$(cat "$BASE/nginx-port.txt")"
NOVNC_PORT="$(cat "$BASE/novnc-port.txt")"

cat > "$BASE/nginx-secure-browser.conf" <<EOF
server {
    listen 127.0.0.1:$NGINX_PORT;
    server_name _;

    auth_basic "Secure VPS Browser";
    auth_basic_user_file /etc/nginx/secure-browser.htpasswd;

    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "no-referrer" always;
    add_header Cache-Control "no-store" always;

    location = / { return 404; }

    location /$PATH_TOKEN/ {
        rewrite ^/$PATH_TOKEN/(.*) /\$1 break;
        proxy_pass http://127.0.0.1:$NOVNC_PORT;
        proxy_http_version 1.1;
        proxy_set_header Upgrade \$http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host \$host;
        proxy_set_header X-Real-IP \$remote_addr;
        proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto \$scheme;
        proxy_read_timeout 86400;
    }

    location / { return 404; }
}
EOF

sudo cp "$BASE/nginx-secure-browser.conf" /etc/nginx/sites-available/secure-browser
sudo ln -sf /etc/nginx/sites-available/secure-browser /etc/nginx/sites-enabled/secure-browser
sudo nginx -t
sudo systemctl reload nginx

7. Start stack and tunnel

"$BASE/start-secure-browser.sh"

Check bindings:

ss -ltnp | grep -E ":($(cat "$BASE/vnc-port.txt")|$(cat "$BASE/novnc-port.txt")|$(cat "$BASE/nginx-port.txt"))\b"

Start tunnel in background if possible:

NGINX_PORT="$(cat "$BASE/nginx-port.txt")"
cloudflared tunnel --url "http://127.0.0.1:$NGINX_PORT" --loglevel info 2>&1 | tee "$BASE/cloudflared.log"

Extract public URL:

TUNNEL_URL="$(grep -o 'https://[-a-z0-9]*\.trycloudflare\.com' "$BASE/cloudflared.log" | tail -1)"
PATH_TOKEN="$(cat "$BASE/path-token.txt")"
CONTROL_URL="$TUNNEL_URL/$PATH_TOKEN/vnc.html?autoconnect=true&resize=scale&path=$PATH_TOKEN/websockify"
printf '%s\n' "$CONTROL_URL" > "$BASE/control-url-latest.txt"
chmod 600 "$BASE/control-url-latest.txt"

8. Verify before replying

BASIC_USER="$(cat "$BASE/basic-user.txt")"
BASIC_PASS="$(cat "$BASE/basic-password.txt")"
CONTROL_URL="$(cat "$BASE/control-url-latest.txt")"
TUNNEL_URL="$(grep -o 'https://[-a-z0-9]*\.trycloudflare\.com' "$BASE/cloudflared.log" | tail -1)"

curl -s -o /dev/null -w '%{http_code}\n' "$TUNNEL_URL/"
# must be 404

curl -s -o /dev/null -w '%{http_code}\n' "$CONTROL_URL"
# must be 401

curl -s -u "$BASIC_USER:$BASIC_PASS" -o /tmp/vnc.html -w '%{http_code}\n' "$CONTROL_URL"
# must be 200

grep -qi '<title>noVNC' /tmp/vnc.html

If nginx returns 500, check:

sudo tail -50 /var/log/nginx/error.log

Usually fix:

sudo cp "$BASE/.htpasswd" /etc/nginx/secure-browser.htpasswd
sudo chmod 640 /etc/nginx/secure-browser.htpasswd
sudo chown root:www-data /etc/nginx/secure-browser.htpasswd
sudo nginx -t && sudo systemctl reload nginx

9. How to open a page in the VPS browser

BASE="$HOME/secure-browser"
DISPLAY_NUM="$(cat "$BASE/display-num.txt")"
export DISPLAY=":$DISPLAY_NUM"
BROWSER="$(command -v google-chrome || command -v chromium || command -v chromium-browser)"
"$BROWSER" --display=":$DISPLAY_NUM" --user-data-dir="$BASE/chrome-profile" \
  --new-window https://example.com >/tmp/open-url.log 2>&1 || true

10. Save persistent memory

If memory is available, save exactly this kind of non-secret operational knowledge after success:

Secure self-hosted VPS browser setup: use $HOME/secure-browser. Start with $HOME/secure-browser/start-secure-browser.sh; stop with $HOME/secure-browser/stop-secure-browser.sh. Port/display values are stored in nginx-port.txt, novnc-port.txt, vnc-port.txt, display-num.txt. Latest generated link format is: <cloudflared-url>/<path-token>/vnc.html?autoconnect=true&resize=scale&path=<path-token>/websockify. To tell the user how to use it: send the control URL, Basic Auth username/password, and VNC password if prompted; warn that cloudflared quick tunnel URLs are ephemeral and change on restart. To open a URL in the VPS browser, run Chrome/Chromium with DISPLAY set from display-num.txt and user-data-dir=$HOME/secure-browser/chrome-profile. Never save Basic Auth password, VNC password, path token, tunnel URL, cookies, or session data to memory.

Do not save secret values.

11. Final response format

After verification, respond:

Done. I set up a secure self-hosted browser on this VPS.

Control URL:
<CONTROL_URL>

HTTP Basic Auth:
username: <BASIC_USER>
password: <BASIC_PASS>

VNC password, if prompted:
<VNC_PASS>

Stop:
$HOME/secure-browser/stop-secure-browser.sh

Restart local stack:
$HOME/secure-browser/start-secure-browser.sh

Restart tunnel:
cloudflared tunnel --url http://127.0.0.1:<NGINX_PORT> --loglevel info 2>&1 | tee $HOME/secure-browser/cloudflared.log

Security:
- Browser runs locally on the VPS.
- VNC/noVNC/nginx listen only on localhost.
- Public access goes through a free HTTPS tunnel to nginx.
- nginx requires Basic Auth and a secret URL path.
- Raw VNC is not public.

Note: the free quick tunnel URL is ephemeral and changes when restarted.

freQuensy23-coder/secure-vps-browser-agent-prompt.md

Select an option

No results found