OpenClaw Voice Call with Twilio + OpenAI Realtime Voice

OPENCLAW_VOICE_CALL_SETUP.md

OpenClaw Voice Calls with Twilio and OpenAI Realtime

This is a practical setup guide for getting inbound phone calls working with the built-in OpenClaw voice-call plugin, Twilio, ngrok, and OpenAI realtime voice.

The working flow is:

Caller
  -> Twilio phone number
  -> OpenClaw voice-call webhook
  -> OpenClaw voice-call plugin
  -> OpenAI realtime voice
  -> Twilio media stream
  -> Caller hears the agent

This guide is for the OpenClaw-native voice-call path. It is not the same thing as putting an ElevenLabs Agent in front of OpenClaw.

What You Need

• OpenClaw installed and running.

• A Twilio account.

• A Twilio phone number with voice capability.

• An OpenAI API key with realtime model access.

• A public HTTPS tunnel to your local machine, such as ngrok.

  Architecture

sequenceDiagram
  participant Caller
  participant Twilio as Twilio Phone Number
  participant Ngrok as ngrok publicUrl<br/>https://YOUR-NGROK-DOMAIN.ngrok-free.dev
  participant OC as OpenClaw voice-call plugin<br/>localhost:3334
  participant OpenAI as OpenAI Realtime

  Caller->>Twilio: Places phone call
  Twilio->>Ngrok: POST /voice/webhook
  Note over Ngrok: publicUrl + path
  Ngrok->>OC: POST /voice/webhook
  OC-->>Twilio: TwiML with WebSocket URL

  Twilio->>Ngrok: WSS /voice/stream/realtime/{callId}
  Note over Ngrok: publicUrl + streamPath
  Ngrok->>OC: WebSocket /voice/stream/realtime/{callId}

  OC->>OpenAI: Realtime audio session
  OpenAI-->>OC: Realtime audio response
  OC-->>Twilio: Audio over media stream
  Twilio-->>Caller: Agent voice

Loading

Config Mapping

JSON value	Meaning	Used by
webhook.publicUrl	Public HTTPS base URL for your local voice-call server	Twilio reaches this through ngrok
webhook.path	HTTP webhook path for inbound call setup	Twilio sends POST <publicUrl><path>
realtime.streamPath	WebSocket path for live call audio	OpenClaw returns wss://...<streamPath>/<callId> in TwiML
webhook.port	Local port where OpenClaw voice-call listens	ngrok forwards public traffic to this port

OpenClaw Config Shape

In openclaw.json, configure the voice-call plugin under:

{
  "plugins": {
    "entries": {
      "voice-call": {
        "enabled": true,
        "config": {}
      }
    }
  }
}

A working realtime-style configuration looks like this:

{
  "plugins": {
    "entries": {
      "voice-call": {
        "enabled": true,
        "config": {
          "provider": "twilio",
          "webhook": {
            "port": 3334,
            "path": "/voice/webhook",
            "publicUrl": "https://YOUR-NGROK-DOMAIN.ngrok-free.dev"
          },
          "twilio": {
            "accountSid": "YOUR_TWILIO_ACCOUNT_SID",
            "authToken": "YOUR_TWILIO_AUTH_TOKEN",
            "phoneNumber": "+15555550123"
          },
          "inbound": {
            "enabled": true,
            "mode": "allowlist",
            "allowlist": [
              "+15555550199"
            ]
          },
          "streaming": {
            "enabled": false
          },
          "realtime": {
            "enabled": true,
            "provider": "openai",
            "streamPath": "/voice/stream/realtime",
            "providers": {
              "openai": {
                "apiKey": "YOUR_OPENAI_API_KEY",
                "model": "gpt-realtime-1.5",
                "voice": "alloy",
                "silenceDurationMs": 500,
                "vadThreshold": 0.5
              }
            }
          },
          "responseModel": "anthropic/claude-sonnet-4-6"
          }
        }
      }
    }
  }

Notes:

• publicUrl should be the base public URL, not the full webhook path.
• Twilio should point to <publicUrl>/voice/webhook.
• The voice-call webhook server is separate from the main OpenClaw gateway dashboard.
• The example uses port 3334 for the voice-call webhook server.
• Prefer silenceDurationMs; do not use silenceDurationSeconds.
• Keep secrets out of git, gists, screenshots, and logs.

Start ngrok

Forward ngrok to the voice-call plugin port:

ngrok http 3334

Copy the HTTPS forwarding URL, for example:

https://YOUR-NGROK-DOMAIN.ngrok-free.dev

Put that base URL in webhook.publicUrl.

Configure Twilio

In the Twilio Console, open your phone number's voice settings.

Set the inbound voice webhook to:

https://YOUR-NGROK-DOMAIN.ngrok-free.dev/voice/webhook

Use:

HTTP POST

Validate and Restart OpenClaw

Validate the config:

openclaw config validate

Restart the daemon:

openclaw daemon restart

Then confirm the voice-call server is listening:

lsof -nP -iTCP -sTCP:LISTEN | rg ':(3334|18789|4040)\b|ngrok|node'

You want to see:

• OpenClaw gateway on 18789.
• OpenClaw voice-call plugin on 3334.
• ngrok inspector on 4040, if ngrok is running locally.

What Good Logs Look Like

Tail the logs:

tail -F ~/.openclaw/logs/gateway.log ~/.openclaw/logs/gateway.err.log

On startup, look for lines like:

[voice-call] Webhook server listening on http://127.0.0.1:3334/voice/webhook
[plugins] [voice-call] Realtime voice provider: openai
[plugins] [voice-call] Runtime initialized

On an inbound call, look for:

[voice-call] Inbound call accepted
[voice-call] Created inbound call record

ngrok's inspector should also show:

• A POST /voice/webhook returning TwiML.
• A WebSocket upgrade to /voice/stream/realtime/... returning HTTP 101 Switching Protocols.

You can inspect ngrok requests at:

http://127.0.0.1:4040

or via API:

curl -sS http://127.0.0.1:4040/api/requests/http

Debug Checklist

If calls do not work, check these in order:

1. Is OpenClaw running?
2. Is the voice-call plugin enabled?
3. Is the voice-call plugin listening on the port you are forwarding with ngrok?
4. Is ngrok forwarding to the voice-call port, not the main OpenClaw gateway port?
5. Does webhook.publicUrl exactly match the current ngrok HTTPS base URL?
6. Does Twilio point to <publicUrl>/voice/webhook with method POST?
7. Is the caller's phone number allowed by your inbound allowlist?
8. Does ngrok show Twilio hitting /voice/webhook?
9. Does ngrok show a WebSocket upgrade to /voice/stream/realtime/...?
10. Do OpenClaw logs show the realtime provider initialized?
11. Does your OpenAI API key have access to the configured realtime model?

Common Failure Modes

Twilio Never Hits OpenClaw

Check:

• Twilio webhook URL.
• ngrok is still running.
• ngrok URL has not changed.
• Twilio is using POST.

OpenClaw Rejects the Call

Check:

• inbound allowlist mode.
• caller number format, usually E.164 like +15555550199.
• Twilio signature validation and publicUrl.

Webhook Works but No Audio

Check ngrok for the WebSocket upgrade. The webhook can succeed while the media stream fails.

For realtime mode, the stream URL should look like:

wss://YOUR-NGROK-DOMAIN.ngrok-free.dev/voice/stream/realtime/...

Split STT/TTS Times Out

If using the older split streaming setup, you may see errors like:

Telephony TTS synthesis timed out
OpenAI realtime transcription connection timeout

In that case, try the simpler realtime provider path:

{
  "streaming": {
    "enabled": false
  },
  "realtime": {
    "enabled": true,
    "provider": "openai"
  }
}

In testing, the OpenAI realtime path was lower friction and gave a fast, natural response.

Known Working Baseline

The baseline that worked:

• Twilio inbound voice webhook pointed at /voice/webhook.
• ngrok forwarded to local port 3334.
• OpenClaw voice-call plugin used realtime mode.
• streaming.enabled was false.
• realtime.enabled was true.
• realtime.provider was openai.
• realtime.streamPath was /voice/stream/realtime.
• OpenAI realtime model was gpt-realtime-1.5.

Security Notes

• Do not paste real API keys into public examples.
• Do not publish Twilio auth tokens.
• Do not publish your personal phone number or caller allowlist.
• For a stable setup, use a fixed public domain or a reserved ngrok domain. Free ngrok URLs may change.
• If your public URL changes, update both OpenClaw webhook.publicUrl and the Twilio webhook URL.

Reboot and ngrok Notes

If you started ngrok manually, it probably will not keep running after your computer restarts.

After a restart, check:

lsof -nP -iTCP -sTCP:LISTEN | rg ':(3334|4040)\b|ngrok|node'

If ngrok is not running, start it again:

ngrok http 3334

If the ngrok public URL changed, update both places:

openclaw.json: webhook.publicUrl
Twilio Console: Voice webhook URL

For a durable setup, use a reserved/static ngrok domain and run ngrok from your system's startup manager, such as a macOS LaunchAgent.

Great write up.
Trying to get STT/TTS working, and have no idea what to use.

What is the path of least resistance?
What is the path of least cost?

Can you say a few words about how to do this?

• Get A Twilio account.
• Get A Twilio phone number with voice capability.

Also, why is this needed:

• A public HTTPS tunnel to your local machine, such as ngrok.