RHAII Early Access: Poolside Laguna S 2.1

A guide for serving Poolside's Laguna S 2.1 model using the Red Hat AI Inference Server with vLLM.

About Laguna S 2.1

Laguna S 2.1 is a 118B parameter Mixture-of-Experts agentic coding model with 8B parameters active per forward pass. It uses 256 experts with top-10 routing across 48 layers (36 sliding-window + 12 global attention in a 3:1 ratio) and supports up to 256K context length.

The FP8 quantized version weighs ~121 GB on disk and fits comfortably on 4x H100 80GB GPUs.

Model card: poolside/Laguna-S-2.1-FP8 on Hugging Face
vLLM recipe: Laguna S 2.1
Container: quay.io/vllm/rhaiis-early-access:laguna-s-2.1

Build Information

Field	Value
Model	`poolside/Laguna-S-2.1-FP8`
Container Image (early access)	`quay.io/vllm/rhaiis-early-access:laguna-s-2.1`
Container Image (run ID)	`quay.io/vllm/automation-vllm:cuda-29766140891`
Container Image (commit)	`quay.io/vllm/automation-vllm:cuda-8fab4cb8d786b874b26014ae15d6e2b2eedb5ace`
nm-vllm-ent branch	`sync-v0.25.1`
nm-cicd branch	`sync-v0.25.1`
GH Actions Run ID	`29766140891`
Target Device	CUDA
Python Version	3.12

Prerequisites

4x NVIDIA H100 80GB GPUs (or 2x H200 141GB, or 2x B200)
~121 GB disk for FP8 weights
Podman with NVIDIA CDI support
Hugging Face account with access to poolside/Laguna-S-2.1-FP8

Download the Model

HF_HOME=/path/to/huggingface \
  huggingface-cli download poolside/Laguna-S-2.1-FP8

Pull the Image

podman pull quay.io/vllm/rhaiis-early-access:laguna-s-2.1

Start the Server

podman run -d \
  --name vllm-laguna-s21 \
  --device nvidia.com/gpu=0 \
  --device nvidia.com/gpu=1 \
  --device nvidia.com/gpu=2 \
  --device nvidia.com/gpu=3 \
  --security-opt=label=disable \
  --shm-size=10g \
  -p 8000:8000 \
  -v /path/to/huggingface:/hf:Z \
  -e HF_HUB_OFFLINE=1 \
  -e HF_HOME=/hf \
  -e FLASHINFER_DISABLE_VERSION_CHECK=1 \
  -e VLLM_BLOCKSCALE_FP8_GEMM_FLASHINFER=0 \
  -e CUDA_VISIBLE_DEVICES=0,1,2,3 \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  quay.io/vllm/rhaiis-early-access:laguna-s-2.1 \
    --model poolside/Laguna-S-2.1-FP8 \
    --tensor-parallel-size 4 \
    --max-model-len 4096 \
    --gpu-memory-utilization 0.9 \
    --enforce-eager \
    --trust-remote-code \
    --enable-auto-tool-choice \
    --tool-call-parser poolside_v1 \
    --reasoning-parser poolside_v1 \
    --host 0.0.0.0 --port 8000

Model loading takes ~24 seconds on 4x H100 80GB. Monitor with:

podman logs -f vllm-laguna-s21

Ready when you see Application startup complete.

Try It Out

Health check

curl http://127.0.0.1:8000/health

Chat completion

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "poolside/Laguna-S-2.1-FP8",
    "messages": [
      {"role": "user", "content": "Write a Python function that returns the fibonacci sequence up to n terms."}
    ],
    "max_tokens": 256
  }'

Completions endpoint

curl -s http://127.0.0.1:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "poolside/Laguna-S-2.1-FP8",
    "prompt": "def hello_world():",
    "max_tokens": 64
  }'

Tool calling

Laguna S 2.1 supports agentic tool calling natively. The server is started with --enable-auto-tool-choice --tool-call-parser poolside_v1 for automatic tool use.

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "poolside/Laguna-S-2.1-FP8",
    "messages": [
      {"role": "user", "content": "What is the weather in Burlington, Vermont?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather for a location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {"type": "string", "description": "City and state"}
            },
            "required": ["location"]
          }
        }
      }
    ],
    "max_tokens": 256
  }'

Reasoning mode

Reasoning is off by default. Enable per-request:

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "poolside/Laguna-S-2.1-FP8",
    "messages": [
      {"role": "user", "content": "Explain the difference between a mutex and a semaphore."}
    ],
    "max_tokens": 512,
    "extra_body": {
      "chat_template_kwargs": {"enable_thinking": true}
    }
  }'

Cleanup

podman stop vllm-laguna-s21
podman rm vllm-laguna-s21

Notes and Gotchas

VLLM_BLOCKSCALE_FP8_GEMM_FLASHINFER=0 is required for the FP8 variant — without it, FlashInfer may use an incompatible GEMM path.
--enforce-eager is recommended on H100 80GB to avoid CUDA graph profiling OOM. H200s with more memory may not need this flag.
--max-model-len 4096 is a conservative default for smoke testing. The model supports up to 256K context; scale up gradually based on your GPU memory headroom. With 4x H100 80GB and FP8, KV cache allows ~424x concurrency at 4096 tokens.
MoE config warning: You may see Using default MoE config. Performance might be sub-optimal! — this is expected. A tuned MoE config for H100 with 256 experts is not yet available. Inference works correctly with defaults.
RoPE warnings: Unrecognized keys in rope_parameters warnings from transformers are benign and do not affect model behavior.
Tokenizer regex warning: A warning about incorrect regex pattern (referencing Mistral) may appear. This does not affect tokenization for standard use cases.
Model weights must be on local disk, not NFS. Rootless podman UID remapping breaks NFS mounts.
Use 127.0.0.1, not localhost — some hosts try IPv6 first and fail.

Validated Configuration

Field	Value
Hardware	4x NVIDIA H100 80GB HBM3
Tensor Parallelism	4
Model Memory	28.36 GiB
KV Cache Memory	39.91 GiB
Max Model Length	4096 (tested)
Load Time	~24 seconds
System Fingerprint	`vllm-0.24.1.dev588+rhaiv.1.g7c4bc7737-tp4-65ab4a17`

This is an unsupported preview release. Not intended for production use. No ongoing maintenance or updates. Model support will be included in the next official RHAIIS release.

dougbtv/laguna-s-2.1-usage-guide.md

Select an option

No results found