s-sajid-ali

dirge sandbox setup fails with "no layers found in manifest" for multi-arch GHCR images

Summary

dirge sandbox setup --image <multi-arch OCI image> fails with:

Error: no layers found in manifest for <owner>/<repo>/<image>:<tag>

s-sajid-ali

Fixing "model does not support tool calling" for LFM2 on mlx-lm

Symptom

Running an OpenAI-compatible client (e.g. dirge or any openai SDK client) against mlx_lm.server serving LiquidAI/LFM2.5-2.6B-MLX-nvfp4 (or other LFM2 MLX conversions) with a tools=[...] payload produces this server-side warning, and the model never actually emits a tool call:

s-sajid-ali

#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.10"
# dependencies = [
#     "openai>=1.50",
#     "pydantic-ai-slim[mcp,openai]>=0.4",
#     "rich",
# ]
# ///
"""Minimal repro: MCP Gateway tool calls don't join the AI Gateway trace.

s-sajid-ali

Bug: portkey-ai 2.3.2 — TypeError: Object of type Omit is not JSON serializable

Affects: portkey-ai==2.3.2 used together with pydantic-ai>=2.9.0 and openai>=2.18.0
Symptom: TypeError raised inside portkey_ai._vendor.openai._utils._json before any network call is made
Reproducer: uv run --no-project reproducer.py (PEP 723 inline deps, no credentials needed)

---

Root cause

s-sajid-ali

In the ancient times ...

We had two kinds of traffic being handled as:

  graph TB
  subgraph traffic-from-outside
  GCP-LB-->|"restrict to specific IP addresses"|MetalLB
  end
  subgraph on-prem-boundary
 MetalLB-->|"bind to service via NodePort"|LLMGateway

s-sajid-ali

Contents of input JSONL:

> bat test_dataset.jsonl
─────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     │ File: test_dataset.jsonl
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ {"custom_id":"request-1","method":"POST","url":"/v1/chat/completions","body":{"model":"gemini-3-flash-preview","messages":[{"role":"user","content":"Where is NYU located
     │ ?"}],"max_tokens":2048}}
   2 │ {"custom_id":"request-2","method":"POST","url":"/v1/chat/completions","body":{"model":"gemini-3-flash-preview","messages":[{"role":"user","content":"What resources are a
     │ vailable for genAI research at nyu?"}],"tools":[{"type":"function","function":{"name":"google_search","description":"Retrieves search results from internet to expand con

s-sajid-ali

Input

Input `veo_request.py`

from portkey_ai import Portkey
import json
import os

s-sajid-ali

CSV Changes Required to Replicate Commit d23fe45 Behavior

Summary

Commit d23fe45 added facet matches to 4 storage services. However, the old system used numeric facet IDs while the new refactored system uses semantic facet IDs. Additionally, some old facets no longer exist in the new system.

Old Facet ID to Description Mapping

Based on the old facet-tree.json:

| Old ID | Facet Category | Description |

s-sajid-ali

# basic version from claude-3.7
#!/bin/bash

# Define the dump file path
DUMP_FILE="/slurm-dump/f-k8s-dev.cfg"

# Function for error handling
handle_error() {
    local exit_code=$1
    local error_message=$2

s-sajid-ali

{
  "spec": {
    "_meta": {
      "version": 3
    },
    "nodes": [
      {
        "name": "openpmd-api",
        "version": "0.15.1",
        "arch": {