maas-bbr-multi-turn.md

Multi-Turn BBR+MaaS Validation Output

• Anthropic multi-turn validation (the errors at turn 8 are TRLP and expected as part of the test)

HOST="https://maas.$(kubectl get ingress.config.openshift.io/cluster -o jsonpath='{.spec.domain}')"
TOKEN=$(oc whoami -t)

$ echo "=== Anthropic multi-turn (10 messages) ==="
MESSAGES='[{"role":"user","content":"My name is Bob. Remember it."}]'
for i in $(seq 1 10); do
  echo "--- Turn $i ---"
  RESP=$(curl -sSk "$HOST/claude-sonnet-4-20250514/v1/chat/completions" \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $API_KEY" \
    -d "{\"model\":\"claude-sonnet-4-20250514\",\"messages\":$MESSAGES,\"max_tokens\":50}")
  CONTENT=$(echo "$RESP" | jq -r '.choices[0].message.content // .error.message // "ERROR"')
  HTTP_CODE=$(echo "$RESP" | jq -r '.error.code // "ok"')
  echo "  Assistant: $CONTENT"
  if [[ "$HTTP_CODE" != "ok" && "$HTTP_CODE" != "null" ]]; then
    echo "  ERROR: $HTTP_CODE — aborting multi-turn"
    break
  fi
  MESSAGES=$(echo "$MESSAGES" | jq --arg c "$CONTENT" '. + [{"role":"assistant","content":$c},{"role":"user","content":"Turn '"$i"': What is my name? Also tell me what turn number this is."}]')
done
echo ""
echo "Final message count: $(echo "$MESSAGES" | jq length)"
=== Anthropic multi-turn (10 messages) ===
--- Turn 1 ---
  Assistant: Got it, Bob! I'll remember your name for our conversation. How can I help you today?
--- Turn 2 ---
  Assistant: This is Turn 2, and your name is Bob!
--- Turn 3 ---
  Assistant: This is Turn 3, and your name is Bob!
--- Turn 4 ---
  Assistant: This is Turn 4, and your name is Bob!
--- Turn 5 ---
  Assistant: This is Turn 5, and your name is Bob!
--- Turn 6 ---
  Assistant: This is Turn 6, and your name is Bob!
--- Turn 7 ---
  Assistant: This is Turn 7, and your name is Bob!
--- Turn 8 ---
jq: parse error: Invalid numeric literal at line 1, column 4
jq: parse error: Invalid numeric literal at line 1, column 4
  Assistant:
  ERROR:  — aborting multi-turn

Final message count: 15

• OAI multi-turn validation (the errors at turn 8 are TRLP and expected as part of the test)

HOST="https://maas.$(kubectl get ingress.config.openshift.io/cluster -o jsonpath='{.spec.domain}')"
TOKEN=$(oc whoami -t)

$ echo "=== OpenAI multi-turn (10 messages) ==="
MESSAGES='[{"role":"user","content":"My name is Alice. Remember it."}]'
for i in $(seq 1 10); do
  echo "--- Turn $i ---"
  RESP=$(curl -sSk "$HOST/gpt-4o/v1/chat/completions" \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $API_KEY" \
    -d "{\"model\":\"gpt-4o\",\"messages\":$MESSAGES,\"max_tokens\":50}")
  CONTENT=$(echo "$RESP" | jq -r '.choices[0].message.content // .error.message // "ERROR"')
  HTTP_CODE=$(echo "$RESP" | jq -r '.error.code // "ok"')
  echo "  Assistant: $CONTENT"
  if [[ "$HTTP_CODE" != "ok" && "$HTTP_CODE" != "null" ]]; then
    echo "  ERROR: $HTTP_CODE — aborting multi-turn"
    break
  fi
  # Accumulate history
  MESSAGES=$(echo "$MESSAGES" | jq --arg c "$CONTENT" '. + [{"role":"assistant","content":$c},{"role":"user","content":"Turn '"$i"': What is my name? Also tell me what turn number this is."}]')
done
echo ""
echo "Final message count: $(echo "$MESSAGES" | jq length)"
=== OpenAI multi-turn (10 messages) ===
--- Turn 1 ---
  Assistant: Hello, Alice! I won't be able to remember your name beyond this conversation, but feel free to remind me if we chat again. How can I assist you today?
--- Turn 2 ---
  Assistant: Your name is Alice, and this is Turn 2. How can I assist you further?
--- Turn 3 ---
  Assistant: Your name is Alice, and this is Turn 3. What would you like to do next?
--- Turn 4 ---
  Assistant: Your name is Alice, and this is Turn 4. How can I assist you today?
--- Turn 5 ---
  Assistant: Your name is Alice, and this is Turn 5. What else can I help you with?
--- Turn 6 ---
  Assistant: Your name is Alice, and this is Turn 6. Let me know if there's anything else you'd like to discuss!
--- Turn 7 ---
  Assistant: Your name is Alice, and this is Turn 7. How may I assist you further?
--- Turn 8 ---
jq: parse error: Invalid numeric literal at line 1, column 4
jq: parse error: Invalid numeric literal at line 1, column 4
  Assistant:
  ERROR:  — aborting multi-turn

Final message count: 15