Skip to content

Instantly share code, notes, and snippets.

@ben-vargas

ben-vargas/reasoningEffort_and_reasoningSummary.md

Last active October 3, 2025 07:52
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save ben-vargas/1c4f5badd59cfc9ba3de1d4186b19b8e to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save ben-vargas/1c4f5badd59cfc9ba3de1d4186b19b8e to your computer and use it in GitHub Desktop.
Download ZIP
gpt-5 and gpt-5-codex OAuth (backend-api) research
Raw
reasoningEffort_and_reasoningSummary.md

● Complete Testing Results - Clarified for Plugin Normalization

GPT-5 Model (via gpt-5-nano)

reasoningEffort auto concise detailed none
minimal ✅ Success ❌ Unsupported ✅ Success ❌ Invalid
low ✅ Success ❌ Unsupported ✅ Success ❌ Invalid
medium ✅ Success ❌ Unsupported ✅ Success ❌ Invalid
high ✅ Success ❌ Unsupported ✅ Success ❌ Invalid
none ❌ Invalid effort ❌ Invalid effort ❌ Invalid effort ❌ Invalid effort

GPT-5-Codex Model (with Plugin Normalization)

reasoningEffort auto concise detailed none Notes
minimal ✅ Success ❌ Unsupported ✅ Success ❌ Invalid Plugin converts to low
low ✅ Success ❌ Unsupported ✅ Success ❌ Invalid Same as minimal
medium ✅ Success ❌ Unsupported ✅ Success ❌ Invalid
high ✅ Success ❌ Unsupported ✅ Success ❌ Invalid
none ❌ Invalid effort ❌ Invalid effort ❌ Invalid effort ❌ Invalid effort

Important: For gpt-5-codex, reasoningEffort: "minimal" is automatically normalized to "low" by the plugin before being sent to the API. They are functionally identical.

Simplified Valid Configurations

GPT-5:

  • Effort: minimal, low, medium, high
  • Summary: auto, detailed

GPT-5-Codex:

  • Effort: minimal (→ low), low, medium, high
  • Summary: auto, detailed

🤔 Surprising Discovery:

The error messages are misleading! The API says "Supported values are: 'concise', 'detailed', and 'auto'" but then rejects concise when you actually try to use it, saying it's "not supported with the model."

Recommended Safe Configuration:

  {
    "reasoningEffort": "medium",     // or: minimal, low, high
    "reasoningSummary": "auto"       // or: detailed
  }

Test Results:

{
  "test_timestamp": "2025-10-03T01:11:52-06:00",
  "results": {
    "gpt-5": {
      "minimal": {
        "auto": {"status": "success"}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-2025-08-07' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" -- [openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-2025-08-07' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" "}
,
        "detailed": {"status": "success"}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" -- [openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" "}
      }
,
      "low": {
        "auto": {"status": "success"}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-2025-08-07' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" -- [openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-2025-08-07' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" "}
,
        "detailed": {"status": "success"}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" -- [openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" "}
      }
,
      "medium": {
        "auto": {"status": "success"}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-2025-08-07' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" "}
,
        "detailed": {"status": "success"}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" "}
      }
,
      "high": {
        "auto": {"status": "success"}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-2025-08-07' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" "}
,
        "detailed": {"status": "success"}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" -- [openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" "}
      }
,
      "none": {
        "auto": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" -- [openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" "}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" "}
,
        "detailed": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" -- [openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" "}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" -- [openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" "}
      }
    }
,
    "gpt-5-codex": {
      "minimal": {
        "auto": {"status": "success"}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-codex' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" "}
,
        "detailed": {"status": "success"}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" "}
      }
,
      "low": {
        "auto": {"status": "success"}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-codex' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" "}
,
        "detailed": {"status": "success"}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" "}
      }
,
      "medium": {
        "auto": {"status": "success"}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-codex' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" "}
,
        "detailed": {"status": "success"}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" "}
      }
,
      "high": {
        "auto": {"status": "success"}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Unsupported value: 'concise' is not supported with the 'gpt-5-codex' model. Supported values are: 'detailed'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"unsupported_value\" "}
,
        "detailed": {"status": "success"}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'concise', 'detailed', and 'auto'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.summary\",     \"code\": \"invalid_value\" "}
      }
,
      "none": {
        "auto": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" "}
,
        "concise": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" "}
,
        "detailed": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" "}
,
        "none": {"status": "error", "message": "[openai-codex-plugin] 400 error: {   \"error\": {     \"message\": \"Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.\",     \"type\": \"invalid_request_error\",     \"param\": \"reasoning.effort\",     \"code\": \"invalid_value\" "}
      }
    }
  }
}
Raw
textVerbosity_test.md

● textVerbosity Testing Results - All Values Tested

GPT-5 Model

textVerbosity Status Error Type Error Message
none ❌ Error API 400 Invalid value: 'none'. Supported values are: 'low', 'medium', and 'high'.
low ✅ Success - -
medium ✅ Success - -
high ✅ Success - -
minimal ❌ Error API 400 Invalid value: 'minimal'. Supported values are: 'low', 'medium', and 'high'.

GPT-5-Codex Model

textVerbosity Status Error Type Error Message
none ❌ Error SDK Validation AI_InvalidArgumentError: invalid openai provider options
low ❌ Error API 400 Unsupported value: 'low' is not supported with the 'codex' model. Supported values are: 'medium'.
medium ✅ Success - -
high ❌ Error API 400 Unsupported value: 'high' is not supported with the 'codex' model. Supported values are: 'medium'.
minimal ❌ Error SDK Validation AI_InvalidArgumentError: invalid openai provider options

Summary

✅ Valid Values:

GPT-5:

  • low, medium, high

GPT-5-Codex:

  • medium ONLY

❌ Invalid Values (Both Models):

  • none - Rejected by SDK validation (never reaches API)
  • minimal - Rejected by SDK validation (never reaches API)

Key Findings:

  1. GPT-5 supports 3 verbosity levels: low, medium, high
  2. GPT-5-Codex supports ONLY medium - API explicitly rejects all other values
  3. SDK validates and rejects "none" and "minimal" before sending to API
  4. This is very different from reasoningEffort, where both models support multiple values

Recommended Configuration:

  {
    "provider": {
      "openai": {
        "options": {
          "textVerbosity": "medium"  // Safe for both models
        },
        "models": {
          "gpt-5": {
            "options": {
              "textVerbosity": "medium"  // or: low, high
            }
          },
          "gpt-5-codex": {
            "options": {
              "textVerbosity": "medium"  // MUST be medium
            }
          }
        }
      }
    }
  }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment