openAI API

what_is_it.md

Chat completion API is not JSON parsable

I'm trying to address two problems here:

• how do you extend the messages with an assistant's answers?
• what is the format of the messages

Format of messages

Let's answer the second question first. The format given by the playground, using the /v1 API, is to have a content object:

response = client.chat.completions.create(
  model="gpt-4o",
  messages=[
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "can you give me some numbers?"
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "{\n  \"numbers\": [\n    42,\n    18,\n    7,\n    64,\n    35\n  ],\n  \"reason\": \"These numbers are often associated with interesting mathematical properties or cultural significance. For example:\\n\\n1. **42**: Known as 'the answer to the ultimate question of life, the universe, and everything,' according to Douglas Adams' 'The Hitchhiker's Guide to the Galaxy.' \\n\\n2. **18**: In many cultures, it's associated with coming of age or the start of adulthood, often celebrated in special ways.\\n\\n3. **7**: This is a number often considered lucky or spiritual, with numerous references in religious texts and folklore.\\n\\n4. **64**: This is not only 2 to the power of 6 but is also significant in computing, representing the bit-length of architecture in computers.\\n\\n5. **35**: While this number may seem arbitrary, it's sometimes relevant in milestones, such as anniversaries or age thresholds. \\n\\nThese numbers showcase a blend of significance in various contexts from literature and culture to technology.\"\n}"
        }
      ]

but the API reference uses a string straight:

completion = client.chat.completions.create(
  model="o3-mini-2025-01-31",
  messages=[
    {"role": "developer", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]
)

It turns out that the API (at least the beta one I'm using client.beta.chat.completions.parse) can take a lot of different combinations:

ChatCompletionMessageParam: TypeAlias = Union[
    ChatCompletionDeveloperMessageParam,
    ChatCompletionSystemMessageParam,
    ChatCompletionUserMessageParam,
    ChatCompletionAssistantMessageParam,
    ChatCompletionToolMessageParam,
    ChatCompletionFunctionMessageParam,
]

where the UserMessage one for example:

class ChatCompletionUserMessageParam(TypedDict, total=False):
    content: Required[Union[str, Iterable[ChatCompletionContentPartParam]]]
    """The contents of the user message."""

    role: Required[Literal["user"]]
    """The role of the messages author, in this case `user`."""

    name: str

with

ChatCompletionContentPartParam: TypeAlias = Union[
    ChatCompletionContentPartTextParam, ChatCompletionContentPartImageParam, ChatCompletionContentPartInputAudioParam
]

class ChatCompletionContentPartTextParam(TypedDict, total=False):
    text: Required[str]
    """The text content."""

    type: Required[Literal["text"]]
    """The type of the content part."""

so the final answer is that you can pass both, either a {type,text} object or a string!

Furthermore, if a message is a string, you don't need the object!

Extending messages

You must extend messages with any responses from the assistant as well as your responses. This means that if the assistant calls a function, and you execute that function, both the call and the result of the function have to be messages that extend the messages so far.

The problem is that client.beta.chat.completions.parse, which returns a structured output, doesn't return a serializable response as it returns a type.

But if you use pydantic you can simply use the model_dump_json() function to do this:

# the type of messages
from openai.types.chat import ChatCompletionMessageParam
conversation_history: list[ChatCompletionMessageParam] = []

# later on, append any message and response...

if len(completions.choices) > 1:
  raise ValueError("we are assuming a single choice here")

if completions.choices[0].message.parsed is not None:
  parsed = completions.choices[0].message.parsed
  content = parsed if isinstance(parsed, str) else parsed.model_dump_json()
  conversation_history.append(
      {
          "role": "assistant",
          "content": content,
      }
  )
if completions.choices[0].message.tool_calls is not None:
  tool_calls = []
  for tool_call in completions.choices[0].message.tool_calls:
    tool_calls.append(tool_call.model_dump_json())
    conversation_history.append(
        {"role": "assistant", "content": [], "tool_calls": tool_calls}
    )

    # append your answer...
    conversation_history.append(
        {
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": result, # your result
        }
    )

the "content" in the tool function call result can also be an object but I suspect that if it's of type text, then a string is good enough!