MemGPT provides you a free endpoint for trying. It's at https://inference.memgpt.ai/chat/completions From official docs it's claimed that the free endpoint is running on variants of Mixtral 8x7b:
MemGPT Free Endpoint: select this if you'd like to try MemGPT on a top open LLM for free (currently variants of Mixtral 8x7b!)
However, after I manually try running Mixtral 8x7b on my own machine, I saw that the model cannot be compared to the free endpoint in term of accuracy (function calling + response). This makes me want to find out the real model behind this endpoint.
Free endpoint just forwards your call to OpenAI's ChatGPT 3.5. However, I cannot be sure if they log our requests or not.
Inspecting the request using Fiddler, here is the JSON request:
{
"model": "memgpt-openai",
"messages": [
{
"content": "Hello! Tell me about you!",
"role": "user"
}
],
"frequency_penalty": 0,
"logprobs": false,
"n": 1,
"presence_penalty": 0,
"stream": false,
"temperature": 1,
"top_p": 1,
"user": "00000000-0000-0000-0000-18937ffe6b0f",
"tools": [
{
"type": "function",
"function": {
"name": "send_message",
"description": "Sends a message to the human user.",
"parameters": {
"type": "object",
"properties": {
"message": {
"type": "string",
"description": "Message contents. All unicode (including emojis) are supported."
}
},
"required": [
"message"
]
}
}
},
{
"type": "function",
"function": {
"name": "pause_heartbeats",
"description": "Temporarily ignore timed heartbeats. You may still receive messages from manual heartbeats and other events.",
"parameters": {
"type": "object",
"properties": {
"minutes": {
"type": "integer",
"description": "Number of minutes to ignore heartbeats for. Max value of 360 minutes (6 hours)."
}
},
"required": [
"minutes"
]
}
}
},
{
"type": "function",
"function": {
"name": "conversation_search",
"description": "Search prior conversation history using case-insensitive string matching.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "String to search for."
},
"page": {
"type": "integer",
"description": "Allows you to page through results. Only use on a follow-up query. Defaults to 0 (first page)."
},
"request_heartbeat": {
"type": "boolean",
"description": "Request an immediate heartbeat after function execution. Set to 'true' if you want to send a follow-up message or run a follow-up function."
}
},
"required": [
"query",
"request_heartbeat"
]
}
}
},
{
"type": "function",
"function": {
"name": "conversation_search_date",
"description": "Search prior conversation history using a date range.",
"parameters": {
"type": "object",
"properties": {
"start_date": {
"type": "string",
"description": "The start of the date range to search, in the format 'YYYY-MM-DD'."
},
"end_date": {
"type": "string",
"description": "The end of the date range to search, in the format 'YYYY-MM-DD'."
},
"page": {
"type": "integer",
"description": "Allows you to page through results. Only use on a follow-up query. Defaults to 0 (first page)."
},
"request_heartbeat": {
"type": "boolean",
"description": "Request an immediate heartbeat after function execution. Set to 'true' if you want to send a follow-up message or run a follow-up function."
}
},
"required": [
"start_date",
"end_date",
"request_heartbeat"
]
}
}
},
{
"type": "function",
"function": {
"name": "archival_memory_insert",
"description": "Add to archival memory. Make sure to phrase the memory contents such that it can be easily queried later.",
"parameters": {
"type": "object",
"properties": {
"content": {
"type": "string",
"description": "Content to write to the memory. All unicode (including emojis) are supported."
},
"request_heartbeat": {
"type": "boolean",
"description": "Request an immediate heartbeat after function execution. Set to 'true' if you want to send a follow-up message or run a follow-up function."
}
},
"required": [
"content",
"request_heartbeat"
]
}
}
},
{
"type": "function",
"function": {
"name": "archival_memory_search",
"description": "Search archival memory using semantic (embedding-based) search.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "String to search for."
},
"page": {
"type": "integer",
"description": "Allows you to page through results. Only use on a follow-up query. Defaults to 0 (first page)."
},
"request_heartbeat": {
"type": "boolean",
"description": "Request an immediate heartbeat after function execution. Set to 'true' if you want to send a follow-up message or run a follow-up function."
}
},
"required": [
"query",
"request_heartbeat"
]
}
}
},
{
"type": "function",
"function": {
"name": "core_memory_append",
"description": "Append to the contents of core memory.",
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Section of the memory to be edited (persona or human)."
},
"content": {
"type": "string",
"description": "Content to write to the memory. All unicode (including emojis) are supported."
},
"request_heartbeat": {
"type": "boolean",
"description": "Request an immediate heartbeat after function execution. Set to 'true' if you want to send a follow-up message or run a follow-up function."
}
},
"required": [
"name",
"content",
"request_heartbeat"
]
}
}
},
{
"type": "function",
"function": {
"name": "core_memory_replace",
"description": "Replace the contents of core memory. To delete memories, use an empty string for new_content.",
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Section of the memory to be edited (persona or human)."
},
"old_content": {
"type": "string",
"description": "String to replace. Must be an exact match."
},
"new_content": {
"type": "string",
"description": "Content to write to the memory. All unicode (including emojis) are supported."
},
"request_heartbeat": {
"type": "boolean",
"description": "Request an immediate heartbeat after function execution. Set to 'true' if you want to send a follow-up message or run a follow-up function."
}
},
"required": [
"name",
"old_content",
"new_content",
"request_heartbeat"
]
}
}
}
],
"tool_choice": "auto",
"parallel_tool_calls": false
}It can just be shortened to a minimum form:
{
"model": "memgpt-openai",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Are you ChatGPT 4?"
}
],
"user": "00000000-0000-0000-0000-18937ffe6b0f"
}This is basically minimum OpenAI JSON + the field user (can be any random UUID). And surprisingly, the answer is:
{
"id": "chatcmpl-A2EiNLl7WvR6qBs6PisLvxdmlTqsE",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"content": "As an AI developed by OpenAI, I'm based on the GPT-3 model. So, I am not ChatGPT-4. If a newer version is released in the future, my capabilities could go beyond the current model.",
"tool_calls": null,
"role": "assistant",
"function_call": null
},
"logprobs": null
}
],
"created": "2024-08-31T09:29:43Z",
"model": "memgpt",
"system_fingerprint": null,
"object": "chat.completion",
"usage": {
"completion_tokens": 49,
"prompt_tokens": 25,
"total_tokens": 74
}
}I wonder if the id field is used to log user's messages. But since user is the same for every time I create a new user, I don't think they log our messages (or at least they log it anonymously). But anyway, it's better not to chat with sensitive information.