README.md

🛠️ Mistral API Proxy for OpenWebUI

Fixes 400 and 422 errors when using Mistral models in OpenWebUI

🔍 Problem

If you're using OpenWebUI with Mistral AI, you may run into issues like:

• 422: OpenWebUI: Server Connection Error when defining a Mistral model as your base model.
• 400: OpenWebUI: Server Connection Error when clicking "Continue Response".

These errors happen because OpenWebUI assumes OpenAI API semantics that Mistral's API doesn't fully support — such as unsupported fields or improperly structured messages.

✅ Solution

This Python-based proxy acts as a middleware between OpenWebUI and the Mistral API:

• Cleans up unsupported fields in the payload (e.g., logit_bias, user, etc.).
• Ensures Continue Response requests are handled properly by appending a "Continue response" message if needed.
• Proxies all other requests transparently, maintaining headers and stream support.

🚀 Quick Start

1. Install dependencies

   pip install flask requests python-dotenv

2. Create a .env file with your Mistral API key:

   MISTRAL_API_KEY=your_actual_key_here
   

3. Run the proxy server

   python proxy.py

   By default it runs at:

   http://localhost:8880
   

4. Configure OpenWebUI

   • Go to Settings → Connections → OpenAI API

   • Set:

     • API Base: http://localhost:8880/v1
     • API Key: Anything (it will be ignored)

   • Use one of Mistral’s models as your base model for a Knowledge or Chat session.

💡 Why this works

OpenWebUI expects OpenAI-like behavior, but Mistral's API:

• Doesn’t support certain fields
• Can’t continue from assistant messages the same way
• Requires stricter message validation

This proxy bridges the gap — giving you a smooth dev experience with OpenWebUI and Mistral models.

🧩 Notes

• This is a minimal, hacky fix – not a production-grade proxy.
• Tested with open-webui (latest) and mistral-small / mistral-medium models.

python.py

from flask import Flask, request, Response, stream_with_context
import requests
import os
from dotenv import load_dotenv
load_dotenv()

app = Flask(__name__)
MISTRAL_API_BASE = "https://api.mistral.ai"
MISTRAL_API_KEY = os.environ.get("MISTRAL_API_KEY", "XXXXXXXXXXXXXXXXXXXXX")  # set via env or replace directly

def build_streaming_response(response):
    excluded_headers = {
        'content-encoding', 'transfer-encoding', 'content-length',
        'connection', 'keep-alive', 'proxy-authenticate',
        'proxy-authorization', 'te', 'trailers', 'upgrade'
    }

    def generate():
        for chunk in response.iter_content(chunk_size=8192):
            if chunk:
                yield chunk

    content_type = response.headers.get('Content-Type', 'application/json')
    flask_response = Response(
        stream_with_context(generate()),
        status=response.status_code,
        content_type=content_type
    )

    for k, v in response.headers.items():
        if k.lower() not in excluded_headers and k.lower() != 'content-type':
            flask_response.headers[k] = v

    return flask_response

def inject_auth_header(original_headers):
    headers = dict(original_headers)
    headers["Authorization"] = f"Bearer {MISTRAL_API_KEY}"
    headers.pop("Host", None)
    headers.pop("Accept-Encoding", None)  # prevent gzip
    return headers


@app.route('/v1/chat/completions', methods=["POST"])
def intercept_and_forward():
    headers = inject_auth_header(request.headers)

    json_payload = request.get_json(force=True, silent=True)
    if not json_payload:
        return Response("Invalid JSON", status=400)

    # Remove undesired fields
    json_payload.pop("logit_bias", None)
    json_payload.pop("user", None)
    json_payload.pop("additionalProp1", None)

    # Fix for Mistral API: ensure last message is NOT from assistant
    messages = json_payload.get("messages", [])
    if messages and messages[-1].get("role") == "assistant":
        messages.append({"role": "user", "content": "Continue response"})
        json_payload["messages"] = messages

    response = requests.post(
        f"{MISTRAL_API_BASE}/v1/chat/completions",
        json=json_payload,
        headers=headers,
        stream=True
    )

    response.raw.decode_content = True
    return build_streaming_response(response)

@app.route('/', defaults={'path': ''}, methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"])
@app.route('/<path:path>', methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"])
def catch_all(path):
    method = request.method
    headers = inject_auth_header(request.headers)
    url = f"{MISTRAL_API_BASE}/{path}"

    kwargs = {
        "headers": headers,
        "params": request.args if method == "GET" else None,
        "data": request.get_data(),
    }

    response = requests.request(
        method,
        url,
        headers=headers,
        data=request.get_data(),
        stream=True
    )
    response.raw.decode_content = True
    return build_streaming_response(response)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8880)