laravel echolabs prism ai

prism-llms-full.txt

# Prism - Laravel LLM Integration

Prism is a powerful Laravel package for integrating Large Language Models (LLMs) into your applications. It provides a unified interface to work with various AI providers, enabling you to focus on building innovative AI features without getting bogged down in provider-specific implementation details.

## Core Concepts

### Installation and Configuration

```php
// Installation
composer require prism-php/prism

// Publish config
php artisan vendor:publish --tag=prism-config
```

Configure in `config/prism.php`:
```php
return [
    'prism_server' => [
        'middleware' => [],
        'enabled' => env('PRISM_SERVER_ENABLED', true),
    ],
    'providers' => [
        'anthropic' => [
            'api_key' => env('ANTHROPIC_API_KEY', ''),
            'version' => env('ANTHROPIC_API_VERSION', '2023-06-01'),
            'default_thinking_budget' => env('ANTHROPIC_DEFAULT_THINKING_BUDGET', 1024),
            'anthropic_beta' => env('ANTHROPIC_BETA', null),
        ],
        'openai' => [
            'url' => env('OPENAI_URL', 'https://api.openai.com/v1'),
            'api_key' => env('OPENAI_API_KEY', ''),
            'organization' => env('OPENAI_ORGANIZATION', null),
        ],
        // Other providers...
    ],
];
```

### Text Generation

Basic text generation:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt('Tell me a short story about a brave knight.')
    ->asText();

echo $response->text;
```

With system prompts:
```php
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withSystemPrompt('You are an expert mathematician who explains concepts simply.')
    ->withPrompt('Explain the Pythagorean theorem.')
    ->asText();
```

Using Laravel views for complex prompts:
```php
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withSystemPrompt(view('prompts.math-tutor'))
    ->withPrompt('What is calculus?')
    ->asText();
```

Message chains for conversations:
```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\AssistantMessage;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withMessages([
        new UserMessage('What is JSON?'),
        new AssistantMessage('JSON is a lightweight data format...'),
        new UserMessage('Can you show me an example?')
    ])
    ->asText();
```

### Tools & Function Calling

Tools extend AI capabilities by providing access to specific functions:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Tool;

$weatherTool = Tool::as('weather')
    ->for('Get current weather conditions')
    ->withStringParameter('city', 'The city to get weather for')
    ->using(function (string $city): string {
        // Your weather API logic here
        return "The weather in {$city} is sunny and 72°F.";
    });

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
    ->withMaxSteps(2)
    ->withPrompt('What is the weather like in Paris?')
    ->withTools([$weatherTool])
    ->asText();
```

Tool parameter types:
- `withStringParameter` - For text inputs
- `withNumberParameter` - For numeric values
- `withBooleanParameter` - For true/false flags
- `withArrayParameter` - For lists of items
- `withEnumParameter` - For restricted value sets
- `withObjectParameter` - For complex objects
- `withParameter` - For schema-based parameters

Tool choice options:
```php
use Prism\Prism\Enums\ToolChoice;

// Let the AI decide whether to use tools
->withToolChoice(ToolChoice::Auto)
// Force the AI to use a tool
->withToolChoice(ToolChoice::Any)
// Force the AI to use a specific tool
->withToolChoice('weather');
```

Complex tool implementation:
```php
namespace App\Tools;

use Prism\Prism\Tool;
use Illuminate\Support\Facades\Http;

class SearchTool extends Tool
{
    public function __construct()
    {
        $this
            ->as('search')
            ->for('useful when you need to search for current events')
            ->withStringParameter('query', 'Detailed search query.')
            ->using($this);
    }

    public function __invoke(string $query): string
    {
        // Implementation...
    }
}
```

### Structured Output

Define structured responses with schemas:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$schema = new ObjectSchema(
    name: 'movie_review',
    description: 'A structured movie review',
    properties: [
        new StringSchema('title', 'The movie title'),
        new StringSchema('rating', 'Rating out of 5 stars'),
        new StringSchema('summary', 'Brief review summary')
    ],
    requiredFields: ['title', 'rating', 'summary']
);

$response = Prism::structured()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->withSchema($schema)
    ->withPrompt('Review the movie Inception')
    ->asStructured();

// Access your structured data
$review = $response->structured;
echo $review['title'];    // "Inception"
echo $review['rating'];   // "5 stars"
echo $review['summary'];  // "A mind-bending..."
```

Available Schema Types:
- `StringSchema` - For text values
- `NumberSchema` - For integers and floating-point numbers
- `BooleanSchema` - For true/false values
- `ArraySchema` - For lists of items
- `EnumSchema` - For restricted value sets
- `ObjectSchema` - For complex, nested data structures

### Streaming Output

Stream responses in real-time:
```php
use Prism\Prism\Prism;

$response = Prism::stream()
    ->using('openai', 'gpt-4')
    ->withPrompt('Tell me a story about a brave knight.')
    ->asStream();

// Process each chunk as it arrives
foreach ($response as $chunk) {
    echo $chunk->text;
    // Flush the output buffer to send text to the browser immediately
    ob_flush();
    flush();
}
```

In a Laravel controller:
```php
use Prism\Prism\Prism;
use Illuminate\Http\Response;

public function streamResponse()
{
    return response()->stream(function () {
        $stream = Prism::stream()
            ->using('openai', 'gpt-4')
            ->withPrompt('Explain quantum computing step by step.')
            ->asStream();

        foreach ($stream as $chunk) {
            echo $chunk->text;
            ob_flush();
            flush();
        }
    }, 200, [
        'Cache-Control' => 'no-cache',
        'Content-Type' => 'text/event-stream',
        'X-Accel-Buffering' => 'no', // Prevents Nginx from buffering
    ]);
}
```

With Laravel 12 Event Streams:
```php
Route::get('/chat', function () {
    return response()->eventStream(function () {
        $stream = Prism::stream()
            ->using('openai', 'gpt-4')
            ->withPrompt('Explain quantum computing step by step.')
            ->asStream();

        foreach ($stream as $response) {
            yield $response->text;
        }
    });
});
```

### Embeddings

Generate vector representations of text:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-large')
    ->fromInput('Your text goes here')
    ->asEmbeddings();

// Get your embeddings vector
$embeddings = $response->embeddings[0]->embedding;

// Check token usage
echo $response->usage->tokens;
```

Generate multiple embeddings:
```php
$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-large')
    // First embedding
    ->fromInput('Your text goes here')
    // Second embedding
    ->fromInput('Your second text goes here')
    // Third and fourth embeddings
    ->fromArray([
        'Third',
        'Fourth'
    ])
    ->asEmbeddings();

/** @var Embedding $embedding */
foreach ($embeddings as $embedding) {
    // Do something with your embeddings
    $embedding->embedding;
}
```

From file:
```php
$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-large')
    ->fromFile('/path/to/your/document.txt')
    ->asEmbeddings();
```

### Input Modalities

#### Images

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\Support\Image;

// From a local file
$message = new UserMessage(
    "What's in this image?",
    [Image::fromPath('/path/to/image.jpg')]
);

// From a URL
$message = new UserMessage(
    'Analyze this diagram:',
    [Image::fromUrl('https://example.com/diagram.png')]
);

// From a Base64
$image = base64_encode(file_get_contents('/path/to/image.jpg'));

$message = new UserMessage(
    'Analyze this diagram:',
    [Image::fromBase64($image)]
);

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withMessages([$message])
    ->generate();
```

#### Documents

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\Support\Document;

Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withMessages([
        // From base64
        new UserMessage('Here is the document from base64', [
            Document::fromBase64(base64_encode(file_get_contents('tests/Fixtures/test-pdf.pdf')), 'application/pdf'),
        ]),
        // Or from a path
        new UserMessage('Here is the document from a local path', [
            Document::fromPath('tests/Fixtures/test-pdf.pdf'),
        ]),
        // Or from a text string
        new UserMessage('Here is the document from a text string (e.g. from your database)', [
            Document::fromText('Hello world!'),
        ]),
    ])
    ->generate();
```

### Testing

Using the fake implementation:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\ResponseMeta;
use Prism\Prism\Enums\FinishReason;
use Prism\Prism\Text\Response as TextResponse;

it('can generate text', function () {
    // Create a fake text response
    $fakeResponse = new TextResponse(
        text: 'Hello, I am Claude!',
        steps: collect([]),
        responseMessages: collect([]),
        toolCalls: [],
        toolResults: [],
        usage: new Usage(10, 20),
        finishReason: FinishReason::Stop,
        meta: new Meta('fake-1', 'fake-model'),
        messages: collect([]),
        additionalContent: []
    );

    // Set up the fake
    $fake = Prism::fake([$fakeResponse]);

    // Run your code
    $response = Prism::text()
        ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
        ->withPrompt('Who are you?')
        ->asText();

    // Make assertions
    expect($response->text)->toBe('Hello, I am Claude!');
});
```

Using ResponseBuilder:
```php
use Prism\Prism\Text\ResponseBuilder;
use Prism\Prism\Text\Step;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\ResponseMeta;
use Prism\Prism\ValueObjects\ToolCall;

Prism::fake([
    (new ResponseBuilder)
        ->addStep(new Step(
            text: "Step 1 response text",
            finishReason: FinishReason::Stop,
            toolCalls: [/** tool calls */],
            toolResults: [/** tool results */],
            usage: new Usage(1000, 750),
            meta: new Meta(id: 123, model: 'test-model'),
            messages: [/** messages */],
            systemPrompts: [/** system prompts */],
            additionalContent: ['test' => 'additional']
        ))
        ->toResponse()
]);
```

Testing tools:
```php
it('can use weather tool', function () {
    // Define the expected tool call and response sequence
    $responses = [
        // First response: AI decides to use the weather tool
        new TextResponse(
            text: '', // Empty text since the AI is using a tool
            steps: collect([]),
            responseMessages: collect([]),
            toolCalls: [
                new ToolCall(
                    id: 'call_123',
                    name: 'weather',
                    arguments: ['city' => 'Paris']
                )
            ],
            toolResults: [],
            usage: new Usage(15, 25),
            finishReason: FinishReason::ToolCalls,
            meta: new Meta('fake-1', 'fake-model'),
            messages: collect([]),
            additionalContent: []
        ),
        // Second response: AI uses the tool result to form a response
        new TextResponse(
            text: 'Based on current conditions, the weather in Paris is sunny with a temperature of 72°F.',
            steps: collect([]),
            responseMessages: collect([]),
            toolCalls: [],
            toolResults: [],
            usage: new Usage(20, 30),
            finishReason: FinishReason::Stop,
            meta: new Meta('fake-2', 'fake-model'),
            messages: collect([]),
            additionalContent: []
        ),
    ];

    // Set up the fake
    $fake = Prism::fake($responses);

    // Create the weather tool
    $weatherTool = Tool::as('weather')
        ->for('Get weather information')
        ->withStringParameter('city', 'City name')
        ->using(fn (string $city) => "The weather in {$city} is sunny with a temperature of 72°F");

    // Run the actual test
    $response = Prism::text()
        ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
        ->withPrompt('What\'s the weather in Paris?')
        ->withTools([$weatherTool])
        ->asText();

    // Assert the correct number of API calls were made
    $fake->assertCallCount(2);

    // Assert tool calls were made correctly
    expect($response->steps[0]->toolCalls)->toHaveCount(1);
    expect($response->steps[0]->toolCalls[0]->name)->toBe('weather');
    expect($response->steps[0]->toolCalls[0]->arguments())->toBe(['city' => 'Paris']);

    // Assert tool results were processed
    expect($response->toolResults)->toHaveCount(1);
    expect($response->toolResults[0]->result)
        ->toBe('The weather in Paris is sunny with a temperature of 72°F');

    // Assert final response
    expect($response->text)
        ->toBe('Based on current conditions, the weather in Paris is sunny with a temperature of 72°F.');
});
```

### Prism Server

Expose your Prism-powered AI models through a standardized API:

Enable in `config/prism.php`:
```php
'prism_server' => [
    'middleware' => [],
    'enabled' => env('PRISM_SERVER_ENABLED', true),
]
```

Register models in a service provider:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\PrismServer;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function boot(): void
    {
        PrismServer::register(
            'my-custom-model',
            fn () => Prism::text()
                ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
                ->withSystemPrompt('You are a helpful assistant.')
        );
    }
}
```

Endpoints:
- Chat Completions: `http://your-app.com/prism/openai/v1/chat/completions`
- List Models: `http://your-app.com/prism/openai/v1/models`

### Error Handling

Basic error handling:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Exceptions\PrismException;
use Throwable;

try {
    $response = Prism::text()
        ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
        ->withPrompt('Generate text...')
        ->asText();
} catch (PrismException $e) {
    Log::error('Text generation failed:', ['error' => $e->getMessage()]);
} catch (Throwable $e) {
    Log::error('Generic error:', ['error' => $e->getMessage()]);
}
```

Provider-specific exceptions:
- `PrismStructuredDecodingException` - Invalid JSON for structured requests
- `PrismRateLimitedException` - Rate limit or quota hit
- `PrismProviderOverloadedException` - Provider capacity issues
- `PrismRequestTooLargeException` - Request too large

## Advanced Features

### Rate Limit Handling

Catching rate limit exceptions:
```php
use Prism\Prism\Prism;
use Prism\Enums\Provider;
use Prism\Prism\ValueObjects\ProviderRateLimit;
use Prism\Prism\Exceptions\PrismRateLimitedException;

try {
    Prism::text()
        ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
        ->withPrompt('Hello world!')
        ->generate();
}
catch (PrismRateLimitedException $e) {
    /** @var ProviderRateLimit $rate_limit */ 
    foreach ($e->rateLimits as $rate_limit) {
        // Loop through rate limits...
    }
    
    // Log, fail gracefully, etc.
}
```

Finding which rate limit you've hit:
```php
use Prism\Prism\ValueObjects\ProviderRateLimit;
use Illuminate\Support\Arr;

try {
    // Your request
}
catch (PrismRateLimitedException $e) {
    $hit_limit = Arr::first($e->rateLimits, fn(ProviderRateLimit $rate_limit) => $rate_limit->remaining === 0);
}
```

### Custom Providers

Creating a custom provider:
```php
namespace App\Prism\Providers;

use Prism\Prism\Contracts\Provider;
use Prism\Prism\Embeddings\Request as EmbeddingsRequest;
use Prism\Prism\Embeddings\Response as EmbeddingsResponse;
use Prism\Prism\Structured\Request as StructuredRequest;
use Prism\Prism\Structured\Response as StructuredResponse;
use Prism\Prism\Text\Request as TextRequest;
use Prism\Prism\Text\Response as TextResponse;

class MyCustomProvider implements Provider
{
    public function text(TextRequest $request): TextResponse
    {
        // Implementation...
    }

    public function structured(StructuredRequest $request): StructuredResponse
    {
        // Implementation...
    }

    public function embeddings(EmbeddingsRequest $request): EmbeddingsResponse
    {
        // Implementation...
    }
}
```

Registering a custom provider:
```php
namespace App\Providers;

use App\Prism\Providers\MyCustomProvider;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function boot(): void
    {
        $this->app['prism-manager']->extend('my-custom-provider', function ($app, $config) {
            return new MyCustomProvider(
                apiKey: $config['api_key'] ?? '',
            );
        });
    }
}
```

Add provider configuration:
```php
// config/prism.php
return [
    'providers' => [
        // ... other providers ...
        'my-custom-provider' => [
            'api_key' => env('MY_CUSTOM_PROVIDER_API_KEY'),
        ],
    ],
];
```

## Provider-Specific Features

### Anthropic

#### Prompt Caching

```php
use Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\Tool;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\SystemMessage;

Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withMessages([
        (new SystemMessage('I am a long re-usable system message.'))
            ->withProviderMeta(Provider::Anthropic, ['cacheType' => 'ephemeral']),

        (new UserMessage('I am a long re-usable user message.'))
            ->withProviderMeta(Provider::Anthropic, ['cacheType' => 'ephemeral'])
    ])
    ->withTools([
        Tool::as('cache me')
            ->withProviderMeta(Provider::Anthropic, ['cacheType' => 'ephemeral'])
    ])
    ->asText();
```

#### Extended Thinking

```php
use Prism\Enums\Provider;
use Prism\Prism\Prism;

Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet-latest')
    ->withPrompt('What is the meaning of life?')
    // enable thinking
    ->withProviderMeta(Provider::Anthropic, ['thinking' => ['enabled' => true]]) 
    ->asText();

// With custom budget
Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet-latest')
    ->withPrompt('What is the meaning of life?')
    ->withProviderMeta(Provider::Anthropic, [
        'thinking' => [
            'enabled' => true, 
            'budgetTokens' => 2048
        ]
    ]);
```

Accessing thinking output:
```php
$response->additionalContent['thinking'];

// Or on a specific step
$response->steps->first()->additionalContent->thinking;
```

#### Citations

```php
use Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\Support\Document;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withMessages([
        new UserMessage(
            content: "Is the grass green and the sky blue?",
            additionalContent: [
                Document::fromChunks(
                    chunks: ["The grass is green.", "Flamingos are pink.", "The sky is blue."],
                    title: 'The colours of nature',
                    context: 'The go-to textbook on the colours found in nature!'
                )
            ]
        )
    ])
    ->withProviderMeta(Provider::Anthropic, ['citations' => true])
    ->asText();
```

Accessing citations:
```php
use Prism\Prism\Providers\Anthropic\ValueObjects\MessagePartWithCitations;
use Prism\Prism\Providers\Anthropic\ValueObjects\Citation;

$messageChunks = $response->additionalContent['messagePartsWithCitations'];

$text = '';
$footnotes = '';

$footnoteId = 1;

/** @var MessagePartWithCitations $messageChunk  */
foreach ($messageChunks as $messageChunk) {
    $text .= $messageChunk->text;
    
    /** @var Citation $citation */
    foreach ($messageChunk->citations as $citation) {
        $footnotes[] = [
            'id' => $footnoteId,
            'document_title' => $citation->documentTitle,
            'reference_start' => $citation->startIndex,
            'reference_end' => $citation->endIndex
        ];
    
        $text .= '<sup><a href="#footnote-'.$footnoteId.'">'.$footnoteId.'</a></sup>';
    
        $footnoteId++;
    }
}
```

### OpenAI

#### Strict Tool Schemas

```php
Tool::as('search')
    ->for('Searching the web')
    ->withStringParameter('query', 'the detailed search query')
    ->using(fn (): string => '[Search results]')
    ->withProviderMeta(Provider::OpenAI, [
      'strict' => true,
    ]);
```

#### Strict Structured Output Schemas

```php
$response = Prism::structured()
    ->withProviderMeta(Provider::OpenAI, [
        'schema' => [
            'strict' => true
        ]
    ]);
```

### VoyageAI

#### Input Type

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;

// For search / querying
Prism::embeddings()
    ->using(Provider::VoyageAI, 'voyage-3-lite')
    ->fromInput('The food was delicious and the waiter...')
    ->withProviderMeta(Provider::VoyageAI, ['inputType' => 'query'])
    ->generate();

// For document retrieval
Prism::embeddings()
    ->using(Provider::VoyageAI, 'voyage-3-lite')
    ->fromInput('The food was delicious and the waiter...')
    ->withProviderMeta(Provider::VoyageAI, ['inputType' => 'document'])
    ->generate();
```

## Common Configuration Parameters

### Generation Parameters

- `withMaxTokens` - Maximum number of tokens to generate
- `usingTemperature` - Controls randomness (0 = deterministic, higher = more random)
- `usingTopP` - Nucleus sampling alternative to temperature
- `withClientOptions` - HTTP client options (e.g., timeouts)
- `withClientRetry` - Automatic retry configuration
- `usingProviderConfig` - Override provider configuration

### Response Properties

- `$response->text` - Generated text content
- `$response->finishReason` - Why generation stopped (Stop, Length, ContentFilter, ToolCalls, Error, Other, Unknown)
- `$response->usage` - Token usage statistics
- `$response->steps` - For multi-step generations
- `$response->responseMessages` - Message history
- `$response->toolCalls` - Tool calls made by the AI
- `$response->toolResults` - Results from tool executions
- `$response->structured` - Parsed structured data (for structured output)

## Supported Providers

- Anthropic (Claude)
- DeepSeek
- Gemini
- Groq
- Mistral
- Ollama
- OpenAI
- VoyageAI
- xAI

Each provider has specific features, limitations, and configuration options. Check the provider-specific documentation for details.