billybonks

# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: "Custom Provider"

prompts:
  - "say {{item}}"

providers:
  - id: file://./provider.js

tests:

billybonks

## Bug: G-Eval assertions with LiteLLM provider do not trigger API calls

### Describe the bug

When using promptfoo with LiteLLM, G-Eval assertions configured to use a LiteLLM provider (referenced by its string ID, e.g., `litellm:gemini-pro`) do not appear to trigger any API calls to the LiteLLM server for the evaluation step.

The initial prompt's response generation call to LiteLLM works correctly, and the call is logged by the LiteLLM server. However, the subsequent G-Eval call is missing from the LiteLLM server logs. This results in the `gradingResult` often being null or showing an error like "No output", indicating that the G-Eval LLM was not invoked.

---

billybonks

You are an experienced software developer tasked with creating a commit message based on a git diff. Your goal is to produce a clear, concise, and informative commit message.

First, carefully analyze the following git diff:

<git_diff>
diff --git a/.gitattributes b/.gitattributes
index 3f263d62be..f4542e6b25 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -1,6 +1,9 @@

billybonks

{
  "$schema": "https://vega.github.io/schema/vega/v5.json",
  "autosize": { "type": "fit", "resize": true },
  "width": 600,
  "height": 400,
  "data": [
    {
      "name": "userFilteredData",
      "values": [
        {

billybonks

{
  "$schema": "https://vega.github.io/schema/vega/v5.json",
  "description": "A basic bar chart example, with value labels shown upon pointer hover.",
  "width": 400,
  "height": 200,
  "padding": 5,

  "data": [
    {
      "name": "table",

billybonks

{% for variant in variants %}
  {{variant.product_name}} {{ variant.name }} | {{ variant.sku }}  | {{ variant.price }} <br />
{% endfor %}

billybonks

for f in **/*.ts; do 
    mv -- "$f" "${f%.ts}.js"
done

for f in **/*.js; do
    mv -- "$f" "${f%.js}.ts"
done

billybonks

Space Jam

billybonks

Build an application that consumes/accepts a CSV (Comma Separated Values) file of N-Rows. Each Row is comprised of:

• object_id: is a unique identifier for a specific object.
• object_type: denotes the object type.
• timestamp: needs no explanation
• object_changes: serialized json comprised of properties that changed at timestamp, and their accompanying values.

Here's an example CSV:

object_id,object_type,timestamp,object_changes

billybonks

Goal

We want to build a MVP of a battleships grid. If you have not played battleships here is an example http://en.battleship-game.org/

Features

The application should have 3 routes.

• A home page where i can choose to either enter: placement test route, or test game route.

Placement test route

• When i enter the placement test route it should automatically place the Ships.