0-intro-tracing.ipynb

0-intro-tracing.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/caleb-kaiser/5b1d7f2f019d0ee4be299eda4bf1050a/0-intro-tracing.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/comet-ml/opik/main/apps/opik-documentation/documentation/static/img/opik-logo.svg\" width=\"250\"/>"
      ],
      "metadata": {
        "id": "ALLVb0Gl10XR"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3L95CZYXbKMT"
      },
      "source": [
        "# Logging Traces with the Open AI Integration\n",
        "\n",
        "In this exercise, you'll log some basic traces with Opik. You can use OpenAI or open source models via LiteLLM. You can find [the full documentation for Opik's integration with OpenAI here](https://www.comet.com/docs/opik/tracing/integrations/openai). You can find [the full documentation for Opik's integration with LiteLLM here](https://www.comet.com/docs/opik/cookbook/litellm)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rZC2yFjVbKMV"
      },
      "source": [
        "# Imports & Configuration"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "26FJE2wtbKMU"
      },
      "outputs": [],
      "source": [
        "%pip install opik openai --quiet"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import opik\n",
        "import os\n",
        "import getpass\n",
        "\n",
        "# Define project name to enable tracing\n",
        "os.environ[\"OPIK_PROJECT_NAME\"] = \"logging_traces_demo\""
      ],
      "metadata": {
        "id": "a-YtJBxp87ap"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "if \"OPIK_API_KEY\" not in os.environ:\n",
        "    os.environ[\"OPIK_API_KEY\"] = getpass.getpass(\"Enter your Opik API key: \")\n",
        "\n",
        "opik.configure()"
      ],
      "metadata": {
        "id": "vLLeF-Fa9Mai"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "if \"OPENAI_API_KEY\" not in os.environ:\n",
        "    os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")"
      ],
      "metadata": {
        "id": "QovBOUFNqZF3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5zWLzUfibKMW"
      },
      "source": [
        "# Tracking OpenAI Calls"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xbG-lxs-bKMW"
      },
      "outputs": [],
      "source": [
        "from opik.integrations.openai import track_openai\n",
        "from openai import OpenAI\n",
        "\n",
        "openai_client = OpenAI(api_key=os.getenv(\"OPENAI_API_KEY\"))\n",
        "openai_client = track_openai(openai_client)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "prompt=\"Hello, world!\"\n",
        "\n",
        "response = openai_client.chat.completions.create(\n",
        "    model=\"gpt-3.5-turbo\",\n",
        "    messages=[\n",
        "      {\"role\":\"user\", \"content\":prompt}\n",
        "    ],\n",
        "    temperature=0.7,\n",
        "    max_tokens=100,\n",
        "    top_p=1,\n",
        "    frequency_penalty=0,\n",
        "    presence_penalty=0\n",
        ")\n",
        "\n",
        "print(response.choices[0].message.content)"
      ],
      "metadata": {
        "id": "3hP2oybz9beN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Use Open Source Models With LiteLLM\n",
        "Opik also integrates with LiteLLM, which allows you to use free open-source models and supports LLM APIs from all of the major providers (Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, Groq, etc.) using the OpenAI format. [See here for a full list of LLM providers supported by LiteLLM as well as how to call them.](https://docs.litellm.ai/docs/providers)\n",
        "\n",
        "In the following example we'll use Meta's `Llama-3.1-8B-Instruct` model hosted on the Hugging Face hub.\n",
        "\n",
        "**If you have already run the OpenAI code above, you will need to restart your kernel before running the following code**"
      ],
      "metadata": {
        "id": "yg8N6_8Abd-g"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zvhvjuQBbKMW"
      },
      "outputs": [],
      "source": [
        "%pip install opik litellm --quiet"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import opik\n",
        "import os\n",
        "import getpass\n",
        "\n",
        "os.environ[\"OPIK_PROJECT_NAME\"] = \"logging-traces-litellm\"\n",
        "\n",
        "if \"OPIK_API_KEY\" not in os.environ:\n",
        "    os.environ[\"OPIK_API_KEY\"] = getpass.getpass(\"Enter your Opik API key: \")\n",
        "\n",
        "opik.configure()"
      ],
      "metadata": {
        "id": "XMf1_TCHfMc1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iouHRlmJbKMX"
      },
      "outputs": [],
      "source": [
        "from litellm.integrations.opik.opik import OpikLogger\n",
        "from opik.opik_context import get_current_span_data\n",
        "from opik import track\n",
        "import litellm\n",
        "\n",
        "opik_logger = OpikLogger()\n",
        "# In order to log LiteLLM traces to Opik, you will need to set the Opik callback\n",
        "litellm.callbacks = [opik_logger]"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "import getpass\n",
        "\n",
        "if \"HF_TOKEN\" not in os.environ:\n",
        "    os.environ[\"HF_TOKEN\"] = getpass.getpass(\"Enter your Hugging Face API key: \")"
      ],
      "metadata": {
        "id": "fdWUie6kroz_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vL8Ze4ShbKMX"
      },
      "outputs": [],
      "source": [
        "messages = [{ \"content\": \"There's a llama in my garden 😱 What should I do?\",\"role\": \"user\"}]\n",
        "\n",
        "response = litellm.completion(\n",
        "    model=\"huggingface/meta-llama/Meta-Llama-3.1-8B-Instruct\",\n",
        "    messages=messages\n",
        ")\n",
        "\n",
        "print(response.choices[0].message.content)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "comet-eval",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.15"
    },
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "rZC2yFjVbKMV",
        "5zWLzUfibKMW",
        "yg8N6_8Abd-g"
      ],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}