Bootstrap Labels with GPT-4

Bootstrap Labels with GPT-4.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bootstrapping Labels using GPT-4\n",
    "This is a quick notebook to show how you can use the OpenAI API to bootstrap labels for your data. In this case, we'll be labeling sentiment examples. Before you begin, you will need an [OpenAI account](https://platform.openai.com/) to obtain an API key. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install OpenAI Python client\n",
    "!pip install openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai\n",
    "import re\n",
    "\n",
    "openai.api_key = \"<YOUR_API_KEY>\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is our function to create a prompt and request for the GPT models to respond to. Note we are using GPT-3.5-turbo here, but we can replace that model with GPT-4 as well. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_sentiment(input_text):\n",
    "    prompt = f\"Respond in the json format: {{'response': sentiment_classification}}\\nText: {input_text}\\nSentiment (positive, neutral, negative):\"\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=\"gpt-3.5-turbo\",\n",
    "        messages=[\n",
    "            {\"role\": \"user\", \"content\": prompt}\n",
    "        ],\n",
    "        max_tokens=40,\n",
    "        n=1,\n",
    "        stop=None,\n",
    "        temperature=0.5,\n",
    "    )\n",
    "    response_text =  response.choices[0].message['content'].strip()\n",
    "    sentiment = re.search(\"negative|neutral|positive\", response_text).group(0)\n",
    "    # Add input_text back in for the result\n",
    "    return {\"text\": input_text, \"response\": sentiment}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result\n",
      " {'text': 'I had a terrible time at the party last night!', 'response': 'negative'}\n"
     ]
    }
   ],
   "source": [
    "# Test single example\n",
    "sample_text = \"I had a terrible time at the party last night!\"\n",
    "sentiment = get_sentiment(sample_text)\n",
    "print(\"Result\\n\",f\"{sentiment}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To verify this data, we'll use the following function to convert the results into the Label Studio format. Then we can load the data into Label Studio for human evaluation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convert_ls_format(input_dict):\n",
    "    \"\"\"\n",
    "    Convert sentiment analysis output from a simple format to Label Studio's prediction format.\n",
    "    \n",
    "    Args:\n",
    "        input_dict (dict): A dictionary containing text and response keys. Example:\n",
    "            {\n",
    "                \"text\": \"I love going to the park on a sunny day.\",\n",
    "                \"response\": \"positive\"\n",
    "            }\n",
    "            \n",
    "    Returns:\n",
    "        dict: A dictionary in Label Studio's prediction format. \n",
    "    \"\"\"\n",
    "    \n",
    "    score_value = 1.00  # We don't know the model confidence\n",
    "    output_dict = {\n",
    "        \"data\": {\n",
    "            \"text\": input_dict[\"text\"]\n",
    "        },\n",
    "        \"predictions\": [\n",
    "            {\n",
    "                \"result\": [\n",
    "                    {\n",
    "                        \"value\": {\n",
    "                            \"choices\": [\n",
    "                                input_dict[\"response\"].capitalize()\n",
    "                            ]\n",
    "                        },\n",
    "                        \"from_name\": \"sentiment\",\n",
    "                        \"to_name\": \"text\",\n",
    "                        \"type\": \"choices\"\n",
    "                    }\n",
    "                ],\n",
    "                \"score\": score_value,\n",
    "                \"model_version\": \"gpt-3.5-turbo\"\n",
    "            }\n",
    "        ]\n",
    "    }\n",
    "    return output_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data': {'text': 'I had a terrible time at the party last night!'}, 'predictions': [{'result': [{'value': {'choices': ['Negative']}, 'from_name': 'sentiment', 'to_name': 'text', 'type': 'choices'}], 'score': 1.0, 'model_version': 'gpt-3.5-turbo'}]}\n"
     ]
    }
   ],
   "source": [
    "# Convert to Label Studio format\n",
    "print(convert_ls_format(sentiment))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Putting it all together, we'll create a file with some examples in it (one per line) that we'll evaluate the sentiment on. Finally, we'll write all of our examples out to a file which can be imported by Label Studio. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting input_texts.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile input_texts.txt\n",
    "I love going to the park on a sunny day.\n",
    "The customer service was terrible; they were rude and unhelpful.\n",
    "I am neither happy nor sad about the new policy changes.\n",
    "The cake was delicious and the presentation was fantastic.\n",
    "I had a really bad experience with the product; it broke after two days."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "input_file_path = \"input_texts.txt\"\n",
    "output_file_path = \"output_responses.json\"\n",
    "\n",
    "with open(input_file_path, \"r\") as input_file, open(output_file_path, \"w\") as output_file:\n",
    "    examples = []\n",
    "    for line in input_file:\n",
    "        text = line.strip()\n",
    "        if text:\n",
    "            examples.append(convert_ls_format(get_sentiment(text)))\n",
    "    output_file.write(json.dumps(examples))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}