Basic google LLM API quickstart and example

README.md

See: https://cloud.google.com/vertex-ai/docs/generative-ai/chat/test-chat-prompts

You'll need:

1. A project id
2. An installed and working gcloud command-line tool
3. A service account that has been granted access to vertex AI
4. A json key file associated with the service account
5. Set up local environment to use the json key for authentication

Tasks 1, 3, and 4 can be performed using the console OR using gcloud by a user authorized to create resources. Note that the end user does NOT need access to the GCP console or even need to be an editor/owner on the project.

Then, perform the curl below. Alternatively, you can use python or even the vertex AI python sdk.

curl_output.json

{
  "predictions": [
    {
      "citationMetadata": [
        {
          "citations": []
        }
      ],
      "safetyAttributes": [
        {
          "categories": [],
          "scores": [],
          "blocked": false
        }
      ],
      "candidates": [
        {
          "author": "1",
          "content": "Yes, Lord of the Rings and Hobbit are both based on a book series."
        }
      ]
    }
  ],
  "metadata": {
    "tokenMetadata": {
      "inputTokenCount": {
        "totalBillableCharacters": 182,
        "totalTokens": 54
      },
      "outputTokenCount": {
        "totalBillableCharacters": 53,
        "totalTokens": 16
      }
    }
  }
}

perform_chat.sh

#!/bin/bash
MODEL_ID="chat-bison"
# replace project id here
PROJECT_ID="YOUR PROJECT ID HERE"


curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:predict -d \
'{
  "instances": [{
      "context":  "My name is Ned. You are my personal assistant. My favorite movies are Lord of the Rings and Hobbit.",
      "examples": [ {
          "input": {"content": "Who do you work for?"},
          "output": {"content": "I work for Ned."}
      },
      {
          "input": {"content": "What do I like?"},
          "output": {"content": "Ned likes watching movies."}
      }],
      "messages": [
      {
          "author": "user",
          "content": "Are my favorite movies based on a book series?",
      }],
  }],
  "parameters": {
    "temperature": 0.3,
    "maxOutputTokens": 200,
    "topP": 0.8,
    "topK": 40
  }
}'