roh26it

Portkey Integration

Portkey.ai is a full-stack LLMOps platform to monitor & log all anyscale requests for latency, costs and accuracy. It also provides capabilities to use multiple models from different providers and setup load balancer, fallbacks and a/b tests between them.

To use Portkey with Anyscale endpoints, use the python SDK.

!pip install portkey-ai

lavishsaluja

"""
Description: code file to get required JSONs for various applications using Open AI large language models

How-to-use:
1. replace the following variables for your use case
    - sample_app_description
    - sample_user_messages
    - supported_animal_list
    - json_examples
    - json_key_value_descriptions