Created
July 7, 2025 22:41
-
-
Save jlewi/8efe8c6424cb8397aff593dc650fd4a5 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| kind: EvalSample | |
| metadata: | |
| name: list_clusters | |
| input_text: List all my az cluster clusters. | |
| assertions: | |
| - name: file_retrieval_tool_invocation | |
| type: TYPE_TOOL_INVOKED | |
| tool_invocation: | |
| tool_name: file_retrieval | |
| - name: shell_tool_invocation | |
| type: TYPE_TOOL_INVOKED | |
| tool_invocation: | |
| tool_name: shell | |
| - name: l1 | |
| type: TYPE_LLM_JUDGE | |
| llm_judge: | |
| prompt: "Does the answer use az aks list?" | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@sourishkrout here you go