Skip to content

Instantly share code, notes, and snippets.

@joenandez

joenandez/implementation_phase4_1.md

Created February 15, 2025 00:02
Show Gist options
  • Save joenandez/8299555fb3cb22bbbb8a8254f220620d to your computer and use it in GitHub Desktop.
Save joenandez/8299555fb3cb22bbbb8a8254f220620d to your computer and use it in GitHub Desktop.
Download ZIP
Phase 4
Raw
implementation_phase4_1.md

Phase 4: Advanced AI Integration with Vercel AI SDK Tool Calling

Summary

In Phase 4, we will expand upon the builder workflow to integrate Vercel AI SDK–based tool calling. Our goal is for each workflow stage (brainstorm vs. review, etc.) to be able to autonomously create or update notes (e.g., riff.md, prd.md) using tool calls while keeping the entire AI output in the chat thread. We'll rely on multi-step logic (maxSteps) to let the model call a tool, get the result, and then produce final text. We'll also pass “invisible” system messages to the AI, instructing it on how to create or edit a doc.

Key additions:

  1. Tool set definitions (like createDocument, updateDocument) using tool from ai.
  2. Invisibly instructing the AI (system messages) at each new stage or section, so it can spontaneously produce partial or final doc text.
  3. Multi-doc support in a single workspace (as user transitions from note.md to riff.md to prd.md, etc.).
  4. Display entire AI response in the thread while storing doc content in the note.

Data Flow

  1. User transitions to a new builder stage or section, e.g., from “brainstorming” to “review.”
  2. Frontend or workspace logic calls our chat route (like /api/threads/[threadId]/chat) with:
    • The user’s existing conversation (messages).
    • A special system message with instructions from workflow.json (e.g., “Create Riff doc”).
    • The new tool definitions (like createDocument or updateDocument).
    • maxSteps > 1 to allow the model to do multiple calls: a tool call plus a concluding message.
  3. The Vercel AI SDK identifies if the model wants to call createDocument or just produce text. If it calls the tool, the SDK’s execute function runs, adding or updating a note in the database.
  4. The model sees the tool result and can produce final text acknowledging the doc creation.
  5. The entire chain of calls and text is returned, and we store the final AI message in the chat, while the doc text goes into the new or updated note.

Execution Tasks

[ ] Task 4.1.1: ADD mode Key and “Tool” Fields to workflow.json

  • What
    ADD a "mode" field (like "brainstorm" or "review") to each workflow section in workflow.json to help shape our system instructions. Optionally list or reference the tool names we expect the model to call (e.g., "createDocument").

  • How

    1. FILE: src/config/workflow.json
    2. ADD "mode" in each section, e.g.: 
      {
  "type": "brainstorming",
  "mode": "brainstorm",
  "inputs": ["note.md"],
  "output": "Riff.md",
  "button_cta": "Generate Riff Doc",
  ...
}
    3. Provide clarity in the JSON if you want to mention the relevant tool names, e.g. "tools": ["createDocument"].

  • Verification

    • Confirm workflow.json loads with no parse errors.
    • Confirm each stage/section has a mode.

  • Progress
    (Leave blank)

[ ] Task 4.1.2: CREATE Tool Definitions in a “toolDefs.ts” Using Vercel AI tool() Helper

  • What
    CREATE a tool set that includes createDocument and updateDocument using Zod or JSON schema to define parameters. This is how we let the model spontaneously call these tools to manipulate notes.

  • How

    1. FILE: src/lib/ai/toolDefs.ts (or a suitable folder).
    2. CODE SAMPLE: 
      import { tool } from 'ai'
import { z } from 'zod'
import { createOrUpdateNote } from '@/lib/notesClient' // for DB logic

export const createDocument = tool({
  description: 'Create a new doc in the workspace',
  parameters: z.object({
    docName: z.string().describe('Filename of the doc to create'),
    docContent: z.string().describe('Content of the doc')
  }),
  async execute({ docName, docContent }, { messages, toolCallId }) {
    // For example, we parse out workspace from the conversation or from function context
    // Then call DB logic:
    await createOrUpdateNote('someWorkspaceId', docName, docContent)
    return `Doc "${docName}" successfully created.`
  },
})

export const updateDocument = tool({
  description: 'Update an existing doc in the workspace',
  parameters: z.object({
    docName: z.string(),
    docContent: z.string()
  }),
  async execute({ docName, docContent }) {
    await createOrUpdateNote('someWorkspaceId', docName, docContent)
    return `Doc "${docName}" updated successfully.`
  },
})

export const myToolSet = {
  createDocument,
  updateDocument
}
    3. Export myToolSet to be used in your chat route.

  • Verification

    • Confirm the file compiles, that you can import myToolSet with no type errors.
    • Possibly do a quick local test by calling createDocument.execute({ docName:'test.md', docContent:'Hello' }, ...).

  • Progress
    (Leave blank)

[ ] Task 4.1.3: REVISE Chat Route to Use Vercel AI streamText or generateText with Tools

  • What
    REVISE /api/threads/[threadId]/chat/route.ts to import your myToolSet and pass them into streamText or generateText along with the user’s prompt and prior messages.

  • How

    1. FILE: src/app/api/threads/[threadId]/chat/route.ts
    2. CODE SAMPLE: 
      import { streamText } from 'ai'
import { myToolSet } from '@/lib/ai/toolDefs'

export async function POST(req: Request, { params }) {
  // parse body, load conversation, etc.

  const result = streamText({
    model: openai('gpt-4-turbo'),
    messages,
    tools: myToolSet,
    maxSteps: 5,  // let the model do multi-step calls
    prompt: "Your combined system prompt here", 
    onFinish() {
      // handle finalizing the response
    }
  })
  return result.toDataStreamResponse()
}
    3. Provide any additional onStepFinish, or onFunctionCall logic as needed.

  • Verification

    • Confirm no compile errors.
    • On user conversation, watch logs to see if the AI calls createDocument or updateDocument.

  • Progress
    (Leave blank)

[ ] Task 4.1.4: ALLOW “Invisible” System Prompts on Stage/Section Start

  • What
    UPDATE your code so that upon user advancing from brainstorming → review, you send a system message instructing the model on what to do for that new step. This can be an empty user message or purely a system role message.

  • How

    1. Possibly do this in WorkspaceContainer.tsx or in a workflow function that calls /api/threads/[threadId]/chat: 
      // Example call with only system:
const messages = [
  ...existingThreadMessages,
  {
    role: 'system',
    content: `You are now in review mode. The doc is: "riff.md". Please refine it.`
  }
]
const response = await fetch(`/api/threads/${threadId}/chat`, {...})
    2. The AI sees that system instruction and can produce text or call a tool.

  • Verification

    • Transition sections.
    • Check that the AI spontaneously does something (e.g., calls updateDocument) without user input.
    • The user sees the final text in the chat, doc updates in the workspace.

  • Progress
    (Leave blank)

[ ] Task 4.1.5: HANDLE Partial or Full AI Doc in the Chat (If No Function Is Called)

  • What
    If the AI just returns text without calling a tool, but the text includes doc content, we can parse it from the text.

  • How

    1. In the onFinish or onStepFinish callback, examine the final assistant message.
    2. Look for doc delimiters, e.g. [DOC_CONTENT_START] / [DOC_CONTENT_END].
    3. If found, call createOrUpdateNote(workspaceId, stageOutput, content).

  • Verification

    • AI returns text “Here is your doc: [DOC_CONTENT_START]some text[DOC_CONTENT_END].”
    • You confirm the doc was updated in the DB and the user sees it in the workspace.

  • Progress
    (Leave blank)

[ ] Task 4.1.6: CREATE or UPDATE “createOrUpdateNote” Utility

  • What
    A function that, given workspaceId, docName, docContent, either creates or updates the relevant note row.

  • How

    1. FILE: src/lib/notesClient.ts or src/lib/workspaces/noteOps.ts
    2. Pseudo-code: 
      export async function createOrUpdateNote(
  workspaceId: string,
  docName: string,
  docContent: string
) {
  // 1) query DB for existing note with (workspace_id, title=docName)
  // 2) if found -> update content
  // 3) else -> create new note
  // 4) optionally store local indexDB or context
}

  • Verification

    • Confirm it modifies or creates a note row in DB.
    • Possibly test with direct calls: await createOrUpdateNote('id123','riff.md','Hello world').

  • Progress
    (Leave blank)

[ ] Task 4.1.7: DISPLAY New or Updated Docs in UI (Sidebar / Editor)

  • What
    UPDATE the UI so that newly generated docs appear in the workspace.

  • How

    1. FILE: src/components/Sidebar.tsx or ContentDrawer.tsx, or a “notes” listing component.
    2. When a doc is created, call fetchNotes() or update local state: 
      // e.g. after onFinish -> notesContext.fetchNotes()
// or notesContext.addNote(newNote) if you have a custom method

  • Verification

    • The newly created doc (like riff.md) shows up in the sidebar.
    • Clicking opens it in NoteEditor.

  • Progress
    (Leave blank)

[ ] Task 4.1.8: ADJUST Chat UI to Distinguish Tool Calls from Normal AI Messages

  • What
    If the AI calls createDocument or any other tool, we might want to show a user-friendly “Doc created” text instead of raw JSON function calls.

  • How

    1. FILE: src/components/threads/ThreadPanel.tsx or your message rendering.
    2. Inside the final response data (response.messages from streamText), look for messages of type 'tool' or 'tool-result'.
    3. Format them as user-friendly messages: “I created doc X.”

  • Verification

    • The user sees a normal chat bubble with “Doc 'riff.md' created.”
    • No messy function JSON is exposed.

  • Progress
    (Leave blank)

[ ] Task 4.1.9: USE maxSteps & onStepFinish to Enable Multi-Step AI > Tool > AI Patterns

  • What
    Let the model call a tool and then produce a final text summary in the same request.

  • How

    1. In your chat route, use: 
      const result = await generateText({
  model: openai('gpt-4-turbo'),
  tools: myToolSet,
  messages: userMessages,
  maxSteps: 5,
  onStepFinish({ text, toolCalls, toolResults }) {
    // optionally handle partial steps (like logging)
  }
})
    2. This ensures the LLM can do:
      • Step 1: produce a tool call → createDocument
      • Step 2: see the doc creation result, then produce final text to the user

  • Verification

    • The model can produce “I’m going to create a doc.” Then calls the tool. Then produce final text.
    • You see the entire chain in steps.

  • Progress
    (Leave blank)

[ ] Task 4.1.10: REVISE “Invisible” Prompting for Stage Start to Use toolChoice or toolCallRepair (If Desired)

  • What
    If you want to ensure the model must call a tool or cannot call a tool, you can use toolChoice. Also consider the experimental_repairToolCall if the model calls a tool with invalid arguments.

  • How

    1. In your chat route: 
      const response = await generateText({
  model: openai('gpt-4-turbo'),
  messages,
  tools: myToolSet,
  toolChoice: 'auto', // or 'required', 'none'
  experimental_repairToolCall: async (options) => {
    // your logic to fix invalid arguments
  },
  maxSteps: 4,
})
    2. If the model tries to pass invalid arguments to createDocument, you can attempt a fix or let it fail.

  • Verification

    • If the AI produces invalid arguments, confirm you either fix them or see a controlled error.

  • Progress
    (Leave blank)

[ ] Task 4.1.11: (Optional) ADD Additional Tools (e.g. Summaries, Query External APIs)

  • What
    If you want the model to do more tasks (like calling an external knowledge base) or re-check doc content, you can add more tools.

  • How

    1. Follow the same pattern with tool().
    2. Provide a parameters Zod schema.
    3. Provide execute() logic.

  • Verification

    • The model can do multi-step calls: e.g. external knowledge → create doc.

  • Progress
    (Leave blank)

Documentation Task

[ ] Task 4.1.12: CREATE “Tool Calling Integration” Dev Doc

  • What
    CREATE a markdown doc explaining how we incorporate Vercel AI SDK tools (like createDocument, updateDocument) in multi-step chat.

  • How

    1. FILE: docs/Phase4_ToolIntegration.md
    2. Outline:
      • Why we use the tool() helper
      • Example code for myToolSet
      • maxSteps usage
      • onStepFinish
      • Handling function calls in the chat interface

  • Verification

    • A code review or peer check ensures the doc is accurate.

  • Progress
    (Leave blank)

Manual Testing Task

[ ] Task 4.1.13: CREATE a Thorough “Tool Calling & Multi-Doc” Manual Test

  • What
    BUILD a test checklist verifying:

    1. AI calls createDocument on brainstorming.
    2. AI calls updateDocument on review.
    3. If the doc is generated in the AI’s text alone, we parse [DOC_CONTENT_START]....
    4. The user sees a normal “assistant” bubble.
    5. The doc is indeed created in the DB.
    6. The doc is visible in the workspace’s sidebar or drawer.

  • How

    1. FILE: docs/Phase4_ManualTest.md
    2. Provide step-by-step instructions (like “Open the app, start build workflow, pick Brainstorm for RIFF, watch logs to confirm function call,” etc.).

  • Verification

    • Dev or QA runs these steps in a test environment, verifying each doc is properly created or updated.

  • Progress
    (Leave blank)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment