decagondev

Vectorizing and Indexing Structured JSON for RAG

Introduction

Retrieval-Augmented Generation (RAG) systems rely heavily on high-quality, efficiently retrievable vector embeddings. Using structured JSON as a source for vectorization can be very effective—provided the structure is leveraged appropriately.

This document outlines best practices, potential pitfalls, and implementation examples for vectorizing and indexing structured JSON data, with an emphasis on downstream use in RAG pipelines.

When is JSON a Good Source for Vectorization?

JSON is a great candidate for vectorization if:

• The schema is consistent across entries.

decagondev

Utility Profit Workflow Scripts

• initial_curls.sh (script to perform a flow of curls to get the initial json data from the api)

• merge_forms.py (script to merge the json data files in to one that can be sent to ombiform)

  Merge Process

flowchart TD

decagondev

Project Plan: AI-Powered Iterative Report Writer using LangGraph

Overview

This plan outlines the development steps, timelines, and responsibilities for building the AI-powered report writing and feedback system using LangGraph. The system will iterate over reports to enhance quality based on user-defined iteration counts.

---

Phase 1: Project Setup

Tasks

• Create GitHub repository and define branch strategy

decagondev

Product Requirements Document (PRD)

Project Title

AI-Powered Iterative Report Writer using LangGraph

---

Objective

Develop an AI system using LangGraph that automates the process of writing a report, providing feedback on the report, and rewriting it iteratively for a user-defined number of times (n). The graph will consist of modular LangGraph nodes that interact to write, critique, and improve the report in a stateful, traceable workflow.

decagondev

In a LangGraph-based multi-agent setup, when a researcher agent produces an output, this output is passed into a supervisor agent. The supervisor uses the output to determine which edge of the graph to traverse next. This often involves wrapping the researcher's output in a structured message or passing it as part of a system prompt.

---

🔄 How Context Propagation Works

• Step 1: Researcher agent queries tools (e.g., Tavily) and returns a summary.
• Step 2: This summary becomes the input context for the supervisor.
• Step 3: The supervisor LLM is prompted with this context + a system prompt asking for decision routing.
• Step 4: Based on reasoning, the supervisor picks the next step in the graph.

decagondev

LangGraph is absolutely usable in user-facing applications — but certain patterns and architectural strategies help make it more responsive. When full runs take upwards of 76 seconds, the key is handling perceived latency through streaming, asynchronous execution, or background task management.

---

⏱️ Is LangGraph Too Slow for UX?

No, but raw sequential execution without streaming or feedback can lead to poor UX. For responsive UIs, consider:

• Streaming partial results (especially from LLMs)
• Background execution + progress polling

decagondev

Using LangSmith with LangGraph enables robust observability in multi-agent systems, especially when teams (like research teams using tools such as Tavily) are coordinated via a supervisor agent. Observability ensures transparency, debuggability, and traceability of workflows across the graph.

---

🧠 Why Use Observability in a LangGraph?

• Debug execution flows across agents and tools.
• Visualize agent decisions, tool calls, and responses.
• Track latency, success/failure, and metadata.
• Compare behavior across runs or task types.

decagondev

When designing agent specialization through system prompts, the choice between hardcoding prompts at setup and dynamically generating them at runtime often depends on the required flexibility, interpretability, and control.

Strategy 1: Hardcoded System Prompts (Static)

• Use Case: When agents serve stable, well-defined roles.
• Advantages:
  • Easy to audit and debug.
  • Predictable behavior.
  • Simpler deployment.
• Disadvantages:

decagondev

When building a supervisor for a system that includes a large language model (LLM), the preferred approach usually depends on the requirements for reliability, transparency, and control.

Preferred Approach: Deterministic Logic Outside the Graph

• Reason: Deterministic logic offers predictable and debuggable behavior, which is crucial when supervising, orchestrating, or enforcing policies around the use of LLMs.
• Use Cases: Supervising task delegation, routing, retries, safety checks, compliance enforcement, logging, and fallback mechanisms.
• Advantages:
  • Easier to test and verify.
  • Transparent decision-making.
  • Easier integration with existing systems.

decagondev

In the context of systems involving large language models (LLMs), the aspect that is typically defined as "artificial intelligence" (AI) depends on the scope and intent of the definition being applied. Here's a breakdown:

1. The LLM Itself

• Defined as AI: Yes.
• Reason: A large language model, such as GPT, is considered AI because it demonstrates capabilities associated with intelligent behavior, such as understanding and generating human-like language, summarizing information, answering questions, and even reasoning within probabilistic and learned parameters.

2. The Deterministic State Machine for Preprocessing Input

• Defined as AI: Not typically.
• Reason: A deterministic state machine follows fixed, rule-based logic without learning or adapting. It is considered a traditional programming construct, not AI, because it lacks traits like learning, adaptation, or probabilistic inference.