Skip to content

Instantly share code, notes, and snippets.

@skryvets

skryvets/jconf-2024-building-custom-ai-chatbot-with-spring-boot-react-LangChain4j.md

Last active September 27, 2024 00:42
Show Gist options
  • Save skryvets/23deb010134c3c1b6971c0f77af2480e to your computer and use it in GitHub Desktop.
Save skryvets/23deb010134c3c1b6971c0f77af2480e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
jconf-2024-building-custom-ai-chatbot-with-spring-boot-react-LangChain4j.md

Building a Custom AI Chatbot with Spring Boot, React, and LangChain4j

Speaker: Marcus Hellberg

YouTube Link: Watch the video
GitHub Repository: java-ai-playground

Key Concepts and Insights

Vaading Perspective

  • LLMs (Large Language Models) are generic, but your business is unique.

Tools for Java Developers to Build AI-Powered Applications

Incorporating LLMs into Java applications requires several tools and components to manage resources efficiently:

  • Tools (Programs): Manage various AI processing functions.
  • Vector Store (Disk): Stores embeddings on disk or other storage mediums for retrieval.
  • CPU (Processing): Powers the execution of LLMs.
  • Search Tools (Ethernet): Helps retrieve relevant information.
  • Other LLMs: Can be integrated to provide more advanced functionalities.
  • Context Window (RAM): Controls how much context is retained and processed during interactions.

IMG_1187

AI for Engineers: APIs and Libraries

  • LangChain4j: A Java-based AI library facilitating LLM integrations.
  • Hilla: A full-stack framework for building modern web applications, combining Spring Boot on the server side and TypeScript on the client side. It simplifies the development of reactive web apps.

Agent Autonomy Levels

  1. Chatbot:
    • Knows a lot of things but lacks specific domain expertise.
  2. Copilot:
    • Acts on your behalf in predefined scenarios.
    • Monitors and assists users but doesn’t control the entire workflow.
    • Most business apps will fall into this category.
  3. Fully Autonomous: Fully capable of making decisions without human intervention.

IMG_1188

Context Window

  • A context window is essential for determining the amount of information processed by the LLM in one session.
  • Components include:
    • System Prompt: Sets the behavior or identity of the AI.
    • History: Previous interactions.
    • Prompt: The user's input.
    • Relevant Information: Critical to the specific task at hand.
    • Response Space: Ensuring enough space is left for the AI's response.

Tokens

  • Definition: A token is a chunk of text (ranging from a character to a word) used for processing by the AI.
  • Token Count: The token count is typically about 25% greater than the word count.

IMG_1189

RAG (Retrieval Augmented Generation)

  • Definition: A method where the AI retrieves specific case-related information before generating responses.
  • LLMs Know Two Things:
    1. Content they’re trained on.
    2. Specific information retrieved via RAG.

Embedding Model

  • Purpose: Transforms text into vectors for processing.
  • Vector Store: Essential for storing and retrieving relevant vectorized information.

LangChain4j Specifics

  • LangChain Interface: Works similarly to Spring Data JPA, allowing developers to query LLMs.
  • Reactive Nature: LangChain allows real-time token streaming during responses.

Key Components

  1. System Prompt: Defines a new AI identity for a session.
  2. Config: Provides access to the necessary data and configuration parameters.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment