chat.md

Theory	Description
Collaborative Intelligence	Combining the outputs of various models through a structured process of proposals and aggregations to enhance performance.
Iterative Refinement	Each layer of LLM agents refines the outputs from the previous layer to improve the overall quality.
Specialization Limitation	Individual models excel in specific tasks but struggle with others, necessitating the combination of multiple models.
Soft Splits in Decision Trees	Traditional decision trees create rigid structures, while soft splits allow inputs to traverse multiple paths with certain probabilities.
Low-Rank Decomposition Methods	Techniques for model compression that create compact models with fewer parameters, enhancing efficiency.
Active Sampling	A data selection method designed to choose the most representative portion of a dataset for a specific task.
Bounded Rationality	Limitations in computational capacity and working memory constrain decision-making processes.
Retrieval Augmented Generation (RAG)	Combining retrieval and generation methods to improve performance without extensive retraining.