Research Summary on Small and Large Language Models

# Small and Large Language Models

## Research Question
Understanding the distinctions, collaboration, performance, and application of small language models (SLMs) versus large language models (LLMs).

## Summary of Work

1. **Dynamic Scoring with Enhanced Semantics for Training-Free HOI Detection** explores Vision-Language models leveraging small and large features for interaction detection without training, emphasizing multi-head attention to fuse visual and textual features.

2. **DynaSearcher** proposes a dynamic knowledge graph augmented search agent using multi-reward reinforcement learning. It uses small models effectively for retrieval tasks, achieving results comparable to large models but with fewer resources.

3. **R-Stitch** presents a hybrid decoding method switching between small and large LMs during chain-of-thought reasoning, improving efficiency by using the small model by default and large model when low confidence is detected.

4. **R4ec** introduces a reasoning, reflection, and refinement framework that helps LLMs perform complex recommendation tasks by evolving towards System-2 like thinking, augmenting model capabilities.

5. **Reinforcement Learning Fine-Tunes a Sparse Subnetwork in LLMs** shows that RL updates only a small subnetwork of a large model, questioning the assumption that fine-tuning modifies most parameters.

6. **Evaluating Nationality Bias in LLMs via Name-Based Benchmarks** highlights that small models exhibit more bias and less accuracy compared to large models, underlining the robustness of larger LLMs in ambiguous contexts.

7. **Collaborative Inference and Learning between Edge SLMs and Cloud LLMs** surveys techniques for edge-cloud collaboration where small and large LMs interact for improved latency, privacy, cost, and personalization during inference and training.

8. **ReMeREC** advances multi-entity referring expression comprehension by jointly leveraging visual and textual cues using large LMs to model inter-entity relations.

9. **Re:Form** explores reducing human priors in formal software verification via reinforcement learning in LLMs, achieving strong results even with smaller models.

10. **Beyond Label Semantics for Few-shot Action Recognition** utilizes LLMs to dissect action labels into atomic components, combining with visual features for improved performance in few-shot learning.

## Papers

- Dynamic Scoring with Enhanced Semantics for Training-Free Human-Object Interaction Detection (arXiv:2507.17456v1)
- DynaSearcher: Dynamic Knowledge Graph Augmented Search Agent via Multi-Reward Reinforcement Learning (arXiv:2507.17365v1)
- R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning (arXiv:2507.17307v1)
- R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systems (arXiv:2507.17249v1)
- Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Models (arXiv:2507.17107v1)
- Obscured but Not Erased: Evaluating Nationality Bias in LLMs via Name-Based Bias Benchmarks (arXiv:2507.16989v1)
- Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges (arXiv:2507.16731v1)
- ReMeREC: Relation-aware and Multi-entity Referring Expression Comprehension (arXiv:2507.16877v1)
- Re:Form -- Reducing Human Priors in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny (arXiv:2507.16331v1)
- Beyond Label Semantics: Language-Guided Action Anatomy for Few-shot Action Recognition (arXiv:2507.16287v1)

[View Papers on arXiv](https://arxiv.org/search/?query=small+language+models+large+language+models&searchtype=all&abstracts=show&order=-announced_date_first&size=50)