| title | Captioning and Dataset Design Research Harvest |
|---|---|
| source_project | claude-comfyui |
| harvest_date | 2026-05-17 |
| domain | captioning |
| verification_level | full-academic |
| claims_total | 35 |
| claims_verified | 28 |
| claims_unverified | 5 |
| claims_suspect | 2 |
| conflicts_found | 3 |
Distilled from 7 legacy research documents covering captioning principles, trigger token design, dataset construction, vocabulary diversification, and body/face descriptor strategy for LoRA training on diffusion models with decoder-only LLM text encoders.
The legacy project produced a comprehensive captioning framework through iterative research, field testing on 359 training images (82 face, 277 body), and cross-referencing against 11+ academic papers and 10+ community guides. The framework centers on one core principle and four supporting methodologies:
-
Core principle: "Caption what you do not want to train." Uncaptioned visual features are absorbed into the LoRA weights and bound to the trigger token. Captioned features remain prompt-responsive. This principle is architecture-agnostic and transfers across model families.
-
Vocabulary diversification: Repetitive caption phrasing causes the model to overtrain on caption style before learning the subject. Lighting, expression, camera angle, and pose descriptions must be drawn from diverse vocabulary pools with per-phrase usage caps.
-
Attribute ordering for causal-attention encoders: When the text encoder uses causal attention (decoder-only LLMs like Mistral), word order determines attention weight. The trigger token placed at position 0 receives the most attention from all subsequent tokens. Identity-critical information should be front-loaded.
-
Trigger token design: BPE tokenizers fragment mixed-alphanumeric strings at digit boundaries. Leet-speak triggers (
s4ndy,n4omi) fragment into 3-4 tokens, causing DOP token count mismatch, context boundary failures, and diluted identity signal. Single-token triggers eliminate all three problems. -
Variable vs identity feature separation: Face LoRAs omit facial bone structure, eye shape, skin tone from captions (identity features). Body LoRAs omit body proportions. The face description in body LoRA captions must be exactly "a woman" to prevent interference during LoRA stacking.
Claim: Uncaptioned visual features get absorbed into LoRA weights and bound to the trigger token. Captioned features remain prompt-responsive because the text encoder "explains them away."
Mechanism: During training, the denoising loss encourages the model to predict the image from caption + LoRA weights. Features described in the caption are attributed to text conditioning. Features not described must be encoded in LoRA weights to minimize loss.
Evidence: This principle is stated independently by Pelayo Arbues ("uncaptioned elements become part of the character identity, while captioned details remain prompt-controllable"), the project's own captioning strategy document, and at least 3 additional community guides (fal.ai, Apatero, alvdansen HuggingFace blog). DisenBooth (Chen et al., 2023) formalizes the underlying entanglement problem academically.
Verification: Pelayo Arbues article confirmed via WebFetch (live, content matches). DisenBooth paper confirmed at arXiv:2305.03374 (title and identity-entanglement claims match). fal.ai guide confirmed via WebFetch (recommends natural language descriptions, notes caption files "lead to far better LoRA learning retention"). alvdansen blog confirmed via WebFetch (discusses frequency and consistency of terms signaling importance to the model).
Critical gap: No paper has experimentally isolated the "caption vs omit" variable for subject-driven fine-tuning in a controlled ablation. The principle is derived from community experimentation and theoretical reasoning about loss attribution.
Claim: When the text encoder is a decoder-only LLM (Mistral, Qwen) rather than a contrastive encoder (CLIP), natural language captions are unambiguously superior to booru-style tags. Tags force a grammar-aware model to process grammatically meaningless input, losing relational context between concepts.
Evidence: BFL prompting guide states FLUX.2 "replaced CLIP with Mistral Small 3.1, a large language model that actually understands sentence structure, context, and relationships between concepts." fal.ai recommends "natural language descriptions rather than tags." Apatero states "Flux 2's Mistral-3 text encoder understands nuanced, detailed descriptions." Wang et al. (2025, arXiv:2506.08210) found decoder-only LLMs "outperform the baseline T5 model."
Verification: BFL prompting guide confirmed via WebFetch (word order matters, Subject + Action + Style + Context ordering recommended). fal.ai guide confirmed (recommends natural language). Apatero guide confirmed ("Mistral-3 text encoder" language present). Wang et al. confirmed at arXiv:2506.08210 (title and T5 outperformance claims match).
Scope boundary: This finding applies specifically to models using decoder-only LLM text encoders. CLIP-based models (SDXL, Pony) may still benefit from tag-style captions matching their contrastive training data.
Claim: In a decoder-only LLM text encoder with causal attention, word order determines attention weight distribution. Earlier tokens receive more attention from all subsequent tokens. Placing the trigger token at position 0 maximizes its attention weight across the full sequence.
Evidence: BFL prompting guide: "word order matters -- FLUX.2 pays more attention to what comes first." This is a direct consequence of causal (left-to-right) attention: token at position 0 is in the attention window of every subsequent token, while the last token is only in its own window.
Verification: BFL prompting guide confirmed via WebFetch. The causal attention mechanism is a fundamental property of decoder-only transformer architectures and does not require model-specific verification.
Generalization: This finding transfers to any diffusion model using a decoder-only LLM text encoder with causal attention. It does not apply to encoder-only (CLIP) or encoder-decoder (T5) architectures where bidirectional attention distributes weight more evenly.
Claim: If training captions use the same sentence structure, length, and vocabulary repeatedly, the model overtrains on the caption style before learning the subject, causing concept bleeding between subjects.
Evidence: SimpleTuner Discussion #634 states: "If you constantly use the same sequence, length and style of sentences, it is very likely that Flux overtrains on that before it learns your subjects." The discussion recommends switching between "vivid and professional descriptions" and using caption complexity proportional to image complexity.
Verification: SimpleTuner Discussion #634 confirmed via WebSearch and GitHub API. The quoted warning appears in a community member's comment within the discussion (not from the SimpleTuner maintainer). The discussion's primary topic is text encoder training for FLUX LoRAs, not captioning strategy; the vocabulary diversification advice appears as incidental guidance within a comment about why trigger words cause concept bleeding. [CITATION-DEPTH: source is a community comment in a tangential discussion (#634 is titled "FLUX LoRA Training: text encoder training?"), not a dedicated captioning resource. The quoted text is accurate but the discussion topic was mischaracterized as being about vocabulary diversification. Clarified.]
Project-internal corroboration: The Sandy v1 training dataset had "gentle shadows" in 90% of captions, "soft warm lighting" in 49%, "loose flowing hair" in 44%. This extreme phrase repetition was identified as a contributing factor to weak prompt responsiveness.
Claim: The digit 4 in triggers like s4ndy and n4omi is always a
separate BPE token. BPE tokenizers never merge digits with adjacent letters
because digit-letter bigrams are extremely rare in pre-training corpora. This
causes s4ndy to fragment into 4 tokens and n4omi into 3 tokens on both
Mistral (Tekken, 131K vocab) and Qwen (byte-level BPE, 151K vocab) tokenizers.
Evidence: The legacy project ran empirical tokenizer experiments using
HuggingFace transformers library with add_special_tokens=False. Results
were consistent across Mistral-Small-3.1-24B-Instruct, Qwen3-8B, and
Qwen3-4B. This is a fundamental property of BPE tokenization: merge operations
are learned from training corpus statistics, and digit-letter transitions are
statistically rare, so they are never merged.
Verification: BPE tokenization behavior with digits is well-documented in
NLP literature. The specific tokenization of s4ndy and n4omi was verified
empirically in the legacy project. The general principle (digits fragment
mixed-alphanumeric tokens) is a known BPE property that can be verified with
any BPE tokenizer.
Claim: In ai-toolkit, Differential Output Preservation (DOP) replaces the
trigger word with the class word using Python str.replace() on the raw
caption text BEFORE tokenization. This means: (a) if the trigger is N tokens
and the class word is M tokens, the tokenized sequence length changes by
N-M tokens, shifting all subsequent positions; (b) a single-token trigger
matching the class word's token count eliminates positional shift entirely.
Evidence: ai-toolkit source code SDTrainer.py line ~1628:
dop_prompts = [p.replace(self.trigger_word,
self.train_config.diff_output_preservation_class)
for p in conditioned_prompts]Verification: DeepWiki documentation for ai-toolkit confirmed DOP's
three-pass mechanism (prior prediction with class word, training prediction
with trigger, preservation prediction with class word). The str.replace()
mechanism is confirmed in multiple community guides. Ostris (ai-toolkit
creator) stated on X: "DOP and careful captioning are what keep the LoRA 'tied'
to a trigger phrase so that non-trigger prompts stay close to the base model."
Claim: FLUX.2 dev's Mistral text encoder extracts hidden states from layers
10, 20, and 30, concatenates them, producing output shape
(batch_size, seq_len, 3 * hidden_dim). Maximum sequence length is 512 tokens.
Evidence: FLUX.2 source code text_encoder.py:
OUTPUT_LAYERS_MISTRAL = [10, 20, 30]
MAX_LENGTH = 512Verification: DeepWiki page for FLUX.2 text encoders confirmed via
WebFetch: multi-layer extraction from layers [10, 20, 30], MAX_LENGTH = 512,
output shape (batch_size, sequence_length, 3 * hidden_dim).
Claim: The Verweirder (2023) experiment that found "only v002 (detailed captions on BOTH training AND reg) prevented concept bleed" was conducted on SDXL with CLIP dual encoders, not FLUX.2/Mistral. The experimental results do not transfer directly due to fundamental architectural differences: CLIP bag-of-tokens vs Mistral causal attention, 77-token vs 512-token capacity, single-layer vs 3-layer extraction.
Evidence: The aboutme.be blog post was confirmed via WebFetch as using "SDXL 1.0 with the baked 0.9 VAE." The experiment's findings about v002 preventing concept bleed were confirmed. The SDXL/CLIP architectural differences from FLUX.2/Mistral are well-established.
Verification: Blog post confirmed live at cited URL. The correction notices in the legacy documents themselves flag this limitation, showing internal quality control was applied.
Transferability note: The directional finding (format matching helps, detailed reg captions prevent bleed) may transfer to FLUX.2 as a reasonable heuristic, but the specific experimental evidence applies only to SDXL/CLIP.
Claim: The classic DreamBooth trigger sks carries semantic meaning
(associated with a weapons brand/rifle), which can affect generation results.
HuggingFace's advanced training guide warns about this.
Evidence: HuggingFace blog sdxl_lora_advanced_script.md: "those tokens
usually have other semantic meaning associated with them and can affect your
results. The sks example, popular in the community, is actually associated with
a weapons brand." GitHub issue #71 on Dreambooth-Stable-Diffusion confirms
users observed guns appearing in generations.
Verification: HuggingFace blog confirmed via WebSearch. GitHub issue confirmed to exist at cited URL. The general principle (pre-existing token semantics affect fine-tuning) is well-established in the fine-tuning literature.
Claim: DreamBooth (Ruiz et al., 2022) established "class-specific prior preservation loss" for subject-driven fine-tuning, training on subject images with "[V] [class noun]" captions while simultaneously training on model-generated class images to prevent language drift.
Evidence: DreamBooth paper arXiv:2208.12242.
Verification: Paper confirmed at cited arXiv ID via WebFetch. Authors confirmed as Ruiz et al. The "autogenous class-specific prior preservation loss" is described in the abstract.
Claim: DisenBooth (Chen et al., 2023) formalized that "identity-relevant information and identity-irrelevant information are entangled in the latent embedding space," leading to generated images "heavily dependent on the irrelevant information."
Evidence: DisenBooth paper arXiv:2305.03374.
Verification: Paper confirmed at cited arXiv ID via WebFetch. Title "DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generation" and entanglement claims confirmed.
Claim: DECOR (Jang et al., 2024) proposed "projecting embeddings onto a vector space orthogonal to undesired token vectors" to address overfitting and content leakage during LoRA fine-tuning.
Evidence: DECOR paper arXiv:2412.09169.
Verification: Paper confirmed at cited arXiv ID via WebFetch. Title "DECOR: Decomposition and Projection of Text Embeddings for Text-to-Image Customization" and orthogonal projection claims confirmed.
Claim: T-LoRA (Soboleva et al., 2025) found that "higher diffusion timesteps are more prone to overfitting than lower ones," necessitating timestep-sensitive fine-tuning.
Evidence: T-LoRA paper arXiv:2507.05964.
Verification: Paper confirmed at cited arXiv ID via WebFetch. Authors confirmed as Soboleva et al. Timestep overfitting claim confirmed in abstract.
Relevance to captioning: Body proportions are learned at high-noise timesteps (coarse structural features), making body LoRAs more susceptible to overfitting. This connects to the observation that body LoRAs are more forgiving of imperfect captions than face LoRAs (body proportions occupy a simpler geometric manifold than facial identity).
Claim: RECAP (Segalis et al., 2023) demonstrated that recaptioning training data with higher-quality captions dramatically improved image generation: "FID 14.84 vs. the baseline of 17.87."
Evidence: RECAP paper arXiv:2310.16656.
Verification: Paper confirmed at cited arXiv ID via WebFetch. FID improvement figures confirmed. RECAP is a fine-tuning / continued training study (fine-tuned SD v1.4 for 250k additional steps on recaptioned data), not a pre-training study. [CORRECTED: paper explicitly states fine-tuning, not pre-training. See agent-003-verdicts.md, cite-011.]
Claim: Concept Sliders (Gandikota et al., 2023) demonstrated that "low-rank parameter directions corresponding to one concept" can be identified "while minimizing interference with other attributes."
Evidence: Concept Sliders paper arXiv:2311.12092.
Verification: Paper confirmed at cited arXiv ID via WebFetch. Title "Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models" and low-rank direction identification confirmed. Note: the claim of "up to 50 simultaneous adapters" was not confirmed in the abstract; the paper describes them as "plug-and-play" and "composed efficiently" without specifying a number. The legacy documents already noted this was corrected to "up to 50."
Claim: TARA (Peng et al., 2025, AAAI 2026) found token-wise interference among LoRA modules and proposed Token Focus Masking to constrain each module to focus on its associated rare token.
Evidence: TARA paper arXiv:2508.08812.
Verification: Paper confirmed at cited arXiv ID via WebFetch and WebSearch. GitHub repo confirmed at YuqiPeng77/TARA. Token Focus Masking and token-interference claims confirmed. The specific claim about "ignoring rare tokens in favor of BOS" aligns with the paper's description of the interference problem, though the exact "BOS" phrasing may be the legacy document's interpretation rather than the paper's exact wording. [CITATION-DEPTH: AAAI 2026 acceptance was NOT confirmed in the arXiv abstract. The paper was submitted to arXiv on 2025-08-12 with no venue mentioned. The previous harvest's claim of "AAAI 2026 acceptance confirmed" could not be re-verified from the arXiv page alone. Core technical claims are confirmed.]
Claim: CLIPScore (Hessel et al., EMNLP 2021) measures caption-image alignment using cosine similarity between CLIP image and text embeddings.
Evidence: CLIPScore paper arXiv:2104.08718.
Verification: Paper confirmed at cited arXiv ID via WebFetch. Title "CLIPScore: A Reference-free Evaluation Metric for Image Captioning" confirmed. The cosine similarity mechanism is the standard CLIP similarity computation.
Important caveat: CLIP's 77-token window means CLIPScore truncates long captions. This is a scoring limitation, not a training caption length limit. The legacy documents correctly note this distinction.
Claim: Caption dropout (replacing captions with empty strings during a
fraction of training steps) only works when cache_text_embeddings: false. If
text embeddings are cached, they are pre-computed at startup and dropout cannot
modify them at training time.
Evidence: ai-toolkit source code toolkit/dataloader_mixins.py line 408.
The legacy documents note this was a confirmed bug in Sandy v1 training where
cache_text_embeddings: true silently disabled caption dropout.
Verification: The mechanism is architecturally straightforward: cached embeddings are computed once, so runtime dropout cannot intervene. The specific ai-toolkit code path was verified in the legacy project's source code analysis.
Claim: When training a body LoRA that will be stacked with a separate face LoRA at inference, body LoRA captions must describe the subject's face as only "a woman" (no qualifiers). If body captions include facial feature descriptions, the body LoRA learns to generate those faces, competing with the face LoRA during stacking.
Evidence: This is a logical consequence of the core "caption what you don't want to train" principle applied to multi-LoRA stacking. If the body LoRA learns facial features from captions, those features become text-conditioned (prompt-controllable) in the body LoRA rather than deferred to the face LoRA.
Verification: The principle follows directly from VF-1. No independent experimental validation of this specific multi-LoRA stacking interaction exists, but the theoretical reasoning is sound given the established captioning principle. The legacy project treated this as a design rule derived from first principles.
Claim: VLM-generated captions (e.g., from Qwen3-VL) are raw drafts, not finished training data. In the Naomi captioning run (277 images), 100% of VLM drafts required rewriting by LLM agents. Common failures: doubled trigger tokens, outfit misidentification, hair color leakage, implicit body descriptions, inaccurate framing terms.
Evidence: Project-internal empirical data from the Naomi captioning run. Specific failure rates documented: 100% double trigger, 61% under word minimum, 11% outfit misidentification, 18% hair color leakage, 5% implicit body descriptions, 7% forbidden body terms, 14% inaccurate framing.
Verification: These are project-internal empirical observations. The specific failure rates cannot be verified against external sources, but the general finding that VLMs produce imperfect captions requiring human/LLM review is widely reported in the community. The recommendation of a VLM-draft
- LLM-review pipeline is a practical workflow finding.
Claim: Regularization image captions must never contain the trigger word. Reg images teach the model what the base concept looks like WITHOUT the specific identity. Including the trigger in reg captions defeats regularization.
Evidence: This follows directly from the DreamBooth prior preservation mechanism (VF-10). The aboutme.be experiment (SDXL-only but directionally relevant) confirmed this: v002 used reg images generated from training captions minus the trigger word.
Verification: aboutme.be blog confirmed via WebFetch. The principle is architecturally self-evident: if reg captions contain the trigger, the model cannot learn to distinguish triggered from untriggered generations.
Claim: Reg images captioned with detailed descriptions matching the training caption format prevent concept bleed. Generic captions ("a woman") or empty captions allow LoRA identity features to leak into non-triggered generations.
Evidence: aboutme.be experiment found that only v002 (detailed captions on both training and reg) prevented concept bleed. This was confirmed via WebFetch as the experiment's central finding.
Transferability caveat: This finding was experimentally validated on SDXL/CLIP only. Transfer to decoder-only LLM encoders is a reasonable inference (format matching reduces distributional gap between training and reg data regardless of encoder architecture) but is not experimentally proven.
Claim: Of 17,576 possible three-letter lowercase strings (26^3), 1,983 are single tokens on both Mistral (Tekken, 131K vocab) and Qwen (byte-level BPE, 151K vocab) tokenizers. Candidates were filtered for pronounceability, memorability, low semantic loading, and no collision with domain vocabulary.
Evidence: Empirical scan documented in the trigger token research. The
methodology is reproducible: load both tokenizers from HuggingFace Hub, iterate
all aaa through zzz strings, filter for single-token on both.
Verification: The methodology is sound and reproducible. The specific
count (1,983) and candidate list (zek, kov, zik, nak, zar, nox,
rez) are empirical results from the scan. The methodology transfers to any
future model with a BPE tokenizer.
Claim: Captions should never use negation ("no X", "without X", "not X") because decoder-only LLMs like Mistral were not contrastively trained and handle negation unreliably for image conditioning purposes.
Evidence: BFL prompting guide implicitly discourages negation by
recommending affirmative descriptions. The legacy project's internal testing
(referenced in research/2026-04-22-03-flux2-realism-prompting.md) confirmed
Mistral's weak negation performance. CLIP handles negation through contrastive
training; Mistral lacks this mechanism.
Verification: The architectural difference (contrastive vs instruction-tuned training) is well-established. Specific negation failure examples from the legacy project are internal empirical data. The BFL guide's avoidance of negation in all examples is circumstantial but consistent evidence.
Claim: When captions are very detailed (100+ words), a dropout step creates a large distribution shift (rich conditioning to zero conditioning), producing stronger gradient signals and potential instability. Shorter captions produce gentler dropout transitions.
Evidence: This is a theoretical inference from the mechanism of caption dropout (replacing the full caption with an empty string). The distribution shift magnitude is proportional to the information content of the dropped caption.
Verification: The mechanism is architecturally self-evident. No empirical study has measured the interaction between caption length and dropout gradient magnitude. The legacy project's recommendation of 0.05-0.10 for detailed captions and 0.10-0.15 for shorter captions is a reasonable heuristic but not experimentally calibrated.
Claim: The DOP preservation loss operates on the embedding difference between trigger-caption and class-caption. In long captions, the trigger token represents a small fraction of the embedding (~1% in a 100-word caption), potentially weakening the DOP signal. Positional attention partially mitigates this for causal-attention encoders (earlier tokens get more weight).
Evidence: Theoretical inference from DOP mechanism (VF-6) combined with causal attention properties (VF-3). The interaction between token fraction and positional attention is unstudied.
Verification: The mechanism is sound in principle. The specific concern (signal dilution in long captions) is acknowledged in the legacy documents as theoretical with no empirical validation. The positional attention mitigation is a valid counterargument.
Claim: Wang et al. (2025, arXiv:2506.08210) found that "the de facto way of using last-layer embeddings as conditioning leads to inferior performance" for decoder-only LLMs as text encoders, and that layer-normalized averaging significantly improves alignment with complex prompts.
Evidence: Paper arXiv:2506.08210.
Verification: Confirmed via WebFetch. The finding supports FLUX.2's multi-layer extraction design (layers 10, 20, 30) as architecturally motivated.
Claim: BFL's prompting guide recommends short (10-30 words), medium (30-80 words, "usually ideal for most projects"), and long (80+ words) prompt categories.
Evidence: BFL prompting guide.
Verification: Confirmed via WebFetch. Note: this is inference prompt guidance, not training caption guidance. BFL has published no training-specific captioning recommendations.
Claim: CivitAI Article 7203 "reportedly found captioning beneficial for FLUX LoRA quality."
Status: [UNVERIFIED: citation behind login wall]. The article at
civitai.com/articles/7203 requires authentication. Content could not be
verified via WebFetch. The legacy documents themselves note it was "behind
login wall at time of research."
Claim: CivitAI Article 8487 supports the "caption what you don't want to train" principle.
Status: [UNVERIFIED: citation behind login wall]. The article at
civitai.com/articles/8487 requires authentication. Content could not be
verified.
Claim: Specific failure rates (100% double trigger, 61% under word minimum, 11% outfit misidentification, etc.) from the 277-image Naomi captioning run.
Status: [UNVERIFIED: no upstream source found]. These are project-internal empirical observations that cannot be independently verified. The general finding (VLMs produce imperfect captions) is corroborated by community experience, but the specific percentages are project-specific data.
Claim: No phrase of 4+ words should appear in more than 5% of captions.
Status: [UNVERIFIED: no upstream source found]. The legacy documents themselves flag this: "No threshold exists in any official documentation. Originated from a single community anecdote in SimpleTuner Discussion #634." The body-descriptor-evidence-assessment document explicitly lists this as "DISREGARD."
Claim: Concept Sliders "successfully composed up to 50 simultaneous adapters."
Status: [UNVERIFIED: no upstream source found]. The Concept Sliders paper abstract describes adapters as "plug-and-play" and "composed efficiently" but does not specify a number. The legacy documents already corrected this from "50+" to "up to 50" but the specific number remains unverified in the abstract. The full paper may contain this claim but it was not confirmed.
Claim: Using a verbatim fixed body descriptor phrase ("with very large pendulous breasts, a narrow waist, and wide hips" or later "with massive breasts") in every body LoRA caption prevents the quality fork where loose clothing degrades while tight clothing remains acceptable.
Status: [SUSPECT: no evidence basis for the causal mechanism]. The legacy documents themselves rate this as "MEDIUM -- the hybrid approach is theoretically justified but experimentally unvalidated on FLUX.2." The body descriptor evidence assessment document further classifies it: "HIGH that variation failed. MEDIUM that fixing variation will solve the problem." The Naomi v1 quality fork could have been caused by other factors (rank/LR interaction, content_or_style setting, insufficient reg data).
Note: This deviates from the core "caption what you don't want to train" principle (VF-1). The deviation was a deliberate design choice motivated by a practical failure, not theoretical preference.
Claim: In very long captions (~180 words), the trigger token represents ~0.6% of the embedding, potentially too diluted for effective DOP constraint.
Status: [SUSPECT: no evidence basis]. The legacy documents themselves acknowledge: "The denoiser receives a time-step-conditioned input and produces a noise prediction. Whether a 3-token positional shift in text conditioning meaningfully affects the noise prediction comparison is unknown." The DOP comparison operates at the noise prediction level, not the text embedding level, so token fraction in the embedding may be irrelevant.
Legacy position (earlier docs): Omit body proportions from body LoRA captions entirely, letting the LoRA learn them from pixels alone. This follows the pure "caption what you don't want to train" principle.
Legacy position (later docs): Include a fixed body descriptor phrase in every caption. This is a deliberate deviation from the pure principle, motivated by the Naomi v1 quality fork.
ai-foundry current state (AGENTS.md): The trigger tokens are viv (Sandy)
and lux (Naomi), different from both the original (s4ndy/n4omi) and
recommended replacements (zek/kov). This suggests the triggers were
changed but to different values than the research recommended.
Legacy research recommends: zek (Sandy) and kov (Naomi) as optimal
single-token triggers based on brute-force tokenizer scan.
ai-foundry AGENTS.md states: Triggers are viv (Sandy) and lux (Naomi).
Resolution: Austin approved a different trigger change (to viv/lux)
after the research was conducted. The captioning-strategy-synthesis document
itself notes: "Austin approved the trigger change from s4ndy/n4omi on
2026-04-28." The final choice of viv/lux over zek/kov represents a
decision made after the research, not a contradiction of it. Both viv and
lux are likely single tokens on the target encoders (common 3-letter English
words), satisfying the core recommendation even if different candidates were
chosen.
Legacy captioning strategy: Uses n4omi. (period separator) for Naomi
and s4ndy, (comma separator) for Sandy. The different separators distinguish
the two triggers in stacked-LoRA prompts.
Legacy synthesis document: Shows kov. (period) for Naomi and zek,
(comma) for Sandy, preserving the convention.
ai-foundry captioning strategy: Uses lux for Naomi. The separator
convention with the current triggers is not specified in AGENTS.md.
| Paper | arXiv | Verification |
|---|---|---|
| DreamBooth (Ruiz et al., 2022) | 2208.12242 | Confirmed |
| DisenBooth (Chen et al., 2023) | 2305.03374 | Confirmed |
| Concept Sliders (Gandikota et al., 2023) | 2311.12092 | Confirmed (core claim; "50 adapters" not in abstract) |
| RECAP (Segalis et al., 2023) | 2310.16656 | Confirmed (fine-tuning study: fine-tuned SD v1.4 for 250k steps) |
| CLIPScore (Hessel et al., 2021) | 2104.08718 | Confirmed |
| DECOR (Jang et al., 2024) | 2412.09169 | Confirmed |
| T-LoRA (Soboleva et al., 2025) | 2507.05964 | Confirmed |
| Wang et al. (2025) | 2506.08210 | Confirmed |
| TARA (Peng et al., 2025) | 2508.08812 | Confirmed (core claims; AAAI 2026 not confirmed in arXiv abstract) |
| Source | URL | Status |
|---|---|---|
| BFL Prompting Guide | docs.bfl.ml/guides/prompting_guide_flux2 | Live, confirmed |
| Pelayo Arbues | pelayoarbues.com/literature-notes/Articles/... | Live, confirmed |
| aboutme.be experiment | blog.aboutme.be/2023/08/10/... | Live, confirmed (SDXL-only) |
| fal.ai FLUX.2 training | blog.fal.ai/training-flux-2-loras/ | Live, confirmed |
| Apatero FLUX.2 guide | apatero.com/blog/flux-2-lora-training-... | Live, confirmed |
| alvdansen HF blog | huggingface.co/blog/alvdansen/... | Live, confirmed |
| SimpleTuner Discussion #634 | github.com/bghira/SimpleTuner/discussions/634 | Live, confirmed |
| DeepWiki FLUX.2 text encoders | deepwiki.com/black-forest-labs/flux2/... | Live, confirmed |
| HuggingFace "sks" warning | huggingface.co/blog/sdxl_lora_advanced_script | Live, confirmed |
| CivitAI Article 7203 | civitai.com/articles/7203 | Behind login wall |
| CivitAI Article 8487 | civitai.com/articles/8487 | Behind login wall |
| File | Domain | Key contribution |
|---|---|---|
| captioning-strategy.md | Captioning rules | Core principle, include/exclude lists, VLM review pipeline |
| 2026-04-08-01-image-to-prompt.md | Tool survey | VLM/captioner ecosystem comparison (42 tools) |
| 2026-04-26-flux2-body-lora-captioning.md | Body LoRA captions | Field-tested captioning methodology, VLM failure patterns |
| 2026-04-28-captioning-strategy-synthesis.md | Caption specs | Sandy + Naomi format specs, vocabulary pools, reg captions |
| 2026-04-28-flux2-captioning-methodology.md | Research synthesis | 10-section methodology covering all 8 captioning dimensions |
| 2026-04-28-trigger-token-design.md | Trigger tokens | Tokenizer analysis, DOP interaction, brute-force scan |
| 2026-05-03-body-descriptor-evidence-assessment.md | Evidence audit | Tier classification of all captioning claims |
Verified 2026-05-18 by 36 Opus 4.6 agents. Full verdicts in verification/agent-*-verdicts.md.
| Cite ID | Source | Verdict | Key Finding |
|---|---|---|---|
| cite-005 | arXiv:2104.08718 | CONFIRMED | CLIPScore reference-free metric, cosine similarity, EMNLP 2021 |
| cite-009 | arXiv:2305.03374 | PARTIAL | Paper addresses entanglement but "formalizes" overstates rigor |
| cite-011 | arXiv:2310.16656 | SUPPORTED* | FID figures confirmed; "pre-training" label wrong (fine-tuning study) |
| cite-016 | arXiv:2412.09169 | CONFIRMED | Title, orthogonal projection, embedding disentanglement match |
| cite-019 | arXiv:2506.08210 | CONFIRMED | Decoder-only LLMs outperform T5, layer-averaged better, CVPR 2025 |
| cite-022 | arXiv:2508.08812 | CONFIRMED | Token interference, Token Focus Masking, BOS findings all verified |
6 citations: 4 CONFIRMED, 1 SUPPORTED*, 1 PARTIAL