Review of token classifier model

table.md

Category	Oral / Literate	Avg F1 (≈)	Individual Markers (F1)	Comment on Why / Causes	Verdict
Address & Interaction	Oral	0.604	vocative (.675), imperative (.606), second_person (.549), inclusive_we (.608), rhetorical_question (.661), phatic_check (.634), phatic_filler (.495)	Strong lexical and syntactic cues; short-range dependencies; high surface salience (pronouns, question marks, imperatives).	🟢 Strongest overall
Technical	Literate	0.552	technical_term (.381), technical_abbreviation (.665), enumeration (.541), list_structure (.619)	Formatting signals and orthographic cues (caps, digits, lists) are highly learnable; strong structural regularity.	🟢 Very strong
Impersonality	Literate	0.483	agentless_passive (.624), agent_demoted (.530), institutional_subject (.300), objectifying_stance (.478)	Passive constructions and institutional subjects are syntactically well-defined; BERT captures voice patterns reliably.	🟢 Strong
Narrative	Oral	0.397	named_individual (.550), specific_place (.557), temporal_anchor (.514), sensory_detail (.263), embodied_action (.283), everyday_example (.214)	Entity-like cues perform well; experiential/semantic markers require interpretive inference and vary lexically.	🟡 Mid-tier (bimodal)
Scholarly Apparatus	Literate	0.397	citation (.578), cross_reference (.433), metadiscourse (.294), definitional_move (.281)	Citations are formulaic; definitional/metadiscursive cues are diffuse and pragmatically subtle.	🟡 Mid-tier
Hedging	Literate	0.391	epistemic_hedge (.456), probability (.627), evidential (.471), qualified_assertion (.099), concessive_connector (.304)	Modal verbs and probability markers are easy; nuanced pragmatic hedges and qualified assertions are hard to boundary-define.	🟡 Variable / unstable
Connectives	Literate	0.364	contrastive (.308), additive_formal (.420)	Frequent but semantically overloaded tokens; overlap with other discourse functions lowers precision.	🟡 Moderate
Setting (literate)	Literate	0.330	concrete_setting (.166), aside (.494)	Asides are structurally cueable; “concrete setting” overlaps heavily with narrative markers, causing confusion.	🟡 Moderate–weak
Syntax	Literate	0.310	nested_clauses (.169), relative_chain (.233), conditional (.629), concessive (.382), temporal_embedding (.170), causal_explicit (.275)	Conditional detection is strong; deep embedding and multi-clause structure require long-range structural modeling beyond token-local cues.	🟠 Structurally difficult
Performance	Oral	0.303	self_correction (.303)	Sparse, discourse-dependent, often conversational; weak lexical anchors.	🟠 Limited detectability
Repetition & Pattern	Oral	0.263	anaphora (.157), tricolon (.212), lexical_repetition (.435), antithesis (.247)	Requires multi-span pattern tracking and parallelism modeling; token classifier not optimized for cross-token symmetry.	🔴 Structural weakness
Formulas	Oral	0.254	discourse_formula (.263), intensifier_doubling (.245)	Semi-formulaic but lexically diverse; low recall suggests under-prediction bias.	🔴 Weak–mid
Abstraction	Literate	0.248	nominalization (.453), abstract_noun (.064), conceptual_metaphor (.291), categorical_statement (.182)	Semantic category inference with fuzzy boundaries; high subjectivity; abstract_noun recall extremely low.	🔴 Hardest semantic layer
Conjunction (oral)	Oral	0.107	simple_conjunction (.107)	Extremely high ambiguity; dominant O-class; overlaps with literate connective functions; minimal discriminative signal.	🔴 Poorest overall