Jan Philip Wahle jpwahle

How fast is "LLM-as-a-Judge" growing at ACL? (2024–2026)

Code + data to measure how many accepted ACL papers (main conference long+short + Findings of ACL) use LLM-as-a-judge-style and LLM evaluator-style terminology — in the title, abstract, body (text before the References section), and full text (everything, incl. bibliography and appendix).

Data source: the official ACL Anthology (metadata via the acl-anthology Python package; full text from the anthology's open-access PDFs, CC BY 4.0). Anthology data as of 2026-08-04.

Key results

Term families (case-insensitive; spaces/hyphens interchangeable; line-break hyphenation handled):

	# Copyright 2022 by Jan Philip Wahle, https://jpwahle.com/
	# All rights reserved.

	import colorcet as cc
	import dask_ml.feature_extraction.text
	import datashader as ds
	import pandas as pd
	from datasets import load_dataset
	from datashader.utils import export_image
	from openTSNE import TSNE