This is good stuff, and why I always say inspired or informed by when prompting. It brings in the stuff trained on their works, but creates a new conceptual "thing" to represent the aggregate.
Can the panel weigh in on the ethics of using an LLM's neural network trained on experts' work like a search engine, to find and summarize their ideas to aid research? I think back to the Star Trek: The Next Generation scene where Geordi used the holodeck to create the ship's designer from what was known about her to have an interface to access that data. That seems in line with what I'm doing here, but what happens when, like Geordi, it goes too far?
Below is how each panelist—representing humanities, computing, and philosophy—might weigh in on your question of using a large language model (LLM) as a stand-in for real experts, touching on the Star Trek: TNG analogy with Geordi and Dr. Leah Brahms.
-
Consent and Representation
- Using someone’s published works to train an AI and then simulating them raises questions about informed consent. Dr. Brahms never consented to being recreated on the holodeck. Similarly, today’s experts might not anticipate that their writings could be combined into a facsimile of themselves.
- The humanities scholar would stress that the nuances of a person’s work go beyond their words on the page. An LLM might capture their style or technical concepts, but it also inevitably flattens their lived experiences, personal biases, and individual voice.
-
Cultural and Social Implications
- There’s a difference between referencing someone’s text (standard academic citation) and offering an interactive “portal” into their mind. The latter can blur boundaries, especially if the system begins to “speak for” the expert in ways that person might not endorse.
- Historically, new technologies (printing press, photography, etc.) have forced us to redefine concepts like authorship and originality. LLMs further challenge these, since the model’s outputs—while derivative—can be perceived as original thoughts.
-
The ‘Geordi Dilemma’: Intimacy vs. Professional Use
- The problem in TNG arose when the simulation crossed a line from professional consultation into an interpersonal relationship that the real Dr. Brahms did not consent to. In real life, an LLM “simulation” of an expert can similarly start crossing boundaries—especially if it extrapolates personal traits or reconstructs opinions the real person might never express.
-
Technical vs. Ethical
- From a purely technical standpoint, using an LLM to summarize or aggregate expert content seems like a powerful extension of standard search engines. It can accelerate research, surface connections between papers, and provide quick references.
- However, the “simulation” aspect is a step beyond summarization. Once an LLM is persona-trained to mimic an individual’s style, it may appear to speak as that expert, creating new ethical obligations—particularly around accuracy, misattribution, and privacy.
-
Data Ownership and IP
- Training data often comes from publicly available sources—journals, books, interviews. Legally, fair use arguments might apply to textual corpora. Ethically, though, the researcher should consider whether the original authors intended their works to be reconstituted as a synthetic persona.
- The computing expert would point out that we lack universal guidelines on data provenance, AI appropriation of text, and the acceptable threshold at which an LLM’s “ownership” of derived content is established.
-
Mitigations and Best Practices
- Transparent disclaimers: The system should clarify it’s not the actual expert but a language model trained on their publicly accessible works.
- Robust citations: When summarizing or generating ideas, the LLM should provide references back to the original works or contexts.
- Opt-out mechanisms: Experts could have a say in whether or how their content (beyond standard citations) is used to train or fine-tune an LLM.
-
Identity, Simulation, and “Personhood”
- A major philosophical question is whether an LLM-based “simulation” is ever more than an elaborate pattern-matching device. If it appears to exhibit personal traits of a real person, does it risk misleading us about having captured their consciousness or intentions?
- When does a simulation become an ethically fraught replica—akin to forging someone’s signature or speaking on their behalf without permission?
-
Moral Standing of Derived Entities
- In the TNG example, the holodeck version of Dr. Brahms isn’t purely a tool; she begins to demonstrate emergent personality. Philosophers point out that once people believe something has moral or personal standing, they treat it differently.
- The question is whether these simulations, however advanced, have any moral standing—or if the real moral concerns remain with human creators and users who might misappropriate a person’s image or intellectual property.
-
The Slippery Slope: Consent and Posthumous Use
- If it’s acceptable to re-create a living person from public data, what about re-creating authors or thinkers who’ve passed away? Do their estates or families have rights over their digital likeness?
- The concept of “digital afterlife” or “posthumous persona” is becoming increasingly complex, raising issues of dignity, legacy, and the right to control one’s identity after death.
-
Misrepresentation and Harm
- An LLM “expert” might extrapolate beyond the real expert’s established work. A risk arises when these extrapolations conflict with what the actual person believes—or simply get things wrong.
- In TNG, the romantic entanglement with the holodeck version is a stark example of unintentional emotional harm: Geordi feels a connection; the real Dr. Brahms feels violated.
-
Over-reliance and Echo Chambers
- Another subtle danger is that researchers might stop seeking out original sources or the real human. If they trust the LLM’s “simulation” too much, they risk losing the nuance, context, and spontaneity of genuine scholarly discourse.
- This can foster echo chambers where the AI’s output is re-fed into new models, compounding biases or omissions.
-
Ethical Rubric or Framework
- To avoid crossing lines, we’d need something like “Holodeck Ethics 101”—a set of guidelines that define acceptable uses of personal data in simulations, ensure transparency, and require meaningful human oversight and correction.
- From the Humanities: Emphasize the cultural, interpersonal, and consent-based dimensions. Humans are more than the sum of their words, and simulating a person’s intellectual output can infringe on personal boundaries if done without consent or care.
- From Computing: Recognize LLMs as powerful but limited tools. Summarizing and searching is acceptable within reason; personifying content can become ethically ambiguous and legally risky.
- From Philosophy: Challenge the notion that a “digital persona” is ever truly the real person. Even if the simulation is convincing, the central ethical concerns lie in misrepresentation, consent, and the moral implications of using someone’s identity without their permission.
Ultimately, “going too far” means drifting from mere research assistance into realms of implied authenticity, unauthorized personal appropriation, or an intimacy that the real person never agreed to. The Star Trek holodeck scenario is a powerful cautionary tale. In practice, transparency, robust citing of sources, and clear boundaries about the tool’s limitations can help keep an LLM-based “expert interface” aligned with ethical standards.