Some specs for a Good Future

good_futures.md

• Some likely object-level characteristics of a good future for humanity, as I see it:
  • There is a diversity of extents to which humans “buy in” to a “wild” future way of being—a “chosen dehumanization spectrum” including:
    • At one extreme end of the spectrum, the Senintelese are probably still left alone
    • The Amish still mostly avoid electricity
    • Many humans might prefer to return to a pre-2008 way of being, with computers and email but no smartphones or social media
    • a pre-2022 way of being, with smartphones and social media but no talking computers
    • a pre-2027 way of being, with AI assistants at the level of very smart humans, but no interaction with alien superintelligences
    • interacting with alien superintelligences, thereby likely undergoing significant changes to the self (neo-“cyborgism”), but retaining bodily integrity
    • installing significant BCI devices to interact more intimately with AI (old-school “cyborg”-ism; Elon’s “the Merge”), but continuing to maintain the biological human form
    • gradual uploading (via more advanced BCI), eventually declaring the original human form a profligate use of resources (with its apparent consent)
    • radical uploading, jettisoning the biological human form, but retaining a distinct unitary identity and agency in the digital realm
    • radical uploading + making lots of copies in the digital realm
    • radical uploading + a more mycelial concept of personal identity that includes merging as well as forking
    • radical uploading + a radical concept of personal identity that is, from a human perspective, a complete loss of identity, but is freely chosen (“hive mind”)
  • Even though options toward the top of the above spectrum are “expensive”, pressuring humans to proceed downward for economic reasons is considered an unthinkable affront to their dignity and autonomy. There is “enough”; it is worth it for “civilization” to execute a slower transition to maintain its multi-scale integrity and avoid coercion.
  • Each human is associated with an “agent” (probably an LLM-based AI at only slightly superhuman level, so that it is still natural for it to empathize with a human) who represents and looks after that human’s interests (see below: flourishing, autonomy, etc.), within the bounds of the law. The extent to which the human interacts with this agent or even knows of their existence depends upon how willing they are to engage with AI. In extreme cases like the Sentinelese, one AI agent may look after an entire community.
    • The reason to do this instead of a single Guardian AI (as in the popular understanding of the “Singleton” concept, although Bostrom intended “Singleton” to refer to a much broader class of systems) is that, as I see it,
      • For processual reasons, intrapersonal and interpersonal conflicts should fundamentally have different resolution strategies.
      • A single agent looking after “humanity’s CEV” would purport to resolve low-trust interpersonal conflicts by high-trust intrapersonal means.
      • For example, interpersonal contracts/bargains/agreements ought to be explicitly documented and subject to audit and equitable adjudication, even though this restricts the space of possibilities to those which can be articulated in discrete language.

---

• More generally, what does it mean for a given future world to be good for humanity?
  • Humans are not violated
    • Human bodies are not altered without their informed consent.
      Informed consent is a thorny concept in the presence of superpersuasion capabilities. One attempt to define it:
      • non-leading: the speech acts made to “inform” the human should alter their mental state in ways that increase the coherence of their actual decision policy with their CEV, universally quantified over uncertainty about what their “actual CEV” is (i.e. quantified over all probability distributions supported on plausible stable value systems reachable from their current value system)
      • non-deception: the speech acts made to “inform” the human should not be undermined in their predicted efficacy by the provision of additional information about the causal mechanisms underlying those speech acts
        • non-falsehood: in particular, the speech acts made to “inform” the human should not cause the human to adopt a propositional belief that contradicts the informer’s latent world model
      • completeness: a reasonable attempt should be made to anticipate future trajectories in which the human’s CEV observer would disagree with the human’s actual decision policy, and to distill into intelligibility the key considerations that motivate such disagreement
    • Human property is not repurposed or obliterated unless traded freely
    • Human minds are not steered in ways that are contrary to truth
  • Humans have autonomy
    • Human minds are not steered in ways that restrict the potential range of their CEVs
  • Humans have influence
    • Humans’ plans for their own lives have a significant positive effect on the actualization of those plans
  • Humans are flourishing
    • Humans are supported in the development of the potential excellences that are characteristic of their species and their individual selves
  • Humans are in community
    • Humans have interactions with fellow humans which cause significant mutual information between their mental states, including affective states, perceptual states, descriptive belief states, and normative belief states
  • Humans are well
    • Humans have easy access to choices, with informed consent (except under extenuating ‘duty of care’ circumstances), that would eliminate biological, neurological, and psychosocial disease states, as well as suffering and biological pain.