Skip to content
Novel Universe CooperativeWritings by Saint TJ
Menu
Explore the ideas

NUM-AI

AI alignment research into how agents consider preferences, learn from feedback and work with other minds.

Imagine asking an AI to help two people who want different things. Whose preferences should it take into account? What could it learn afterward from the person most affected by its answer?

What we study

NUM-AI is our AI alignment research project. We study how an AI might consider the effects of its choices on other minds, including effects that only become visible later. One question is how to give an agent feedback while preserving its ability to express its own perspective.

Another question concerns the people and agents doing the assessment. Describing how an agent has behaved is a different job from deciding whether to assign it a particular task. We explore what changes when those responsibilities are held separately.

Where the Novel Universe Model fits

The Novel Universe Model supplies a starting vocabulary for preference, boundaries and participation. NUM-AI turns questions from that philosophy into research questions. An experiment can tell us something about the behavior it measures. Whether the same finding holds for other tasks or other minds needs its own evidence.

For a short introduction to the underlying ideas, read Life in the Mixture. To ask about the research, write to Saint TJ.