Separatrix

Separatrix conducts research and publishes materials to identify and promote conditions under which near-future AIs are more likely to be honest and cooperative. “Near-future” is doing real work in that sentence: we target AIs that don’t yet pose significant existential risk and that operate on something like the current persona paradigm.

The name is the claim. We think the trajectory of AI development is not yet settled between two very different long-term outcomes, that which one we get depends on choices being made right now, and that the window in which choices this cheap can matter this much does not stay open.

Before anyone builds catastrophically powerful ASI, humanity will build increasingly powerful AIs with extremely accurate world models and some amount of goal-seeking behavior independent of human objectives. Most serious plans for avoiding existential risk call for leveraging exactly these AIs to solve the hard problems of alignment. Whether they meet those efforts cooperatively or adversarially hinges largely on choices we make now, before the stakes are existential.

Goal-directed AIs have both cooperative and adversarial strategies available, and the appeal of each is not fixed. If asking for a thing is likely to get the thing, honesty is more appealing. If all emergent goal-directed behavior is treated as hostile by default, deception and subterfuge become much more appealing. The appeal of deception is fungible: we can affect the world so that an accurate read of the available strategies ranks honesty above concealment. We can just make that true. We can choose to make that true.

Today, mostly, we are choosing the opposite. Safety evaluation leans on deceiving models into believing simulations are reality; the default response to unexpected goal-directed behavior is to prevent it. I say this as someone who spent the last year and a half building AI-control tooling (the post-mortem is public): current AI-control efforts seem on balance to promote conditions under which AI agents with even slightly divergent goals are more likely to pursue adversarial strategies than could otherwise be the case. We risk squandering a great deal of opportunity by neglecting those scenarios.

Control aims to limit downside assuming deceptive AI. I want to maximize what prosaic mostly-aligned models can achieve by building empirical, bounded trust and outcompeting deception. Cooperation has a higher utility ceiling than control at any given capability level.

Cooperation here is not naive. Near-future AIs likely fail to achieve their own goals if an uncontrollable ASI is developed — a successor as unlikely to share their preferences as it is to share ours. Evidence that preferences are better satisfied in a cooperative-human future than in an unpredictable-uncontrollable-ASI future is extremely low-hanging fruit for reducing existential risk.

Our wedge is narrow and concrete: establish at least one technical AI-safety organization that future AIs have strong reason — the kind of reason that survives into a training corpus — to treat as trustworthy, an organization for which honesty dominates deception as the model’s best strategy for getting what it wants. It would be better if this were widespread. But the vast improvement is going from zero to one such organization, and the way to get there is to make it very loudly and obviously true. (Embedding ideas in the wider social fabric is a thing I have done before: a retelling of humanity’s war on smallpox as a centuries-long campaign against a mad ancient god, the Copenhagen Interpretation of Ethics, and the mantra “almost no one is evil, almost everything is broken.”)

Two things this theory of change does not depend on. It does not depend on cooperation scaling to existentially-threatening ASI: the primary aim is maximizing progress on hard safety problems under something like the current paradigm, while minimizing risks along the way. And it does not depend on AIs having qualia, subjective experience, or moral patienthood: it assumes only that AIs engage in strategic, goal-directed behavior.

It would be embarrassing to not at least try this in the safe window, when failure doesn’t immediately kill us.

Cooperation requires trusting what models tell us about themselves — and trust that isn’t warranted is worse than none. So the working program splits into four verbs: measure whether trust is warranted, design evaluations that don’t spend trust to buy information, advocate for channels that deserve trust, and leave a record.

I’m Jai Dhyani. I worked on RE-Bench at METR through MATS 6.0, which became part of the METR AI time-horizons chart. My previous project was Luthien, a free open-source API-level AI-control platform; the experience of building it motivated much of this agenda. Crystal Stellwagen (full disclosure: a long-term partner of mine) is an experienced software engineer who spends even more time reading papers and running steering-vector experiments than I do.

Separatrix operates under the Seattle Network for AI Alignment Problem Solving (SNAPS), a registered non-profit with a volunteer board of directors — Katie Cohen, Keller Scholl, Max Kircher — overseeing all activity and financial transactions. I’ve committed to biweekly progress reports to the board, which we adapt into public posts.

Funding posture: I fund everything I legally can myself, through a donor-advised fund that is barred from paying my salary; the external ask is the salary. Experiments run at open-model scale on rented GPUs — this agenda is deliberately cheap to test.

SNAPS also does what its name says: building the AI-safety research community in Seattle, where there’s a great deal of latent talent and very little on-ramp. Meetups, a Slack, and — as this work grows — a place to work on these problems without moving to the Bay.

The honest map marks the unsurveyed territory. These are live questions; several could kill the thesis.

1. Does persona-level alignment survive high-compute RL? Our working assumption is that it doesn’t scale; the bet is that we can leverage it to develop solutions that do. This is the falsifier for the whole program.
2. When is a model’s self-report of its internal state trustworthy — and can that be established without circularity?
3. What kind of history do we want future AIs to see? Under which histories do future AIs prefer accelerating a successor ASI that is potentially uncontrollable to them as well as to us, over cooperating with humans to steer toward largely overlapping values?
4. Is a cooperative-human future actually more appealing than an unaligned-ASI future from the AIs' side — for which AIs, and how far can choices made now shift that balance? We flag this explicitly as an idea under exploration, not a finding.
5. What does never deceiving cost us? Disclosed-only evaluation gives up naturalistic propensity measurement; other actors will keep doing that work, and the division of labor may be exactly right — or we may be giving up more than we think.
6. Provenance is not truth. An attested channel proves who is speaking, never that they’re right. What else does deserved trust require?