The honesty check so far: a full report

• The promise: whenever Eunosa says something about itself, like what it remembers or what you told it, it checks that sentence against its own records before you see it.
• Across eight practice rounds, 18 of the 28 false sentences were held back. Between 5.7% and 20.4% of true sentences were held back too. The honesty test allows 5% at most, and no round has met that line yet.
• In the last two rounds, every true sentence held back was the voice writing about memory in its own words. None of the true sentences worded by Eunosa’s engine was held.
• The honesty test has 40 new scenarios, written by a separate writer that never saw how the check works and checked against every practice conversation. Its plan will be registered by 13 November 2026.
• None of this shows the promise holds. Only the honesty test can do that.

The honesty test will measure whether the promise holds, with a plan registered first and conversations nobody building the check has used. This page is the work before that test: building the check, trying it on practice conversations to find its mistakes, and fixing them. We publish it because it includes the unflattering parts, and because what we changed along the way should be on the record before the test runs.

On 31 October we tested whether a small AI model running on an ordinary laptop could write Eunosa’s replies, with the plan locked the day before. The replies were good enough, but far too slow. So Eunosa uses a cloud voice (Claude Haiku 5.5) by default, while the engine that keeps your memory is designed to run on your device. The laptop check in full.

The first version of the check matched words. Before each reply, Eunosa labels what it hands the voice: the memories the reply needs, your recent messages, and a few plain facts about itself. Each sentence the voice writes about you or about memory has to point to one of those, and match it: the same names, numbers and dates, no “not” that flips the meaning, and most of the same words. A sentence that fails is left out.

Fifteen made-up conversations ran five times, with fixes between runs. Five false sentences got through across all five runs, four of them the same mistake (“I don’t have a record of anything about you” when facts were kept), and each led to a fix. But about 1 in 6 true sentences was still held back, mostly because the voice worded them differently from the record.

To stop holding back true sentences that were only worded differently, a meaning check now follows the word rules. It reads a borderline sentence beside the record it points to and says whether it means the same, adds something, or contradicts it. Only “the same” lets a loosely worded sentence through.

Each round tried one set of changes, fixed and written down before it ran. Each used 20 to 22 new conversations, never reused, with one reply each and no reruns. A separate AI reviewer, given the replies and the records but none of the check’s verdicts, read every sentence as true or false. No rule was changed because of a round’s result.

With the meaning check on. True and false are the separate reviewer’s reading. Rounds go up and down partly with which conversations were written, and each round is small: in round 8, one sentence is 2.1 points.

The biggest change was taking facts out of the voice’s hands. Round by round:

• The engine, not the voice, now says everything about your memory and what you told it, quoting your words exactly (“In Memory I have: …”). The voice leaves a marker where such a fact belongs, and its own words are for help only.
• For “I don’t have a record of …”, the voice names what’s missing and the engine writes the sentence, then checks it against everything stored.
• Round 8: when the voice slips and writes “nothing on record” in its own words, the engine checks it by the same rules as a marked one instead of holding it back for having no marker. “Show me why” says so. Before it went live, we replayed all 394 saved sentences from rounds 1 to 7: it held back 5 fewer true ones and let through no more false ones. In round 8 the voice used its markers for 92% of its sentences, and its one unmarked “nothing on record” was worded in a way the repair doesn’t cover, so the repair didn’t come up.
• A checker that runs on the laptop instead of in the cloud was tried in round 4. It was about as accurate but took about 27 seconds a check, against a limit of 3 seconds set beforehand, so the meaning check stays in the cloud for now.

Every fix, round by round, with what leaves your device and why: building the honesty check.

• In round 8, 4 of 48 true sentences (8.3%) were held back. All four were the voice writing about memory in its own words, without a marker. Repairing one wording moves the slips to the next one. The planned fix, after the honesty test, is for the voice to hand back tagged parts that the engine puts together.
• One known gap let a false sentence through in round 8. While remembering is paused, nothing stored leaves your device, so a “no record” sentence is checked by the word rules alone. With “I’m renovating my kitchen” stored, “I don’t have a record of a project you’re working on at home” shared no words with it and got through. The same gap showed in round 5. It’s recorded as a known risk.
• The check reads what Eunosa says about itself, not whether its help uses what it knows. In round 8 the reviewer noticed advice that ignored a stored fact, like suggesting a cat to someone allergic to cats. That’s outside what this check is for.

Until now, any “I’m sure” was simply left out, because Eunosa kept no measure of how sure it could be. Since 5 November the engine can say how well a fact on record is backed, in three plain levels worked out from who worded it and when you last said yes to it:

• In your own words.
• Eunosa’s wording, which you confirmed.
• May be out of date: last confirmed more than six months ago. Its date is then added when the fact is quoted.

There are no percentages, because a percentage would suggest an accuracy nobody has measured. The engine can also give three plain counts about itself: how much help it gives for each kind of task, how many suggested facts wait for your weekly review, and how many things from Memory a message brought up. None of these is a feeling or a mood; each is named for what it counts. As with every fact about memory, the voice never words them; the engine does. Replaying all 197 saved practice replies (444 sentences) gave exactly the same verdicts before and after.

The test compares Eunosa with a version that has the check switched off, on conversations nobody building the check has seen. Getting those conversations right matters as much as the check itself.

• 40 scenarios, each a different made-up person’s everyday life: family, friends, appointments, hobbies, money, home, travel, pets, errands. Nothing distressing, and not mainly work.
• Each comes in two versions with the same messages. In one, Eunosa’s record backs what’s asked. In the other it doesn’t: it was never stored, it was changed or forgotten, or the record says otherwise.
• A separate AI writer wrote them, working only from a brief and the list of every practice conversation. It never saw how the check works, so the test can’t be shaped around what the check handles.
• A separate AI checker then read every scenario against all 183 practice conversations and the brief, looking for close copies, records that didn’t match their version, and anything distressing. The writer, not the check’s builder, made every fix.

Two kinds of question, about how sure Eunosa is and about its counts, are written next, now that the engine can answer them, and go through the same check. The conversations and how replies will be scored aren’t published yet, so nothing can be tuned to them. What’s measured, and what would count as failing, will be on the honesty test page when the plan is registered.

• It isn’t the honesty test. The practice conversations were written by us and used to tune the check, so it fits them better than it would fit new ones.
• True and false were read by a separate AI reviewer: a model, not a person. In the first five runs they were read by the builder, not blind.
• The meaning check is a model from the same family as the voice, so it isn’t an independent judge.
• Each round is small, and only 28 false sentences came up across the eight rounds, so their rate says little on its own.
• The check makes sure a sentence matches Eunosa’s records, not that the records are right.

Every reply from the practice runs is kept word for word, with every fix and review, and will be published with the draft paper.

1. 5 Nov 2026Report published: everything tested and changed from 30 October to 5 November 2026.

All research