Why a tutor that answers is the wrong product

The obvious AI tutor answers questions. A child is stuck, the child asks, the tutor explains. It demos beautifully, it is far easier to build, and at nine o'clock on a Tuesday it is exactly what a stuck child wants.

We built the one that asks a question back instead. This is the argument for that, including the part where it costs us something.

What actually happens when you are given the answer

A child stuck on a reading question is not usually missing a fact. They have applied a strategy that nearly works. The commonest one in reading comprehension is taking a striking detail for the main idea — the passage mentions a storm, the storm is the most vivid thing in it, so the question about what the passage is about gets answered with the storm.

Being told "the answer is B" resolves the question and leaves the strategy untouched. The next passage produces the same error. The child has been unblocked, which feels like help and is not the same as learning.

Being asked "is the writer telling us what happened, or why it mattered?" is more annoying and it addresses the thing that will happen again.

The part that makes this a product rather than a preference

Anyone can write a prompt telling a model not to give away the answer. It will comply for a while and then drift, because prompts drift — under a rephrased question, a long conversation, a child who asks four times.

So the rule is not in the prompt. Every response is compared against the item's recorded correct answer before a child sees it, and one that contains the answer is regenerated with a corrective instruction. If the second attempt also fails, the child gets a safe response and the failure is written to the audit log.

That comparison is possible because the item bank knows what the right answer is, and knows what each wrong answer represents. Every incorrect option carries a code for the specific misunderstanding it was written to catch, and an item cannot be served to a child until a person has approved it and every distractor has one.

That annotation is slow, human work. It is also the substrate the whole behaviour rests on: without it, "Archie names the misconception" degrades into "Archie is encouraging", which is what most of these products already are.

What it costs us

Some children will bounce off it.

A child who wants the answer and is asked a question instead can close the app and open something that will simply tell them. We do not have a mechanism to prevent that and would not want one — the mechanisms that prevent it are the ones §6 of our own design brief bans.

We would rather lose that child than build the thing that helps them finish their homework without learning anything. That product exists several times over and none of them are education, whatever their marketing says.

The honest caveat

We do not have an efficacy number.

We have built the instrument that would produce one — a pre-assessment, a post-assessment drawn from a different exam year with a matched distribution of standards so improvement cannot be explained by familiarity, and a measure of how often a child gets it right after being asked a better question.

What we do not have is data from real classrooms, because there have not been any yet. When there are, the number goes on the site whatever it says.