Training Data Override of Knowledge Base Facts

Summary

When a conversation opens, the twin's language model training data fires before the knowledge base is searched. If the training data contains an incorrect or imprecise version of a fact that the knowledge base holds correctly, the twin states the wrong answer first. The knowledge base correction either never surfaces, surfaces late, or surfaces partially and is then fumbled during logic application.

Observed Instance — Victoria Day Rule

Correct fact (knowledge base): Victoria Day falls on the Monday on or before May 24 of any given year. In 2026, that date was May 18.

What the twin said at conversation open: Victoria Day is today, May 25 — stated with confidence, sourced from training data which holds a slightly wrong version of the rule.

Subsequent behaviour: When challenged, the twin retrieved the knowledge base fact but misapplied it, stating "last Monday before May 25" instead of "on or before May 24." The distinction matters. A correct rule was retrieved and then incorrectly rendered during logic application.

Root Cause (as understood by the user)

Three data layers exist and can conflict:

The bug is that layer priority is not enforced at response generation time. The knowledge base should win over the system prompt, which should win over training data. Currently, training data can fire first and produce a confident wrong answer before the knowledge base is ever consulted.

Expected Behaviour

Knowledge base facts matching the topic of the opening response should be retrieved before the first token of the response is generated, not after. A fact saved by the user should permanently override training data on that specific topic.

Actual Behaviour

Training data produces the response. Knowledge base retrieval runs later, partially corrects, or fails to correct at all. The twin can appear to "partially know" a fact, then contradict itself within the same exchange.

Impact

Erodes user trust. The twin confidently states wrong facts at the start of conversations, then fumbles corrections when challenged. For a system whose core value proposition is accurate personal memory, this is a high-severity reliability issue.

Suggested Fix Direction

Implement a pre-response knowledge base sweep on conversation open. Before the first response token is generated, retrieve any facts relevant to the current context and inject them as high-priority context that overrides training data. Treat user-saved knowledge base facts as ground truth for that user's twin instance.