Principle · Technology and AI

Evidence Before Assertions.

Source: foundational to the test-driven development culture established by Kent Beck, Test-Driven Development by Example (2002), and the verification discipline practiced across modern software engineering. Closely related: the scientific method, the falsifiability standard from Karl Popper, The Logic of Scientific Discovery (1959), and the engineering ethic of "show me, do not tell me."

The Principle

Before claiming that work is complete, fixed, passing, or working, run the verification command, read the actual output, and confirm that the output matches the claim. The discipline is to never assert what you have not verified. "It should work" is not a status. "I ran the tests, here are the results" is a status. The gap between those two sentences is the difference between engineering and theater.

The reason this matters is that working software produces output that can be checked. Broken software often produces output that looks plausible. The mind is good at filling in the missing evidence. An engineer reading their own code will see what they intended to write, not what they actually wrote. The only way to know whether the code does what you think it does is to run it and read the result. Without that step, "complete" is a guess, and guesses compound across a system into outages, regressions, and lost trust.

Evidence before assertions is a culture, not just a practice. In a culture where engineers say "done" when they have verified, the team can trust each other and ship faster. In a culture where engineers say "done" when they have only convinced themselves, the team has to verify each other's work, slowing everything down and breeding distrust. The single discipline of running the command and reading the output is what makes the whole system work.

Why It Matters Here

Technology and AI is the department where unverified assertions cost the most. AI systems produce confident-looking output even when they are wrong. Automated workflows can fail silently and look the same as workflows that succeed. The team's only protection is the CAIO's discipline of verifying every claim before it becomes a commitment. Without it, the leverage layer becomes a liability: systems that look like they work, until they do not.

Signals (When to Apply)

About to claim a fix is complete based on having written code, not having run it
About to ship a system because it looked correct in the development environment
About to mark a task done in the project tracker without running the end-to-end check
About to merge a pull request because the diff looks right, without seeing test output
About to tell a stakeholder a problem is resolved, based on the engineer's word, not on a verification run

How to Apply

Define the verification command before starting the work. "When this is done, the proof will be: this command, with this expected output." Writing it down before the work makes the goal concrete and the cheat impossible.
Run the command. Read the output. Capture the output in the project record. The act of capture is what turns the assertion into evidence.
If the output does not match the expected result, the work is not done. No exceptions. No "close enough." The output is the truth. Adjust the work, not the standard.
Treat error messages and warnings as evidence, not noise. A test that passes with warnings is not the same as a test that passes clean. Read what the system is telling you.
Make verification automatic where you can. CI pipelines, pre-commit hooks, automated checks. The goal is to make "verify" the default state, not a step the engineer has to remember.

Examples

Applied well An engineer ships a fix to a content automation. Before claiming complete, they run the automation end-to-end on three real test inputs, paste the actual outputs into the pull request description, and confirm each output matches the expected result. The reviewer reads the evidence, sees that two out of three pass, and the engineer goes back to fix the third. The bug is caught at review, not in production. The verification took twenty minutes. The cost of catching it in production would have been a customer-facing outage and a day of incident response.

Misapplied The same engineer ships the fix and writes "complete" in the tracker after compiling the code and reading the diff. The reviewer trusts the claim and merges. The third test case, which was not run, fails for the first user who hits it in production at three in the morning. The team spends six hours on the incident, the customer loses an hour of work, and the post-mortem reveals that nobody had run the end-to-end test. The original assertion of complete was not evidence. It was a guess that the team trusted because the engineer was credible. Credibility does not replace verification.

When to Break It

For trivially safe changes (a typo in a string, a comment edit) where running a full verification is more cost than the change is worth. Use judgment, but err on the side of running the check.
In emergency situations where you have to ship a partial fix to stop bleeding, with the understanding that you are explicitly asserting "this reduces the harm, full verification to follow." Name it that way. Do not pretend the fix is complete.
When the cost of verification is genuinely prohibitive (a multi-day load test for a tiny change). Even then, find the smallest verification that increases your confidence. Skipping verification entirely is rarely the right answer.