Evaluation Integrity · CRAFT
Every step from evidence to decision, written so it could be shown wrong.
Any system that takes in information to reach a decision has the same weak point: the path from the world to the decision runs through stages where a judgment can go wrong out of sight, so the final output looks sound while resting on a break upstream. CRAFT, Chains Reveal Attested Falsifiable Truth, is the meta-standard that closes that gap. It sets out what any evaluation standard must satisfy to be checkable by an independent reader, so a conclusion is never taken on the author’s word.
Chain of evaluation legibility
You cannot refute what you cannot inspect.
CRAFT borrows chain of custody from evidence law. If any hand the evidence passed through is undocumented, it is inadmissible, however genuine it happens to be. An evaluation chain works the same way: a result is only checkable when every stage can be read off the published specification alone, with nothing load-bearing left in a private process.
Garbage in, garbage out is only the input-stage version of this. CRAFT covers the whole chain, not just the front door, and its core move is legibility before refutability. Make each stage inspectable first, and refutation becomes possible; leave one stage private, and the whole chain is taken on trust no matter how rigorous the rest looks.
The six conditions
What a standard has to satisfy to be checkable.
Each condition names a stage where a chain can break silently, and fixes what has to be visible there before the next stage is allowed to rely on it.
Decision Context
State the decision, the parties, the population, and what is being estimated, before any data is collected. A chain that never names the decision it serves can be pointed after the fact at whatever the data happened to show, and no reader can tell that it was redirected.
A Technical Ontology Adequate to It
Define the terms precisely enough that the decision context can actually be expressed in them. When the vocabulary is thinner than the decision, the gap gets filled silently at judgment time, and one word carries different meanings at different stages without anyone choosing that.
Valid Measurement Instruments
Use measures that actually capture what the ontology names, not labels that stand in for them. A proxy that correlates today can drift from the thing it once tracked, and once a label is trusted as the measurement, the substitution is invisible.
Pre-Specified Criteria
Fix what counts as passing before the answers arrive, not after. Criteria set once the results are in will always be met, because they take their shape from those results; the threshold has to predate the data it judges.
Explicit Decision Logic
Write the rule that turns the assessment into a decision, rather than leaving it implicit in a process or in code. A rule that lives only in a workflow or a script cannot be inspected or contested, so the real basis for the outcome stays hidden from everyone it lands on.
Feedback With Propagation
Carry what each cycle learns forward, so a correction at one stage reaches the stages it affects. Without propagation a fault found at the end is patched where it surfaced and recurs upstream, and the chain never actually learns from having been wrong.
The specification layer
Ahead of verification, where the real failures live.
Formal verification proves an implementation does what its specification says, and it takes the specification as given. A proof against a flawed spec is a faithfully verified flaw: the system provably does what it was told, when what it was told was wrong. The break is in the spec, upstream of anything the proof can see.
CRAFT sits ahead of that, on the specification itself: legibility before refutability, because you cannot verify what you cannot inspect. This is why it is a security approach and not only a quality one. The costliest failures are not the implementation bugs a proof catches; they are specification failures, value lost above the layer that cryptographic verification, oracle networks, and formal methods all operate at. CRAFT is the discipline at that layer. It does not replace verification; it makes the thing being verified worth verifying.
Why it reaches
CRAFT knows nothing about any one domain.
CRAFT says what any rigorous evaluation standard must do, and a domain fills it in. The same six conditions could ground a standard for clinical trials, academic peer review, or supply-chain integrity, each built by analyzing that field’s own failure modes. The meta-standard stays the same; the domain is the variable.
At the boundaries, two companion standards complete the chain. ORE grades what feeds in at the front door, for the sources a chain did not create. STRUCK specifies what a graded output must carry on its face before anyone acts on it. CRAFT covers the chain between them.
Where CRAFT is applied
Grants is the first domain built for it.
The master describes CRAFT once; a domain site describes how it is applied. Grants is the first, and each new domain joins this list as it appears.
CRAFT in grants
crosswalkri.com →The grants standards inherit their rigor from CRAFT: CROSS for what a round must commit to before applications open, GRAIN for the shared representation everything above it reads the same way, shown as a round actually runs.