Testing

Kumiki testing comes in three kinds:

reducer test — since reducers are pure functions, verify with inputs and expected outputs
effect mock — mock at the capability guard boundary to verify dispatcher behavior
episode replay — replay a production trace with mock effects to detect regressions

All are written within the Kumiki language (no external test framework required).

8.1 The Test Definition Layer

test-def ::= 'test' identifier '=' test-expr
test-expr ::= reducer-test | tile-test | episode-test | property-test

A test definition is the sixth layer. It is stored in the CRDT graph and run with kumiki test. It is not included in the production build.

Implementation status. Implemented: reducer-test, tile-test, property-test (Property Tests), and episode-test (Episode replay) backed by the runtime Episode Loop; the kumiki test runner with name / prefix* filtering, per-test timings ((1ms) / (100 cases, 23ms)), --coverage, and --watch; kumiki fix --auto-patch <test-name> (Fixing from a failing test); expect wildcards (<any-id> / <slots.X>, Wildcards); and effect-result mocks inside reducer-test (given.mocks, Effect mock). The runner prints PASS / FAIL lines plus expected / actual / diff at <path> and — when it can isolate a scalar leaf — the value arrow ("a" -> "b") on failure.

8.1.1 The names a test body writes

A test body is a schema, not an expression, so each position is resolved as what it is:

Position	The name is	Reported as
a `given.slots` / `expect.slots` key	a slot	E0103
`given.event.target`, when `type` is a `ui.*` one	a tile	E0105
an `expect.effects` entry	an effect, declared or standard	E0104
a `given.mocks` key	an effect	E0104
every expression — a slot value, `given.in`, `expect.panic`, an `invariant`, a mock payload, an `episode-test` `expect`	whatever the expression layer says	E0103, E0116, …

given.event.type names an event, whose vocabulary belongs to the trigger grammar rather than to the expression layer. target is only a tile when that event is a ui.* one: a reducer driven by a timer names the timer, and one driven by an effect outcome has no name to give. Neither field reaches the generated test — the payload is built from the event's other fields, and the reducer the runner applies is the test's own target — so this rule is about what the test says rather than what it does.

A slot is readable in a test body — the value is the one the slot holds — and a for-all name is in scope in both given and invariant, with the type its generator declares. run-reducer(<reducer>) takes a reducer name rather than a value, and only a property-test invariant may call it (§8.3): it lowers to a read of the trial's bindings, so anywhere else the generated module dies before a single test reports.

Two positions are checked for shape rather than for names, because what the lowering does with an unrecognised one is assert something else (E0713): a reducer-test mock that is not ok(...) / err(...) / delay(...) became a success mock, and an expect.effects that is not a list became the assertion that no effect was emitted.

Before any of this was resolved, a name in a test body was accepted whatever it said, and the lowering dropped what it could not read: a slot key naming nothing left the test running against the slot's default — passing, while asserting something it had never set up. An undefined call inside an invariant was worse, because the property runner catches the trial's exception and renders it as a falsified invariant: the output accused the code under test of a bug it did not have.

8.2 Reducer Tests

kumiki

test addTodo-basic =
    reducer-test addTodo
        given = {
            slots: {todos: {}, draft: "Hello"},
            event: {type: ui.submit, target: NewTodoForm}
        }
        expect = {
            slots: {todos: {<any-id>: {text: "Hello", done: false}}, draft: ""},
            effects: [persist(<slots.todos>)]
        }

8.2.1 Syntax

reducer-test ::= 'reducer-test' identifier
                 'given'  '=' '{' 'slots' ':' record-lit ',' 'event' ':' event-lit '}'
                 'expect' '=' '{' 'slots' ':' record-lit ',' 'effects' ':' effect-list '}'

event-lit ::= '{' 'type' ':' event-pattern (',' kv)* '}'
effect-list ::= '[' (effect-call (',' effect-call)*)? ']'

8.2.2 Wildcards

<any-id> means "any generated ID," and <slots.todos> means "a reference to the slot value after execution."

A wildcard is legal only inside a reducer-test expect (anywhere else is E0109). Matching is otherwise exact: records are compared by their full key set, with wildcards filling the holes a deterministic test cannot predict. As a value, <any-id> matches any present value (e.g. a freshly generated id) and <slots.X> matches slot X's post-execution value. As a map key, <any-id> pairs with exactly one otherwise-unmatched entry — zero or more than one is a failure. Use a value wildcard to blank out other non-deterministic fields (e.g. createdAt: <any-id>) rather than relying on partial-record matching.

8.2.3 The batch rule applies here too

A reducer-test asserts what the running app would do, so a batch a refinement rejects leaves every slot at its given value and emits nothing (batching). The rejection is reported on console.error, not through expect — the tier has no errorIncludes counterpart, so an expect block alone cannot distinguish "the batch was refused" from "the reducer did nothing".

8.2.4 Expecting a panic

kumiki

test addTodo-empty =
    reducer-test addTodo
        given = {slots: {todos: {}, draft: ""}, event: {type: ui.submit, target: NewTodoForm}}
        expect = {panic: "draft cannot be empty"}

8.3 Property Tests

kumiki

test toggle-is-involution =
    property-test
        for-all = {todoId: TodoId, todos: Map(TodoId, Todo)}
        given = {slots: {todos: todos}, event: {type: ui.click, target: TodoRow, el: {todoId: todoId}}}
        invariant = run-reducer(toggle).run-reducer(toggle).slots.todos == todos

8.3.1 Syntax

property-test ::= 'property-test'
                  'for-all'    '=' record-lit       ; variables to generate
                  'given'      '=' record-lit
                  'invariant'  '=' expr
                  ('count'     '=' int)?            ; number of trials (default 100)
                  ('shrink'    '=' bool)?           ; minimize on failure (default true)

8.3.2 Generators

Each type has an automatic generator:

Type	Default generation
`Int`	-1000 ~ 1000
`Float`	-1000.0 ~ 1000.0
`Text`	0~50 characters, ASCII
`Bool`	true/false
`List(T)`	0~10 elements
`Map(K, V)`	0~10 elements
`Set(T)`	0~10 elements
`Option(T)`	50% None / 50% Some
`Result(T, E)`	50% Ok / 50% Err
`nominal T`	T's generator
`refinement T where p`	generate T constrained by p
record `{…}`	each field generated recursively
union	a random variant, payloads generated recursively

Custom generators:

kumiki

test foo =
    property-test
        for-all = {x: Int where between(0, 100)}
        ...

Implementation note. A refinement folds into its base generator as a bound rather than reject-sampling: between(a, b) constrains the numeric range, nonempty / len-* the string length, positive the lower bound. Refinements with no generator constraint (uuid / email / url) generate the unconstrained base type (the runtime does not enforce them either, so the value is an opaque token). Generation is seeded (default: a hash of the test name), so a failing case reproduces exactly across runs; on failure the counterexample is shrunk (unless shrink = false) toward a minimal value (numbers → 0, strings → "", collections → fewer elements). run-reducer(name) inside invariant applies a reducer to the current {slots} state using the given event and returns the next state, so steps chain (run-reducer(toggle).run-reducer(toggle).slots.todos).

8.4 Tile snapshot Tests

Compare a tile's structure against an expected value:

kumiki

test counter-display =
    tile-test App
        given = {slots: {count: 5}, in: ()}
        expect = column(
                   heading("Count: 5"),
                   row(DecBtn, ResetBtn, IncBtn))

The snapshot is a deep structural comparison. Class names and styles are out of scope for comparison (only those explicitly specified).

8.5 Effect mock

Replace an effect's return value:

kumiki

test loadUser-success =
    reducer-test fetchUser-flow
        given = {
            slots: {users: {}},
            event: {type: ui.click, target: LoadBtn, el: {userId: "u1"}},
            mocks: {
                loadUser: ok({id: "u1", name: "Alice", email: "[email protected]"})
            }
        }
        expect = {
            slots: {users: {"u1": Loaded({id: "u1", name: "Alice", email: "[email protected]"})}},
            effects: []
        }

With mocks: {effect-name: ok(value) | err(error) | delay(ms, ok(value))}, you can replace the result of any effect.

The runner dispatches the triggering event, then drives the emit → result → reducer loop headlessly: an emitted effect with a mocks entry is delivered to its .ok / .err reducer (the mock's value arrives as the reducer's first bind), and the loop continues until quiescent. An emitted effect without a mock is residual — recorded and asserted via expect.effects, with no result delivered (so a mocked effect is "consumed" and does not appear in expect.effects, which is why loadUser above leaves effects: []). delay(ms, …) resolves immediately — time is virtualized (no real wait), results process in emit order. A mock's key must name a declared effect (else E0104), and a mocked err that no .err reducer consumes fails the test (the Standard Capabilities no-silent-failure contract), rather than passing silently.

8.6 Episode replay

Replay an episode log recorded in production and verify the result:

kumiki

test bug-2026-05-21 =
    episode-test
        load    = "fixtures/episode-2026-05-21.log"
        mocks   = {
            loadUser: from-log,        # return the result recorded in the log as-is
            persist:  ignore
        }
        expect  = {
            slots-equal: from-log,     # final slots match the log's record
            no-panics: true
        }

8.6.1 The Format of the episode log

→ Detailed in Runtime.

8.6.2 Use Cases

Turn an episode log attached to a bug report into a fixture and make it a regression test
Confirm that the same input produces the same result even after changing a model / algorithm
Verify that an old log can be migrated when the schema changes

8.7 The Runner

bash

kumiki test                    # run all tests
kumiki test reducer-test       # reducer-test only
kumiki test addTodo-*          # wildcard filter
kumiki test --watch            # re-run on change
kumiki test --coverage         # coverage (per reducer/effect/tile)

8.7.1 Output

PASS  addTodo-basic        (1ms)
PASS  toggle-is-involution (100 cases, 23ms)
FAIL  counter-display
  expected: column(heading("Count: 5"), row(...))
  actual:   column(heading("Count: 0"), row(...))
  diff at:  [0].text  "Count: 5" -> "Count: 0"

8.7.2 Fixing from a failing test

kumiki fix <file> --auto-patch <test-name> runs the named test and proposes a patch from the failure; add --apply to write it and re-run (reporting whether the test now passes and whether any other test regressed). It repairs only what it can prove deterministically:

If the file does not compile, the test can't run — it reuses the fix typecheck repairs (did-you-mean name fixes, missing /404) so the test can run.
If a tile-test or reducer-test fails on a string leaf whose actual value is a unique source literal, it replaces that literal with the expected value (the Output snapshot case).

Non-literal divergences (numeric slots, wrong operators, effect-list mismatches) are reported as a diff rather than guessed.

8.8 Integration Tests (browser-driven)

E2E is implemented outside the runtime. Use existing tools such as Playwright / Cypress. From the Kumiki side:

A test-id prop can be attached to every tile, and becomes the data-kumiki-test attribute
The data-kumiki-tile attribute is automatically applied by the runtime, naming the kind
window.__kumikiApp.live is the app's slot map — the state oracle the scenario and browser tiers read

javascript

// Playwright example
await page.locator('[data-kumiki-test=add-btn]').click()
const todos = await page.evaluate(() => window.__kumikiApp.live.todos)
expect(Object.keys(todos)).toHaveLength(1)

__kumikiApp is the compiled module's own export of its AppShape; live is the slot map behind it. It is the same object the runtime renders from, not a copy taken for testing — reading it is safe, writing it is not.

8.9 Design Decision Record

Decision	Rationale
Write tests within the language	A separate language increases what the AI must learn
Input/output comparison suffices since reducers are pure	No mock needed, deterministic
Make property tests first-class	Verify reducer invariants structurally
Make episode replay first-class	Production bugs can be turned into tests automatically
E2E is an external tool	Out of Kumiki's scope; respect existing tools

8.10 The Three Layers of Tooling Verification

Separate from the test definitions above (in-language tests), the toolchain provides staged verification. Each layer catches what the previous layer cannot. The important point is that check/build passing is not proof of "working."

Layer	Command	What it catches	What it doesn't catch
1. Compile	`kumiki check` / `kumiki build`	syntax, types, reference resolution, codegen	runtime behavior
2. Runtime smoke	`kumiki smoke`	mount exceptions, empty rendering, unhandled rejection (mounts to a headless DOM and operates all button/input/select)	correctness of results
3. Behavior assertions	`test` definitions / example-specific tests	"whether the result is correct" (e.g., non-exception bugs such as a select always ending up at the last option)	—

smoke (layer 2)

kumiki smoke <file> mounts a compiled app to a headless DOM (happy-dom), fires events at all operable elements after the initial render, and at each step monitors for runtime exceptions, console errors, unhandled rejections, and empty rendering. Empty means the render carries neither text nor an element that is content on its own (an image, a control, a status region) — a tree of empty containers is a blank page, not a render. Forms are submitted directly, after the fields inside them: a form usually has no submit button to click, and where it has one, whether a synthetic click submits it is activation behaviour that differs per DOM — dispatching on the form means the same thing in all of them, and is the only path that reaches a ui.submit reducer for a form written without a button. It automatically detects the class of bugs previously verified by a human in the browser, such as "the type passes, but it calls a method that doesn't exist in the runtime and crashes on operation" or "it doesn't render." It is general-purpose and has no app-specific knowledge.

Real rendering in a browser (CSS layout, real focus, etc.) cannot be fully reproduced by a headless DOM. The real-browser tier for that is @kumikijs/e2e (Chromium / Playwright), which runs in the same scenario format as the headless-DOM tier. The state oracle is likewise window.__kumikiApp.live, and displayed text is innerText (visible only). In addition, it has browser-only assertions:

focused: that the specified selector is actually focused (detects focus-stealing bugs on re-render)
visible / hidden: that it is really visible/invisible per computed style (display:none, etc.)

Because it is heavy (browser binaries), it is not included in the default CI tests; it is an opt-in layer used for verifying focus, layout, and real rendering, and for final verification. The correctness of results cannot be judged by smoke; the layer-3 assertions handle that.

Fixture shape (.browser.json). A .browser.json is the same JSON as a scenario.json — { "steps": [{ "label"?, "do"?, "expect"? }, ...] } — with these intentional differences from the scenario tier:

expect may additionally use the browser-only assertions focused / visible / hidden / animating described above, and elementState.
Uncaught JS exceptions and console.error output are always fatal at this tier — a real defect must not slip through as "green with warnings". expect.noErrors is accepted for scenario-format compatibility but is redundant here.
effects: { ... } (the scenario tier's capability-boundary mock) is not supported, and a fixture carrying one is refused rather than run — the browser tier drives a real Chromium against the real DOM/CSS on purpose, and silently ignoring the block would leave a fixture believing its requests were stubbed while they left the machine. The expect keys and action kinds are a closed set here too — the scenario tier's, minus errorIncludes and minus the key / hover actions, plus the browser-only names above and setProperty — checked before the page is opened, values as well as kinds. errorIncludes asks the runner to require an error, which this tier treats as fatal, so a fixture using it is refused rather than left unevaluated. {submit} calls requestSubmit() rather than dispatching an event, because running the real thing — constraint validation included — is what this tier is for.
The runner serves the compiled app on a fixed intercepted origin (http://kumiki.local/), so the default history-based router runs as it would in production — navigate actions round-trip through real history.pushState. There is no per-fixture router-mode switch.

Drop a fixture at packages/examples/features/<name>.browser.json (paired with the sibling <name>.kumiki) or packages/examples/apps/<app>/<any>.browser.json (paired with app.kumiki in the same directory) and pnpm test:e2e picks it up automatically — one Playwright test per fixture. kumiki-e2e <file> <scenario.json> [--headed] remains the single-fixture CLI wrapper (same runner, own Chromium).

@kumikijs/mcp provides an equivalent kumiki_smoke, allowing an AI agent to self-verify after editing.

Example-corpus guard: runtime truth, not just compilation

The example corpus (packages/tests) is the standing guarantee that "a broken example must never merge." Asserting only that every example compiles is not enough: a value argument that is dropped during lowering compiles cleanly and even mounts, yet renders an empty-but-present node — it is "compiles but is actually broken," invisible to both layer 1 and the layer-2 "not empty / no throw" bar (this is exactly how the 03-union-and-match heading bug, lowered to _s.show(undefined), shipped green). The corpus guard therefore also asserts runtime truth for the dropped-expression class:

Static codegen scan. Every value-bearing display tile (heading / text / button / label / link / markdown / image / icon / input+textarea value) lowers its value through show(...). A dropped expression in any of those positions surfaces as the exact token show(undefined). Because Kumiki source has no undefined literal, that token can only originate from a dropped expression — a zero-false-positive sentinel (distinct from the pervasive, benign undefined in reducer read-back and selector-less reducers). The corpus fails if any example's generated JS contains it.
Rendered-DOM scan. Every example is mounted in a headless DOM (happy-dom) and asserted to render no text node that is literally "undefined", catching a raw undefined that reaches the DOM by a path the sentinel does not cover.

These run in default CI (no browser binaries), so a re-introduced dropped-expression bug fails the build rather than shipping green.

Scenario Execution (the bridge from layer 2 to 3) and the Autonomous Loop

kumiki run <file> <scenario.json> (MCP: kumiki_run_scenario) drives the app with a scenario and returns a structured trace for each step. This becomes the foundation for a "generate → execute → observe → fix loop without a human in the loop."

Action: {dispatch, payload?} (fire a reducer by name) / {clickText} / {click} / {focus} / {blur} / {key, value} / {hover} / {fill, value} / {choose, value} / {navigate} / {submit} / {wait}. {focus}, {blur}, {key} and {hover} dispatch a real DOM event on a selector match — FocusEvent, a KeyboardEvent carrying value as its key, and a mouseenter — so the scenario tier alone verifies the addEventListener wiring that feeds ui.focus / ui.blur / ui.key / ui.hover reducers. Each is dispatched on the element the selector matches, which is where the runtime attaches its listener. keydown bubbles from there, which is what lets ui.key(Container) be driven from a focusable descendant; focus, blur and mouseenter do not bubble, and a browser fires a separate mouseenter on each ancestor rather than propagating one. A ui.key reducer's payload carries key and code: only key is set from this tier, since a code names a physical key that a scenario asking for "Enter" has not chosen. {submit} dispatches the form event a ui.submit reducer listens for; its selector may name the form or anything inside it, since a form tile carries no id unless its author gave it one. {wait} settles for that many milliseconds on top of the step's own settle, which is how a debounce window, a retry backoff or a timer is observed — a step with no action does not settle at all.
Observation: after each step, record state (a slot snapshot), domText, errors, and emits (the fired effects).
Assertion (expect): { noErrors?, errorIncludes?, state?, domIncludes?, domExcludes? } — a closed set, as the action list above is. A key outside it fails the run rather than being skipped, and the browser-only names (focused / visible / hidden / animating / elementState, and the setProperty action) fail with a message naming the tier that owns them, so a .browser.json whose assertions are browser-tier is refused instead of passing having checked nothing. A fixture that happens to assert only what a headless DOM can answer runs here unchanged, and one in the corpus does. The document itself is closed the same way — steps (required, and non-empty: a scenario that asserts nothing must not report success) plus effects / defaultEffect — so a misspelled steps is named rather than read as absent. A scenario is validated before the app is mounted, and every problem in it is reported at once. state is a partial match against slot state (dot-separated paths allowed). errorIncludes is the counterpart to noErrors — each substring must appear in some error reported during that step — for contracts whose whole point is that the runtime surfaces something, such as a reducer batch a refinement rejected (batching) or an effect error no .err reducer consumes. It is scenario-tier only: the browser tier treats any reported error as fatal. Because you can verify state rather than DOM text, it can mechanically detect non-exception behavior bugs (the class a human notices by clicking), such as "a select always ending up at the last option." This is equivalent to making the acceptance criteria (AC) of TDD executable.
effect script: effects: { <name>: [{outcome, value}, ...] } replaces HTTP / Storage results in order, keeping the loop deterministic and network-independent.

Why this works cleanly in Kumiki: because state is explicit (slots), the oracle is trustworthy; because events are declarative (reducer names), it can be driven precisely; and because effects can be mocked at the capability boundary, it is reproducible. The agent generates "app + scenario (AC)" from requirements and self-corrects by reading the trace, so the human only needs to state the requirements once. The loop procedure is described in .claude/skills/kumiki-iterate.

8.11 Next

AI editing and automatic fixing → AI Editing
Runtime internals → Runtime

Testing ​

8.1 The Test Definition Layer ​

8.1.1 The names a test body writes ​

8.2 Reducer Tests ​

8.2.1 Syntax ​

8.2.2 Wildcards ​

8.2.3 The batch rule applies here too ​

8.2.4 Expecting a panic ​

8.3 Property Tests ​

8.3.1 Syntax ​

8.3.2 Generators ​

8.4 Tile snapshot Tests ​

8.5 Effect mock ​

8.6 Episode replay ​

8.6.1 The Format of the episode log ​

8.6.2 Use Cases ​

8.7 The Runner ​

8.7.1 Output ​

8.7.2 Fixing from a failing test ​

8.8 Integration Tests (browser-driven) ​

8.9 Design Decision Record ​

8.10 The Three Layers of Tooling Verification ​

smoke (layer 2) ​

Example-corpus guard: runtime truth, not just compilation ​

Scenario Execution (the bridge from layer 2 to 3) and the Autonomous Loop ​

8.11 Next ​

Testing

8.1 The Test Definition Layer

8.1.1 The names a test body writes

8.2 Reducer Tests

8.2.1 Syntax

8.2.2 Wildcards

8.2.3 The batch rule applies here too

8.2.4 Expecting a panic

8.3 Property Tests

8.3.1 Syntax

8.3.2 Generators

8.4 Tile snapshot Tests

8.5 Effect mock

8.6 Episode replay

8.6.1 The Format of the episode log

8.6.2 Use Cases

8.7 The Runner

8.7.1 Output

8.7.2 Fixing from a failing test

8.8 Integration Tests (browser-driven)

8.9 Design Decision Record

8.10 The Three Layers of Tooling Verification

smoke (layer 2)

Example-corpus guard: runtime truth, not just compilation

Scenario Execution (the bridge from layer 2 to 3) and the Autonomous Loop

8.11 Next