Test results

Recorded 2026-10-04T12:24:20.669Z. This page is a saved snapshot.

Source revision
777abacc9d837ebc8f6ea5374f8162094c05901a

Automated checks

4 of 4 checks passed

Unit and browser tests use fixtures. These checks do not make live Jev calls.

Passed npm run typecheck 0.7 s

Started 2026-10-04T12:24:07.090Z · Exit 0

Command completed successfully.
Passed npm run test 1.1 s

Started 2026-10-04T12:24:07.780Z · Exit 0

tests: 35
pass: 35
fail: 0
cancelled: 0
skipped: 0
todo: 0
Passed npm run test:history 0.1 s

Started 2026-10-04T12:24:08.874Z · Exit 0

History tests: 3
Passed npm run test:e2e 11.5 s

Started 2026-10-04T12:24:08.976Z · Exit 0

Browser tests passed: 20

Routing results

Routes are compared with authored synthetic expectations. Agreement does not measure task success or prove which model is best. The decision table reuses saved assessments and makes no API calls.

Split caveat: policy-5 and policy-6 repeat the database recovery scenario in demanding-1 and demanding-2. The current holdout is not fully independent.

tuning split

Recorded 2026-10-04T10:55:14.927Z · Evidence: Saved live responses to five explicitly synthetic prompts. Selected examples only.

Route agreement
SelectionAccepted / completedMissed clarificationUnnecessary clarification
Jev4 / 500
Decision table5 / 500

0 service errors · 19 cases not attempted · 4 identical selections

1 model selections and 1 effort selections have confidence below 50%. Confidence does not measure task success.

Cases
Case / familyJev selectionTable selectionJev route acceptedAssessment differencesLatency / tokens in + out
routine-3 / routinegpt-6-luna · lowgpt-6-luna · lowYesNone417 ms / 2450 + 307
implementation-1 / implementationgpt-6.1-sol · mediumgpt-6.1-sol · mediumYesNone441 ms / 2489 + 308
synthesis-2 / synthesisgpt-6-astra · highgpt-6-astra · highYesNone479 ms / 2502 + 307
demanding-1 / demandinggpt-6.1-sol · highgpt-6-astra · highNoNone462 ms / 2494 + 310
context-1 / contextAsk for contextAsk for contextYesNone472 ms / 2454 + 305
Expected routes and assessment values
Assessment details
CaseAcceptable routesTask type: actual / expectedComplexity: actual / rangeScope: actual / rangeConsequence: actual / range
routine-3gpt-6-luna: low, mediumoperations / operations0.05 / 0–1.20.01 / 0–1.50.96 / 0–1.5
implementation-1gpt-6.1-sol: medium, highimplementation / implementation1.09 / 0.7–21.15 / 0.5–1.81.37 / 0.5–2
synthesis-2gpt-6.1-sol: medium, high; gpt-6-astra: higharchitecture / research, architecture2.86 / 2–31.77 / 0.7–2.52.05 / 0.5–2.5
demanding-1gpt-6-astra: high, xhigh, maxdebugging / debugging2.97 / 2.2–31.05 / 0.8–2.81.92 / 1–3
context-1Ask for contextdebugging / implementation, debugging, other1.41 / 0–30.2 / 0–31.43 / 0–3
Selection confidence and missing information
Uncertainty details
CaseModel confidenceEffort confidenceMissing-information probability
routine-396.0%100.0%32.0%
implementation-175.0%66.0%52.0%
synthesis-273.0%92.0%16.0%
demanding-144.0%68.0%41.0%
context-171.0%33.0%96.0%
Dataset and question fingerprints
Dataset: 6b1f9f0a900ab49c57d4ab65d92be614ce25b7746c62c1e455759d04bd3e9b1b
Assessment: 72b8cada7fb1372418b5ca5cca9313db5f01e52cbff02f43bc9de6fe0797d977
Selection: 2d4bfd7510f64ba99e4551fb687be55d087d3d7e67ffe15b072fff2d939b1f88
Policy: bdf2a7715df957cb5a740da93d595913e81e544a269459fb120450b3f944aeb1

Matching fingerprints check compatibility. They do not prove that a result came from a live call.