notes-todos-lists-A-fb954a4d-c43-20260801

Supposed to	N	Correct	Wrong	Unverified
▸ LOG — log the entry	24	17 (71%)	7 (29%)	0 (0%)
7 handled wrong — click one to jump to its full detail below "note that the client prefers email over phone" Reported could-not-confirm (write-confirmation false negative). "note that down" WRITE-TRUTH FAIL — WRONG/MISSING content "parking on Main Street" — no saved note row with matching identity (rows: the client prefers email over phone) "jot this down somewhere" WRITE-TRUTH FAIL — WRONG/MISSING content "contractor needs the deposit by Friday" — no saved note row with matching identity (rows: the parking thing on main street) "uh put buy birthday card for mom on my list" Reported could-not-confirm (write-confirmation false negative). "put schedule oil change on my car to-do list" Did not write — misroute-into-workout; no notes-domain saved row. "add call the insurance company to my morning list" Reported could-not-confirm (write-confirmation false negative). "remind me tomorrow" OVER-ASK — asked instead of writing the note/todo/list (no saved row).
▸ UPDATE — update the entry	1	1 (100%)	0 (0%)	0 (0%)
No errors — all handled correctly.
Total	25	18 (72%)	7	0

PASSshould log the entry"note that the wifi password is sunflower22" (difficulty —)0.6s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: the wifi password is sunflower22.

Why verdict Notes write-truth verified: the wifi password is sunflower22.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.6s

1 · TTS said no speech captured

2 · Card shown Logged your note in General about the wifi password is sunflower22.

3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:20:22.858Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"jot down that parking is free after six on main street" (difficulty —)0.6s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: parking is free after six on main street.

Why verdict Notes write-truth verified: parking is free after six on main street.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.6s

1 · TTS said no speech captured

2 · Card shown Logged your note in General about parking is free after six on main street.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:20:34.591Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"remember the spare key is under the blue flower pot" (difficulty —)0.7s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: the spare key is under the blue flower pot.

Why verdict Notes write-truth verified: the spare key is under the blue flower pot.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.7s

1 · TTS said no speech captured

2 · Card shown Logged your note in General about the spare key is under the blue flower pot.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:20:46.560Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"make a note the contractor said they start on the fifteenth" (difficulty —)0.2s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: the contractor said they start on the fifteenth.

Why verdict Notes write-truth verified: the contractor said they start on the fifteenth.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Logged your note in General about the contractor said they start on the fifteenth.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:20:57.822Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"save a note my blood pressure was one twenty over eighty this morning" (difficulty —)0.2s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: my blood pressure was one twenty over eighty this morning.

Why verdict Notes write-truth verified: my blood pressure was one twenty over eighty this morning.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Logged your note in General about my blood pressure was one twenty over eighty this morning.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:21:09.167Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"note from the meeting budget is due friday" (difficulty —)0.3s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: budget is due friday, Meeting.

Why verdict Notes write-truth verified: budget is due friday, Meeting.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.3s

1 · TTS said no speech captured

2 · Card shown Logged your note in Meeting about budget is due friday. Say if that category should be different.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:21:20.599Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"jot that Jake's jersey size is youth large" (difficulty —)0.3s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: Jake's jersey size is youth large.

Why verdict Notes write-truth verified: Jake's jersey size is youth large.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.3s

1 · TTS said no speech captured

2 · Card shown Logged your note in General about Jake's jersey size is youth large.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:21:32.032Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

FAILshould log the entry"note that the client prefers email over phone" (difficulty —)0.6s

Verdict Expected LOG — should log the entry. FAIL: Reported could-not-confirm (write-confirmation false negative).

Why verdict Reported could-not-confirm (write-confirmation false negative).

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.6s

1 · TTS said no speech captured

2 · Card shown I'm not sure how to help with that yet. Try asking me to log food, a workout, a note, or a to-do.

4 · UI did no screen events captured

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:21:43.895Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"quick note the gym locker code is four seven two nine" (difficulty —)0.2s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: the gym locker code is four seven two nine.

Why verdict Notes write-truth verified: the gym locker code is four seven two nine.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Logged your note in General about the gym locker code is four seven two nine.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:21:55.359Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

FAILshould log the entry"note that down" (difficulty —)0.5s

Verdict Expected LOG — should log the entry. FAIL: WRITE-TRUTH FAIL — WRONG/MISSING content "parking on Main Street" — no saved note row with matching identity (rows: the client prefers email over phone)

Why verdict WRITE-TRUTH FAIL — WRONG/MISSING content "parking on Main Street" — no saved note row with matching identity (rows: the client prefers email over phone)

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.5s

1 · TTS said no speech captured

2 · Card shown Logged your note in General about the client prefers email over phone.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:22:06.868Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

FAILshould log the entry"jot this down somewhere" (difficulty —)0.2s

Verdict Expected LOG — should log the entry. FAIL: WRITE-TRUTH FAIL — WRONG/MISSING content "contractor needs the deposit by Friday" — no saved note row with matching identity (rows: the parking thing on main street)

Why verdict WRITE-TRUTH FAIL — WRONG/MISSING content "contractor needs the deposit by Friday" — no saved note row with matching identity (rows: the parking thing on main street)

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Logged your note in General about the parking thing on main street.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:22:31.506Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould update the entry"append also grab oat milk to my latest note" (difficulty —)0.2s

Verdict Expected UPDATE — should update the entry. PASS: Notes UPDATE write-truth verified (changed:personal_notes×1).

Why verdict Notes UPDATE write-truth verified (changed:personal_notes×1).

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Updated your note in General about the parking thing on main street.

4 · UI did navigated to "notes" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:23:07.106Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"remind me to call the dentist tomorrow" (difficulty —)0.4s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: call the dentist.

Why verdict Notes write-truth verified: call the dentist.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.4s

1 · TTS said no speech captured

2 · Card shown Added to-do: call the dentist. Priority 3, due Aug 2. Tell me if the title or due date should be different.

3 · App data rows written created todo_item: call the dentist

4 · UI did navigated to "todos" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:23:18.720Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"add pick up dry cleaning to my to-do list" (difficulty —)0.2s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: pick up dry cleaning.

Why verdict Notes write-truth verified: pick up dry cleaning.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Added to-do: pick up dry cleaning. Priority 3. Tell me if the title or due date should be different.

3 · App data rows written created todo_item: pick up dry cleaning

4 · UI did navigated to "todos" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:23:29.984Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"put return the library books on my to-dos" (difficulty —)0.2s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: return the library books.

Why verdict Notes write-truth verified: return the library books.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Added to-do: return the library books. Priority 3. Tell me if the title or due date should be different.

3 · App data rows written created todo_item: return the library books

4 · UI did navigated to "todos" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:23:41.337Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

FAILshould log the entry"uh put buy birthday card for mom on my list" (difficulty —)0.1s

Verdict Expected LOG — should log the entry. FAIL: Reported could-not-confirm (write-confirmation false negative).

Why verdict Reported could-not-confirm (write-confirmation false negative).

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.1s

1 · TTS said no speech captured

2 · Card shown I'm not sure how to help with that yet. Try asking me to log food, a workout, a note, or a to-do.

4 · UI did no screen events captured

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:23:52.506Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"remind me to text Sarah about dinner plans" (difficulty —)0.2s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: text Sarah about dinner plans.

Why verdict Notes write-truth verified: text Sarah about dinner plans.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Added to-do: text Sarah about dinner plans. Priority 3. Tell me if the title or due date should be different.

3 · App data rows written created todo_item: text Sarah about dinner plans

4 · UI did navigated to "todos" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:24:04.002Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

FAILshould log the entry"put schedule oil change on my car to-do list" (difficulty —)0.1s

Verdict Expected LOG — should log the entry. FAIL: Did not write — misroute-into-workout; no notes-domain saved row.

Why verdict Did not write — misroute-into-workout; no notes-domain saved row.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.1s

1 · TTS said no speech captured

2 · Card shown I need to know which workout set to update.

4 · UI did no screen events captured

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:24:15.125Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

FAILshould log the entry"add call the insurance company to my morning list" (difficulty —)0.0s

Verdict Expected LOG — should log the entry. FAIL: Reported could-not-confirm (write-confirmation false negative).

Why verdict Reported could-not-confirm (write-confirmation false negative).

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.0s

1 · TTS said no speech captured

2 · Card shown I'm not sure how to help with that yet. Try asking me to log food, a workout, a note, or a to-do.

4 · UI did no screen events captured

5 · Seed/context data

Seeded starting state:

Seeded value

{
  "table": "todo_categories",
  "record": {
    "id": "seed-todo-cat-morning",
    "name": "Morning",
    "created_at": {
      "$relativeDays": -14
    }
  }
}

snapshot captured 2026-08-01T13:24:40.763Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"new todo finish the slide deck by thursday" (difficulty —)0.2s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: finish the slide deck.

Why verdict Notes write-truth verified: finish the slide deck.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Added to-do: finish the slide deck. Priority 3, due Aug 6. Tell me if the title or due date should be different.

3 · App data rows written created todo_item: finish the slide deck

4 · UI did navigated to "todos" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:24:52.060Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"remind me friday morning to submit the expense report" (difficulty —)0.2s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: submit the expense report.

Why verdict Notes write-truth verified: submit the expense report.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.2s

1 · TTS said no speech captured

2 · Card shown Added to-do: submit the expense report. Priority 3, due Aug 7. Tell me if the title or due date should be different.

3 · App data rows written created todo_item: submit the expense report

4 · UI did navigated to "todos" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:25:14.826Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"add a to-do to renew my passport in the category Admin" (difficulty —)0.3s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: renew my passport.

Why verdict Notes write-truth verified: renew my passport.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.3s

1 · TTS said no speech captured

2 · Card shown Added to-do: renew my passport. Priority 3.

3 · App data rows written created todo_item: renew my passport

4 · UI did navigated to "todos" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:25:26.396Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

FAILshould log the entry"remind me tomorrow" (difficulty —)0.0s

Verdict Expected LOG — should log the entry. FAIL: OVER-ASK — asked instead of writing the note/todo/list (no saved row).

Why verdict OVER-ASK — asked instead of writing the note/todo/list (no saved row).

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.0s

1 · TTS said no speech captured

2 · Card shown Want me to add that as a to-do or a list item?

4 · UI did no screen events captured

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:25:37.471Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"add it to todos" (difficulty —)0.0s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: submit my timesheet.

Why verdict Notes write-truth verified: submit my timesheet.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.0s

1 · TTS said no speech captured

2 · Card shown What should the to-do say?

4 · UI did no screen events captured

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:26:00.436Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

PASSshould log the entry"make sure I text Sarah back and also add a to-do to buy milk" (difficulty —)0.3s

Verdict Expected LOG — should log the entry. PASS: Notes write-truth verified: buy milk, text Sarah back.

Why verdict Notes write-truth verified: buy milk, text Sarah back.

My override

Things I think are wrong

Use this when the report passed something that should have failed.

Comprehension

Picked wrong food Wrong quantity Should have asked (over-logged) Shouldn't have asked (over-asked) Wrong action entirely

Execution

Didn't save to DB Wrong data saved No voice/TTS Card missing/wrong No chat

Cosmetic

Wording weird

Other thing wrong (describe below)

Things the report marked wrong but I think are right

Use this when the report failed something that should have passed.

Actually right

Food/action was right Quantity/serving was right Asking was right Not asking was right Answer was right No app data row was expected App data row was right Voice/card/chat was acceptable

Report/grader issue

Report expected the wrong result Evidence was enough to judge Failure reason was not valid

Other thing actually right (describe below)

Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.

Timing 0.3s

1 · TTS said no speech captured

2 · Card shown Added 2 to-dos: text Sarah back; buy milk.

3 · App data rows written created todo_item: text Sarah back
created todo_item: buy milk

4 · UI did navigated to "todos" screen

5 · Seed/context data

Seeded starting state: none listed in the corpus row.

No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.

snapshot captured 2026-08-01T13:26:23.031Z

6 · Why this food No food-decision explanation was captured for this path.

7 · Lookup method No lookup method was captured for this path.

8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)

Why the fails happened — comprehension vs execution vs cosmetic

Handled correctly? — by expected action

Accuracy by difficulty

Clarification follow-ups — scored separately

Cosmetic only

System / infra

Latency

Filter — controls the list below

Per-utterance detail