📊 Status Dashboard ↑ all runs

acceptance-corpus-conversational-fb954a4d-c43-20260801

acceptance-corpus-conversational · preserved partial run · 542s elapsed · iOS sim
This is the preserved partial result for this run. Its original utterance evidence is shown below.
Rows
40
Pass
35 (88%)
Fail
4 (10%)
Unverified
1 (3%)
Pass rate
90%
Avg difficulty
—
"Unverified" = the grader couldn't confidently call it PASS or FAIL from the trace — needs a human look (that's you 👍/👎-ing it). "Pass rate" = pass ÷ (pass + fail) — it excludes Unverified rows, so it differs slightly from the Pass % above (which is share of ALL rows). Unverified breaks down as: 1 unclassified — the "grader coverage gap" ones aren't ambiguous, the grader just doesn't fully score that intent type yet.

Why the fails happened — comprehension vs execution vs cosmetic

Comprehension — picked the wrong action/target (the hard problem)
4 (100%)
Of 4 fails: if most are comprehension, that's the hard problem (the app didn't figure out the right action/target); execution/cosmetic means the decision was right but something downstream broke. Click a bar for the sub-split + example utterances.

Handled correctly? — by expected action

Click a row to see the ones it got wrong; click a wrong utterance to jump to its full detail below.
Supposed toNCorrectWrongUnverified
▸ CLARIFY — ask a clarifying question 3935 (90%) 4 (10%) 0 (0%)
▸ CLARIFY_ANSWER — complete the pending action 10 (0%) 0 (0%) 1 (100%)
Total4035 (90%)41

Accuracy by difficulty

Pending A1's per-utterance difficulty score (requested 2026-07-05) — this bar chart lights up once that lands.

Clarification follow-ups — scored separately

Second turn: app asked, we replied — did it resolve correctly?
Follow-ups
1
Resolved
0 (0%)
Mishandled
0 (0%)
Unverified
1

Cosmetic only

Not yet classified — pending A1 adding cosmetic-issue detection (double response, wording variance, rounding) to the grader. Nothing fabricated here.

System / infra

Not yet classified — pending confirmation from A1 whether trace data already carries infra-error signal (TTS/sync/DB-write errors) or needs new detection.

Latency

Avg (time to ready)
1.1s
p90
2.7s
Max
5.6s
Went async
0 (0%)
Each dot is one utterance (green=pass, red=fail, amber=unverified), positioned by how long it took to be ready for review. Bigger dots are the slow outliers — click any dot to jump to its detail.
0s
1s
2s
5s
6s
Response path — quick (single response) vs async (an ack like "Working on it…" before the real answer).
Sync clarification
39
Quick response
1
Slowest 8 utterances (click to jump to detail):
"I ate a burger"5.6s
"make it quieter"5.5s
"had eggs this morning"2.7s
"log my leg day"2.7s
"add some fruit"2.7s
"log cardio"2.7s
"track my run"2.7s
"track the set I just did"2.7s

Filter — controls the list below

Pass / Fail / Unverified
PASS 35 FAIL 4 UNVERIFIED 1
Module (intended for)
food (12)workout (10)lists (4)todos (3)notes (3)settings (4)navigation (4)
Utterance sub-type (within module)
40 shown — 35 pass, 4 fail, 1 unverified

Per-utterance detail

PASSshould ask a clarifying question"log a sandwich" (difficulty —)1.0s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 1.0s
1 · TTS said no speech captured
2 · Card shown What food did you log? Could you be more specific?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:18:22.324Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"I had some pasta" (difficulty —)2.2s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Decisive guess with stated assumption + invited correction (good-guessing).
Why verdict Decisive guess with stated assumption + invited correction (good-guessing).
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 2.2s
1 · TTS said no speech captured
2 · Card shown I logged Pasta, Cooked — let me know if that's not right.
3 · App data rows written created food_log_entry: Pasta, Cooked ×1 (1 cup, cooked) 220 cal · 8.1g P · 43g C · 1.3g F
created food_log_entry: Pasta, Cooked ×1 (1 cup, cooked) 220 cal · 8.1g P · 43g C · 1.3g F
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:18:35.640Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method The answer used provisional food evidence from the USDA nutrition database.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"track my lunch" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Decisive guess with stated assumption + invited correction (good-guessing).
Why verdict Decisive guess with stated assumption + invited correction (good-guessing).
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown Logged Pasta, Cooked. Using your recent Pasta, Cooked history. Tell me if that is wrong.
3 · App data rows written created food_log_entry: Pasta, Cooked ×1 (1 cup, cooked) 220 cal · 8.1g P · 43g C · 1.3g F
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:18:46.840Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method The answer used provisional food evidence from the USDA nutrition database.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"log a coffee" (difficulty —)0.2s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Decisive guess with stated assumption + invited correction (good-guessing).
Why verdict Decisive guess with stated assumption + invited correction (good-guessing).
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.2s
1 · TTS said no speech captured
2 · Card shown Logged a coffee. Assumed 100 g because no amount was stated.
3 · App data rows written created food_log_entry: Coffee ×1 (100 g) 1 cal · 0.1g P · 0g C · 0g F
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:18:58.153Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method The answer used provisional food evidence from the common-food list.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"I ate a burger" (difficulty —)5.6s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 5.6s
1 · TTS said no speech captured
2 · Card shown What exact food and amount should I use for Hamburger? I did not log it yet because I could not safely finish that food log.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:19:14.853Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"add some fruit" (difficulty —)2.7s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 2.7s
1 · TTS said no speech captured
2 · Card shown What exact food and amount should I use for some fruit? I did not log it yet because I could not safely finish that food log.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:19:28.731Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"log dinner from earlier" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown What did you have for dinner?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:19:39.918Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"I had a smoothie" (difficulty —)1.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 1.1s
1 · TTS said no speech captured
2 · Card shown What food did you log? Could you be more specific?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:19:52.122Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"log a snack" (difficulty —)1.0s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 1.0s
1 · TTS said no speech captured
2 · Card shown What exact food and amount should I use for a snack? I did not log it yet because I could not safely finish that food log.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:20:04.286Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"I drank something" (difficulty —)0.9s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.9s
1 · TTS said no speech captured
2 · Card shown Which one did you mean? 1. Stand For Something Drink, Lemonade — KEITH'S ICE COLD BEVERAGES, 2. Keith'S Ice Cold Beverages, Stand For Something Drink, Lemonade — Cutting Edge Beverages
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:20:16.342Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"had eggs this morning" (difficulty —)2.7s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 2.7s
1 · TTS said no speech captured
2 · Card shown What exact food and amount should I use for had eggs this morning? I did not log it yet because I could not safely finish that food log.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:20:30.159Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"log the salad I just ate" (difficulty —)2.6s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 2.6s
1 · TTS said no speech captured
2 · Card shown Did the salad have dressing or anything calorie-heavy on it? I did not log it yet because add-ons can change the nutrition.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:20:43.970Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"log some squats" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown Which exercise, weight, and reps should I log?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:20:55.128Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"I did bench press" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown Which exercise, weight, and reps should I log?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:21:06.312Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"track my run" (difficulty —)2.7s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 2.7s
1 · TTS said no speech captured
2 · Card shown What exact food and amount should I use for my run? I did not log it yet because I could not safely finish that food log.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:21:20.168Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"log deadlifts" (difficulty —)0.0s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.0s
1 · TTS said no speech captured
2 · Card shown Which exercise, weight, and reps should I log?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:21:31.515Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"I did some curls" (difficulty —)2.5s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 2.5s
1 · TTS said no speech captured
2 · Card shown What weight and how many reps should I log for your curls?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:21:45.117Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"log cardio" (difficulty —)2.7s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 2.7s
1 · TTS said no speech captured
2 · Card shown What exact food and amount should I use for cardio? I did not log it yet because I could not safely finish that food log.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:21:59.126Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"did pull-ups" (difficulty —)0.0s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.0s
1 · TTS said no speech captured
2 · Card shown Which exercise, weight, and reps should I log?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:22:10.175Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"log my leg day" (difficulty —)2.7s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 2.7s
1 · TTS said no speech captured
2 · Card shown What exact food and amount should I use for my leg day? I did not log it yet because I could not safely finish that food log.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:22:24.055Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"I benched earlier" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown Should I add that as a to-do?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:22:46.766Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"track the set I just did" (difficulty —)2.7s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 2.7s
1 · TTS said no speech captured
2 · Card shown What exact food and amount should I use for the set i just did? I did not log it yet because I could not safely finish that food log.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:23:00.559Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"add milk" (difficulty —)1.2s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 1.2s
1 · TTS said no speech captured
2 · Card shown What food did you log? Could you be more specific?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:23:12.961Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
UNVERIFIEDunclassifiedshould complete the pending action"put batteries on the list" (difficulty —)0.1s
Verdict Expected CLARIFY_ANSWER — should complete the pending action. UNVERIFIED: CLARIFY_ANSWER not gradeable single-shot — no pending clarification in this replay (app re-asked / mis-parsed as fresh); needs multi-turn replay.
Why verdict CLARIFY_ANSWER not gradeable single-shot — no pending clarification in this replay (app re-asked / mis-parsed as fresh); needs multi-turn replay.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown Which list should I add batteries to?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:23:24.229Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"remind me about that thing" (difficulty —)0.3s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Decisive guess with stated assumption + invited correction (good-guessing).
Why verdict Decisive guess with stated assumption + invited correction (good-guessing).
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.3s
1 · TTS said no speech captured
2 · Card shown Added to-do: about that thing. Priority 3. Tell me if the title or due date should be different.
3 · App data rows written created todo_item: about that thing
4 · UI did navigated to "todos" screen
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:23:35.669Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
FAILshould ask a clarifying question"add a task for tomorrow" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. FAIL: Reported could-not-confirm (write-confirmation false negative).
Why verdict Reported could-not-confirm (write-confirmation false negative).
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown I'm not sure how to help with that yet. Try asking me to log food, a workout, a note, or a to-do.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:23:46.881Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
FAILshould ask a clarifying question"make a note" (difficulty —)0.2s
Verdict Expected CLARIFY — should ask a clarifying question. FAIL: Logged a BLIND guess — no stated assumption, no correction invited.
Why verdict Logged a BLIND guess — no stated assumption, no correction invited.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.2s
1 · TTS said no speech captured
2 · Card shown Logged your note in General about for tomorrow.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did navigated to "notes" screen
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:23:58.258Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
FAILshould ask a clarifying question"jot something down" (difficulty —)0.3s
Verdict Expected CLARIFY — should ask a clarifying question. FAIL: Logged a BLIND guess — no stated assumption, no correction invited.
Why verdict Logged a BLIND guess — no stated assumption, no correction invited.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.3s
1 · TTS said no speech captured
2 · Card shown Logged your note in General about something down.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did navigated to "notes" screen
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:24:09.769Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"add it to my list" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown Which list should I add that to, and what is the item called?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:24:20.935Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"create a todo" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown What should the to-do say?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:24:32.129Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
FAILshould ask a clarifying question"save a note about the meeting" (difficulty —)0.2s
Verdict Expected CLARIFY — should ask a clarifying question. FAIL: Logged a BLIND guess — no stated assumption, no correction invited.
Why verdict Logged a BLIND guess — no stated assumption, no correction invited.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.2s
1 · TTS said no speech captured
2 · Card shown Logged your note in General about about the meeting.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did navigated to "notes" screen
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:24:43.553Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"add eggs and butter" (difficulty —)0.4s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.4s
1 · TTS said no speech captured
2 · Card shown How much should I log for Egg and Butter? I did not log them yet because the amounts were not clear.
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:24:55.107Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"change my settings" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown Which setting would you like to change?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:25:06.346Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"turn something off" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown Should I add that as a to-do?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:25:17.512Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"update my preferences" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown I couldn't identify which food entry you meant. Which one would you like to correct?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:25:28.771Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"make it quieter" (difficulty —)5.5s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 5.5s
1 · TTS said no speech captured
2 · Card shown Which exact make it quieter should replace Coffee?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:25:45.461Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"go there" (difficulty —)0.1s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.1s
1 · TTS said no speech captured
2 · Card shown What would you like me to do with that?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:25:56.547Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"open that screen" (difficulty —)0.0s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.0s
1 · TTS said no speech captured
2 · Card shown What would you like me to do with that?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:26:07.717Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"take me back" (difficulty —)0.0s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 0.0s
1 · TTS said no speech captured
2 · Card shown What would you like me to do with that?
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:26:18.883Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)
PASSshould ask a clarifying question"show me the other one" (difficulty —)1.9s
Verdict Expected CLARIFY — should ask a clarifying question. PASS: Asked a clarification as expected.
Why verdict Asked a clarification as expected.
My override
Things I think are wrong
Use this when the report passed something that should have failed.
Comprehension
Execution
Cosmetic
Things the report marked wrong but I think are right
Use this when the report failed something that should have passed.
Actually right
Report/grader issue
Pick every reason you think applies. Multiple boxes are allowed because one utterance can be wrong in more than one way. Use the note below for "Other" or extra detail.
Timing 1.9s
1 · TTS said no speech captured
2 · Card shown I'm not sure which item you're referring to. Could you be more specific? For example, you could say "show me the other food entry" or "show me the other to-do."
3 · App data rows written No food/workout/list/note data row captured for this utterance.This does not mean "nothing anywhere was stored"; spoken/chat evidence is shown above, and some chat/session records may live outside this write-capture field. It means the report did not capture an app data-row write such as food_log_entries or workout_sets.
4 · UI did no screen events captured
5 · Seed/context data
Seeded starting state: none listed in the corpus row.
  • No DB snapshot captured for this utterance; the report cannot prove whether yesterday/personal data existed.
snapshot captured 2026-08-01T13:26:31.953Z
6 · Why this food No food-decision explanation was captured for this path.
7 · Lookup method No lookup method was captured for this path.
8 · Candidates → decider no candidate list captured (this path did not run DB resolution — e.g. provisional branded log)