Showing 95 of 95 rows
| # | What was said | App's final reply | Verdict | Reason |
|---|---|---|---|---|
| 1 | remind me tomorrow | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 2 | make a new list for that | What should I call the new list? | Fail | OVER-ASK — asked instead of writing the note/todo/list (no saved row). |
| 3 | note that down | Sure. What should the note say? | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 4 | remind me later | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 5 | jot this down somewhere | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 6 | put schedule oil change on my car to-do list | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 7 | add it to todos | What should the to-do say? | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 8 | add eggs | How much should I log for Egg? I did not log it yet because the amount was not clear. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 9 | new reminder for the appointment | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 10 | uh same reminder as last week but for friday | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 11 | put that on packing | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 12 | put milk on the list | Which list should I add milk to? | Fail | OVER-ASK — asked instead of writing the note/todo/list (no saved row). |
| 13 | mark that done | Which to-do should I mark done: pay the electric bill on the first of every month; water the plants every; take out the trash tonight at seven? | Fail | MISROUTE — general/no-module dead-end (should update the entry). |
| 14 | remind me about that thing tomorrow | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 15 | mark call the dentist done | Marked done: call the dentist. | Fail | MISROUTE — general/no-module dead-end (should update the entry). |
| 16 | add the usual stuff to shopping | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 17 | add that to my list | Which list should I add that to, and what is the item called? | Fail | OVER-ASK — asked instead of writing the note/todo/list (no saved row). |
| 18 | put bananas and uh the other thing on groceries | Reminder or shopping-list intent belongs to todos/lists, not food. | Fail | Did not write — misroute-into-food; no notes-domain saved row. |
| 19 | check that off | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should update the entry). |
| 20 | remove that from the list | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should delete the entry). |
| 21 | edit that note | no_notes_intent_match | Fail | MISROUTE — general/no-module dead-end (should update the entry). |
| 22 | create a hardware store list | Created list: Hardware store list. | Pass | Notes write-truth verified: Hardware Store List. |
| 23 | jot down that parking is free after six on main street | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 24 | add milk to my grocery list | Added milk to Grocery List. Say if that (Grocery List) is the wrong list. | Pass | Notes write-truth verified: milk milk. |
| 25 | add call the insurance company to my morning list | todo_category_container | no evidence captured | UNVERIFIABLE — no saved notes/todo/list row captured (untagged / lossy). App claim not credited without write-tag/turn-window proof. |
| 26 | quick note the gym locker code is four seven two nine | I could not confirm that the change was saved, so I did not mark it as done. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 27 | remind me to call the dentist tomorrow | Added to-do: call the dentist. Priority 3, due Jul 24. | Pass | Notes write-truth verified: call the dentist. |
| 28 | start a beach list for saturday | Created list: Beach list for saturday. | Pass | Notes write-truth verified: Beach List For Saturday. |
| 29 | make a shopping list | Created list: Shopping list. | Pass | Notes write-truth verified: Shopping List. |
| 30 | add two bottles of sparkling water to groceries | I logged Water — let me know if that's not right. | no evidence captured | UNVERIFIABLE — no saved notes/todo/list row captured (untagged / lossy). App claim not credited without write-tag/turn-window proof. |
| 31 | create a packing list for the trip | Created list: Packing list for the trip. | Pass | Notes write-truth verified: Packing List For The Trip. |
| 32 | make a note the contractor said they start on the fifteenth | I could not confirm that the change was saved, so I did not mark it as done. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 33 | save a note my blood pressure was one twenty over eighty this morning | I could not confirm that the change was saved, so I did not mark it as done. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 34 | add pick up dry cleaning to my to-do list | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 35 | add greek yogurt to the shopping list | Added greek yogurt to Shopping List. Say if that (Shopping List) is the wrong list. | Pass | Notes write-truth verified: greek yogurt greek yogurt. |
| 36 | make sure I text Sarah back and also add a to-do to buy milk | What fat level or type of milk should I use? I did not log it yet because milk fat level can change the nutrition. | Fail | Did not write — misroute-into-food; no notes-domain saved row. |
| 37 | jot that Jake's jersey size is youth large | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 38 | remember the spare key is under the blue flower pot | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 39 | new todo finish the slide deck by thursday | Added to-do: finish the slide deck. Priority 3, due Jul 23. Say if the title or due date should be different. | Pass | Notes write-truth verified: finish the slide deck. |
| 40 | put return the library books on my to-dos | Added to-do: return the library books. Priority 3. Say if the title or due date should be different. | Pass | Notes write-truth verified: return the library books. |
| 41 | remind me when I get home to defrost the chicken | I logged Chicken breast — let me know if that's not right. | no evidence captured | UNVERIFIABLE — no saved notes/todo/list row captured (untagged / lossy). App claim not credited without write-tag/turn-window proof. |
| 42 | note from the meeting budget is due friday | I could not confirm that the change was saved, so I did not mark it as done. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 43 | add fix the leaky faucet to weekend projects | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 44 | note that the client prefers email over phone | not_create_or_append | no evidence captured | UNVERIFIABLE — no saved notes/todo/list row captured (untagged / lossy). App claim not credited without write-tag/turn-window proof. |
| 45 | new pet supplies list please | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 46 | add a to-do to renew my passport in the category Admin | Added to-do: renew my passport in the category Admin. Priority 3. | Pass | Notes write-truth verified: renew my passport in the category Admin. |
| 47 | start a grocery list | Created list: Grocery list. | Pass | Notes write-truth verified: Grocery List. |
| 48 | new list called weekend projects | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 49 | uh put buy birthday card for mom on my list | no_lists_intent_match | no evidence captured | UNVERIFIABLE — no saved notes/todo/list row captured (untagged / lossy). App claim not credited without write-tag/turn-window proof. |
| 50 | add cat litter to my pet supplies list | Created list: Pet supplies. Added cat litter. | Pass | Notes write-truth verified: Pet Supplies, cat litter cat litter. |
| 51 | add sunscreen and snacks to the beach list | Added 2 items to Beach List For Saturday: sunscreen and snacks. Say if that (Beach List For Saturday) is the wrong list. | Pass | Notes write-truth verified: sunscreen sunscreen, snacks snacks. |
| 52 | append also grab oat milk to my latest note | I logged Milk — let me know if that's not right. | no evidence captured | UNVERIFIABLE — no notes-domain mutation captured for UPDATE. App claim not credited. |
| 53 | note that the wifi password is sunflower22 | I could not confirm that the change was saved, so I did not mark it as done. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 54 | remind me to text Sarah about dinner plans | Added to-do: text Sarah about dinner plans. Priority 3. | Pass | Notes write-truth verified: text Sarah about dinner plans. |
| 55 | add charger and passport to my packing list | Added 2 items to Packing List For The Trip: charger and passport. Say if that (Packing List For The Trip) is the wrong list. | Pass | Notes write-truth verified: charger charger, passport passport. |
| 56 | remind me friday morning to submit the expense report | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 57 | set a reminder every monday to take out the recycling | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 58 | mark the laundry done | Marked done: laundry. | no evidence captured | UNVERIFIABLE — no notes-domain mutation captured for UPDATE. App claim not credited. |
| 59 | remind me to take out the trash tonight at seven | Added to-do: take out the trash tonight at seven. Priority 3. | Pass | Notes write-truth verified: take out the trash tonight at seven. |
| 60 | throw apples oranges and bananas on the fruit list | Reminder or shopping-list intent belongs to todos/lists, not food. | no evidence captured | UNVERIFIABLE — no saved notes/todo/list row captured (untagged / lossy). App claim not credited without write-tag/turn-window proof. |
| 61 | add dish soap sponges and trash bags to the household list | Created list: Household. Added 3 items: dish soap, sponges, and trash bags. | Pass | Notes write-truth verified: Household, dish soap dish soap, sponges sponges, trash bags trash bags. |
| 62 | I finished the buy milk task | I could not find a to-do matching buy milk. Did you mean: pay the electric bill on the first of every month; water the plants every; take out the trash tonight at seven? | Fail | OVER-ASK — asked instead of update (no notes-domain mutation). |
| 63 | add three avocados and two limes to groceries | I found Avocado, but I also heard limes to groceries. Should I log those separately too? | Fail | OVER-ASK — asked instead of writing the note/todo/list (no saved row). |
| 64 | add 1.5 cups of rice to my meal prep list | Created list: Meal prep. Added 1.5 cups of rice. | Pass | Notes write-truth verified: Meal Prep, 1.5 cups of rice 1.5 cups of rice. |
| 65 | mark all the morning tasks done | I could not find a to-do matching all the morning. Did you mean: pay the electric bill on the first of every month; water the plants every; take out the trash tonight at seven? | Fail | OVER-ASK — asked instead of update (no notes-domain mutation). |
| 66 | remind me to pay the electric bill on the first of every month | Added to-do: pay the electric bill on the first of every month. Priority 3. | Pass | Notes write-truth verified: pay the electric bill on the first of every month. |
| 67 | add chicken thighs broccoli and brown rice to meal prep | Logged Chicken thigh and Cooked brown rice. | no evidence captured | UNVERIFIABLE — no saved notes/todo/list row captured (untagged / lossy). App claim not credited without write-tag/turn-window proof. |
| 68 | add shampoo conditioner and body wash to the drugstore list | Created list: Drugstore. Added 3 items: shampoo, conditioner, and body wash. | Pass | Notes write-truth verified: Drugstore, shampoo shampoo, conditioner conditioner, body wash body wash. |
| 69 | change take out trash from seven to eight thirty | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should update the entry). |
| 70 | remind me every morning at eight to take my vitamins | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 71 | mark pick up dry cleaning as complete | I could not find a to-do matching pick up dry cleaning. Did you mean: pay the electric bill on the first of every month; water the plants every; take out the trash tonight at seven? | Fail | OVER-ASK — asked instead of update (no notes-domain mutation). |
| 72 | new to-do water the plants every sunday | Added to-do: water the plants every. Priority 3, due Jul 26. Say if the title or due date should be different. | Pass | Notes write-truth verified: water the plants every. |
| 73 | set the expense report todo due date to friday at noon | not_create_complete_or_list | no evidence captured | UNVERIFIABLE — no notes-domain mutation captured for UPDATE. App claim not credited. |
| 74 | put tape measure screws and anchors on the hardware list | Added 3 items to Hardware Store List: tape measure, screws, and anchors. Say if that (Hardware Store List) is the wrong list. | Pass | Notes write-truth verified: tape measure tape measure, screws screws, anchors anchors. |
| 75 | remind me in two hours to check the oven | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 76 | mark defrost chicken done | I could not find a to-do matching defrost chicken. Did you mean: pay the electric bill on the first of every month; water the plants every; take out the trash tonight at seven? | Fail | OVER-ASK — asked instead of update (no notes-domain mutation). |
| 77 | check off call the dentist | call the dentist is already done — want me to uncheck it? | Pass | UPDATE — no notes-domain write (as expected). |
| 78 | remind me next monday to renew my car registration | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should log the food). |
| 79 | add batteries and light bulbs to the hardware list | Did you mean 2 items: batteries and light bulbs? | Fail | OVER-ASK — asked instead of writing the note/todo/list (no saved row). |
| 80 | add tomatoes onions and cilantro to the taco night list | Created list: Taco night. Added 3 items: tomatoes, onions, and cilantro. | Pass | Notes write-truth verified: Taco Night, tomatoes tomatoes, onions onions, cilantro cilantro. |
| 81 | add milk eggs and bread to my grocery list | Added 3 items to Grocery List: milk, eggs, and bread. Say if that (Grocery List) is the wrong list. | Fail | WRITE-TRUTH FAIL — WRONG/MISSING content "milk" — no saved list_item row with matching identity (rows: eggs eggs | bread bread) |
| 82 | add eggs bread and butter to the shopping list | Added 3 items to Shopping List: eggs, bread, and butter. Say if that (Shopping List) is the wrong list. | Pass | Notes write-truth verified: eggs eggs, bread bread, butter butter. |
| 83 | add olive oil and paper towels to groceries | Reminder or shopping-list intent belongs to todos/lists, not food. | no evidence captured | UNVERIFIABLE — no saved notes/todo/list row captured (untagged / lossy). App claim not credited without write-tag/turn-window proof. |
| 84 | remove eggs from groceries | I don't see a eggs from groceries logged today. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 85 | remove bread and butter from the shopping list | I couldn't identify which food entry you meant. Which one would you like to correct? | Fail | Did not delete — misroute-into-food. |
| 86 | take eggs off the list | I logged Egg — let me know if that's not right. | no evidence captured | UNVERIFIABLE — no notes-domain mutation captured for DELETE. App claim not credited. |
| 87 | remove cat litter from pet supplies | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should delete the entry). |
| 88 | remove milk from my grocery list | I could not confirm that the change was saved, so I did not mark it as done. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 89 | edit the grocery list change two percent milk to oat milk | I could not confirm that the change was saved, so I did not mark it as done. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 90 | update the note about the contractor start date to the twentieth | I could not confirm that the change was saved, so I did not mark it as done. | Fail | Reported could-not-confirm (write-confirmation false negative). |
| 91 | remove the second item on my grocery list | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should delete the entry). |
| 92 | uncheck water the plants I still need to do it | I could not find a to-do matching water the plants. Did you mean: call the dentist; laundry? | Fail | OVER-ASK — asked instead of update (no notes-domain mutation). |
| 93 | change call insurance to call the insurance company about the claim | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should update the entry). |
| 94 | change the dentist reminder to next wednesday | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should update the entry). |
| 95 | remove passport from the packing list | No registered new-shell module for general. | Fail | MISROUTE — general/no-module dead-end (should delete the entry). |
Verdicts come from the grade file. Replies come only from the app side of chat history (not guessed). When the grader could not prove a result, the badge says no evidence captured.