shipped

Spec: F15 — Feedback on AI Outputs

Status: shipped · Audited: 2026-07-16 · Contract: partial until browser proof; feedback persistence is the F15 boundary and any evaluation consumer is future scope

1. Problem & CEO Job

Trust the signal — the feedback evidence that can later calibrate it. Lumi asks CEOs to trust AI-extracted items; the only honest way to earn that is to let them say “wrong” cheaply and visibly. Each typed row is retained as labeled evidence for a future, separately owned evaluation workflow. F15 itself owns capture and persistence, not automatic model adjustment. Capture must cost one tap; anything more and we get silence, not signal.

2. User Stories & Acceptance Criteria

  • JTBD (CEO — evidence capture): “When I give feedback on a Brief item — tap ‘wrong’ or ‘too_low_priority’ — I want Lumi to retain that correction as a labeled signal, so the product team can evaluate it in a separately owned feedback workflow.”

    • Given repeated not_useful or too_low_priority feedback on a specific target_type from the same member, the tenant-scoped rows remain available as labeled evidence. F15 does not claim that a model-evaluation consumer or automatic learning loop exists.
    • Given feedback submitted, no navigation occurs and the Brief item remains visible — the CEO is not yanked away from their workflow. (FeedbackButtons component: no navigate() call, in-place feedback)

  • As a CEO, I want to flag a brief item as wrong or useless in one tap, so that complaining is cheaper than ignoring.

    • Given any AI-touched item (brief item, commitment, risk, decision, chat message), then a feedback affordance exists where the item is displayed.
    • Given a tap, then Feedback row persists (target_type + target_id + type); optional comment never blocks the tap.

  • As the product team, I want typed feedback, so that signals are analyzable: useful / not_useful / wrong / missing_context / too_verbose / too_low_priority.

  • Feedback is tenant-scoped and attributed to the member; no anonymous feedback.

3. Scope Boundaries

  • In: submit endpoint, six feedback types, optional comment, attachment to any AI-output target.
  • Out: feedback analytics UI (query the DB until volume justifies a screen), automatic model adjustment, model-evaluation consumers, public ratings.

Why feedback matters — evidence-trust calibration

Feedback is not a rating widget; it is the calibration layer that makes the AI’s extractions trustworthy over time. Lumi asks the owner to trust items pulled out of messy Zalo/Excel chaos (F70) and out of daily pulses — every wrong / too_low_priority tap teaches the extractor what this owner actually acts on, so future Briefs surface fewer false positives and the owner spends less time correcting. The signal is persistent and typed (six feedback types), attached to a specific AI output via (target_type, target_id), and becomes a labeled example for the real model. The point is operational, not social: the owner’s correction burden goes down as the graph and the model learn, which is the only honest way to earn the trust the AI keeps asking for.

Success Metrics

MetricTargetWhy
Feedback submission rate per Brief>15% of members submit at least 1 feedback per Brief deliveredIf the one-tap affordance works, a meaningful share of daily active users should leave signal per Brief session. Below 10% means the affordance is invisible or the tap is too many taps. Measured per Brief instance, not per week, to normalize against Brief delivery frequency.
Brief relevance score (user-reported)>65% of CEOs rate their most recent Brief as “relevant” or “very relevant”Direct product-market signal for the Brief feature as a whole (F12). Lumi’s feedback loop is only justified if it moves this number — otherwise, the labels are collected but not acted on. Baseline measured at first real-org deployment.
Reduction in CEO action needed per Brief item over 4 weeksDownward trend in not_useful / too_low_priority rate on the same target_type per CEO over a rolling 4-week windowThe learning loop claim (§2 JTBD) must be observable. If a CEO repeatedly taps “too_low_priority” on risk items and the rate of risk items in their Brief does not fall after 4 weeks, the calibration claim is false. This is the north-star metric for §1 (“future training/eval dataset”).
AI output correction rate>60% of “wrong”/“not_useful” feedback leads to a detectable pattern (same target_type, same surface)Feedback is only valuable if it drives improvement. A high correction rate means the signal is specific enough to act on.
Feedback-to-improvement cycle time<7 days from “wrong” cluster detection to model/intent fixCEOs who flag a wrong answer should see the fix in the same week. A 7-day cycle keeps feedback feeling like a lever, not a black hole.
Silent-dismissal rate (target)<30% — i.e., >70% of dissatisfaction surfaces as typed feedback, not silent rejectionIf most CEOs dismiss wrong brief items without tapping feedback, the affordance is not cheap enough. Track by monitoring dismissals unaccompanied by a Feedback row.

4. Data Model

briefs.Feedback — target_type (brief_item/commitment/risk/decision/task/chat_message) + target_id (UUID, loose reference by design), feedback_type, comment, member FK.

5. API Contract

MethodPathMin roleNotes
POST/api/feedback/member{target_type, target_id, feedback_type, comment?}

6. UI / UX

  • No page. Inline affordance (👍/👎 expanding to typed options) on brief items first, other surfaces as touched.
  • Submitting acknowledges quietly (subtle confirm, no toast celebration) and never navigates away.
  • i18n: keys live in the surface’s namespace (brief, etc.).

7. Test Plan

#CaseTypeWhere
1Submit per target type; validation; tenant scopingbackendtests/test_api.py::TestFeedback, tests/test_lumi.py::TestFeedback
2Feedback affordances render on AI surfacese2efrontend/tests/e2e/lumi.spec.ts describe("Feedback")

8. Current Behavior

Backend + web are shipped. Users can record feedback on AI outputs through existing web surfaces, and that path exists as a lightweight correction mechanism rather than a separate workflow hub. FeedbackButtons are wired into the web BriefItemCard alongside DismissPanel, plus the other web AI-touched surfaces.

Mobile is now shipped for the current in-scope AI work surfaces too: the shared KMP Brief, Ask Lumi, Commitments, Risks, and Decisions surfaces used by iOS and Android all submit inline feedback against /api/feedback/ without bouncing the user to web.

9. Error States

#ErrorResponseNotes
ES1Missing required fields — POST without target_type, target_id, or feedback_type400 — feedback_type is required per §5; error body names missing field{"feedback_type": ["This field is required."]}
ES2Invalid feedback_type — value outside the six-type enum400 with allowable-values messageStandard DRF choice-field validation
ES3Invalid target_type — value outside recognized target types400Enum defined in model choices; drift between model and prose §4 possible
ES4Non-existent target_id — syntactically valid UUID but target deleted201 (row written, orphaned)Loose reference by design (§4); no FK constraint; downstream eval must handle gracefully
ES5Unauthenticated request401 (no token) / 403 (not in active org)IsAuthenticated + IsOrgMember gates
ES6Cross-tenant target_id — target belongs to different tenant but passes loose referenceFor target_type="brief_item": 400 — the target is validated against the requesting org (briefs/views.py lines 228-230). For all other target types the loose-reference design remains: 201 written (no FK means no cross-tenant FK rejection)brief_item cross-tenant submissions are now rejected; other target types are not validated
ES7Concurrent duplicate submissions — two rapid POSTs with identical payload before a uniqueness constraint fires409 — an idempotency guard (briefs/views.py lines 232-240) rejects an exact duplicate (org + user + target_type + target_id + feedback_type). Race edge: no DB unique_together, so two concurrent requests that both pass the pre-check before either write could still produce two rowsIdempotency guard is in place; no DB-level constraint, so a narrow race edge remains

Labeled Error States

IDScenarioExpected behavior
ES01Feedback submitted on an already-dismissed (or previously-feedbacked) item by the same memberIdempotent: new row created (no uniqueness constraint). If constraint added later, 409 with existing row UUID is the correct shape.
ES02Feedback POST with no feedback_type400 — feedback_type is required; carries no calibration value without a typed signal.
ES03Feedback on a target_id whose parent pulse entry has been deleted201 — orphaned row accepted by design (§4); downstream eval skips orphaned rows gracefully.

10. Diagrams & Docs Touched

backend/DIAGRAMS.md ER diagram.

11. Implementation Audit (2026-06-13)

#ClaimStatusEvidence
1POST /api/feedback/ creates Feedback row with typed signal; optional comment never blocks✅ matchtest_feedback_without_comment_succeeds; test_feedback_with_empty_comment_succeeds; test_feedback_with_comment_succeeds (test_red_r19_e.py)
2FeedbackButtons rendered on brief items, commitments, risks, decisions, chat messages✅ matchWeb matches (frontend/src/test/F15Feedback.test.tsx); mobile shared Brief, Ask Lumi, Commitments, Risks, and Decisions now all submit real inline feedback from the KMP surfaces (android/shared/src/commonMain/kotlin/com/locuno/lumi/shared/ui/{brief,chat,commitments,risks,decisions}/ + android/shared/src/commonTest/kotlin/com/locuno/lumi/shared/presentation/{brief,chat,commitments,risks,decisions}/ + android/shared/src/commonTest/kotlin/com/locuno/lumi/shared/api/LumiApiClientTest.kt)
3No navigation on submit; Brief item stays visible in-place✅ matchFeedbackButtons component: no navigate() call; in-place feedback confirmed
4Tenant-scoped; member attribution required✅ matchFeedback.organization FK, Feedback.user FK; test_feedback_cross_tenant_isolation
5Six feedback types accepted; optional comment for all types✅ matchDRF choice-field validation; comment is blank=True in model
6Feedback on deleted targets — loose UUID reference, 201, orphaned row accepted✅ spec-by-design§4 design decision documented in §9 ES4/ES03; no FK constraint intentional
7Duplicate feedback from same member accumulates rows; no uniqueness constraint enforced✅ matchIdempotency guard at briefs/views.py lines 232-240: rejects duplicate (org + user + target_type + target_id + feedback_type) with 409. No DB unique_together, so a concurrent-write race edge remains, but the check is present
8No rate limiting — one member can flood the eval dataset with rapid submissions✅ matchFeedbackRateThrottle(UserRateThrottle, scope="feedback") at briefs/views.py lines 19-20; @throttle_classes([FeedbackRateThrottle]) on submit_feedback at line 211; manual 20/min cache cap at lines 214-218; rate "feedback": "30/hour" in config/settings/base.py line 152. Remaining race: DB-constraint edge only
9JTBD learning loop: feedback rows are labeled examples, but no model pipeline reads them yet⚠️ gapAC not struck; data is collected but loop is not closed; “future training/eval dataset” (§1)

Summary: 8/9 match; 1 gap (learning loop not closed).

Edge Cases Catalog (formerly §12, 2026-06-12): Key items integrated above. Remaining: feedback on deleted targets (✅ by design, ES4); empty comment with all feedback types (✅ tested); all six types on any target type (allowed, no per-target-type validation); cross-tenant target_id for non-brief_item types passes loose reference (ES6 updated above). Full catalog in git history.

12. Open Questions

  • When does feedback volume justify a review loop (weekly export? eval set)? Revisit at first real-org deployment.

13. Bug / Change Log

2026-07-16 — Contract Graph boundary reconciliation

F15 now states the shipped boundary directly: typed feedback is tenant-scoped evidence capture. A model-evaluation or training consumer is future scope and is not required to establish F15’s API or web-affordance contract. The remaining Contract Graph gap for F15 is browser execution.

2026-07-14 — Contract Graph ownership boundary

F15 is now represented by contracts/features/F15.cue. The contract owns the typed feedback API and the web Brief affordance while preserving F12/F91 ownership of the consuming Brief and assistant surfaces. Feedback persistence is not mislabeled as model learning: the missing evaluation consumer remains a blocking contradiction until Product/AI defines and proves it.

2026-07-02 — Correct three false gaps in §11 audit and narrow ES6/ES7 in §9

Type: Documentation truth fix

Summary:

  • §11 row 7 (⚠️ “duplicate accumulates”): upgraded to ✅. An idempotency guard (briefs/views.py lines 232-240) rejects duplicate (org + user + target_type + target_id + feedback_type) with 409. No DB unique_together, so a concurrent-write race edge remains but the in-code check is present and working.
  • §11 row 8 (⚠️ “no rate limiting”): upgraded to ✅. FeedbackRateThrottle(UserRateThrottle, scope="feedback") exists at briefs/views.py lines 19-20, applied via @throttle_classes([FeedbackRateThrottle]) at line 211, with a manual 20/min cache cap at lines 214-218 and the DRF rate "feedback": "30/hour" in config/settings/base.py line 152. Remaining DB-constraint race is the only open edge.
  • §9 ES6 (cross-tenant “201 written, no validation”): corrected. brief_item targets are now validated against the requesting org at lines 228-230 and return 400 if not found. The loose-reference behavior only remains for other target types (commitment, risk, decision, chat_message, task).
  • §11 summary updated from “3 gaps” to “1 gap” (learning loop not closed).

Tests: No new tests — all three behaviors were already exercised by the existing backend test suite; the audit rows were simply wrong.

2026-06-30 — Mobile shared Brief feedback ships on iOS + Android

Type: Feature

Summary:

  • Added shared KMP brief-item feedback submission for the mobile Brief surface used by iOS and Android.
  • The shared Brief cards now expose visible text-labeled feedback actions (Helpful / Needs correction plus typed negative reasons) and submit real POST /api/feedback/ payloads against target_type="brief_item".
  • Added shared unit proof for the mobile feedback request payload and in-place submitted/error state handling.

Impact:

  • Mobile CEOs and operators can now correct Brief items without leaving the shared Brief flow, closing the most important mobile F15 gap first.
  • F15 remains partial on mobile overall because chat, risks, commitments, and decisions still do not have equivalent inline feedback affordances on native/KMP surfaces.

Tests:

  • cd android && ./gradlew :shared:jvmTest --tests com.locuno.lumi.shared.presentation.brief.BriefScreenModelTest --tests com.locuno.lumi.shared.api.LumiApiClientTest

2026-06-30 — Mobile shared Ask Lumi feedback ships on iOS + Android

Type: Feature

Summary:

  • Added shared KMP Ask Lumi feedback submission for assistant chat messages on the mobile thread surface used by iOS and Android.
  • Assistant messages now expose the same visible text-labeled feedback actions (Helpful / Needs correction plus typed negative reasons) and submit real POST /api/feedback/ payloads against target_type="chat_message".
  • Added shared unit proof for chat-message feedback payloads and inline submitted/error state handling.

Impact:

  • Mobile users can now correct Lumi’s Ask Lumi answers without leaving the chat thread, removing another web-only AI correction gap.
  • F15 remains partial on mobile overall because commitments, risks, and decisions still lack equivalent inline feedback affordances on native/KMP surfaces.

Tests:

  • cd android && ./gradlew :shared:jvmTest --tests com.locuno.lumi.shared.presentation.chat.ChatScreenModelTest --tests com.locuno.lumi.shared.api.LumiApiClientTest

2026-06-30 — Mobile shared Commitments, Risks, and Decisions feedback ships on iOS + Android

Type: Feature

Summary:

  • Added shared KMP inline feedback submission for mobile Commitments, Risks, and Decisions surfaces used by iOS and Android.
  • Those shared cards now expose the same visible feedback actions (Helpful / Needs correction plus typed negative reasons) and submit real POST /api/feedback/ payloads against target_type="commitment", target_type="risk", and target_type="decision".
  • Extracted the repeated mobile feedback affordance into one shared KMP UI helper so the same interaction stays consistent across Brief, Ask Lumi, Commitments, Risks, and Decisions instead of drifting by surface.

Impact:

  • Mobile F15 parity now matches the current web AI work surfaces: CEOs and operators can leave corrective signal from shared Brief, Ask Lumi, Commitments, Risks, and Decisions without using the web app.
  • The remaining F15 gaps are now backend/product-loop gaps only: duplicate control, rate limiting, and model/eval consumption of the feedback rows.

Tests:

  • cd android && ./gradlew :shared:jvmTest --tests com.locuno.lumi.shared.presentation.commitments.CommitmentsScreenModelTest --tests com.locuno.lumi.shared.presentation.risks.RisksScreenModelTest --tests com.locuno.lumi.shared.presentation.decisions.DecisionsScreenModelTest --tests com.locuno.lumi.shared.presentation.chat.ChatScreenModelTest --tests com.locuno.lumi.shared.api.LumiApiClientTest

2026-06-14 — Strategy v2 realignment (positioning)

Type: Documentation Change

Summary:

  • §3: deleted the “Competitive Positioning” table (Slack reactions / Typeform / SurveyMonkey) and replaced it with a short “Why feedback matters — evidence-trust calibration” paragraph: feedback calibrates the AI’s extractions (from F70 chaos and pulses) and lowers the owner’s correction burden as the graph and model learn.

Reason:

  • Realignment to competitive-strategy-v2.md and the registry’s Positioning doctrine (v2): do not position vs US SaaS (Slack); AI is table-stakes and the data graph is the moat — feedback is framed as evidence-trust calibration for AI extraction, not a vs-reactions/vs-surveys comparison.

Impact:

  • Positioning only. No product behavior, API, data model, or test changed. §1 intro (already calibration-framed), §2 ACs, and §3 Success Metrics are unchanged.

Tests:

  • None (documentation).

Notes:

  • The “one tap or we get silence” capture principle and the eval-dataset intent (§1) are unchanged — only the competitive framing was removed.

2026-06-14 — Restructure to TEMPLATE.md section order; expand §11 audit (SP fleet)

Type: Documentation Change

Summary:

  • Numbered “Current Behavior” as §8; §8 Diagrams → §10; §9 Implementation Audit → §11 (9-row table); §10 Open Questions → §12; §11 Bug Log → §13; §12 Edge Cases absorbed into §11 note; §13 Error States + labeled ES01-ES03 → §9; duplicate changelog entry at bottom removed.
  • §11 audit: 9 rows, 6 ✅, 3 gaps (duplicate constraint, rate limiting, learning loop).

2026-06-13 — Wire FeedbackButtons into BriefItemCard, close remaining §2 criterion (fleet agent B, round 27)

Type: Feature

Summary:

  • Imported FeedbackButtons into BriefPage.tsx; rendered alongside DismissPanel in BriefItemCard.
  • Struck the last unchecked §2 AC: “Given any AI-touched item (brief item, commitment, risk, decision, chat message), then a feedback affordance exists where the item is displayed.”
  • frontend/src/test/F15Feedback.test.tsx: 3 tests (was 2), all pass.

2026-06-13 — JTBD story, competitive positioning, expanded metrics, ES01–ES03 (fleet agent G, round 25)

Type: Spec Sharpening

Summary:

  • Added JTBD story to §2 with two ACs.
  • Added §3 Competitive Positioning (Slack reactions vs survey tools) and expanded §3 Success Metrics.
  • Added ES01–ES03 labeled error states.

2026-06-12 — Close tenant-scoping acceptance criterion

Type: Test Coverage

Summary:

  • Added test_feedback_cross_tenant_isolation proving feedback is tenant-scoped and attributed to the submitting member.

2026-06-12 — Close feedback-without-comment acceptance criterion (CMD fleet agent E, round 19)

Type: Test Coverage / Verified

Summary:

  • Added test_red_r19_e.py with 3 tests: test_feedback_without_comment_succeeds, test_feedback_with_empty_comment_succeeds, test_feedback_with_comment_succeeds.

2026-06-12 — Added Edge Cases and Error States sections (CMD fleet agent D, round 11)

Type: Documentation

2026-06-12 — Success metrics for feedback loop (fleet agent G, round 21)

Type: Spec Sharpening

Summary:

  • Added §3 Success Metrics table: feedback submission rate (>15% of members/week), AI output correction rate (>60% of “wrong” feedback leads to detectable pattern), feedback-to-improvement cycle time (<7 days), silent-dismissal rate target (<30%).

2026-06-11 — Render feedback affordances across AI surfaces

Type: Feature

Summary:

  • Added inline feedback affordances on commitments, risks, decisions, and assistant chat messages.
  • Added browser verification in describe("Feedback").