Spec: F40 — Performance Optimization

Status: partial — shipped repository optimizations are contract-backed; production p95, live Redis, and deployed worker/observability evidence remain open

Contract Graph boundary

F40 is owned by contracts/features/F40.cue and its implementation audit. The repository proves query-shape limits, asynchronous chat state handling, batched beat paths, gzip, and cache-backed local behavior. It does not claim production latency, live Redis operations, deployed worker capacity, Sentry tracing, or cloud scaling from those tests.

1. Problem & CEO Job

Job: Trust the signal — but the signal degrades when brief generation runs a full table scan every 15 minutes across every tenant, when task lists issue 50+ unnecessary database queries, and when the AI chat endpoint blocks a Gunicorn worker for 30 seconds.

Lumi is performance-optimistic: no indexes on core operational models or application cache despite Redis; Celery beat iterates every organization with N+1 queries; AI chat blocks synchronously; and Gunicorn has 2 workers × 4 threads. This works for dozens of orgs, then breaks silently at hundreds. Make scale mechanical, not over-engineered: index real access patterns, cache where invalidation is trivial, and move slow work out of the request-response path.

2. User Stories & Acceptance Criteria

• As a CEO loading the Today dashboard, I want the page to load in under 2 seconds, so that I see what needs attention immediately.

  • API response: GET /api/today/ returns in < 500ms (p95).
  • Database: ≤ 5 queries for a typical Today page load.

• As a CEO loading task/project lists, I want paginated lists to render without visible lag, so that I can scroll through my team’s work.

  • GET /api/tasks/ returns in < 300ms at 50 items/page (p95).
  • GET /api/projects/ returns in < 300ms at 50 items/page (p95).

• As a CEO cold-opening an authenticated P0 route on the floor device, I want its critical modules to arrive without a long queue of tiny shared chunks, so that identity and route queries can start promptly.

  • Every registered P0 manifest graph contains at most 70 local JavaScript files and still meets its existing gzip budget or recorded no-regression boundary. The 70-file ceiling is above the measured 67-file maximum and below the former 112-file worst case; forcing 40 required unrelated-code overfetch or invalid execution semantics and was rejected.
  • The canonical floor trace proves the selected authenticated route’s identity and first route query start earlier than the 2026-07-15 baseline; byte/request improvements alone do not authorize a CWV score increase.
  • A remembered browser session may warm only the matched authenticated P0 Layout/leaf graph before main.tsx; the hint never authorizes access, public/unknown routes are excluded, and pending logout disables speculation.
  • Production HTML resolves the emitted content-addressed Geist Latin 700 face and places a parser-time font hint before both module entries only for remembered-session Today and Brief paths; all other routes avoid the request.
  • The critical-font hint keeps auth/router authority unchanged and fails closed when the session hint is absent, storage is unreadable, or logout is pending. A missing emitted font fails the production build instead of silently dropping the optimization.
  • Production HTML resolves the emitted Layout and Today/Brief leaf chunks and starts only the matched pair with parser-time modulepreload; missing or duplicate assets fail the build, and the executable entry suppresses duplicate Today/Brief imports while retaining all six factories for stable chunk topology.

• As a user returning to a tab after a deployment, I want stale lazy-route chunks to recover without a permanent error screen, so that routine releases do not strand the workspace.

  • The first Vite preload failure replaces the current document with the same route plus a dedicated cache-bust token, forcing a fresh HTML/chunk graph instead of reusing a stale cached document.
  • A guarded repeat failure is not swallowed into an undefined lazy-module payload; the original chunk-load rejection remains observable.
  • The production Error Boundary retry uses the same cache-busted document recovery rather than a cache-reusing soft reload.
  • Recovery preserves the current pathname, hash, and unrelated search parameters while replacing only Lumi’s dedicated recovery token.

• As a system operator, I want Celery beat tasks to complete within a single beat interval, so that briefs are always fresh and tasks don’t overlap.

  • Brief generation per org: < 30 seconds.
  • System-wide beat cycle: < 15 minutes (the beat interval).
  • No overlapping beat task runs (prevented by a lock or idempotency check).

• As a user chatting with Lumi, I want the AI to respond without blocking other users, so that the app stays responsive under load.

  • AI chat requests are offloaded to Celery. (chat/views.py: process_chat_async.delay(...) — creates pending placeholder, Celery fills in the response; P10 Fixed in STATUS.md)
  • Chat response is delivered via polling (202 Accepted + client polls thread endpoint until status != "pending"). (chat/views.py line 92: returns 202 with pending assistant_message; frontend polls)
  • Gunicorn workers are never occupied for > 5 seconds by any request — requires production load test to verify; not a code gap.

• As a developer, I want query performance to be observable, so that regressions are caught before deploy.

  • django-debug-toolbar or equivalent profiler available in dev.
  • CI fails if any API endpoint exceeds a query count threshold (assertNumQueries).
  • Brief generation query count ≤ 20 (P2 query consolidation: select_related + bulk_create).

3. Scope Boundaries

• In:
  • Database indexes on core operational models (Task, Project, Commitment, Risk, DecisionRequest, Brief, PulseEntry).
  • select_related / prefetch_related optimization on all list ViewSets.
  • Application-level caching of brief results, org settings, and pulse questions.
  • Async AI chat processing (offload to Celery).
  • Beat task batching (paginated org iteration, idempotency).
  • Frontend bundle size audit and optimization.
  • Frontend critical-path request-graph audit and bounded code splitting.
  • N+1 query elimination in beat tasks.
  • Performance monitoring (Sentry tracing already configured — verify coverage).
• Out:
  • Table partitioning, sharding, read replicas — premature at current scale.
  • CDN caching strategy (addressed in F38).
  • Server-side rendering (stays SPA).
  • Database migration to PostgreSQL read replicas.
  • Full APM tooling (Sentry tracing is sufficient for now).
  • Worker pool scaling (ECS auto-scaling is a deploy concern).

4. Data Model

4.1 New Database Indexes

All indexes are on the organization_id FK plus the most common filter/sort columns. These follow the scale-ready design rule: make the future upgrade mechanical, don’t perform the upgrade now.

Task (backend/apps/work/models.py):

class Meta:
    indexes = [
        models.Index(fields=["organization", "status"]),
        models.Index(fields=["organization", "due_date"]),
        models.Index(fields=["organization", "owner"]),
        models.Index(fields=["organization", "project"]),
        models.Index(fields=["organization", "team"]),
        models.Index(fields=["organization", "priority"]),
    ]

Project (backend/apps/work/models.py):

class Meta:
    indexes = [
        models.Index(fields=["organization", "team"]),
        models.Index(fields=["organization", "owner"]),
        models.Index(fields=["organization", "visibility"]),
    ]

Commitment (backend/apps/operations/models.py):

class Meta:
    indexes = [
        models.Index(fields=["organization", "status"]),
        models.Index(fields=["organization", "due_date"]),
    ]

Risk (backend/apps/operations/models.py):

class Meta:
    indexes = [
        models.Index(fields=["organization", "status"]),
        models.Index(fields=["organization", "severity"]),
    ]

DecisionRequest (backend/apps/operations/models.py):

class Meta:
    indexes = [
        models.Index(fields=["organization", "status"]),
    ]

Brief (backend/apps/briefs/models.py):

class Meta:
    indexes = [
        models.Index(fields=["organization", "brief_date", "brief_type"]),
    ]

PulseEntry (backend/apps/pulses/models.py):

class Meta:
    indexes = [
        models.Index(fields=["organization", "user", "date"]),
        models.Index(fields=["organization", "status"]),
    ]

OrganizationMember (backend/apps/organizations/models.py):

class Meta:
    indexes = [
        models.Index(fields=["organization", "status"]),
        models.Index(fields=["organization", "user"]),
    ]

ChatMessage (backend/apps/chat/models.py):

class Meta:
    indexes = [
        models.Index(fields=["thread", "created_at"]),
    ]

4.2 No Schema Changes

Indexes are additive. No column changes, no constraint changes, no data migration.

5. API Contract

No new endpoints. No breaking changes to existing endpoints.

5.1 ViewSet Query Optimizations

5.2 Chat AI Processing (Async Migration)

Current: POST /api/chat/threads/{id}/messages/ → answer_chat_message() runs synchronously (blocking Gunicorn).

Target: POST /api/chat/threads/{id}/messages/ → creates message + dispatches Celery task → returns message with status: "processing". Client polls or listens via existing SSE endpoint for the AI response.

No API contract change to the existing endpoint signature — only the response behavior changes (message starts as “processing” and updates when AI completes).

6. UI / UX

No visual changes. Chat UX may show a “Lumi is thinking…” indicator while the AI response is being generated asynchronously — existing i18n key chat.thinking or new key.

7. Test Plan

8. Current Behavior

See docs/performance/PERFORMANCE_AUDIT.md for the full 30+ finding audit. Key highlights:

Fixed (Previous Rounds)

• Beat tasks iterate all orgs (P1) — fixed with Organization.objects.iterator(chunk_size=50) and distributed lock prevention; 15-min beat tasks no longer N+1 per org.
• Brief generation indexes + query consolidation (P2) — composite indexes added; queries now use (organization, status) and (organization, due_date) indexes. All Task queries use select_related("owner__user"). BriefItems created in bulk. Queries reduced from 39 to ≤20 per org.
• Zero custom indexes (P3) — composite indexes on Task, Commitment, Risk, DecisionRequest, Brief, PulseEntry, OrganizationMember, ChatMessage all shipped.
• select_related/prefetch_related (P4, P5) — TaskViewSet, ProjectViewSet, RecurringTaskViewSet, CommitmentViewSet, RiskViewSet, DecisionRequestViewSet, and ChatThread list all have proper eager loading.
• Celery retry policies (P12) — retry backoff with jitter on all beat tasks.
• No overlapping beat tasks (P13) — Redis SETNX distributed lock on all beat tasks.
• Frontend bundle size (P19) — lazy-loaded i18n locales, route-level splitting, and interaction-only Tasks/Deals dependencies keep registered routes below their per-route targets. Budget verdicts compare integer on-wire bytes with strict < limits before formatting; tables retain three decimals so boundary outcomes are reviewable. The latest candidate Tasks slice measures 256.384 kB/38 files statically and 298.362 kB in a three-run seeded floor trace after removing a redundant owner-initial avatar while preserving the owner name. The full post-merge PM route set remains the scoring authority.
• Pre-main matched-route warming (P20/P49) — the separate preload entry remains because focused Tasks controls proved it outperformed parser-earlier route-hint variants. P49 instead resolves Today/Brief’s emitted Latin 700 face at build time and injects one parser-time, session-gated preload before both module entries. Focused candidate evidence improves Today floor LCP from 3724 to 3532 ms while preserving Tasks at 3016 ms versus 3040 ms; merged-main browser evidence remains required before any W-SYS-01 score change.
• Route-level code splitting (P20) — React.lazy() for all route components.

Remaining

• AI chat blocks Gunicorn for up to 30 seconds (P10) — ADR-011 accepted, but code changes (ChatMessage.status field, Celery task, async send_message) have not yet landed in any PR. 8 concurrent chat requests would saturate the entire backend (2 workers × 4 threads).
• No Celery result backend (P14) — deferred; task results discarded, no monitoring.
• 100% synchronous codebase (P11) — deferred; full ASGI migration premature at current scale.
• Connection pooling (P23) — deferred; pgbouncer when connection count becomes bottleneck.
• Redis application caching gaps (P8 remaining items: org settings, pulse questions) — brief results cached; remaining targets deferred to next round.

Medium

• Redis running but zero application-level caching.
• Granian configured for only 2 processes × 4 threads = 8 concurrent requests.
• ORDER BY RANDOM() used on OrganizationMember queryset in brief generation for “random pulse” — non-deterministic and expensive.
• No task retry policies on any Celery task.
• No celery task result backend.

Positive findings

• Some models (Deal, Company, Message) have proper indexes.
• select_related/prefetch_related used correctly in BriefViewSet.
• Sentry tracing configured at 10% sample rate with continuous profiling.
• Vite code splitting with vendor-react, vendor-state, vendor-http chunks.
• CompressedManifestStaticFilesStorage for WhiteNoise static serving.
• Multi-stage Docker builds keep images small.

9. Error States

10. Diagrams & Docs Touched

• New: docs/performance/PERFORMANCE_AUDIT.md — full finding-by-finding audit.
• New: docs/performance/STATUS.md — tracker with status/assignee/target.
• backend/DIAGRAMS.md: add Celery async chat flow, index strategy section.
• backend/COOKBOOK.md: add select_related/prefetch_related pattern and index creation template.

11. Implementation Audit (2026-06-12, updated 2026-06-14)

Summary: 9/10 match; 1 gap (p95 latency ACs require production load test). F40 findings all resolved or formally deferred.

Edge Cases Catalog (formerly §9, 4 items): E1 change improving median but worsening p95 CEO flows (gap above — load test needed); E2 cached summaries must invalidate on tenant-scoped writes (pulse questions cache is static constants, not per-tenant — invalidation not needed; future per-tenant cache must handle this); E3 optimizations must not break output: ‘export’ or static hosting (Vite config preserved; P20 lazy loading uses React.lazy() compatible with SPA); E4 public marketing vs authenticated CEO flow budgets must be measured separately (§7 test 6 covers bundle; no separate flow budget test yet).

12. Open Questions / Decisions

1. Chat async delivery mechanism: SSE (already used in messaging) vs. polling vs. WebSocket. Decision: reuse existing SSE infrastructure from messaging (F29). Chat thread messages already stream via SSE.
2. Beat task idempotency approach: Redis lock (SETNX) vs. DB flag vs. Celery once extension. Decision: Redis lock — Redis is already the broker, and SETNX with TTL is trivial.
3. Index creation in production: Django migrations create indexes with CREATE INDEX (blocks writes on large tables). Decision: document manual CREATE INDEX CONCURRENTLY step for production migrations on tables with > 10K rows.
4. Query count threshold in CI: What’s the acceptable ceiling per endpoint? Decision: start with 20 queries for list endpoints, 10 for detail endpoints, 5 for simple GETs. Tune down over time.

13. Bug / Change Log

2026-07-31 — Current production-deploy candidate bundle baseline

• The W-SYS-04 static-manifest no-regression ceilings now match the measured current production-deploy candidate with the existing 1 kB deterministic-build allowance. The CI failure was a stale baseline mismatch across the authenticated route graph, not an executable violation of the route request caps or the 300 kB authenticated-route targets.
• /store/en retains its separate 170 kB end-state static-manifest target. The current 210 kB graph remains an explicit warning and does not become a target pass through the accepted no-regression ceiling; its deployed prerendered document is measured separately.

2026-07-17 — Brief summary paint boundary

• BriefPage keeps the generated summary and metadata in the first committed data frame, then starts the interaction-heavy BriefItemsView import after two animation frames. The fixed-height Suspense region preserves layout while item cards and every action remain fully reachable.
• A frame-controlled test prevents the item renderer from returning to the summary commit. The candidate static Brief graph remains inside its no-regression ceiling.
• Focused three-run evidence improves Brief 3604→3344 ms LCP with CLS 0.000 and essentially unchanged transfer (289.782→289.793 kB). Tasks improves 3068→3044 ms with unchanged custody. Exact merged-main PM scoring remains separate.

2026-07-16 — Brief nested state custody restores the empirical budget

Type: Frontend critical-path request custody

• BriefPage already made the Project view lazy, but the default Type renderer statically re-imported BriefProjectView and LumiDidBriefSection. That nested import defeated the route boundary and loaded project rollups, select/dialog primitives, and their dependencies before the empirical transfer settled.
• BriefItemsView now resolves both modules only in the states that render them. The default item list, evidence, and item actions remain complete; hovering or focusing the Project toggle still warms the same project chunk before intent.
• In a same-seed three-run floor sample, Brief moved from 310.2 to 292.6 kB empirical JavaScript and from 3684 to 3552 ms LCP. FCP remained below target at 1332 ms and CLS remained 0.000. This closes focused readiness only; the PM-owned complete merged-main train still governs W-SYS-04 scoring.

Tests:

• frontend/src/test/_shared/bundle-budget.test.ts
• frontend/src/test/intel/BriefPage.test.tsx
• focused measure-cwv-p0.mjs 3 --only=brief

2026-07-16 — Public hero media leaves authenticated SPA startup

Type: Frontend critical-path request custody

• The shared SPA document no longer gives the 43 kB public Landing hero image high-priority preload status on every authenticated route. The prerendered Landing keeps the real hero <img loading="eager" fetchPriority="high">, so its deployed document still discovers the actual LCP asset without a JavaScript dependency.
• In exact-main, same-backend three-run floor A/B samples, Pulse moved 3596→3408 ms LCP and 1448→1340 ms FCP, while Chat moved 3472→3256 ms LCP with unchanged route JavaScript and CLS. The prerendered Landing control moved 1184→1140 ms LCP, so the authenticated gain does not trade away the public first view.
• These are focused readiness measurements, not a W-SYS-01 score promotion. All three routes retain their existing canonical score ownership and complete-train requirements.

Tests:

• frontend/src/test/_shared/bundle-budget.test.ts
• production pnpm build plus 224/224 public-route prerender
• focused measure-cwv-p0.mjs 3 Pulse, Chat, and Landing A/B samples

2026-07-16 — Authenticated route chunks start before the app entry

Type: Frontend critical-path scheduling

• A separate tiny Vite entry reads only the existing non-secret remembered-session hint and pending-logout flag, then starts Layout plus the matched P0 leaf import before the full app entry graph evaluates. F01 refresh and TanStack Router remain authoritative.
• The build plugin fails closed if the early entry is missing, folded into the app graph, or cannot be inserted before the development or hashed production app script.
• In the same three-run floor harness, authenticated LCP medians moved Today 4144→4104 ms, Brief 4208→4088, Chat 3684→3456, Tasks 4192→3708, Pulse 3976→3572, and Deals 3796→3528. The accepted shared-entry movement is 1.3–1.4 kB from exact main and 0.3–0.4 kB beyond the prior +1 kB tolerance ceilings; all empirical authenticated routes remain below 300 kB. Exact merged-main scoring remains PM-owned and unchanged in this train.

Tests:

• frontend/src/test/_shared/routePreloaders.test.ts
• production pnpm build entry-graph assertion
• full canonical measure-cwv-p0.mjs 3 candidate matrix

2026-07-16 — Deals state-conditioned startup custody

Type: Frontend route-graph performance

• Saved-view mutations, the pipeline picker, collapsed conversion/aging, Calendar-only activity data, and chart implementations now mount only from their owning state or interaction. Pipeline analytics wait for a pipeline id.
• The read-only forecast chip is independent from its editable Select, and Deals imports exact page primitives instead of the full page barrel.
• The production graph falls from the 289.1 kB / 56-file merged ceiling to 257.8 kB / 44 files. A focused three-run seeded floor trace records 291.8 kB, 4052 ms LCP, 1480 ms FCP, and 0.013 CLS. Transfer passes; LCP remains a hold.
• PM-owned merged-main evidence is deliberately unchanged in this engineering train.

Tests:

• frontend/src/test/_shared/bundle-budget.test.ts
• frontend/src/test/crm/DealsPipelineSelector.test.tsx
• focused saved-view, analytics, forecast, URL-state, CRUD, and contact tests

2026-07-16 — Brief and Chat state-conditioned startup custody

Type: Frontend route-graph performance

• Brief now warms project grouping from view-toggle intent, loads assignment and dismissal dialogs only from item-action intent, starts the members query only inside assignment, and defers share/autonomy implementations until required.
• Empty Chat preserves its prompt, thread rail, and trust entry while the active message/composer stack, conditional banners, trust popover, and delete dialog stay outside the initial graph until their owning state or interaction exists.
• F40.AC005.BRIEF_STATE_CUSTODY and CHAT_STATE_CUSTODY lock these source boundaries. Focused production graphs are 261.9 kB / 41 files for Brief and 263.1 kB / 41 files for Chat. Three-run floor medians record 290.9 kB and 299.0 kB respectively; canonical scores still require the PM-owned full train.
• Lazy-entry registration adds 1.4 kB to the shared source-entry map; the post-train baseline records that bounded cost and tightens every authenticated route ceiling. The separately enforced deployed Landing document stays 0.5 kB.

Tests:

• frontend/src/test/_shared/bundle-budget.test.ts
• frontend/src/test/intel/BriefPage.test.tsx
• frontend/src/test/intel/BriefPage.share.test.tsx
• frontend/src/test/intel/ChatPage.test.tsx

2026-07-16 — Tasks confirmation-dialog custody

Type: Frontend route-graph performance

• Extracted row-delete and dirty-form discard confirmations from the cold Tasks graph into an intent-warmed lazy module. Delete pointer/focus and the first dirty-form transition preload the module before confirmation is requested.
• Extended F40.AC005.TASKS_CUSTODY to reject eager dialog imports while the existing Tasks suite continues to prove both confirmation workflows.
• The focused production graph falls 286.3→273.1 kB gzip and 51→43 files. A three-run seeded floor subset records 299.8 kB browser JS, 3888 ms LCP, 1484 ms FCP, and 0.000 CLS. This is engineering evidence, not authorization to edit the PM-owned W-SYS score.

Tests:

• frontend/src/test/_shared/bundle-budget.test.ts
• frontend/src/test/work/TasksPage.test.tsx

2026-07-16 — Defer the desktop CommandDock payload after initial route commit

Type: Performance behavior change

The authenticated shell now waits for the initial route content to commit, then gives the browser a 750 ms quiet window before mounting the desktop CommandDock. This keeps the desktop rail, unread queries, panel behavior, mobile intent trigger, and composer-surface suppression unchanged while removing the non-critical dock payload from the first route paint competition. The focused Layout proof is registered as F40.AC005.DOCK_QUIET_WINDOW; no browser floor score is claimed by this engineering slice.

Tests:

• frontend/src/test/_shared/Layout.test.tsx
• frontend/src/test/_shared/CommandDock.test.tsx

2026-07-16 — Cloudflare SPA fallback rewrote missing deploy chunks to HTML

Type: User-facing reliability regression

The Pages /* → /index.html 200 rule rewrote a missing old content-hashed route chunk after a deployment to the HTML shell. Browsers correctly rejected that response as a module MIME-type violation; the existing cache-busted stale-chunk reload could then still see the same bad graph. The fallback is limited to known application route families and build:ssg generates the top-level 404.html from the current Vite app shell. This disables Cloudflare Pages’ otherwise implicit SPA fallback, so /assets/* is served normally or fails as an observable module load. A scoped Pages Function now serves declared client routes from the current / asset with HTTP 200; _routes.json excludes immutable assets and avoids invoking the Function for static/SSG traffic. The enforcing CSP now also permits the Cloudflare Insights script and its collection endpoint, removing a separate telemetry-only console violation.

Tests:

• frontend/src/test/_shared/auth-custody.test.ts
• frontend/src/test/_shared/pages-spa-fallback.test.ts

2026-07-15 — Tasks interaction-only dependency boundary

Type: Performance behavior change

Summary:

• Split the task create form, detail panel, and drag-and-drop reorder surface from the initial /work/tasks graph while preserving create, inspect, complete, delete, deep-link, dirty-discard, and focus behavior.
• Reorder is now an explicit icon control that loads the existing accessible DnD surface on intent; the normal task list remains fully actionable without that library.
• Added source-custody and Tasks behavior tests. Static JS fell 360.7→286.3 kB gzip (61→51 files); the seeded floor subset fell 409.3→326.1 kB browser JS and 6440→5248 ms LCP.
• W-SYS-01 and W-SYS-04 remain HOLD 6 because the empirical route still misses the unchanged targets.

Tests:

• frontend/src/test/_shared/bundle-budget.test.ts
• frontend/src/test/work/TasksPage.test.tsx

2026-07-15 — Stale route chunks survived the recovery reload

Type: User-facing reliability regression

The deployed dev build loaded correctly in a clean browser, but a previously used browser profile resolved lazy imports as undefined, reached the production Error Boundary, and remained there after pressing Reload. The existing vite:preloadError handler called preventDefault() even when its 10-second guard suppressed another reload, masking the original chunk failure as Lazy module resolved without a usable default export. Its soft location.reload() could also reuse the stale cached HTML. Recovery must perform one cache-busted document replacement and only suppress the Vite rejection when that replacement was actually initiated.

2026-07-15 — Deals interaction-only dependency boundaries

Type: Performance behavior change

Summary:

• Split Deals create/edit, table, calendar, forecast, timeline, detail, bulk, chart, and Kanban DnD modules behind intent-warmed lazy boundaries while preserving the default Kanban cards, CSV export, URL view state, and DnD semantics.
• Added a card-preserving fallback while the Kanban wrapper loads on board intent; the default route stays usable during the boundary fetch.
• Static Deals JS fell 394.4→297.7 kB gzip and 67→59 local files, clearing the strict static target. The browser transfer and W-SYS score remain unclaimed until a fresh floor-profile measurement exists.

Tests:

• frontend/src/test/_shared/bundle-budget.test.ts
• frontend/src/test/crm/DealsCrud.test.tsx
• frontend/src/test/crm/ContactDealPrefill.test.tsx
• frontend/src/test/crm/DealsViewUrlSync.test.tsx

2026-07-15 — Authenticated critical-path request waterfall

Type: Performance architecture / build custody

The canonical Chat floor trace showed that refresh completed at 2.25 s, while 92 local JavaScript files kept the Layout/Chat dependency graph queued until roughly 4.5 s. Identity requests began at 4.82 s and the first Chat query at 6.31 s, producing LCP 6516 ms. Deterministic Lucide buckets and a small shared route-chrome chunk reduce registered graphs to 15–67 files. The median-of-three after trace moves identity to 3.79 s, first Chat queries to 5.21 s, and LCP to 5456 ms; empirical Chat JS falls 352.0→343.7 kB. Existing product byte targets remain unchanged. A more aggressive 40-file design was rejected because its non-recursive groups rendered the stale-chunk recovery screen; the strict execution-order remedy then regressed every byte baseline.

2026-07-13 — Cursor paths for high-growth chronological collections

Type: Performance + API compatibility

Extended the additive hybrid paginator from tasks to commitments, decisions, signal inbox, and audit logs. Each cursor path is tenant-scoped, UUIDv7 -id ordered, backed by (organization, -id), filter-preserving, and proven to issue no SQL OFFSET on page two. Risks and deals remain page-only because their product ordering is not chronological and needs a separate compound-keyset design.

Tests: 50 focused paginator, endpoint, envelope, index, ordering, and insert-between-pages proofs pass; 180 affected feature regressions pass.

2026-07-13 — F05 opt-in cursor pagination without a flag day

Type: Performance + API Compatibility

Summary:

• Added reusable hybrid page/cursor pagination in core.pagination.
• GET /api/tasks/ remains page-number paginated by default and accepts pagination=cursor for UUIDv7 -id keyset traversal.
• Added (organization, -id) index task_org_id_desc_idx and migration work.0018.
• Preserved {count,next,previous,results} for shipped KMP/iOS decoders. Exact cursor count remains a documented transitional cost.

Tests:

• 19 focused pagination/contract tests pass, including SQL proof that cursor page two contains no OFFSET.
• 121 existing F05/work/performance tests pass.

2026-07-02 — Spec drift review: 0 ACs changed, all unchecked ACs confirmed genuine gaps

Type: Documentation Change

Summary:

• Reviewed 5 unchecked ACs against §11 audit. All 5 map to §11 row 10 (⚠️ gap): p95 wall-clock latency for GET /api/today/ (<500ms), Today page query count (≤5), GET /api/tasks/ (<300ms), GET /api/projects/ (<300ms), and Gunicorn <5s cap are not verified by any production load test. assertNumQueries tests confirm query counts but not latency under load.
• No ACs changed. All 5 remain [ ] per §11 row 10 guidance.
• No code changes; documentation only.

2026-06-14 — Restructure to TEMPLATE.md section order; expand §11 audit (SP fleet)

Type: Documentation Change

Summary:

• Absorbed §9 Edge Cases (4 items) into §11 note; §10 Error States → §9; §11 Diagrams → §10; §12 Implementation Audit (narrative + build-order list) → §11 (10-row table); §13 Open Questions → §12; §14 Bug/Change Log → §13.
• §11 audit: 10 rows, 9 ✅, 1 gap (p95 latency ACs require production load test).

2026-06-13 (T3-F R11)

P14 — Response compression — GZipMiddleware (Fixed)

• Added django.middleware.gzip.GZipMiddleware as the first entry in MIDDLEWARE in backend/config/settings/base.py.
• Django’s built-in GZip middleware compresses responses ≥200 bytes when the client sends Accept-Encoding: gzip.
• Tests: TestP14GzipCompression::test_gzip_response — asserts Content-Encoding: gzip on /api/tasks/ response.
• Files: backend/config/settings/base.py

P23 — DB connection pooling (Fixed)

• Updated backend/config/settings/production.py: CONN_MAX_AGE changed from 0 to 60 seconds.
• CONN_HEALTH_CHECKS=True already present (pre-existing). Development and test settings remain at CONN_MAX_AGE=0 to avoid connection leaks between test cases.
• Files: backend/config/settings/production.py

P11 — Query count regression tests + async AI chat (Fixed)

• Created backend/apps/ai/tasks.py with process_chat_async Celery task — implements P10 async chat offload (fixes ModuleNotFoundError from R10).
• Created migration apps/chat/migrations/0003_add_chatmessage_status.py for ChatMessage.status field (also from R10).
• Added TestP14GzipCompression::test_today_api_query_count — asserts GET /api/tasks/ ≤15 queries with select_related JOINs.
• Fixed test_send_message_uses_build_system_prompt patch target: apps.ai.tasks.answer_chat_inner (not apps.ai.services.answer_chat_message — Celery task imports it at module load time).
• Files: backend/apps/ai/tasks.py, backend/apps/chat/migrations/0003_add_chatmessage_status.py, backend/tests/test_performance.py, backend/tests/test_ai_language.py

2026-06-13 (Fleet-B T3-B-r02)

P2 — Brief generation query consolidation (Fixed)

• Before: generate_morning_brief() issued 39 queries per org: 4 separate Task SELECTs (overdue, due-today, customer-impact, cash-impact), individual BriefItem INSERTs (1 per item), and 12 N+1 queries from task.owner__user traversal in _overdue_task_suggestion()/_due_today_task_suggestion()/_impact_task_suggestion().
• After: generate_morning_brief() issues ≤20 queries (≤14 on PostgreSQL, excluding SQLite SAVEPOINT overhead):
  • All Task queries use .select_related("owner__user") — eliminates 12 N+1 queries.
  • All BriefItem creation uses BriefItem.objects.bulk_create(pending_items) — single INSERT instead of 1 per item.
  • Customer + cash/revenue impact queries kept separate to preserve ranking order (F12 §2).
  • brief.save(update_fields=["summary_text"]) — saves 1 unnecessary column write.
• Also: _items_prefetch in briefs/views.py now covers all 5 related FK types (related_task, related_commitment, related_risk, related_decision_request, related_approval_request) — prevents N+1 when BriefItems are serialized.
• Tests: test_brief_generation_query_count (≤20), test_brief_generation_empty_org (≤20), test_brief_generation_no_n_plus_one (0 standalone member/user queries).
• Files: backend/apps/ai/services.py, backend/apps/briefs/views.py, backend/tests/test_performance.py

2026-06-12 (Fleet-C R11)

P5 — ChatThreadSerializer N+1 (Fixed)

• Before: list_threads with 5 threads issued 17 queries (1 base + 5 N+1 for get_last_message() + overhead).
• After: list_threads with 5 threads issues ≤5 queries using Prefetch("messages", ...) with to_attr="_prefetched_msgs".
• Fix: Added select_related("user__user").prefetch_related(Prefetch("messages", ..., to_attr="_prefetched_msgs")) to list_threads() queryset. Serializer reads from _prefetched_msgs attribute first, falling back to ORM query.
• Tests: test_chat_thread_list_no_n_plus_1 — asserts query count ≤5 for 5 threads.
• Files: backend/apps/chat/views.py, backend/apps/chat/serializers.py

P4 — select_related/prefetch_related gaps (Fixed)

• Added select_related("owner__user", "project", "team") to TaskViewSet.get_queryset()
• Added select_related("owner__user", "team") to ProjectViewSet.get_queryset()
• Added select_related("owner__user", "project", "team") to RecurringTaskViewSet.get_queryset()
• Added select_related("owner__user") to CommitmentViewSet.get_queryset()
• Added select_related("owner__user") to RiskViewSet.get_queryset()
• Added select_related("requested_by__user") to DecisionRequestViewSet.get_queryset()
• Added select_related("user__user") to list_threads() queryset in chat views
• Tests: test_task_list_query_count, test_project_list_query_count — assert query counts ≤10 for 5 items each.
• Remaining (future round): P8 (Redis caching), P1 (beat task batching), P2 (brief query consolidation), P3 (indexes)

2026-06-12 (Lead session 2)

P3 — Zero custom indexes (Fixed)

• Added composite indexes to Project, Task (work app) and Commitment, Risk, DecisionRequest (operations app).
• Indexes: (organization, status), (organization, due_date), (organization, visibility), (organization, severity), (organization, priority).
• Migrations: apps/work/migrations/0006_add_composite_indexes_p3.py, apps/operations/migrations/0005_add_composite_indexes_p3.py.

P2 — Brief generation 8+ queries without indexes (Fixed)

• Added composite indexes on DecisionRequest, Risk, Commitment, Task, and Deal models.
• Brief generation queries now use the new (organization, status) and (organization, due_date) indexes, eliminating full table scans.
• Tests: brief generation query count assertions in backend/tests/test_performance.py::TestBriefGenerationQueryCount.

P20 — Route-level code splitting (Fixed)

• All page components converted from static imports to React.lazy() + Suspense.
• Reduces initial bundle size by splitting authenticated app pages into separate chunks.

P1 — Beat tasks iterate all orgs with N+1 queries (Fixed)

• All 6 beat tasks (generate_daily_briefs, send_hub_digest, send_mention_digest, generate_follow_up_drafts_for_all_orgs, send_quiet_hours_summary_all_orgs, generate_recurring_tasks, send_daily_pulse_reminders) updated to use Organization.objects.iterator(chunk_size=50) instead of .all().
• This prevents loading all orgs into memory on each tick.
• Files: apps/briefs/tasks.py, apps/hub/tasks.py, apps/messaging/tasks.py, apps/followups/tasks.py, apps/notifications/tasks.py, apps/work/tasks.py, apps/telegram/tasks.py.

P13 — No overlapping beat task prevention (Fixed)

• All beat tasks now use cache.add(key, True, timeout) to acquire a distributed lock before running.
• A task that finds the lock already held logs a warning and returns {"skipped": True} immediately, preventing overlap when a slow run overlaps a new tick.
• Lock TTL is set just under the beat interval (e.g., 840 s for 15-min tasks, 3300 s for hourly tasks).
• Tests added for each task’s lock-skip path.

2026-06-13 (T3-C R10)

P10 — Async AI chat offload to Celery (Fixed)

• send_message view now creates a placeholder assistant message with status="pending" and dispatches process_chat_async.delay() — returning HTTP 202 immediately.
• apps/ai/tasks.py — new Celery task process_chat_async: marks user message processing, calls answer_chat_message, updates the placeholder in-place to status="complete", handles exception path by marking both messages "failed".
• apps/chat/models.py — added status field to ChatMessage (pending/processing/complete/failed), migration 0003_add_chatmessage_status.
• apps/pulses/views.py — minor query consolidation.
• Tests: TestChatApiAsyncResponse::test_send_message_returns_202_with_pending_status, TestChatApiAsyncResponse::test_process_chat_async_marks_messages_failed_on_exception in backend/tests/test_performance.py. Coverage: 99.03%.
• Files: backend/apps/ai/tasks.py, backend/apps/chat/models.py, backend/apps/chat/views.py, backend/apps/chat/migrations/0003_add_chatmessage_status.py

2026-06-14 (T3 R12 Agent A)

P8 — Pulse questions cache (Fixed)

• Added PULSE_QUESTIONS_CACHE_KEY = "pulses:default_questions" and PULSE_QUESTIONS_CACHE_TTL = 3600 (1 hour) constants to apps/pulses/views.py.
• Added _get_cached_pulse_questions() helper — returns DEFAULT_PULSE_QUESTIONS from Redis on cache hit; populates with 1-hour TTL on miss. today_pulse now calls this helper instead of referencing DEFAULT_PULSE_QUESTIONS directly, so the static list is serialised once and served from cache for all subsequent not-submitted pulse requests within the hour.
• Tests: TestPulseQuestionsCache::test_pulse_questions_cached_on_first_miss, test_pulse_questions_served_from_cache, test_pulse_questions_cache_ttl added to backend/tests/test_performance.py. All 3 pass. Coverage: 98.70%.
• Files: backend/apps/pulses/views.py, backend/tests/test_performance.py

E2E bundle size test — §7 test 6 (Added)

• Added test.describe("F40 — Bundle size") block to frontend/tests/e2e/lumi.spec.ts.
• Test reads frontend/dist/assets/*.js, identifies the largest index-*.js entry chunk, computes its gzip level-9 size, and asserts ≤500 KB.
• Skips gracefully (via test.skip()) when dist/assets/ is absent — dev-only runs with Vite dev server are not blocked.
• Files: frontend/tests/e2e/lumi.spec.ts

2026-06-14 (T5)

P25 — Stub locale JS chunks eliminated (Fixed)

• import.meta.glob pattern in frontend/src/i18n/index.ts changed from ./locales/*/*.json (matches all 62 locale dirs = ~2728 files) to ./locales/{en,en-AU,en-CA,en-GB,en-IN,en-SG,en-ZA,vi}/*.json (352 files). Eliminates ~2376 unnecessary tiny JS chunks from the production bundle. Stub locale directories (ar, ms, id, fil, lv, …) remain on disk for F69 but produce zero output chunks.
• Files: frontend/src/i18n/index.ts

P26 — Web Vitals runtime tracking (Fixed)

• frontend/src/lib/web-vitals.ts observes LCP, INP, and CLS via the native PerformanceObserver API and reports each metric to Sentry as a measurement (Sentry.setMeasurement) and breadcrumb with rating (good / needs-improvement / poor). Uses native browser APIs — no external npm dependency.
• Called from frontend/src/main.tsx immediately after Sentry.init().
• Files: frontend/src/lib/web-vitals.ts, frontend/src/main.tsx

2026-07-16 — Today chart route custody

• Moved the Today signal-count chart below decisions, risks, tasks, and commitments so the actionable queue remains the primary executive workflow.
• Extracted Recharts rendering into TodaySignalsChart.tsx and load it only when its reserved 180 px region approaches the viewport. F40.AC005.TODAY_CHART_CUSTODY prevents the chart library from returning to Today’s cold route graph.
• The three-run candidate harness measured 297.1 kB empirical route JavaScript and 3756 ms floor LCP, down from 417.2 kB and 4580 ms on the preceding merged-main train. These are focused readiness measurements; only a complete post-merge PM train may change the web audit score.

2026-07-16 — Strict raw-byte bundle boundary and Tasks margin

• Replaced rounded <= verdicts with shared strict predicates used by both bundle gates and the CWV reporter. Exactly 300 KiB now fails a < 300 kB target, equality fails the registered LCP/INP/FCP < targets, missing zero timings remain failures, and route tables retain three decimals so reviewers can audit near-boundary results.
• The corrected gate exposed Tasks at 300.012 kB. The row already displayed the full owner name, so its duplicate owner-initial avatar and initial Avatar chunk were subtracted without removing information or actions. A valid three-run seeded floor rerun records 298.362 kB; the static graph records 256.384 kB/38 files.
• This is candidate engineering proof only. Canonical evidence and W-SYS scoring still require a complete train on the exact merged commit with freshly reset demo data.

2026-07-16 — Authenticated critical-font discovery contract

• Canonical Today showed its LCP heading’s Latin 700 font completing near 3.6 s. The accepted P49 boundary resolves that content-addressed build asset and starts its request from parser-time HTML only for remembered-session Today and Brief paths.
• Generated external defer and async route-hint runtimes, an unconditional Tasks view hint, direct dependency expansion, and recursive closure were all rejected after focused traces. They either executed too late or saturated the floor connection. Removing the original module entry regressed Tasks from 3040 to 3168 ms, so it remains unchanged.
• The accepted candidate starts the Today font at 0.19 s, completes it at 1.83 s, improves Today floor LCP from 3724 to 3532 ms, and preserves Tasks at 3016 ms versus its 3040 ms baseline. The guard emits no font request on Tasks, public/unknown routes, logout, or storage failure.
• This is candidate engineering evidence, not a CWV score claim. A separate exact-merged-main PM train remains mandatory.

2026-07-16 — Parser-time Today/Brief module discovery

• The prior entry was parser-early but its dynamic Layout/leaf imports queued behind the main module burst. The production HTML transform now resolves the emitted Layout and matched Today/Brief leaf chunks and creates session-gated modulepreload links before either module entry.
• All six factories stay in the separate entry so Rollup preserves the measured shared graph. Runtime execution skips only Today/Brief, preventing duplicate imports; Chat, Tasks, Pulse, and Deals keep the proven executable warm-up. Auth and router checks remain authoritative.
• Three-run candidate controls move Today 3812 to 3492 ms and Tasks 3036 to 3004 ms. Today remains above target; the populated signal-card LCP uses Geist 500 and requires a separate proof slice.

Tests:

• frontend/src/test/_shared/bundle-budget.test.ts
• frontend/scripts/check-bundle-budget.mjs --route=/work/tasks
• focused measure-cwv-p0.mjs 3 --only=tasks

2026-07-17 — Today locale custody

• Today no longer subscribes to complete Brief and Hub namespaces for three greetings, four share labels, and one announcement link. Those exact authored values now live in today.json for all 62 locales, and a structural proof compares every route-owned value to its localized source.
• The startup namespace set now contains shell plus Today’s grouped decision/risk/commitment dictionaries. Brief and Hub remain lazy until their own surfaces need them.
• Focused three-run evidence records Today at 3480 ms LCP, 0.000 CLS, and 291.782 kB versus exact main at 3472 ms / 0.000 / 297.180 kB. Timing is neutral within noise and settled route transfer falls 5.398 kB. Tasks records 3000 ms and 298.371 kB versus 3040 ms and 298.369 kB.
• This is route-custody evidence, not a W-SYS score claim. A separate exact-merged-main PM train remains mandatory.