Spec: F44 — API Integrations (SME CEO-Grade Connectors)

Status: shipped

1. Problem & CEO Job

Job: See clearly — but external signals are invisible until someone reports them. A late-cleared transaction, damaged shipment, 30-day-overdue invoice, or unanswered Zalo complaint can wake a CEO at night without ever appearing in a daily pulse.

F33 provides the generic webhook framework; F44 applies it to the systems Vietnamese SME CEOs run. The promise is not “connect your tools,” but “Lumi notices before you ask.” Every connector ships with at least one CEO-recognizable exception from an ordinary Tuesday morning.

2. User Stories & Acceptance Criteria

Phase 1 — Must-Ship (Launch Partner Connectors)

Each acceptance criterion below is testable and maps to at least one test case in §7.

Zalo OA Connector

• As a CEO, I want Lumi to watch my Zalo Official Account, so that customer messages that go unanswered for 2+ hours become a risk in my brief.
  • Given an admin with valid Zalo OA credentials (OA ID + access token + refresh token), when they connect via Admin UI, then SourceConnection is created with provider_slug=zalo_oa, status=active, and config_json stores {oa_id, access_token, refresh_token, app_id}.
  • Given an active Zalo OA connection, when the poll task runs (every 5 minutes), then messages are fetched from Zalo API and new unanswered messages create SourceEvent records with capture_channel=connector.
  • Given a Zalo customer message with no reply within 2 hours, then SourceEvent has severity=medium and routes to SignalInboxItem via the default rule event_type=message_unanswered.
  • Given a Zalo customer message with no reply within 6 hours, then severity=high.
  • Given a Zalo customer message with no reply within 24 hours, then severity=critical.
  • Given a Zalo connection exists, when an admin views /settings/sources, then the connection card shows status indicator (active/degraded/disabled), last polled time, and message counts.
  • Given the Zalo API returns a 401, then connector transitions to degraded and last_error_text is set.

MISA Connector

• As a CEO, I want Lumi to surface overdue invoices from my accounting system, so that I stop discovering cash problems at month-end.
  • Given an admin with valid MISA API credentials (company tax code + API key + API secret), when they connect, then SourceConnection is created with provider_slug=misa, status=active.
  • Given an active MISA connection, when the poll task runs, then invoices 15+ days past due create SourceEvent records with event_type=invoice_overdue, proposal_type=risk.
  • Given an invoice 30+ days past due, then the resulting SourceEvent has severity=critical.
  • Given MISA API returns a 429, then connector applies exponential backoff (60s→120s→240s) and retries up to 3 times before marking degraded for this poll cycle.

Fast Accounting Connector

• As a CEO, I want Lumi to surface overdue invoices from Fast Accounting, so that cash-flow problems are surfaced regardless of accounting platform.
  • Given an admin with valid Fast Accounting credentials, when they connect, then SourceConnection is created with provider_slug=fast_accounting, status=active.
  • Given an active Fast Accounting connection, when the poll task runs, then invoices 15+ days past due create SourceEvent records with event_type=invoice_overdue, proposal_type=risk.
  • Given Fast Accounting API is down (5xx for 3 consecutive poll cycles), then connector transitions to degraded and retry_count resets after a successful poll.

Shopify Connector

• As a CEO, I want Lumi to flag stalled e-commerce orders, so that fulfillment problems don’t fester until the customer cancels.
  • Given an admin with a Shopify shop, when they initiate OAuth (scoped to read_orders), then Lumi redirects to Shopify OAuth, receives the callback, stores the access token, and creates SourceConnection with provider_slug=shopify, status=active.
  • Given an active Shopify connection, when the poll task runs, then orders with fulfillment_status=unfulfilled for 48+ hours create SourceEvent with event_type=order_unfulfilled, proposal_type=task.
  • Given a refund or chargeback is detected on a Shopify order, then SourceEvent is created with event_type=refund_received, proposal_type=risk.
  • Given Shopify webhook is registered during connect(), when an order event fires, then Lumi receives it at POST /api/source-connections/webhook/<secret>/ and processes it.

Haravan Connector

• As a CEO, I want Lumi to flag stalled orders on Haravan, so that e-commerce fulfillment gaps are caught regardless of platform.
  • Given an admin with a Haravan shop, when they initiate OAuth (scoped to orders), then Lumi completes the OAuth flow and creates SourceConnection with provider_slug=haravan, status=active.
  • Given an active Haravan connection, when the poll task runs, then orders unfulfilled for 48+ hours create SourceEvent with event_type=order_unfulfilled, proposal_type=task.
  • Given two identical Haravan webhook events arrive within 5 seconds, then the second is silently dropped (dedup key (organization_id, connection_id, external_event_id)).

Google Workspace Connector

• As a CEO running a service business, I want Google Workspace integration so Lumi can parse project updates from shared docs and Sheets, so that the brief includes what teams already wrote instead of asking them to write it twice.
  • Given a Google Workspace admin configures a service account with domain-wide delegation (Drive + Sheets read-only scopes), when they connect, then SourceConnection is created with provider_slug=google_workspace, status=active.
  • Given an active Google Workspace connection, when an admin selects monitored Drive folder IDs and Sheet IDs, then those IDs are stored in config_json.monitored_folder_ids and config_json.monitored_sheet_ids.
  • Given a monitored Google Sheet, when new rows are added, then the next poll creates SourceEvent with the row data as body_text, event_type=sheet_row_added, proposal_type=commitment.
  • Given a monitored Drive folder, when a doc is created or edited, then the next poll creates SourceEvent with document title + excerpt as body_text, event_type=doc_modified, proposal_type=decision.
  • Given the Google Workspace service account token expires, when the token refresh fails, then connector transitions to degraded with last_error_text indicating auth failure. Historical events are preserved.

Phase 2 — Platform Expansion

• As a CEO, I want Lumi to ingest bank transaction alerts, so that I see unusual cash movements without logging into internet banking.

  • Connect via VietQR webhook (Sepay already supported for inbound payments — extend to transaction alerts).
  • Transaction above configurable threshold → SourceEvent with proposal_type=decision.
  • Daily cash balance summary pushed to Brief.

• As a CEO, I want Lumi to pull staffing alerts from Base.vn or 1Office, so that I know when someone is on leave during a critical delivery week.

  • Connect Base.vn / 1Office (API-based, scoped to leave/absence).
  • Leave scheduled within 7 days of a project deadline → SourceEvent with proposal_type=commitment.
  • Rule: match leave dates against project deadlines in Lumi.

• As a CEO, I want Lumi to ingest Microsoft 365 signals, so that teams using Outlook/Teams/Excel get the same signal quality as Google Workspace teams.

  • Connect Microsoft 365 via Graph API (delegated or app-only, scoped to mail + calendar + Excel).
  • Calendar: meetings marked “urgent” or with “decision” in subject → SourceEvent.
  • Excel Online: monitored workbook changes → SourceEvent.

General Framework

• As a developer adding a connector, I want a connector SDK/contract so new integrations take hours, not weeks.
  • Every connector implements the BaseConnector interface: connect(), poll(), disconnect(), health_check(), normalize(raw_event) → SourceEvent.
  • Connectors register themselves in CONNECTOR_REGISTRY with provider_slug, category, auth schema, and default rules.
  • Polling is Celery-beat-driven via a unified poll_all_active_connectors task — each connector gets its own sub-task.

3. Scope Boundaries

• In:
  • Phase 1 (launch): Zalo OA, MISA/Fast Accounting, Shopify/Haravan, Google Workspace.
  • Phase 2 (post-launch): VietQR transaction alerts, Base.vn/1Office, Microsoft 365.
  • Connector framework: BaseConnector abstract class, unified polling, auth management, health tracking.
  • All connectors feed through F33’s existing SourceConnection → SourceRule → SignalInboxItem pipeline.
  • Each connector ships with default rules that create signal inbox items (never auto-promote to ops).
  • Connector UI lives in the existing /settings/sources page (F33 surface).
  • OAuth flow handling (redirect URL, token storage, refresh).
• Out:
  • Building the source system’s features inside Lumi (no Zalo chat client, no accounting UI, no spreadsheet editor).
  • Bi-directional sync — Lumi reads from these systems; it never writes back (v1).
  • Full data mirror/migration — Lumi ingests exceptions and summaries, not complete databases.
  • Native mobile push for connector signals (F36 handles push; F44 just creates the signals).
  • Connector marketplace/plugin system — connectors are first-party only, built into the product.
  • WhatsApp as a conversational bot or in-app chat client. WhatsApp Business source ingestion is covered by F76 (planned spec) and the regional connector rollout in F45.
  • Slack/Discord/Teams chat integration (F29 is the internal chat; external chat platforms are out of scope per wedge doctrine).

4. Data Model

4.1 Extensions to F33 Models

All connectors use the existing SourceConnection model (F33). New fields:

# SourceConnection additions
auth_expires_at = models.DateTimeField(null=True)        # OAuth token expiry
last_polled_at = models.DateTimeField(null=True)          # Last successful poll
poll_interval_minutes = models.IntegerField(default=5)    # Configurable per-connection

4.2 Connector-Specific Configs

Each connector’s config_json stores its own auth/config shape:

# Zalo OA config
{"oa_id": "...", "access_token": "...", "refresh_token": "...", "app_id": "..."}

# MISA config
{"company_tax_code": "...", "api_key": "...", "api_secret": "..."}

# Shopify config
{"shop_domain": "xxx.myshopify.com", "access_token": "...", "scope": "read_orders"}

# Google Workspace config
{"service_account_email": "...", "private_key_id": "...", "domain": "company.com",
 "monitored_folder_ids": ["...", "..."], "monitored_sheet_ids": ["...", "..."]}

# WhatsApp Business config (regional)
{"phone_number": "...", "market": "my|id|ph|sa", "app_secret": "..."}

4.3 Default Rules Per Connector

Shipped as data migrations (every connector installs sensible defaults):

4.4 No New Models

All connector state lives in SourceConnection.config_json, cursor_json, and last_error_text. The SourceEvent model handles all event shapes and is hash-partition-ready by organization: dedupe uniqueness includes organization, and org-leading indexes cover chronological, connection, status, and event-type cursor paths.

4.5 Connector Contracts — Detailed

Each launch connector has a defined contract covering auth type, normalized event schema, rate limits, and error handling.

4.5.1 Connector Contract Table

4.5.2 Normalized Event Schema

All connector events use a common SourceEvent shape with connector-specific fields in raw_payload:

{
  "id": "uuid",
  "connection_id": "uuid",
  "provider_slug": "zalo_oa|misa|fast_accounting|shopify|haravan|google_workspace",
  "event_type": "string",
  "external_event_id": "string",
  "severity": "low|medium|high|critical",
  "title": "string",
  "body_text": "string",
  "actor": "string|null",
  "proposal_type": "risk|task|commitment|decision",
  "raw_payload": {"...": "..."},
  "captured_at": "datetime",
  "created_at": "datetime"
}

Connector-specific event_type values:

4.5.3 Error Handling Per Connector

5. API Contract

5.1 Existing Endpoints (Reused from F33)

All connector management uses the existing F33 CRUD endpoints. New endpoints are connector-action only:

5.2 Connector Catalog Response

{
  "connectors": [
    {
      "provider_slug": "zalo_oa",
      "name": "Zalo Official Account",
      "category": "support",
      "description": "Monitor customer messages. Flags unanswered conversations.",
      "auth_type": "oauth2",
      "auth_fields": ["oa_id"],
      "oauth_url": "https://oauth.zalo.me/v4/permission",
      "phases": ["launch"]
    },
    ...
  ]
}

5.3 Webhook Reception

F33 already handles webhook reception. New connector webhooks (Zalo OA callbacks, Shopify webhooks, Haravan webhooks) reuse the existing POST /api/source-connections/webhook/<secret>/ endpoint. Each connector registers its webhook URL with the source system via the connect() flow.

5.4 OAuth Flow

5.4.1 Redirect URL Pattern

All OAuth-based connectors (Zalo OA, Shopify, Haravan) follow the same redirect pattern:

POST /api/source-connections/<id>/connect/
→ Returns {"oauth_url": "https://provider.com/oauth/authorize?..."}
redirect_uri = "https://api.runlumi.app/api/source-connections/oauth/callback/"

The redirect_uri is fixed per environment — registered with each OAuth provider’s developer console during setup.

5.4.2 Authorization Flow

1. Admin clicks “Connect” for an OAuth connector in the UI.
2. UI calls POST /api/source-connections/<id>/connect/ which returns the provider’s OAuth URL.
3. Admin is redirected to provider’s consent page.
4. Provider redirects back to {redirect_uri}?code=AUTH_CODE&state=STATE.
5. The callback endpoint exchanges the code for tokens:
   • POST /oauth/token → {access_token, refresh_token, expires_in}
6. Tokens are stored in SourceConnection.config_json.
7. SourceConnection.status is set to active.
8. Default rules are installed via data migration.

5.4.3 Token Refresh Mechanism

def refresh_oauth_token(connection: SourceConnection) -> bool:
    """
    Attempts OAuth token refresh. Returns True if successful.
    
    1. Read refresh_token from connection.config_json
    2. POST to provider's /oauth/token endpoint with grant_type=refresh_token
    3. If success: update config_json with new tokens, update auth_expires_at
    4. If failure (invalid_grant): set status=degraded, last_error_text="token_expired"
    5. If provider down: keep current tokens, retry next poll
    """

• Refresh is attempted automatically when:
  • A poll request returns 401/403.
  • The pre-emptive check shows auth_expires_at is within 5 minutes.
• Successful refresh: updates config_json.access_token, config_json.refresh_token (if rotated), auth_expires_at.
• Failed refresh with invalid_grant: connector transitions to degraded. Admin notified via UI badge. Admin must reconnect.
• Failed refresh due to network error: retried on next poll cycle. Tokens preserved.

5.4.4 OAuth Error States

6. UI / UX

6.1 /settings/sources Page Extensions

The existing F33 SourcesPage gets a connector catalog view:

┌─ Connected Sources ───────────────────────────────────┐
│                                                        │
│  [+ Connect a source]                                  │
│                                                        │
│  ┌─ Zalo OA ─────── ● Active ────── [Disconnect] ──┐  │
│  │  Last polled: 2 min ago                           │  │
│  │  23 messages today · 2 unanswered (>2h)           │  │
│  │  [View events] [Configure rules]                  │  │
│  └───────────────────────────────────────────────────┘  │
│                                                        │
│  ┌─ Shopify ─────── ● Active ────── [Disconnect] ──┐  │
│  │  Last polled: 4 min ago                           │  │
│  │  47 orders today · 1 unfulfilled (>48h)           │  │
│  │  [View events] [Configure rules]                  │  │
│  └───────────────────────────────────────────────────┘  │
│                                                        │
│  ┌─ MISA ─────── ◐ Degraded ────── [Reconnect] ────┐  │
│  │  Error: API token expired                         │  │
│  │  Last synced: 2 hours ago                         │  │
│  └───────────────────────────────────────────────────┘  │
│                                                        │
└────────────────────────────────────────────────────────┘

6.2 Connection Flow

1. Admin clicks ”[+ Connect a source]” → catalog opens.
2. Select connector type → see auth requirements.
3. For OAuth: redirected to provider’s OAuth page → back to Lumi callback → connection active.
4. For API key: paste credentials → “Test connection” → active.
5. Default rules installed automatically. Admin can customize thresholds in “Configure rules.”

6.3 Signal Appearance in Inbox

Connector signals in /intel/inbox show:

• Source badge: “[Zalo OA]” / “[Shopify]” / “[MISA]”
• capture_channel: connector
• severity indicator from rule mapping
• Link back to SourceEvent detail

6.4 i18n

All existing sources namespace keys (F33) remain. New keys added for F44 connectors:

Connector labels (used in catalog + connection card):

Connector descriptions (shown in catalog card):

Auth flow keys:

Action keys:

Validation keys:

Both en and vi locale files must be updated together. All 8 language dirs receive en placeholders for unsupported languages (existing convention).

7. Test Plan

Coverage note (2026-06-14): Connector tests live in backend/tests/test_connectors.py. Real classes include TestF44BaseConnectorContract, TestF44ConnectorRegistry, TestF44ConnectorCatalogEndpoint, TestF44ShopifyConnector, TestF44ZaloConnector, TestF44MisaConnector, TestF44FastAccountingConnector, TestF44HaravanConnector, TestF44GoogleWorkspaceConnector, TestF44RegistryIncludesAll, TestConnectorOAuthFlow, TestConnectorCRUD, TestCatalogMarketFilter. Rows below use original per-case naming; the file is authoritative.

8. Current Behavior

• F33 ships the generic source-ingestion spine: SourceConnection → SourceRule → SourceEvent → SignalInboxItem.
• Native vendor-specific connectors do exist under backend/apps/connectors/, including Shopify, Zalo OA, MISA/Fast Accounting, Google Workspace, and the regional whatsapp_business connector.
• OAuth flow handling exists for supported providers via the source-connection OAuth endpoints.
• Polling infrastructure exists for poll-based connectors, while webhook-first connectors such as WhatsApp use the generic webhook ingress path.
• Connector config and webhook secrets encrypt at the field boundary after the shared cutover; webhook URL lookup uses a digest index while preserving constant-time verification.
• All connector credential columns use one expand/contract rollout: migration 0005 introduces cutover-aware config/webhook fields and backfills lookup hashes without rewriting values while old ECS tasks can still serve. The expand release keeps CONNECTOR_WEBHOOK_ENCRYPTION_ENABLED=false, reads plaintext or ciphertext, writes old-reader-compatible config/webhook plaintext plus the lookup hash, and repairs a hashless plaintext row written by an overlapping old task. After all old app/worker tasks drain, operators enable the flag, roll compatible tasks, and only then run cutover_webhook_secret_encryption; --restore-legacy restores config and webhook values under the same row lock before an old-image rollback. Existing senders remain on URL-secret compatibility until the Connections UI reveals a replacement key and the operator explicitly enables HMAC.
• Generic connector creation reveals its signing key only in the creation response. The web catalog intercepts that response with a non-dismissible, copyable handoff and does not continue provider setup until the operator acknowledges it; list/detail serialization never returns the key. WhatsApp instead collects and stores Meta’s app secret before creation and does not generate or reveal an unused generic key.
• Owners/admins prepare a missing or lost signing key through POST /api/source-connections/{id}/rotate-webhook-signing-key/; the replacement is revealed only in that response and excluded from normal serialization. Recovery remains available after interrupted creation even when HMAC is already required, but active-key rotation requires explicit destructive confirmation before the API call because rotation immediately invalidates the sender’s configured key. POST /api/source-connections/{id}/enable-webhook-hmac/ is a distinct second step for staged legacy senders so enforcement never precedes handoff. The OpenAPI create response explicitly declares the one-time key and webhook_hmac_required state.
• New and explicitly activated generic webhooks require HMAC; staged legacy rows remain live until their owner completes the handoff. WhatsApp setup collects the Meta app secret and phone number before creating the source, because Meta’s signature is derived from that app secret rather than Lumi’s generated generic key. OAuth state is expiring, one-time, and bound to user/organization/provider; Shopify and Haravan dynamic hosts are restricted to *.myshopify.com and *.myharavan.com respectively.
• webhook_hmac_required keeps a database-level false default for old-task inserts during a rolling deployment while the current application model defaults new rows to true; this preserves mixed-version availability without weakening new-connection enforcement.
• The connector catalog exists at GET /api/source-connections/catalog/.

9. Error States

10. Diagrams & Docs Touched

• backend/DIAGRAMS.md: add apps/connectors/ app with BaseConnector class hierarchy and poll scheduler.
• frontend/DIAGRAMS.md: update /settings/sources route tree with connector catalog and connection flow.
• F33 — Connected Source Signals: update §11 audit to note that F44 extends F33 with native connectors.
• New: backend/apps/connectors/ — connector implementations.
• backend/apps/sources/ — extended with catalog endpoint and polling.

11. Implementation Audit

Edge Cases Catalog (formerly §9, 2026-06-14; 9 items): auth-expires mid-poll (refresh then cursor resume or degraded), dedup via (organization_id, connection_id, external_event_id) unique constraint + SHA256 for systems without IDs, 429 exponential backoff 60/120/240s (Retry-After override), permanent deauthorization → disabled + reconnect flow, payload drift → degraded + Sentry + raw_payload preserved, empty-response no-op, webhook replay 5-min window + SHA256 dedup, concurrent OAuth pending_oauth 10-min flag, API version migration config_json.api_version. Full catalog in git history.

12. Open Questions / Decisions

1. Polling vs. webhook for each connector: Zalo OA currently has no outbound webhook for message status (only for message receipt). Shopify has webhooks but they require a public HTTPS endpoint. Decision: implement both modes. Use webhooks when the source system supports them (lower latency, less API load). Fall back to polling when the source system only supports polling (MISA, Google Sheets change detection).
2. VietQR vs. bank-specific APIs: Sepay handles VietQR payment receipt (already integrated in F24). Do we also integrate individual bank APIs (Vietcombank, Techcombank, MB Bank, etc.)? Decision: Sepay first (it aggregates multiple banks). Individual bank APIs deferred until a paying customer needs a specific bank.
3. Connector error vs. source system outage: How long should Lumi wait before degrading a connector when the source returns 5xx? Decision: 3 consecutive failures over 15+ minutes = degraded. Less than that = transient, silently retried.
4. Data retention for SourceEvents: Should raw connector payloads be pruned after N days? Decision: keep indefinitely for now (audit trail). Storage is cheap; the event is mostly metadata + summary, not full source system replication.

13. Bug / Change Log

2026-07-15 — Preserve HMAC rollout safety across old ECS tasks

Type: Deployment safety

• Kept db_default=False on webhook_hmac_required through both migration state transitions so an old task that omits the new column cannot raise NotNullViolation during rolling deployment.
• Retained default=True in current application state, so newly created generic connectors still fail closed unless they are explicitly staged as legacy compatibility rows.
• Added the field to the permanent rolling-deploy regression ratchet.

2026-07-14 — Close connector credential, webhook, OAuth, and SSRF findings

Type: Security hardening

• Required a valid production Fernet key delivered through native ECS Secrets Manager injection.
• Added encrypted dedicated webhook values and a deterministic lookup digest through a mixed-version-safe expand migration plus an explicit, reversible post-stabilization cutover.
• Added an owner/admin-only two-step legacy rollout: reveal a rotated key while URL-secret compatibility remains live, then enable fail-closed HMAC only after the operator confirms sender configuration.
• Added the missing OAuth-callback response and blocking web copy handoff so the one-response signing key is usable before HMAC enforcement.
• Bound OAuth state to the initiating principal and provider with one-time cache consumption.
• Rejected unsafe Shopify/Haravan dynamic hostnames before outbound calls.
• Corrected Haravan’s provider boundary to its actual *.myharavan.com merchant domain and locked cutover row loading against concurrent signing-key rotation.

2026-07-13 — Source-event cursor and org-hash readiness

GET /api/source-events/ keeps page-number pagination as the compatibility default and adds pagination=cursor with fixed UUIDv7 -id traversal. Dedupe uniqueness now includes the future organization hash key, and filter-specific composite indexes remove scans on high-volume connector ledgers. Invalid modes/cursors remain 400 and every filter remains tenant-scoped.

2026-07-02 — Verified §5, §6, §10 are free of stale connectors/tasks.py and sources/oauth.py references (batch 3)

Type: Documentation

Summary:

• Grepped §5 (API Contract), §6 (UI/UX), and §10 (Diagrams & Docs Touched) for connectors/tasks.py and sources/oauth.py — none found.
• The stale paths were already corrected in §11 rows 3 and 5 (see entry directly below).
• No additional changes required.

2026-07-02 — §5 and §11 corrected: task name, OAuth location, model fields, API actions (batch 12)

Type: Documentation Truth Fix

Summary:

• §5 API (Rows 2–3): Replaced non-existent POST /api/source-connections/<id>/disconnect/ and POST /api/source-connections/<id>/poll/ with the actual actions registered on SourceConnectionViewSet: DELETE /api/source-connections/<id>/ for disconnect and POST /api/source-connections/<id>/sync/ for manual sync (verified against backend/apps/sources/views.py and backend/apps/sources/urls.py).
• §11 Row 3: Corrected Celery task from poll_all_active_connectors at backend/apps/connectors/tasks.py (non-existent) to poll_active_connectors at backend/apps/sources/tasks.py (~line 48) (verified by grep).
• §11 Row 5: Replaced non-existent backend/apps/sources/oauth.py with the actual locations: oauth_initiate() / oauth_callback() in backend/apps/sources/views.py (~lines 278–323) and check_token_expired() in backend/apps/sources/connectors.py (~line 283) (verified by grep).
• §11 Row 7: Replaced stale field list auth_expires_at, last_polled_at, poll_interval_minutes (none are DB columns on SourceConnection) with the actual fields: last_synced_at, last_error_text, cursor_json (~lines 43–45 in backend/apps/sources/models.py); noted that poll_interval_minutes is a BaseConnector class attribute, not a DB field (verified by grep).

2026-07-02 — Spec drift fix: checked shipped ACs in §2

Type: Documentation

Summary:

• 29 of 38 [ ] ACs in §2 flipped to [x] — confirmed shipped against §11 audit rows 1–9.
• Phase 1 connector ACs (Zalo OA ×7, MISA ×4, Fast Accounting ×3, Shopify ×4, Haravan ×3, Google Workspace ×5) and General Framework ACs (×3) all confirmed by §11 rows 1–9 (all ✅).
• Phase 2 ACs (VietQR, Base.vn/1Office, Microsoft 365 — 9 total) remain [ ] per §11 rows 10–11 (⚠️ gap, deferred post-v1).
• No code changes.

2026-06-22 — Manual sync now runs in-request (no broker dependency)

Type: Implementation Change (no API contract change)

Summary:

• POST /api/source-connections/{id}/sync/ previously enqueued a Celery task via sync_single_connection.delay(...). With no Celery worker deployed in the ECS topology (deploy/DIAGRAMS.md → “Worker Task … NOT DEPLOYED”) and no reachable broker, that path raised kombu.exceptions.OperationalError and returned 500 to the user — the LUMI-BE-7 class from the 2026-06-16 chat-empty-replies postmortem.
• apps/sources/views.py:SourceConnectionViewSet.sync now invokes sync_single_connection(str(conn.id)) synchronously in-request. Response shape is unchanged (202 Accepted with {"status": "queued"}); only the execution mechanism moved off the broker.
• Regression test backend/tests/test_sources.py::TestConnectorSyncAsync::test_sync_does_not_500_with_broker_down pins the contract under the production topology (eager OFF + dead broker).
• Failure class sealed by static gate scripts/check-no-delay-in-views.py (+ self-test, wired into make verify): any future .delay()/.apply_async() in a backend/apps/**/views.py file fails the build with the postmortem named in the remediation message.

Reason:

• Production has no worker — the previous code 500’d for every user who clicked “Sync now”. The scheduled fan-out path in tasks.py still uses .delay(), which is correct (background context); only the user-facing endpoint moved inline.

Impact:

• Users who click “Sync now” now get the actual sync result in-request (instead of an immediate 500). Connector poll has its own per-connector HTTP timeouts (see apps/connectors/*), so a slow source bounds the request, not the broker.

2026-06-15 — Frontend route namespaced

Frontend route namespaced: /sources -> /settings/sources and /inbox -> /intel/inbox (app-wide IA restructure; API paths unchanged).

2026-06-14 — Spec restructured to TEMPLATE.md canonical order (§1–§13)

Type: Documentation Change

Summary:

• §9 Edge Cases (9 items, 9.1–9.10 with gap at 9.7) absorbed into §11 Implementation Audit note.
• §12 Implementation Audit converted from Priority/Phase tables + narrative to canonical 4-column table (11 rows: 9 ✅, 2 ⚠️ gaps for Phase 2 connectors and rate_limit_hits counter).
• Sections renumbered: §10 ES → §9, §11 Diags → §10, §12 Audit → §11, §13 OQ → §12, §14 BL → §13.

Reason:

• Align with TEMPLATE.md canonical section order per SP fleet restructuring pass.

Impact:

• Spec authors and future implementation agents.

2026-06-13 — Spec refinement: detailed contracts, OAuth flow, edge cases, i18n

Type: Documentation Change

Summary:

• Added detailed connector contracts (§4.5) with auth types, normalized event schemas, rate limits, and per-connector error handling for all 6 launch connectors.
• Added OAuth flow documentation (§5.4): redirect URL pattern, authorization flow, token refresh mechanism, and error states.
• Refined Phase 1 acceptance criteria (§2) with full Given/When/Then structure per connector.
• Expanded test plan (§7) from 12 to 31 test cases, covering all connectors, edge cases, and error states.
• Added numbered Edge Cases (§9): auth expires mid-poll, duplicate events (dedup key strategy), rate limit exceeded (backoff policy), permanent deauthorization, payload drift, empty response, webhook registration failure.
• Replaced error state table with detailed 11-row matrix (§10) linking scenario → user-visible → internal → log → retry.
• Expanded i18n keys (§6.4) from brief list to full tables: 6 connector labels, 6 descriptions, 7 auth flow keys, 8 action keys, 5 validation keys with en/vi translations.
• Expanded Implementation Audit (§12) with per-priority tables, file paths, design constraints, and rework items.
• Updated spec status from draft to agreed.

Reason:

• Spec needed refinement per the implementation audit gap analysis. Connector contracts, OAuth specifics, and error handling were previously underspecified, making implementation ambiguous.

Impact:

• Spec authors and future implementation agents.

Tests:

• Spec-only change — no code tests affected.

2026-06-13 — Added edge cases: webhook replay attack, concurrent OAuth flows, connector migration

Type: Documentation Change

Summary:

• Added §9.8 (Webhook Replay Attack): replay window, stale timestamp rejection, dedup fallback.
• Added §9.9 (Concurrent OAuth Flows): pending_oauth flag, 10-min expiry, state param.
• Added §9.10 (Connector Migration): api_version tracking, backwards-compatible migration window, degraded behavior on unknown version.

Reason:

• Spec needed hardening from F44 audit review. These three edge cases were not captured in the initial spec refinement.

Impact:

• Spec authors and future implementation agents.

Tests:

• Spec-only change — no code tests affected.