bellsystems-cp/AUTOMATION_ENGINE_STRATEGY.md

BellSystems CP — Automation & Notification Engine Strategy

Overview

This document defines the architecture and implementation plan for a three-tier intelligence layer built on top of the existing BellSystems Control Panel. The system consists of:

1. Event Logs — passive, timestamped record of notable system events
2. Notifications — real-time or near-real-time alerts surfaced in the UI
3. Automation Rules — trigger → condition → action pipelines, configurable via UI

The existing tech stack is unchanged: FastAPI + SQLite (aiosqlite) + Firestore + React. Everything new slots in as additional tables in mqtt_data.db, new backend modules, and new frontend pages/components.

---

Architecture

┌──────────────────────────────────────────────────┐
│  Scheduler Loop (runs inside existing FastAPI    │
│  startup, alongside email_sync_loop)             │
│                                                  │
│  Every 60s: evaluate_rules()                     │
│    ↓                                             │
│  Rules Engine                                    │
│    → loads enabled rules from DB                 │
│    → evaluates conditions against live data      │
│    → fires Action Executor on match              │
│                                                  │
│  Action Executor                                 │
│    → create_event_log()                          │
│    → create_notification()                       │
│    → send_email()           (existing)           │
│    → mqtt_publish_command() (existing)           │
│    → update_field()                              │
└──────────────────────────────────────────────────┘
                     ↕ REST / WebSocket
┌──────────────────────────────────────────────────┐
│  React Frontend                                  │
│    - Bell icon in Header (unread count badge)    │
│    - Notifications dropdown/panel                │
│    - /automations page (rule CRUD)               │
│    - Event Log viewer (filterable)               │
└──────────────────────────────────────────────────┘

---

Database Schema (additions to mqtt_data.db)

event_log

Permanent, append-only record of things that happened.

CREATE TABLE IF NOT EXISTS event_log (
    id          TEXT PRIMARY KEY,
    category    TEXT NOT NULL,  -- 'device' | 'crm' | 'quotation' | 'user' | 'system'
    entity_type TEXT,           -- 'device' | 'customer' | 'quotation' | 'user'
    entity_id   TEXT,           -- the ID of the affected record
    title       TEXT NOT NULL,
    detail      TEXT,
    severity    TEXT NOT NULL DEFAULT 'info',  -- 'info' | 'warning' | 'error'
    rule_id     TEXT,           -- which automation rule triggered this (nullable)
    created_at  TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_event_log_category ON event_log(category, created_at);
CREATE INDEX IF NOT EXISTS idx_event_log_entity ON event_log(entity_type, entity_id);

notifications

Short-lived, user-facing alerts. Cleared once read or after TTL.

CREATE TABLE IF NOT EXISTS notifications (
    id          TEXT PRIMARY KEY,
    title       TEXT NOT NULL,
    body        TEXT,
    link        TEXT,           -- optional frontend route, e.g. "/crm/customers/abc123"
    severity    TEXT NOT NULL DEFAULT 'info',  -- 'info' | 'warning' | 'error' | 'success'
    is_read     INTEGER NOT NULL DEFAULT 0,
    rule_id     TEXT,
    entity_type TEXT,
    entity_id   TEXT,
    created_at  TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_notifications_read ON notifications(is_read, created_at);

automation_rules

Stores user-defined rules. Evaluated by the scheduler.

CREATE TABLE IF NOT EXISTS automation_rules (
    id              TEXT PRIMARY KEY,
    name            TEXT NOT NULL,
    description     TEXT,
    enabled         INTEGER NOT NULL DEFAULT 1,
    trigger_type    TEXT NOT NULL,  -- 'schedule' | 'mqtt_alert' | 'email_received'
    trigger_config  TEXT NOT NULL DEFAULT '{}',  -- JSON
    conditions      TEXT NOT NULL DEFAULT '[]',  -- JSON array of condition objects
    actions         TEXT NOT NULL DEFAULT '[]',  -- JSON array of action objects
    cooldown_hours  REAL NOT NULL DEFAULT 0,     -- min hours between firing on same entity
    last_run_at     TEXT,
    run_count       INTEGER NOT NULL DEFAULT 0,
    created_at      TEXT NOT NULL DEFAULT (datetime('now')),
    updated_at      TEXT NOT NULL DEFAULT (datetime('now'))
);

automation_run_log

Deduplication and audit trail for rule executions.

CREATE TABLE IF NOT EXISTS automation_run_log (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    rule_id     TEXT NOT NULL,
    entity_type TEXT,
    entity_id   TEXT,
    status      TEXT NOT NULL,  -- 'fired' | 'skipped_cooldown' | 'error'
    detail      TEXT,
    fired_at    TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_run_log_rule ON automation_run_log(rule_id, fired_at);
CREATE INDEX IF NOT EXISTS idx_run_log_entity ON automation_run_log(entity_type, entity_id, fired_at);

---

Backend Module: automation/

New module at backend/automation/, registered in main.py.

backend/automation/
├── __init__.py
├── router.py        # CRUD for rules, event_log GET, notifications GET/PATCH
├── models.py        # Pydantic schemas for rules, conditions, actions
├── engine.py        # evaluate_rules(), condition evaluators, action executors
├── scheduler.py     # automation_loop() async task, wired into main.py startup
└── database.py      # DB helpers for all 4 new tables

Wiring into main.py

from automation.router import router as automation_router
from automation.scheduler import automation_loop

app.include_router(automation_router)

# In startup():
asyncio.create_task(automation_loop())

---

Rule Object Structure (JSON, stored in DB)

{
  "id": "rule_abc123",
  "name": "Quotation follow-up after 7 days",
  "enabled": true,
  "trigger_type": "schedule",
  "trigger_config": { "interval_hours": 24 },
  "conditions": [
    { "entity": "quotation", "field": "status",              "op": "eq",  "value": "sent" },
    { "entity": "quotation", "field": "days_since_updated",  "op": "gte", "value": 7 },
    { "entity": "quotation", "field": "has_reply",           "op": "eq",  "value": false }
  ],
  "actions": [
    {
      "type": "send_email",
      "template_key": "quotation_followup",
      "to": "{{quotation.client_email}}",
      "subject": "Following up on Quotation {{quotation.quotation_number}}",
      "body": "Hi {{customer.name}}, did you have a chance to review our quotation?"
    },
    {
      "type": "create_notification",
      "title": "Follow-up sent to {{customer.name}}",
      "link": "/crm/customers/{{quotation.customer_id}}"
    },
    {
      "type": "create_event_log",
      "category": "quotation",
      "severity": "info",
      "title": "Auto follow-up sent for {{quotation.quotation_number}}"
    }
  ],
  "cooldown_hours": 168
}

---

Supported Trigger Types

Trigger	How it works
schedule	Evaluated every N hours by the background loop
mqtt_alert	Fires immediately when _handle_alerts() in mqtt/logger.py upserts an alert — hook into that function
email_received	Fires inside sync_emails() in crm/email_sync.py after a new inbound email is stored

Note: mqtt_alert and email_received triggers bypass the scheduler loop — they are called directly from the relevant handler functions, giving near-real-time response.

---

Supported Condition Operators

op	Meaning
eq	equals
neq	not equals
gt / gte / lt / lte	numeric comparisons
contains	string contains
is_null / not_null	field presence
days_since	computed: (now - field_datetime) in days

---

Supported Action Types

Action	What it does	Notes
create_event_log	Writes to event_log table	Always safe to fire
create_notification	Writes to notifications table	Surfaces in UI bell icon
send_email	Calls existing crm.email_sync.send_email()	Uses existing mail accounts
update_field	Updates a field on an entity in DB/Firestore	Use carefully — define allowed fields explicitly
mqtt_publish	Calls mqtt_manager.publish_command()	For device auto-actions
webhook	HTTP POST to an external URL	Future / optional

---

Notification System (Frontend)

Bell Icon in Header

• Polling endpoint: GET /api/notifications?unread=true&limit=20
• Poll interval: 30 seconds (or switch to WebSocket push — the WS infrastructure already exists via mqtt_manager)
• Badge shows unread count
• Click opens a dropdown panel listing recent notifications with title, time, severity color, and optional link

Notification Panel

• Mark as read: PATCH /api/notifications/{id}/read
• Mark all read: PATCH /api/notifications/read-all
• Link field navigates to the relevant page on click

Toast Popups (optional, Phase 3 polish)

• Triggered by polling detecting new unread notifications since last check
• Use an existing toast component if one exists, otherwise add a lightweight one

---

Automation Rules UI (/automations)

A new sidebar entry under Settings (sysadmin/admin only).

Rule List Page

• Table: name, enabled toggle, trigger type, last run, run count, edit/delete
• "New Rule" button

Rule Editor (modal or full page)

• Name & description — free text
• Trigger — dropdown: Schedule / MQTT Alert / Email Received
  • Schedule: interval hours input
  • MQTT Alert: subsystem filter (optional)
  • Email Received: from address filter (optional)
• Conditions — dynamic list, each row:
  • Entity selector (Quotation / Device / Customer / User)
  • Field selector (populated based on entity)
  • Operator dropdown
  • Value input
• Actions — dynamic list, each row:
  • Action type dropdown
  • Type-specific fields (to address, subject, body for email; notification title/body; etc.)
  • Template variables hint: {{quotation.quotation_number}}, {{customer.name}}, etc.
• Cooldown — hours between firings on the same entity
• Enabled toggle

Rule Run History

• Per-rule log: when it fired, on which entity, success/error

---

Event Log UI

Accessible from /event-log route, linked from Dashboard.

• Filterable by: category, severity, entity type, date range
• Columns: time, category, severity badge, title, entity link
• Append-only (no deletion from UI)
• Retention: purge entries older than configurable days (e.g. 180 days) via the existing purge_loop pattern in mqtt/database.py

---

Pre-Built Rules (Seeded on First Run, All Disabled)

These are created on first startup — the admin enables and customizes them.

Rule	Trigger	Condition	Action
Quotation follow-up	Schedule 24h	status=sent AND days_since_updated ≥ 7 AND no reply	Send follow-up email + notify
Device offline warning	Schedule 1h	no heartbeat for > 2h	Create notification + event log
New unknown email	email_received	customer_id IS NULL	Create notification
Subscription expiring soon	Schedule 24h	subscription.expiry_date within 7 days	Notify + send email
Device critical alert	mqtt_alert	state = CRITICAL	Notify + event log + optional MQTT restart
Quotation expired	Schedule 24h	status=sent AND days_since_updated ≥ 30	Update status → expired + notify

---

Implementation Phases

Phase 1 — Foundation (DB + API)

• Add 4 new tables to mqtt/database.py schema + migrations
• Create automation/database.py with all DB helpers
• Create automation/models.py — Pydantic schemas for rules, conditions, actions, notifications, event_log
• Create automation/router.py — CRUD for rules, GET event_log, GET/PATCH notifications
• Wire router into main.py

Phase 2 — Rules Engine + Scheduler

• Create automation/engine.py — condition evaluator, template renderer, action executor
• Create automation/scheduler.py — automation_loop() async task
• Hook email_received trigger into crm/email_sync.sync_emails()
• Hook mqtt_alert trigger into mqtt/logger._handle_alerts()
• Seed pre-built (disabled) rules on first startup
• Wire automation_loop() into main.py startup

Phase 3 — Notification UI

• Bell icon with unread badge in Header.jsx
• Notifications dropdown panel component
• 30s polling hook in React
• Mark read / mark all read

Phase 4 — Automation Rules UI

• /automations route and rule list page
• Rule editor form (conditions + actions dynamic builder)
• Enable/disable toggle
• Run history per rule
• Add "Automations" entry to Sidebar under Settings

Phase 5 — Event Log UI

• /event-log route with filterable table
• Purge policy wired into existing purge_loop
• Dashboard widget showing recent high-severity events

Phase 6 — Polish

• Toast notifications on new unread detection
• Template variable previewer in rule editor
• "Run now" button per rule (for testing without waiting for scheduler)
• Named email templates stored in DB (reusable across rules)

---

Key Design Decisions

Decision	Choice	Reason
Storage	SQLite (same mqtt_data.db)	Consistent with existing pattern; no new infra
Scheduler	asyncio task in FastAPI startup	Same pattern as email_sync_loop and purge_loop already in main.py
Rule format	JSON columns in DB	Flexible, UI-editable, no schema migrations per new rule type
Template variables	{{entity.field}} string interpolation	Simple to implement, readable in UI
Cooldown dedup	automation_run_log per (rule_id, entity_id)	Prevents repeat firing on same quotation/device within cooldown window
Notification delivery	DB polling (30s) initially	The WS infra exists (mqtt_manager._ws_subscribers) — easy to upgrade later
Pre-built rules	Seeded as disabled	Non-intrusive — admin must consciously enable each one
update_field safety	Explicit allowlist of permitted fields	Prevents accidental data corruption from misconfigured rules

---

Template Variables Reference

Available inside action body, subject, title, link fields:

Variable	Source
{{customer.name}}	Firestore crm_customers
{{customer.organization}}	Firestore crm_customers
{{quotation.quotation_number}}	SQLite crm_quotations
{{quotation.final_total}}	SQLite crm_quotations
{{quotation.status}}	SQLite crm_quotations
{{quotation.client_email}}	SQLite crm_quotations
{{device.serial}}	Firestore devices
{{device.label}}	Firestore devices
{{alert.subsystem}}	MQTT alert payload
{{alert.state}}	MQTT alert payload
{{user.email}}	Firestore users

---

Notes

• crm/email_sync.send_email() is reused as-is for the send_email action type. The engine constructs the call parameters.
• update_field actions start with an allowlist of: quotation.status, user.status. Expand deliberately.
• For MQTT auto-restart, mqtt_manager.publish_command(serial, "restart", {}) already works — the engine just calls it.
• Firestore is read-only from the automation engine (for customer/device lookups). All writes go to SQLite, consistent with the existing architecture.
• The has_reply condition on quotations is computed by checking whether any crm_comms_log entry exists with direction='inbound' and customer_id matching the quotation's customer, dated after the quotation's updated_at.