project-vesper/vesper/documentation/project-vesper-plan.md

Project Vesper — Manufacturing Automation Master Plan

How to use this document: Work through each Phase in order. Each Phase has self-contained tasks you can hand directly to Claude Code. Phases 1–3 are foundational; don't skip ahead. Phases 4–6 build on top of them.

---

Current Stack (Reference)

Layer	Technology
Microcontroller	ESP32 (ESP8266 on older models, STM32 possible future)
MCU Firmware	Arduino / C++ (moving to PlatformIO)
Tablet App	Flutter / FlutterFlow (Android)
Phone App	Flutter / FlutterFlow (Android + iOS)
Admin Console Backend	Python 3.11+ / FastAPI
Admin Console Frontend	React + Tailwind CSS
Primary Database	Firebase Firestore (via Admin SDK)
Secondary Database	Local SQLite (MQTT logs, etc.)
File Storage	Firebase Storage
MQTT Broker	Mosquitto on VPS
Web Server	NGINX on VPS (OTA files, reverse proxy)
Deployment	Gitea → git pull on VPS, Docker Compose locally

---

Serial Number Format (Locked In)

PV-YYMMM-BBTTR-XXXXX

PV       = Project Vesper prefix
YY       = 2-digit year         (e.g. 26)
MMM      = 3-char date block    = 2-digit month letter + 2-digit day (e.g. A18 = Jan 18)
BB       = Board Type code      (e.g. BC = BellCore, BP = BellPRO)
TT       = Board Type revision  (e.g. 02)
R        = Literal "R"
XXXXX    = 5-char random suffix (A-Z, 0-9, excluding 0/O/1/I for label clarity)

Example: PV-26A18-BC02R-X7KQA

Month Letter Codes: A=Jan, B=Feb, C=Mar, D=Apr, E=May, F=Jun, G=Jul, H=Aug, I=Sep, J=Oct, K=Nov, L=Dec

---

Phase 1 — PlatformIO Migration (Firmware Side)

Goal: Replace Arduino IDE with PlatformIO. All future firmware work happens here. This unlocks scripted builds and the ability to produce .bin files on demand.

Tell Claude Code: "Help me migrate my existing Arduino IDE ESP32 project to PlatformIO with multiple board environments."

Tasks

• Install PlatformIO extension in VS Code
• Create platformio.ini with one [env] block per hardware variant. Example:

[env:bellcore-v2]
platform = espressif32
board = esp32dev
board_build.partitions = partitions/custom_4mb.csv
board_build.flash_mode = dio
board_upload.flash_size = 4MB
build_flags =
    -DBOARD_TYPE="BC"
    -DBOARD_VERSION="02"
    -DPSRAM_ENABLED=1

[env:bellcore-v1]
platform = espressif32
board = esp32dev
board_build.partitions = partitions/custom_2mb.csv
board_build.flash_mode = dout
board_upload.flash_size = 2MB
build_flags =
    -DBOARD_TYPE="BC"
    -DBOARD_VERSION="01"
    -DPSRAM_ENABLED=0

• Move all #include library dependencies into lib_deps in platformio.ini (no more manual library manager)
• Verify pio run -e bellcore-v2 compiles clean
• Confirm .pio/build/bellcore-v2/firmware.bin is produced
• Create a /firmware directory structure on the server (NGINX already serves this):

/srv/ota/
  bellcore-v1/
    latest.bin
    v1.0.0.bin
    v1.0.1.bin
  bellcore-v2/
    latest.bin
    ...

• Update your OTA logic in firmware to pull from /ota/{board_type}/latest.bin
• Add a scripts/build_and_upload.sh that compiles + copies the new .bin to the right NGINX folder (run this after every release)

NVS Partition Generator Setup

• Install esptool and esp-idf NVS tool: pip install esptool
• Grab nvs_partition_gen.py from ESP-IDF tools (or install via idf-component-manager)
• Test generating a .bin from a CSV manually:

key,type,encoding,value
serial_number,data,string,PV-26A18-BC02R-X7KQA
hw_type,data,string,BC
hw_version,data,string,02

python nvs_partition_gen.py generate nvs_data.csv nvs_data.bin 0x6000

• Confirm the ESP32 reads NVS values correctly on boot with this pre-flashed partition
• Note the NVS partition address for your board (check your partition table CSV — typically 0x9000)

---

Phase 2 — MQTT Dynamic Auth (Backend Side)

Goal: Replace mosquitto_passwd manual SSH with automatic credential management. New devices are live on MQTT the moment they exist in your database. Per-device topic isolation enforced automatically.

Tell Claude Code: "Help me set up mosquitto-go-auth on my VPS with a FastAPI backend for dynamic MQTT authentication and ACL enforcement."

Tasks

2a. Install mosquitto-go-auth on VPS

• Install Go on VPS (required to build the plugin)
• Clone and build mosquitto-go-auth:

  git clone https://github.com/iegomez/mosquitto-go-auth
  cd mosquitto-go-auth && make
  

• Update mosquitto.conf to load the plugin:

  auth_plugin /path/to/go-auth.so
  auth_opt_backends http
  auth_opt_http_host localhost
  auth_opt_http_port 8000
  auth_opt_http_getuser_uri /mqtt/auth/user
  auth_opt_http_aclcheck_uri /mqtt/auth/acl
  auth_opt_cache true
  auth_opt_cache_host localhost
  auth_opt_cache_reset true
  auth_opt_auth_cache_seconds 300
  auth_opt_acl_cache_seconds 300
  

2b. Add MQTT Auth Endpoints to FastAPI

• Create /mqtt/auth/user endpoint — Mosquitto calls this on CONNECT:

  • Receives: username (= device SN), password
  • Checks Firestore/SQLite for device record + hashed password
  • Returns: 200 (allow) or 403 (deny)

• Create /mqtt/auth/acl endpoint — Mosquitto calls this on SUBSCRIBE/PUBLISH:

  • Receives: username, topic, acc (1=sub, 2=pub)
  • Rule: username must match the SN segment in the topic
    • Topic pattern: /vesper/{SN}/data or /vesper/{SN}/control
    • Extract {SN} from topic, compare to username
  • Returns: 200 or 403

• For user phone app clients: Add a separate user auth flow

  • Users authenticate with their Firebase UID as MQTT username
  • ACL check: look up which devices this UID owns in Firestore, permit only those SNs in topics

2c. MQTT Password Strategy

Use HMAC-derived passwords so you never have to store or manually set them:

import hmac, hashlib

MQTT_SECRET = os.getenv("MQTT_SECRET")  # Keep in .env, never commit

def derive_mqtt_password(serial_number: str) -> str:
    return hmac.new(
        MQTT_SECRET.encode(),
        serial_number.encode(),
        hashlib.sha256
    ).hexdigest()[:32]

• Device SN is known → password is deterministic → firmware can compute it at boot

• No password storage needed in DB (just re-derive on auth check)

• Changing MQTT_SECRET rotates all passwords at once if ever needed

• Add MQTT_SECRET to your .env and Docker Compose secrets

• Update firmware to derive its own MQTT password using the same HMAC logic (port to C++)

• Remove all existing mosquitto_passwd file entries and disable static auth

2d. Test

• New device connects with correct SN + derived password → allowed
• Device tries to sub/pub on another device's topic → denied (403)
• Wrong password → denied
• Confirm cache is working (check logs, only 1-2 auth calls per session)

---

Phase 3 — Serial Number & Batch Management in Admin Console

Goal: SN generation, DB registration, and MQTT credential provisioning all happen in one flow in the React Console. The Flutter admin app is retired.

Tell Claude Code: "Add a Manufacturing / Batch Management section to our React+FastAPI admin console with the following features..."

Tasks

3a. Backend — New API Routes in FastAPI

• POST /manufacturing/batch — Create a new batch:

  • Input: board_type, board_version, quantity, subscription_plan, available_outputs
  • Generate N serial numbers using the PV-YYMMM-BBTTR-XXXXX format
  • Check Firestore for collisions, regenerate if collision found
  • Write N device documents to Firestore collection devices:

    {
      "serial_number": "PV-26A18-BC02R-X7KQA",
      "hw_type": "BC",
      "hw_version": "02",
      "status": "manufactured",
      "subscription_plan": "standard",
      "available_outputs": 8,
      "created_at": "...",
      "owner": null,
      "users_list": []
    }
    

  • Returns: list of created SNs

• GET /manufacturing/batch/{batch_id} — List devices in a batch with status

• GET /manufacturing/devices — List all devices with filters (status, hw_type, date range)

• POST /manufacturing/devices/{sn}/assign — Pre-assign device to a customer email

• GET /manufacturing/firmware/{hw_type}/{hw_version} — Return download URL for the correct .bin

3b. SN Generator Utility (Python)

# utils/serial_number.py
import random, string
from datetime import datetime

MONTH_CODES = "ABCDEFGHIJKL"
SAFE_CHARS = "ABCDEFGHJKLMNPQRSTUVWXYZ23456789"  # No 0,O,1,I

def generate_serial(board_type: str, board_version: str) -> str:
    now = datetime.utcnow()
    year = now.strftime("%y")
    month = MONTH_CODES[now.month - 1]
    day = now.strftime("%d")
    random_suffix = "".join(random.choices(SAFE_CHARS, k=5))
    return f"PV-{year}{month}{day}-{board_type}{board_version}R-{random_suffix}"

3c. NVS Binary Generation in FastAPI

• Copy nvs_partition_gen.py into your FastAPI project
• Add endpoint GET /manufacturing/devices/{sn}/nvs.bin:
  • Generates a temp CSV for this SN
  • Runs nvs_partition_gen.py to produce the .bin
  • Returns the binary file as a download
• Add endpoint GET /manufacturing/devices/{sn}/firmware.bin:
  • Looks up device's hw_type and hw_version from Firestore
  • Returns the correct firmware .bin from the NGINX folder (or redirects to NGINX URL)

3d. Label Sheet Generation

• Add POST /manufacturing/batch/{batch_id}/labels endpoint
  • Returns a PDF with one label per device
  • Each label contains: SN (human readable), QR code of SN, HW Type, HW Version
  • Use reportlab or fpdf2 Python library for PDF generation
  • QR code: use qrcode Python library

3e. Frontend — Manufacturing Section in React Console

• New route: /manufacturing
• Batch Creator: Form with board type selector, quantity, subscription plan → calls POST /manufacturing/batch → shows created SNs + download label PDF button
• Device List: Filterable table of all devices with status badges (manufactured / sold / claimed / active)
• Device Detail Page: Shows all fields, allows status update, shows assignment history

---

Phase 4 — Browser-Based Flashing (The Provisioning Wizard)

Goal: A single browser tab handles the entire provisioning flow. Plug in ESP32, click through wizard, done. No Arduino IDE, no esptool CLI, no SSH.

Tell Claude Code: "Add a Device Provisioning Wizard to our React admin console using esptool-js and the Web Serial API."

Browser requirement: Chrome or Edge only (Web Serial API). Firefox not supported. This is fine for an internal manufacturing tool.

Tasks

4a. Add esptool-js to React Console

• npm install esptool-js (or use the CDN build)
• Confirm Chrome is used on the manufacturing bench laptop

4b. Provisioning Wizard UI (React Component)

Build a step-by-step wizard. Steps:

Step 1 — Select or Create Device

• Search existing unprovisioned device by SN, OR
• Quick-create single device (calls POST /manufacturing/batch with qty=1)
• Displays SN, HW Type, HW Version

Step 2 — Flash Device

• "Connect Device" button → triggers Web Serial port picker
• Fetches nvs.bin and firmware.bin from your FastAPI backend for this SN
• Shows two progress bars: NVS partition flash + Firmware flash
• Flash addresses (example for standard ESP32):
  • NVS: 0x9000 (verify against your partition table)
  • Firmware: 0x10000
• On completion: updates device status to flashed in Firestore via API call

Step 3 — Verify

• Prompt: "Power cycle device and wait for it to connect"
• Poll Firestore (or MQTT) for first heartbeat/connection from this SN
• Show green checkmark when device phone home
• Updates status to provisioned

Step 4 — Done

• Show summary
• Option: "Provision next device" (loops back to Step 1 with same batch settings)
• Option: "Print label" (downloads single-device PDF label)

4c. esptool-js Flash Logic (Skeleton)

import { ESPLoader, Transport } from "esptool-js";

async function flashDevice(serialPort, nvsArrayBuffer, firmwareArrayBuffer) {
  const transport = new Transport(serialPort);
  const loader = new ESPLoader({ transport, baudrate: 460800 });

  await loader.main_fn();
  await loader.flash_id();

  await loader.write_flash({
    fileArray: [
      { data: nvsArrayBuffer,      address: 0x9000  },
      { data: firmwareArrayBuffer, address: 0x10000 },
    ],
    flashSize: "keep",
    flashMode: "keep",
    flashFreq: "keep",
    eraseAll: false,
    compress: true,
  });

  await transport.disconnect();
}

4d. NGINX CORS Headers

Add to your NGINX config so the browser can fetch .bin files:

location /ota/ {
    add_header Access-Control-Allow-Origin "https://your-console-domain.com";
    add_header Access-Control-Allow-Methods "GET";
}

---

Phase 5 — Email Notifications

Goal: Admin Console can send transactional emails (device assignment invites, alerts, etc.)

Tell Claude Code: "Add email sending capability to our FastAPI backend using Resend (or SMTP)."

Tasks

• Sign up for Resend (free tier: 3000 emails/month, 100/day)
• Add RESEND_API_KEY to .env
• Install: pip install resend
• Create utils/email.py:

import resend
import os

resend.api_key = os.getenv("RESEND_API_KEY")

def send_device_invite(customer_email: str, serial_number: str, customer_name: str = None):
    resend.Emails.send({
        "from": "noreply@yourcompany.com",
        "to": customer_email,
        "subject": "Your Vesper device is ready",
        "html": f"""
            <h2>Your device has been registered</h2>
            <p>Serial Number: <strong>{serial_number}</strong></p>
            <p>Open the Vesper app and enter this serial number to get started.</p>
        """
    })

• Hook into POST /manufacturing/devices/{sn}/assign to send invite automatically
• Add basic email templates for: device assignment, welcome, error alerts

---

Phase 6 — Polish & Retire Legacy Tools

Goal: Clean up. Everything lives in the Console. Nothing is done manually.

Tasks

• Retire Flutter admin app — confirm every function it had is now in the React Console
• Remove static mosquitto password file — all auth is dynamic now
• Add device status dashboard to Console home: counts by status, recent provisioning activity
• Add audit log — every manufacturing action (batch created, device flashed, device assigned) logged to SQLite with timestamp and admin user
• Document your platformio.ini environments — add a FIRMWARE_VARIANTS.md to the firmware repo
• Set up Gitea webhook → on push to main, VPS auto-pulls and restarts Docker containers (replaces manual git pull)

---

Gitea / Docker Compose Deployment Note

Your local Docker Compose setup and VPS production setup are the same codebase — this is correct and will continue to work fine. A few tips:

• Use a .env.production and .env.development file, never commit either
• Your docker-compose.yml should reference ${ENV_VAR} from the env file
• The Gitea webhook for auto-deploy is a simple shell script triggered by the webhook:

  #!/bin/bash
  cd /path/to/project
  git pull origin main
  docker compose up -d --build
  

• Protect this webhook endpoint with a secret token

---

Summary — What Gets Killed

Old Way	Replaced By
Arduino IDE	PlatformIO (VS Code)
Manual mosquitto_passwd via SSH	FastAPI dynamic auth endpoints
Flutter admin app	React Admin Console
Manual SN generation	Console batch creator
Manual DB entry per device	Auto-provisioned on batch creation
Manual firmware flash + config page	Browser provisioning wizard (esptool-js)
Manual NVS entry via HTTP config page	Pre-flashed NVS partition

Estimated Time Per Device (After All Phases Complete)

Task	Time
Generate 15-device batch + print labels	~2 min
Flash each device (plug in, click Flash, done)	~3 min each (parallelizable)
Devices self-verify on lab WiFi	passive, ~1 min each
Total for 15 devices	~20-25 min

vs. current ~20 min per device = ~5 hours for 15.