Files
TapTrack-Hub/.paul/ROADMAP.md
kevin-asprec 132290957c feat(phase-5): on-prem sync poll hardening + stale job reclaim
Harden the Hub-side polling protocol for on-prem TapTrack agents:
- sync/poll: accept job_failed_ids (retry/fail on-prem delivery failures)
- sync/poll: deduct credits + write ledger when on-prem reports sent jobs
- sync/poll: return feature_flags (tier-based) + suspended flag in config
- sync/poll: skip job dispatch for suspended/expired schools
- sms_jobs: add delivered_via (pull|push) + processing_started_at columns
- tasks/sms: new sms.reclaim_stale_jobs task resets processing→pending if on-prem
  goes offline (jobs stuck >5 min), enabling Celery push fallback
- tasks/sms: tag Celery-sent jobs as delivered_via='push'
- worker: schedule reclaim_stale_jobs every 5 minutes
- migration: 001_phase5 adds delivered_via + processing_started_at to sms_jobs
2026-03-16 12:30:37 +08:00

15 KiB

Roadmap: TapTrack Hub

Overview

TapTrack Hub is a cloud-hosted SaaS control plane for managing all on-prem TapTrack school deployments. It handles school licensing, SMS gateway proxying (Semaphore PH), billing/invoicing, support tickets, monthly automated reports, and provides a client-facing school portal.

Two user roles:

  • Super Admin — you (the operator): full control over all schools, licenses, SMS, billing
  • School Admin — each client school's administrator: portal access for billing, SMS, tickets

Architecture: FastAPI + PostgreSQL + Redis + Celery + Vue 3 + Nginx, deployed as Docker Compose on a cloud VPS.

SMS flow: On-prem TapTrack polls Hub every 30s → Hub queues SMS jobs → Hub sends via Semaphore → reports back status.


Current Milestone

v1.0 — Foundation & Core Services Status: Phase 5 complete — Phase 6 next Phases: 5 of 15 complete


Phase Status

Phase Name Plans Status Completed
1 Project Setup & Infrastructure 1 Complete 2026-03-15
2 School Registry + License Mgmt 2 Complete 2026-03-15
3 On-Prem License Validation 1 Complete 2026-03-15
4 SMS Gateway (credits + queue) 2 Complete 2026-03-15
5 On-Prem SMS Polling Agent 1 Complete 2026-03-16
6 Super Admin Dashboard UI TBD Not started
7 School Admin Portal UI TBD Not started
8 Billing Engine + Invoice PDF TBD Not started
9 Email Dispatcher TBD Not started
10 Support Ticket System TBD Not started
11 Monthly Report Generation TBD Not started
12 On-Prem Monthly Report Pull TBD Not started
13 Feature Flags + Suspension TBD Not started
14 Onboarding Wizard + Welcome Email TBD Not started
15 UX Polish + Ops Tools TBD Not started

Phase Details

Phase 1: Project Setup & Infrastructure

Goal: Runnable Docker Compose stack with FastAPI, PostgreSQL, Redis, Celery, and Vue 3 frontend skeleton. Completed: 2026-03-15

What was built:

  • Docker Compose: backend (gunicorn+uvicorn), celery worker, celery beat, frontend (vite build + nginx), PostgreSQL, Redis, nginx reverse proxy
  • Full database schema: hub_users, schools, licenses, sms_jobs, sms_credit_ledger, invoices, invoice_line_items, school_subscriptions, support_tickets, ticket_replies, audit_logs, announcements
  • Auth: JWT (python-jose), bcrypt, role-based dependencies (super_admin, school_admin)
  • All FastAPI routers: auth, schools, licenses, sms, billing, tickets, users, dashboard, school_portal, announcements, sync
  • All Celery tasks: sms.process_queue, billing.generate_monthly_invoices, billing.send_invoice_email, billing.check_overdue, license.check_expiry, reports.send_monthly_reports
  • Email service (SMTP), seed script (super admin)
  • Vue 3 frontend: router (super admin + portal routes), Pinia auth store, full api.ts client, Tailwind CSS, all page skeletons (Login, Dashboard, Schools, SchoolDetail, Licenses, SMS, Billing, Tickets, TicketDetail, Users, Announcements, NotFoundPage + all 5 portal pages)
  • Shared components: AppSidebar, PortalSidebar, SidebarItem, KpiCard, StatusBadge, ToastStack

Phase 2: School Registry + License Management

Goal: Super admin can register schools, issue license keys, set billing plans, add SMS credits, activate/suspend. Depends on: Phase 1

Scope:

  • School create/edit form (modal) in SchoolsPage
  • License detail panel in SchoolDetailPage: change expiry, tier, revoke
  • Subscription setup form per school (monthly fee, SMS cost per message)
  • Add SMS credits form with ledger view
  • Send welcome email on school creation (onboarding stub)
  • School status lifecycle: pending → active → suspended/expired

Plans:

  • 2-01: School create/edit modal + license expiry editor in SchoolDetailPage
  • 2-02: Subscription setup + SMS credit top-up form + ledger table

Phase 3: On-Prem License Validation

Goal: TapTrack on-prem validates its license key against Hub on startup and periodically; Hub returns feature flags and school config. Depends on: Phase 2

Scope:

  • POST /api/licenses/validate endpoint (already built) — verify, update last_seen, return features
  • On-prem TapTrack changes:
    • Celery beat task every 6h to call Hub license validate
    • If expired/revoked: SMS disabled, warning banner in UI
    • Store validated features in Redis (5min TTL) for fast feature flag checks
  • Hardware fingerprint binding (optional — store server MAC on first validation)

Plans:

  • 3-01: On-prem license validation task + feature flag Redis cache (TapTrack side)
  • 3-02: UI warning banner on TapTrack when license invalid/expired

Phase 4: SMS Gateway (Credits + Job Queue)

Goal: Schools submit SMS jobs via Hub; Hub processes them through Semaphore using school's sender name and deducts credits. Depends on: Phase 2

Scope:

  • POST /api/sms/submit (already built) — on-prem posts jobs
  • Celery sms.process_queue (already built) — processes via Semaphore, deducts credits, logs ledger
  • SMS job retry logic (up to 5 retries, exponential backoff)
  • Low credit email alert (already built)
  • Super admin SMS dashboard: jobs by status, school breakdown, Semaphore API health check
  • Credit ledger view per school in SchoolDetailPage

Plans:

  • 4-01: SMS dashboard stats + per-school credit ledger UI in SchoolDetailPage
  • 4-02: SMS retry logic hardening + Semaphore health check endpoint

Phase 5: On-Prem SMS Polling Agent

Goal: TapTrack on-prem polls Hub every 30s for pending SMS jobs; sends them via Hub's Semaphore account; reports back completion. Depends on: Phase 4

Scope:

  • Replace TapTrack's direct Semaphore calls with Hub polling
  • On-prem Celery task: POST /api/sync/poll with license key + completed job IDs
  • Hub returns jobs to send, school config (sender name, credits, status)
  • On-prem sends via Hub-provided sender name (not its own settings)
  • If school status = suspended: skip SMS, show banner
  • Graceful degradation: if Hub unreachable, queue locally and retry

Plans:

  • 5-01: On-prem polling task replacing direct Semaphore calls (TapTrack side)
  • 5-02: Graceful degradation + local queue fallback (TapTrack side)

Phase 6: Super Admin Dashboard UI

Goal: Rich super admin dashboard with KPI overview, school health list, SMS volume chart, revenue snapshot. Depends on: Phase 4, Phase 2

Scope:

  • KPI row: total/active/suspended schools, licenses expiring, open tickets, SMS today, pending invoices
  • School health table: name, status, credits, last seen, license expiry (sortable)
  • SMS volume chart (last 30 days) — bar chart by day
  • Revenue snapshot: MRR, outstanding, overdue total
  • Quick action buttons: add school, trigger SMS queue, check overdue invoices

Plans:

  • 6-01: Dashboard KPI row + school health table
  • 6-02: SMS volume chart + revenue snapshot panel

Phase 7: School Admin Portal UI

Goal: Complete school admin portal — overview, billing history, SMS reports with charts, support tickets, profile. Depends on: Phase 5, Phase 8

Scope:

  • Portal overview: license status card, credit meter with low-credit warning, monthly SMS usage chart
  • Billing tab: full invoice table, PDF download button per invoice, credit top-up request form
  • SMS tab: monthly SMS log, delivery rate chart, credit burn rate chart
  • Tickets tab: create ticket + reply thread view (already built as stubs)
  • Profile tab: change password, view school details

Plans:

  • 7-01: Portal overview enhancements (credit meter, charts, license countdown)
  • 7-02: Billing PDF download + credit top-up request form
  • 7-03: SMS usage charts + delivery rate visualization

Phase 8: Billing Engine + Invoice PDF

Goal: Automated monthly invoice generation per school; PDF export; mark as paid workflow; overdue escalation. Depends on: Phase 2

Scope:

  • billing.generate_monthly_invoices Celery task (already built) — creates draft invoices
  • Invoice PDF generation using Jinja2 HTML template + WeasyPrint
  • Invoice PDF stored at /app/data/invoices/{id}.pdf
  • GET /api/billing/invoices/{id}/pdf — serve PDF
  • Mark as paid workflow (super admin sets paid_at, payment_method, payment_reference)
  • Overdue escalation: 7 days → warning email, 30 days → suspend school
  • Invoice number sequencing: INV-YYYYMM-NNNN

Plans:

  • 8-01: Invoice PDF generation (Jinja2 template + WeasyPrint) + serve endpoint
  • 8-02: Mark-paid workflow + overdue escalation (suspend at 30 days)

Phase 9: Email Dispatcher

Goal: All automated emails (invoices, low credits, license expiry, monthly reports) send correctly via SMTP with proper HTML templates. Depends on: Phase 8

Scope:

  • Jinja2 HTML email templates: invoice, low credit alert, license expiry warning, monthly report, welcome email
  • Email queue via Celery (retry on failure)
  • Email delivery log table (email_logs): to, subject, type, status, sent_at, error
  • GET /api/email-logs — super admin can view all email delivery history
  • Test email endpoint: POST /api/email/test — send a test email to verify SMTP config

Plans:

  • 9-01: Jinja2 HTML email templates + email_logs table + delivery logging
  • 9-02: Email delivery log UI in super admin + test email endpoint

Phase 10: Support Ticket System

Goal: Full-featured support ticket system with SLA tracking, internal notes, priority management, and email notifications. Depends on: Phase 9 (emails)

Scope:

  • Ticket create/reply already built — polish and harden
  • Email notification on new ticket (to super admin) and on reply (to school admin)
  • Internal notes (super admin only, already supported via is_internal flag)
  • SLA tracking: first_response_at already stored — display SLA status in ticket list
  • Priority escalation: tickets open > 48h auto-escalate to high
  • Ticket assignment to specific super admin users
  • Bulk actions: close all resolved, assign multiple

Plans:

  • 10-01: Email notifications on ticket create/reply + SLA display in ticket list
  • 10-02: Ticket assignment + priority auto-escalation + bulk close

Phase 11: Monthly Report Generation + Email

Goal: On the 1st of each month, automatically generate and email a comprehensive report to each active school. Depends on: Phase 9, Phase 12

Scope:

  • reports.send_monthly_reports Celery task (already built as stub) — enhance with real data
  • Report content: attendance summary, SMS usage, credit consumption, invoice for period
  • HTML email template for monthly report (branded, professional)
  • Super admin can manually trigger report for any school: POST /api/reports/send/{school_id}
  • School portal: view past monthly reports (list + detail)

Plans:

  • 11-01: Monthly report data aggregation + HTML email template
  • 11-02: Manual trigger endpoint + report history in school portal

Phase 12: On-Prem Monthly Report Pull

Goal: Hub pulls attendance summary data from each on-prem TapTrack instance to populate monthly reports. Depends on: Phase 11

Scope:

  • New endpoint on TapTrack on-prem: GET /api/hub/monthly-report?key={license_key}&month={YYYY-MM}
    • Returns: total_students, present_days, absent_days, late_days, avg_attendance_rate, sms_sent
  • Hub Celery task: on the 1st at 5am, poll each active school's on-prem for monthly data
  • Store result in school_monthly_stats table for report generation
  • Fallback: if on-prem unreachable, report shows "data unavailable" for attendance section

Plans:

  • 12-01: TapTrack on-prem monthly report endpoint + school_monthly_stats table in Hub
  • 12-02: Hub Celery pull task + fallback handling

Phase 13: Feature Flags + Service Suspension Logic

Goal: Hub controls which features each school's on-prem instance can use, based on tier and payment status. Depends on: Phase 3, Phase 8

Scope:

  • Feature flags per tier (already in _tier_features()) — expose via /api/licenses/validate
  • Suspension flow:
    1. Invoice overdue 30 days → school.status = suspended
    2. On-prem polls Hub → gets status=suspended → disables SMS, shows banner
    3. On-prem attendance still works (not blocked) — only SMS and reports disabled
  • Feature flag UI: per-school overrides in SchoolDetailPage (super admin can enable beta features)
  • feature_overrides JSON column on School model for per-school feature customization

Plans:

  • 13-01: feature_overrides column + per-school feature flag UI in SchoolDetailPage
  • 13-02: Suspension propagation to on-prem via sync poll + on-prem banner

Phase 14: Onboarding Wizard + Welcome Email

Goal: When a new school is registered, automatically send a welcome email with license key and setup instructions; guide super admin through initial school setup. Depends on: Phase 9

Scope:

  • Welcome email template: school name, license key, setup instructions link, Hub portal login URL
  • Triggered automatically on school creation (after license is issued)
  • Super admin onboarding checklist in SchoolDetailPage:
    • School created
    • License issued
    • Billing plan set
    • SMS credits added
    • Welcome email sent
    • School admin account created
  • "Resend welcome email" button
  • "Complete onboarding" button sets school.status = active

Plans:

  • 14-01: Welcome email template + auto-send on school creation
  • 14-02: Onboarding checklist UI in SchoolDetailPage + "activate school" workflow

Phase 15: UX Polish + Super Admin Ops Tools

Goal: Production-ready polish: loading states, empty states, error boundaries, keyboard shortcuts, mobile responsiveness, audit log viewer, bulk operations. Depends on: All previous phases

Scope:

  • Audit log viewer: GET /api/audit-logs — searchable table of all system events
  • Bulk school actions: export CSV, bulk status change
  • Global search: search schools, tickets, invoices by name/number
  • Keyboard shortcuts: / focus search, N new school, Esc close modals
  • Mobile-responsive layouts for portal (school admins may use phone)
  • Error boundary component for API failures
  • Dashboard refresh button with last-updated timestamp
  • Empty state illustrations for zero-data pages
  • Production hardening: rate limiting on auth endpoints, HTTPS redirect

Plans:

  • 15-01: Audit log table + global search + bulk export
  • 15-02: Mobile-responsive portal + error boundary + production hardening