docs: complete project research for NetForge ISP Management SaaS

Files: - STACK.md - FEATURES.md - ARCHITECTURE.md - PITFALLS.md - SUMMARY.md Key findings: - Stack: Next.js 15 + TypeScript + PostgreSQL + Prisma monolith; BullMQ for billing cron jobs; MikroTik client needs library verification (LOW confidence) - Architecture: Modular monolith with shared-DB multi-tenancy; JournalEntryService as sole ledger gateway; derived financial state (no balance fields) - Critical pitfall: Cross-tenant data leakage + fake accounting + billing state machine must all be solved in Phase 1 before any feature work Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 17:21:44 +08:00
parent ec2e583787
commit c407e182f8
5 changed files with 1716 additions and 0 deletions
--- a/.planning/research/STACK.md
+++ b/.planning/research/STACK.md
@@ -0,0 +1,346 @@
+# Technology Stack
+
+**Project:** NetForge — Multi-Tenant ISP Management SaaS
+**Researched:** 2026-03-04
+**Research Confidence Note:** WebSearch, WebFetch, and Context7 tools were unavailable during this session. All version numbers and rationale below are derived from knowledge current as of August 2025 training data plus logical inference from the project's requirements. Treat version numbers as MEDIUM confidence — verify against official changelogs before pinning in package.json.
+
+---
+
+## Recommended Stack
+
+### Core Framework
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| Next.js | 15.x | Full-stack React framework | App Router with server components reduces client JS bundle. API Routes serve as the API layer for n8n/MikroTik integrations. Edge middleware enables per-tenant routing. Deployed easily on Vercel or self-hosted. |
+| React | 19.x | UI rendering | Bundled with Next.js 15. Server Components mean accounting tables and dashboards render server-side, improving cold-load performance for data-heavy views. |
+| TypeScript | 5.x | Type safety | Accounting logic (double-entry journal entries, debit/credit balancing) must never silently pass wrong types. TypeScript catches ledger arithmetic errors at compile time, not runtime. |
+
+**Why Next.js over alternatives:**
+- SvelteKit: Smaller ecosystem for complex data tables and form libraries needed for accounting UI. Fewer multi-tenant SaaS templates to learn from.
+- Remix: Good, but Next.js App Router has converged on similar patterns with broader adoption and tooling.
+- Separate frontend + backend (React + Express): Overkill complexity for this team size. Next.js API Routes cover the API-first requirement adequately for v1; migrate to standalone API if load demands it.
+
+---
+
+### Database
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| PostgreSQL | 16.x | Primary relational database | Double-entry accounting requires ACID transactions — no eventual consistency. PostgreSQL's Row Level Security (RLS) is the right primitive for tenant isolation. JSON columns handle MikroTik config blobs without a separate document store. |
+| Prisma ORM | 5.x | Database access layer | Type-safe queries align with TypeScript across the stack. Prisma Migrate handles schema evolution safely (critical when adding accounting tables incrementally). Prisma's `$transaction` API is correct for double-entry journal entry writes (debit + credit must be atomic). |
+| Redis | 7.x | Session store, job queue backing, rate limiting | Session storage for web app auth, queue backing for billing engine cron jobs, rate limiting for public API endpoints used by n8n. |
+
+**Multi-tenancy approach — `tenant_id` column strategy (not schema-per-tenant):**
+
+Use a single shared schema where every table has a `tenant_id` column. Enforce isolation at the application layer (Prisma middleware that injects `WHERE tenant_id = ?` on every query) and optionally with PostgreSQL Row Level Security as a defense-in-depth second layer.
+
+Why not schema-per-tenant: Managing 50+ ISP schemas becomes a migration nightmare. Prisma does not have first-class support for dynamic schema switching.
+
+Why not database-per-tenant: Operational cost and connection pool explosion. Not warranted at this scale (target: under 500 subscribers per ISP, under ~200 ISPs on platform).
+
+**Confidence:** HIGH for PostgreSQL and Prisma. MEDIUM for Redis version number.
+
+---
+
+### Authentication & Authorization
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| NextAuth.js (Auth.js) | 5.x (v5 beta / stable) | Authentication | Handles session management, credential-based login, and JWT/session tokens. Supports the five-role model via custom session claims. Native Next.js App Router integration. No managed auth vendor lock-in. |
+| CASL | 6.x | Fine-grained RBAC authorization | Five roles (Admin, Office Staff, Collector, Technician, Client) have overlapping permissions with context-dependent rules. CASL allows defining "Collector can read Payment where collectorId = self" without encoding every rule in if/else blocks. Separates auth logic from business logic. |
+
+**Why not Clerk or Auth0:**
+- Clerk/Auth0 are managed services. Multi-tenant SaaS with per-ISP user pools requires either expensive tiers or complex configuration. Auth.js + CASL gives full control at the cost of more initial setup — correct tradeoff for a product with complex role rules.
+- Clerk is viable if team wants to move fast on auth and accept the vendor dependency. Note for roadmap: if Clerk is chosen, budget for migration away if ISP count scales significantly.
+
+**Confidence:** HIGH for Auth.js architecture. MEDIUM for CASL v6 version number.
+
+---
+
+### UI & Component Layer
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| Tailwind CSS | 4.x | Utility-first styling | Fastest path to a consistent, responsive UI across the five role dashboards. No CSS module naming overhead. v4 introduces significant performance improvements in build time. |
+| shadcn/ui | (copy-paste, no package version) | Base component library | Not an installed package — components are copied into the project. This means full control over component customization. Data tables (billing history, journal entries, inventory) and form components (payment entry, job order forms) are the primary shadcn components used. Built on Radix UI primitives for accessibility. |
+| Recharts | 2.x | Dashboard charts | Revenue vs. collected charts, cash flow over time, subscriber growth. Recharts integrates cleanly with React and is the most common charting library in the Next.js/shadcn ecosystem. |
+| React Hook Form | 7.x | Form management | Payment collection forms, ticket creation, inventory entries. RHF minimizes re-renders on large forms (client onboarding has many fields). |
+| Zod | 3.x | Schema validation | Paired with React Hook Form for client-side validation and reused on the API layer for server-side validation. Single source of truth for data shapes shared between form and API. |
+
+**Why not MUI or Ant Design:**
+- MUI and Ant Design are full design systems that fight with custom branding. shadcn/ui with Tailwind gives a professional look without the override battle.
+- For a vertical SaaS (ISP ops), the UI doesn't need to be generic — it should reflect the domain. shadcn gives building blocks; Tailwind gives control.
+
+**Confidence:** HIGH for Tailwind and shadcn approach. MEDIUM for Recharts as chart choice (TanStack Table is also relevant for data-heavy tables).
+
+---
+
+### Data Tables
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| TanStack Table | 8.x | Complex data table logic | Billing history, journal entry ledger, inventory list, subscriber list all require: sorting, filtering, pagination, and column visibility toggles. TanStack Table is headless — works with shadcn/Tailwind rendering. The most capable React table library available as of 2025. |
+
+**Confidence:** HIGH. TanStack Table is the clear standard for complex React tables.
+
+---
+
+### API Layer
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| Next.js API Routes (App Router Route Handlers) | (bundled with Next.js) | REST API endpoints | API-first requirement means all operations are exposed via REST. Route Handlers in Next.js App Router handle this natively without a separate Express server. n8n connects to these endpoints. MikroTik integration uses these endpoints as webhooks. |
+| Zod | 3.x | API request/response validation | Validate all incoming payloads on API routes. Reuse the same schemas from the frontend forms. |
+
+**When to extract to a standalone API:**
+If the platform reaches scale where Next.js server-side rendering and API processing compete for the same compute, extract the API to a standalone service (Fastify or Hono). For v1-v2, keeping them together in Next.js is the correct tradeoff.
+
+**Confidence:** MEDIUM. This is defensible for v1 but the team should monitor API latency as billing engine cron jobs grow.
+
+---
+
+### Background Jobs & Scheduling
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| BullMQ | 5.x | Job queue for billing engine and async tasks | Billing cron jobs (auto-generate invoices on billing cycle date, send SMS reminders) must be reliable and retryable. BullMQ on Redis is the standard for Node.js job queues. Supports delayed jobs (schedule SMS 3 days before due date), retries with backoff, and job priority. |
+| node-cron or BullMQ Scheduler | (bundled) | Scheduling recurring billing runs | Trigger monthly invoice generation per tenant on their configured billing cycle date. |
+
+**Why not Vercel Cron Jobs:**
+Vercel cron jobs have execution time limits (10s on Hobby, 60s on Pro). Billing engine runs that process many subscribers in a single tenant will exceed this. BullMQ with a dedicated worker process is the correct architecture.
+
+**Deployment note:** Background workers must run as a separate process, not as Vercel serverless functions. This means deployment on a VPS (Railway, Render, or a raw VPS) is required for the worker, even if the web app is on Vercel.
+
+**Confidence:** HIGH for BullMQ. MEDIUM for version number.
+
+---
+
+### Billing & Payments
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| Custom billing engine (in-house) | — | ISP billing logic | No off-the-shelf billing library covers ISP-specific patterns: prepaid vs. postpaid cycles per subscriber, collector cash tracking, partial payments, reconnection fees. Must be built custom on top of the accounting layer. |
+| Stripe (for SaaS subscriptions only) | — | Charging ISPs for platform subscription | The SaaS owner charges ISPs monthly. Stripe handles this B2B subscription. Stripe is NOT used for ISP-to-subscriber payments in v1 (subscriber payments are tracked manually as cash/bank transfer). |
+
+**Confidence:** HIGH for architecture decision. MEDIUM for Stripe as the SaaS billing choice (PayMongo is also relevant if targeting Philippines market, given the ISP context suggests Philippines-based operations).
+
+**Philippines market note:** The PRD mentions SMS and door-to-door cash collection patterns common in Philippine ISPs. Consider PayMongo (local payment gateway) for online subscriber payments if that feature is added. GCash and Maya (PayMaya) are the dominant digital wallets.
+
+---
+
+### Double-Entry Accounting Engine
+
+This is the highest-complexity module. No third-party library fully covers it — must be built custom.
+
+| Component | Approach | Why |
+|-----------|----------|-----|
+| Chart of Accounts | Database table: `accounts (id, tenant_id, code, name, type, normal_balance)` | Standard COA structure. Account types: Asset, Liability, Equity, Revenue, Expense. |
+| Journal Entries | Database table: `journal_entries` (header) + `journal_entry_lines` (debit/credit lines). Prisma `$transaction` wraps every write. | Every financial event (invoice created, payment received, expense logged) writes an immutable journal entry. Double-entry constraint: sum(debits) must equal sum(credits) — enforced in application code and optionally as a PostgreSQL check constraint. |
+| Ledger / Trial Balance | Computed from journal_entry_lines by account | Sum debits and credits per account. Trial balance = all accounts with net balance. |
+| Reports | Derived queries: Balance Sheet = asset/liability/equity accounts at a point in time. P&L = revenue/expense accounts for a period. | Standard accounting report queries. |
+
+**What to NOT use:** Do not use an external accounting API (QuickBooks, Xero API) as the accounting backend. The double-entry system must be internal to NetForge — ISPs need their books within the platform, not exported to another service. This is a core product differentiator.
+
+**Confidence:** HIGH for the architectural approach. The patterns here (journal_entries + journal_entry_lines) are the industry-standard ledger schema used by every accounting system from QuickBooks to Odoo.
+
+---
+
+### MikroTik Integration
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| RouterOS API client (Node.js) | `node-routeros` or `mikronode` | Connect to MikroTik routers via RouterOS API | MikroTik exposes a proprietary TCP API (port 8728/8729). Node.js libraries wrap this protocol. Commands: enable/disable PPPoE users, query connection status, update rate limits per plan change. |
+
+**Confidence:** LOW — verify which Node.js RouterOS API client is actively maintained in 2025/2026. `node-routeros` was the most common as of training data but open-source maintenance may have shifted. This needs verification before Phase 1 implementation.
+
+**Fallback:** If no well-maintained Node.js library exists, implement the RouterOS API protocol directly — it is a documented protocol (ASCII-based sentences). This is non-trivial but the protocol is stable.
+
+---
+
+### Infrastructure & Deployment
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| Railway or Render | — | Application hosting | Both support Next.js + PostgreSQL + Redis + background workers as separate services in one project. Railway is simpler to configure for a full-stack monorepo (web + worker + db + redis). Vercel for the Next.js web app is viable but creates split deployment (Vercel + Railway for worker). |
+| Neon or Supabase (PostgreSQL) | — | Managed PostgreSQL | Neon offers serverless PostgreSQL with branching (useful for preview environments). Supabase includes PostgreSQL + Auth + Storage but the auth layer conflicts with Auth.js. Use Neon for managed Postgres if self-hosted Railway is not preferred. |
+| Cloudflare (CDN/DNS) | — | CDN, DDoS protection, DNS | Free tier covers multi-tenant domain routing (each ISP subdomain: `isp-name.netforge.app`). |
+| Docker | 24.x+ | Containerize worker process | Background worker runs as a Docker container alongside the web app in Railway/Render. Ensures consistent environment between dev and production. |
+
+**Self-hosting note:** If the target market is Philippine ISPs with cost sensitivity, a single VPS (DigitalOcean/Linode/Hetzner) with Docker Compose may be more economical than managed services. This is a valid deployment strategy for v1.
+
+**Confidence:** MEDIUM for Railway/Render recommendation. The self-hosted VPS path is equally valid and may be preferable given the market.
+
+---
+
+### SMS
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| Semaphore SMS (Philippines) | — | Billing reminders, payment confirmations | The dominant SMS API provider for the Philippines market. Handles pre-due reminders, overdue alerts, and payment receipt SMS. |
+| Twilio | — | Alternative / fallback | International option. More expensive for Philippine SMS volume but better documented and more reliable for global coverage. |
+
+**Recommendation:** Use Semaphore for initial Philippine market targeting. Abstract behind a `SmsService` interface so the provider can be swapped without touching business logic.
+
+**Confidence:** MEDIUM for Semaphore. This is based on known Philippine ISP/SME tech patterns. Verify Semaphore API is still active and competitively priced in 2026.
+
+---
+
+### Testing
+
+| Technology | Version | Purpose | Why |
+|------------|---------|---------|-----|
+| Vitest | 2.x | Unit and integration tests | Accounting engine (journal entry creation, trial balance calculation) must be unit tested. Vitest is faster than Jest and has native TypeScript support. |
+| Playwright | 1.x | End-to-end tests | Test billing cycle runs, payment collection flows, and role-based access restrictions end-to-end. |
+
+**Confidence:** HIGH for Vitest. The accounting engine particularly demands unit tests — a bug in debit/credit logic corrupts financial records for all tenants.
+
+---
+
+### Supporting Libraries
+
+| Library | Version | Purpose | When to Use |
+|---------|---------|---------|-------------|
+| date-fns | 3.x | Date manipulation | Billing cycle calculations (next due date, days overdue, billing period start/end). Never use moment.js — deprecated. |
+| decimal.js or dinero.js | — | Monetary arithmetic | Never use JavaScript floating-point for money calculations. All financial amounts (invoice totals, payments, account balances) must use decimal-safe arithmetic. |
+| Papa Parse | 5.x | CSV import/export | Bulk subscriber import, payment export for reconciliation, accounting export. |
+| pino | 9.x | Structured logging | Audit trail for financial operations. Every journal entry write, payment record, and billing cycle run should be logged with tenant context for debugging. |
+
+**Confidence:** HIGH for decimal.js/dinero.js requirement — this is non-negotiable for an accounting system. MEDIUM for specific library versions.
+
+---
+
+## Alternatives Considered
+
+| Category | Recommended | Alternative | Why Not |
+|----------|-------------|-------------|---------|
+| Framework | Next.js 15 | Remix | Remix is solid but Next.js has broader ecosystem for multi-tenant SaaS patterns, more shadcn examples, and App Router has now caught up on streaming/loading patterns. |
+| ORM | Prisma | Drizzle ORM | Drizzle is lighter and faster, gaining popularity in 2025. For this project: Prisma's `$transaction` API and Middleware (for injecting tenant_id) are better suited for accounting-grade data integrity. Revisit Drizzle for v2 if Prisma query performance becomes a bottleneck. |
+| ORM | Prisma | TypeORM | TypeORM is older, more complex, slower DX. Not recommended. |
+| Auth | Auth.js | Clerk | Clerk accelerates auth setup but adds per-MAU cost that scales with ISP subscriber count. With 5 roles and complex per-tenant user pools, Auth.js custom implementation is worth the upfront cost. |
+| Job Queue | BullMQ | Inngest | Inngest is a good managed option but adds vendor dependency for a billing-critical process. BullMQ on self-hosted Redis is more predictable for billing reliability. |
+| Database | PostgreSQL | MySQL | PostgreSQL's Row Level Security is a material advantage for multi-tenancy. MySQL lacks this. |
+| Database | PostgreSQL | MongoDB | Double-entry accounting requires relational joins and ACID transactions. MongoDB's eventual consistency is wrong for financial data. |
+| Hosting | Railway | Vercel | Vercel doesn't support persistent workers without workarounds. Billing engine requires a persistent process. Railway supports web + worker + db as first-class services. |
+| Charts | Recharts | Chart.js | Recharts is more idiomatic React (composable components). Chart.js requires imperative configuration. |
+
+---
+
+## Installation
+
+```bash
+# Core Next.js project
+npx create-next-app@latest netforge --typescript --tailwind --eslint --app --src-dir
+
+# ORM and database
+npm install prisma @prisma/client
+npx prisma init
+
+# Auth
+npm install next-auth@beta
+npm install @auth/prisma-adapter
+
+# Authorization
+npm install @casl/ability @casl/react
+
+# UI Components (shadcn — follow shadcn CLI, not npm install)
+npx shadcn@latest init
+
+# Forms and validation
+npm install react-hook-form zod @hookform/resolvers
+
+# Data tables
+npm install @tanstack/react-table
+
+# Charts
+npm install recharts
+
+# Job queue
+npm install bullmq ioredis
+
+# Date handling
+npm install date-fns
+
+# Money arithmetic
+npm install decimal.js
+
+# CSV handling
+npm install papaparse
+npm install -D @types/papaparse
+
+# Logging
+npm install pino pino-pretty
+
+# Testing
+npm install -D vitest @vitejs/plugin-react playwright @playwright/test
+
+# Development
+npm install -D @types/node tsx
+```
+
+---
+
+## Critical Architecture Decisions That Affect Every Phase
+
+### 1. Tenant Context Middleware (Implement in Phase 1)
+
+Every Prisma query must automatically scope to the current tenant. Implement a Prisma extension/middleware that injects `tenant_id` on every read and write:
+
+```typescript
+// prisma/middleware/tenant.ts
+prisma.$use(async (params, next) => {
+  const tenantId = getTenantContext(); // from request context
+  if (params.args.where) {
+    params.args.where.tenantId = tenantId;
+  } else {
+    params.args.where = { tenantId };
+  }
+  return next(params);
+});
+```
+
+If this is bolted on later, every query written before it needs auditing. Do it first.
+
+### 2. Monetary Arithmetic (Non-Negotiable)
+
+All financial values in the database are stored as integers in the smallest currency unit (centavos for PHP, cents for USD). Never store floats for money. The application layer converts to/from human-readable decimal using `decimal.js`. This prevents floating-point rounding errors in accounting reports.
+
+```typescript
+// WRONG
+const total = 100.1 + 200.2; // = 300.30000000000003
+
+// RIGHT — store as integer centavos, compute with decimal.js
+import Decimal from 'decimal.js';
+const total = new Decimal(10010).plus(20020).toNumber(); // = 30030 centavos
+```
+
+### 3. Journal Entry Immutability (Accounting Integrity)
+
+Once a journal entry is written, it must never be deleted or modified. Corrections are made via reversal entries (new journal entry that negates the original). Enforce this at the database level:
+
+```sql
+-- No DELETE allowed on journal_entries
+-- Application code enforces; consider RLS policy as backup
+```
+
+### 4. API-First from Day One
+
+All business operations must go through the API layer, even when called from the Next.js server-side. This ensures n8n and future mobile app can access every operation without a separate integration layer.
+
+---
+
+## Sources
+
+Note: Due to tool restrictions during this research session (WebSearch, WebFetch, Context7 unavailable), all recommendations are based on:
+- Knowledge of the Next.js, React, Prisma, PostgreSQL ecosystem as of August 2025 training data
+- Direct inference from the NetForge PRD requirements (double-entry accounting, multi-tenancy, Philippine ISP market)
+- Standard patterns from the multi-tenant SaaS engineering community
+
+**Confidence summary by area:**
+- Core framework (Next.js + TypeScript + PostgreSQL + Prisma): HIGH — these are the dominant choices for this class of application with strong community validation
+- Auth approach (Auth.js + CASL): MEDIUM-HIGH — correct architecture, version numbers need verification
+- Accounting engine design (journal_entries schema): HIGH — industry-standard pattern independent of implementation stack
+- MikroTik integration (Node.js client): LOW — needs verification of library maintenance status in 2026
+- SMS (Semaphore): MEDIUM — verify pricing and API stability for Philippine market
+- Infrastructure (Railway): MEDIUM — verify pricing tiers and feature parity for worker support
+- Version numbers: MEDIUM — verify all package versions against npm before pinning