feat: initial BantayCam scaffold
- FastAPI backend with async SQLAlchemy - Camera RTSP management (add, start, stop) - Vehicle detection (YOLO + fast-alpr) - Type: car, motorcycle, truck, jeepney - Color detection (HSV) - License plate OCR - Motorcycle person count - Face detection + InsightFace ArcFace embedding - pgvector identity grouping (auto-cluster same face) - Vehicle + person movement trail APIs - Docker Compose with pgvector/pg16 - Models: Camera, VehicleIdentity, VehicleEvent, PersonIdentity, FaceEvent
This commit is contained in:
23
.env.example
Normal file
23
.env.example
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
# BantayCam Environment Variables
|
||||||
|
# Copy to .env and fill in your values
|
||||||
|
|
||||||
|
# Database
|
||||||
|
DATABASE_URL=postgresql+asyncpg://bantaycam:bantaycam@localhost:5432/bantaycam
|
||||||
|
|
||||||
|
# Storage
|
||||||
|
STORAGE_TYPE=local
|
||||||
|
STORAGE_PATH=./snapshots
|
||||||
|
|
||||||
|
# AI Models
|
||||||
|
YOLO_MODEL=yolov8n.pt
|
||||||
|
FACE_MODEL=buffalo_l
|
||||||
|
GPU_DEVICE=0 # Set to -1 for CPU-only
|
||||||
|
|
||||||
|
# Processing tuning
|
||||||
|
PROCESS_EVERY_N_FRAMES=5
|
||||||
|
FACE_SIMILARITY_THRESHOLD=0.5
|
||||||
|
PLATE_CONFIDENCE_THRESHOLD=0.6
|
||||||
|
|
||||||
|
# Alerts (optional)
|
||||||
|
TELEGRAM_BOT_TOKEN=
|
||||||
|
TELEGRAM_CHAT_ID=
|
||||||
107
README.md
Normal file
107
README.md
Normal file
@@ -0,0 +1,107 @@
|
|||||||
|
# 🎥 BantayCam
|
||||||
|
|
||||||
|
**Integrated License Plate + Face Recognition for Philippine CCTV Infrastructure**
|
||||||
|
|
||||||
|
Built with Python, FastAPI, YOLOv8, InsightFace, and PostgreSQL.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- 📡 **RTSP Camera Management** — Add any IP camera by RTSP URL, name it, start/stop anytime
|
||||||
|
- 🚗 **Vehicle Detection** — Detects car, motorcycle, jeepney, truck; identifies type + color
|
||||||
|
- 🔢 **License Plate Recognition** — Reads PH plates; logs every appearance with timestamp + camera
|
||||||
|
- 🏍️ **Motorcycle Person Count** — Counts riders on motorcycles
|
||||||
|
- 👤 **Face Detection** — Detects and embeds every face using InsightFace ArcFace
|
||||||
|
- 🔗 **Identity Grouping** — Auto-groups same face across events using pgvector similarity search
|
||||||
|
- 🗺️ **Movement Trail** — Query where a face or plate has been seen, on which camera, and when
|
||||||
|
- 🏷️ **Labeling** — Register identities with names, mark vehicles as resident/blacklisted
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
### Requirements
|
||||||
|
- Docker + Docker Compose
|
||||||
|
- (Optional) NVIDIA GPU + CUDA for real-time processing
|
||||||
|
|
||||||
|
### Run with Docker
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone http://192.168.1.159:3000/kibin/bantaycam.git
|
||||||
|
cd bantaycam
|
||||||
|
cp .env.example .env
|
||||||
|
docker-compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
API available at: http://localhost:8000
|
||||||
|
Swagger docs: http://localhost:8000/docs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
### Cameras
|
||||||
|
| Method | Endpoint | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| GET | /api/v1/cameras/ | List all cameras |
|
||||||
|
| POST | /api/v1/cameras/ | Register new RTSP camera |
|
||||||
|
| POST | /api/v1/cameras/{id}/start | Start stream |
|
||||||
|
| POST | /api/v1/cameras/{id}/stop | Stop stream |
|
||||||
|
| GET | /api/v1/cameras/streams/status | Live stream status |
|
||||||
|
|
||||||
|
### Vehicles
|
||||||
|
| Method | Endpoint | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| GET | /api/v1/vehicles/identities | List all known vehicles |
|
||||||
|
| GET | /api/v1/vehicles/identities/{id}/trail | Where was this vehicle seen? |
|
||||||
|
| GET | /api/v1/vehicles/events | Recent vehicle detection events |
|
||||||
|
| PATCH | /api/v1/vehicles/identities/{id} | Label/whitelist/blacklist a vehicle |
|
||||||
|
|
||||||
|
### Persons
|
||||||
|
| Method | Endpoint | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| GET | /api/v1/persons/identities | List all detected identities |
|
||||||
|
| GET | /api/v1/persons/identities/{id}/trail | Where was this person seen? |
|
||||||
|
| PATCH | /api/v1/persons/identities/{id} | Register name/watchlist |
|
||||||
|
| GET | /api/v1/persons/events | Recent face detection events |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
Camera (RTSP) → StreamWorker → Frame Queue → DetectionPipeline
|
||||||
|
│
|
||||||
|
┌────────────────────┤
|
||||||
|
│ │
|
||||||
|
VehicleDetector FaceService
|
||||||
|
(YOLO + ALPR) (InsightFace)
|
||||||
|
│ │
|
||||||
|
└────────────────────┤
|
||||||
|
│
|
||||||
|
PostgreSQL + pgvector
|
||||||
|
(Events, Identities, Embeddings)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tech Stack
|
||||||
|
|
||||||
|
| Component | Technology |
|
||||||
|
|---|---|
|
||||||
|
| Backend API | FastAPI + Python 3.11 |
|
||||||
|
| Vehicle Detection | YOLOv8 (Ultralytics) |
|
||||||
|
| License Plate OCR | fast-alpr (YOLOv9 + ONNX) |
|
||||||
|
| Face Recognition | InsightFace ArcFace (buffalo_l) |
|
||||||
|
| Database | PostgreSQL 16 + pgvector |
|
||||||
|
| Containerization | Docker Compose |
|
||||||
|
| Stream Ingestion | OpenCV RTSP |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Development Roadmap
|
||||||
|
|
||||||
|
- [ ] Phase 1: Core — RTSP + vehicle logging + face detection ← **YOU ARE HERE**
|
||||||
|
- [ ] Phase 2: Multi-stream + SMS/push alerts + gate relay
|
||||||
|
- [ ] Phase 3: Web dashboard + mobile app + LTO API integration
|
||||||
16
backend/Dockerfile
Normal file
16
backend/Dockerfile
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
FROM python:3.11-slim
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# System deps for OpenCV + InsightFace
|
||||||
|
RUN apt-get update && apt-get install -y \
|
||||||
|
libgl1 libglib2.0-0 libgomp1 \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY requirements.txt .
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
EXPOSE 8000
|
||||||
|
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||||
0
backend/app/__init__.py
Normal file
0
backend/app/__init__.py
Normal file
0
backend/app/api/__init__.py
Normal file
0
backend/app/api/__init__.py
Normal file
118
backend/app/api/cameras.py
Normal file
118
backend/app/api/cameras.py
Normal file
@@ -0,0 +1,118 @@
|
|||||||
|
"""
|
||||||
|
Camera API — Register, manage, and monitor RTSP cameras.
|
||||||
|
"""
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
from sqlalchemy import select
|
||||||
|
from pydantic import BaseModel
|
||||||
|
from typing import Optional
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from app.core.database import get_db
|
||||||
|
from app.models.camera import Camera, CameraStatus
|
||||||
|
from app.services.stream_manager import StreamManager
|
||||||
|
from app.workers.pipeline import DetectionPipeline
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/cameras", tags=["cameras"])
|
||||||
|
|
||||||
|
|
||||||
|
class CameraCreate(BaseModel):
|
||||||
|
name: str
|
||||||
|
rtsp_url: str
|
||||||
|
location: Optional[str] = None
|
||||||
|
description: Optional[str] = None
|
||||||
|
auto_start: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
class CameraResponse(BaseModel):
|
||||||
|
id: UUID
|
||||||
|
name: str
|
||||||
|
rtsp_url: str
|
||||||
|
location: Optional[str]
|
||||||
|
description: Optional[str]
|
||||||
|
status: str
|
||||||
|
is_enabled: bool
|
||||||
|
|
||||||
|
class Config:
|
||||||
|
from_attributes = True
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/", response_model=list[CameraResponse])
|
||||||
|
async def list_cameras(db: AsyncSession = Depends(get_db)):
|
||||||
|
result = await db.execute(select(Camera).order_by(Camera.name))
|
||||||
|
return result.scalars().all()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/", response_model=CameraResponse, status_code=status.HTTP_201_CREATED)
|
||||||
|
async def add_camera(payload: CameraCreate, db: AsyncSession = Depends(get_db)):
|
||||||
|
"""Register a new RTSP camera and optionally start streaming."""
|
||||||
|
# Check duplicate name
|
||||||
|
existing = await db.execute(select(Camera).where(Camera.name == payload.name))
|
||||||
|
if existing.scalar_one_or_none():
|
||||||
|
raise HTTPException(status_code=400, detail="Camera name already exists")
|
||||||
|
|
||||||
|
camera = Camera(
|
||||||
|
name=payload.name,
|
||||||
|
rtsp_url=payload.rtsp_url,
|
||||||
|
location=payload.location,
|
||||||
|
description=payload.description,
|
||||||
|
status=CameraStatus.inactive,
|
||||||
|
)
|
||||||
|
db.add(camera)
|
||||||
|
await db.commit()
|
||||||
|
await db.refresh(camera)
|
||||||
|
|
||||||
|
if payload.auto_start:
|
||||||
|
await _start_camera(camera)
|
||||||
|
|
||||||
|
return camera
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{camera_id}/start")
|
||||||
|
async def start_camera(camera_id: UUID, db: AsyncSession = Depends(get_db)):
|
||||||
|
camera = await _get_camera(db, camera_id)
|
||||||
|
await _start_camera(camera)
|
||||||
|
camera.status = CameraStatus.active
|
||||||
|
await db.commit()
|
||||||
|
return {"message": f"Camera '{camera.name}' started"}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{camera_id}/stop")
|
||||||
|
async def stop_camera(camera_id: UUID, db: AsyncSession = Depends(get_db)):
|
||||||
|
camera = await _get_camera(db, camera_id)
|
||||||
|
manager = StreamManager.get()
|
||||||
|
await manager.stop_stream(camera_id)
|
||||||
|
camera.status = CameraStatus.inactive
|
||||||
|
await db.commit()
|
||||||
|
return {"message": f"Camera '{camera.name}' stopped"}
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/{camera_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||||
|
async def delete_camera(camera_id: UUID, db: AsyncSession = Depends(get_db)):
|
||||||
|
camera = await _get_camera(db, camera_id)
|
||||||
|
await StreamManager.get().stop_stream(camera_id)
|
||||||
|
await db.delete(camera)
|
||||||
|
await db.commit()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/streams/status")
|
||||||
|
async def stream_status():
|
||||||
|
"""Get real-time status of all active streams."""
|
||||||
|
return StreamManager.get().get_all_status()
|
||||||
|
|
||||||
|
|
||||||
|
# --- Helpers ---
|
||||||
|
|
||||||
|
async def _get_camera(db: AsyncSession, camera_id: UUID) -> Camera:
|
||||||
|
result = await db.execute(select(Camera).where(Camera.id == camera_id))
|
||||||
|
camera = result.scalar_one_or_none()
|
||||||
|
if not camera:
|
||||||
|
raise HTTPException(status_code=404, detail="Camera not found")
|
||||||
|
return camera
|
||||||
|
|
||||||
|
|
||||||
|
async def _start_camera(camera: Camera):
|
||||||
|
manager = StreamManager.get()
|
||||||
|
worker = await manager.start_stream(camera.id, camera.name, camera.rtsp_url)
|
||||||
|
pipeline = DetectionPipeline(camera.id, camera.name, worker)
|
||||||
|
await pipeline.start()
|
||||||
130
backend/app/api/persons.py
Normal file
130
backend/app/api/persons.py
Normal file
@@ -0,0 +1,130 @@
|
|||||||
|
"""
|
||||||
|
Person API — Face identities and sighting trails.
|
||||||
|
See everywhere a face was detected, which cameras, and when.
|
||||||
|
"""
|
||||||
|
from fastapi import APIRouter, Depends, Query, UploadFile, File
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
from sqlalchemy import select, desc
|
||||||
|
from pydantic import BaseModel
|
||||||
|
from typing import Optional
|
||||||
|
from uuid import UUID
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from app.core.database import get_db
|
||||||
|
from app.models.person import PersonIdentity, FaceEvent
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/persons", tags=["persons"])
|
||||||
|
|
||||||
|
|
||||||
|
class PersonIdentityResponse(BaseModel):
|
||||||
|
id: UUID
|
||||||
|
name: Optional[str]
|
||||||
|
label: Optional[str]
|
||||||
|
is_registered: int
|
||||||
|
is_watchlisted: int
|
||||||
|
thumbnail_path: Optional[str]
|
||||||
|
first_seen_at: Optional[datetime]
|
||||||
|
last_seen_at: Optional[datetime]
|
||||||
|
total_sightings: int
|
||||||
|
|
||||||
|
class Config:
|
||||||
|
from_attributes = True
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/identities", response_model=list[PersonIdentityResponse])
|
||||||
|
async def list_identities(
|
||||||
|
name: Optional[str] = Query(None),
|
||||||
|
limit: int = Query(50, le=200),
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
):
|
||||||
|
q = select(PersonIdentity).order_by(desc(PersonIdentity.last_seen_at)).limit(limit)
|
||||||
|
if name:
|
||||||
|
q = q.where(PersonIdentity.name.ilike(f"%{name}%"))
|
||||||
|
result = await db.execute(q)
|
||||||
|
return result.scalars().all()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/identities/{identity_id}/trail")
|
||||||
|
async def person_trail(identity_id: UUID, db: AsyncSession = Depends(get_db)):
|
||||||
|
"""
|
||||||
|
Full sighting trail — every camera this face appeared on and when.
|
||||||
|
Groups by camera to show movement pattern.
|
||||||
|
"""
|
||||||
|
result = await db.execute(
|
||||||
|
select(FaceEvent)
|
||||||
|
.where(FaceEvent.identity_id == identity_id)
|
||||||
|
.order_by(desc(FaceEvent.captured_at))
|
||||||
|
.limit(500)
|
||||||
|
)
|
||||||
|
events = result.scalars().all()
|
||||||
|
return {
|
||||||
|
"identity_id": str(identity_id),
|
||||||
|
"total_sightings": len(events),
|
||||||
|
"trail": [
|
||||||
|
{
|
||||||
|
"camera_id": str(e.camera_id),
|
||||||
|
"captured_at": e.captured_at,
|
||||||
|
"similarity_score": e.similarity_score,
|
||||||
|
"face_snapshot_path": e.face_snapshot_path,
|
||||||
|
"snapshot_path": e.snapshot_path,
|
||||||
|
"vehicle_event_id": str(e.vehicle_event_id) if e.vehicle_event_id else None,
|
||||||
|
}
|
||||||
|
for e in events
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch("/identities/{identity_id}")
|
||||||
|
async def register_identity(
|
||||||
|
identity_id: UUID,
|
||||||
|
name: Optional[str] = None,
|
||||||
|
label: Optional[str] = None,
|
||||||
|
is_watchlisted: Optional[int] = None,
|
||||||
|
notes: Optional[str] = None,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
):
|
||||||
|
"""Give a name/label to an auto-detected unknown identity."""
|
||||||
|
result = await db.execute(select(PersonIdentity).where(PersonIdentity.id == identity_id))
|
||||||
|
identity = result.scalar_one_or_none()
|
||||||
|
if not identity:
|
||||||
|
from fastapi import HTTPException
|
||||||
|
raise HTTPException(status_code=404, detail="Person identity not found")
|
||||||
|
|
||||||
|
if name is not None:
|
||||||
|
identity.name = name
|
||||||
|
identity.is_registered = 1
|
||||||
|
if label is not None:
|
||||||
|
identity.label = label
|
||||||
|
if is_watchlisted is not None:
|
||||||
|
identity.is_watchlisted = is_watchlisted
|
||||||
|
|
||||||
|
await db.commit()
|
||||||
|
return {"message": "Identity registered"}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/events")
|
||||||
|
async def list_face_events(
|
||||||
|
camera_id: Optional[UUID] = Query(None),
|
||||||
|
identity_id: Optional[UUID] = Query(None),
|
||||||
|
limit: int = Query(100, le=500),
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
):
|
||||||
|
q = select(FaceEvent).order_by(desc(FaceEvent.captured_at)).limit(limit)
|
||||||
|
if camera_id:
|
||||||
|
q = q.where(FaceEvent.camera_id == camera_id)
|
||||||
|
if identity_id:
|
||||||
|
q = q.where(FaceEvent.identity_id == identity_id)
|
||||||
|
result = await db.execute(q)
|
||||||
|
events = result.scalars().all()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"id": str(e.id),
|
||||||
|
"camera_id": str(e.camera_id),
|
||||||
|
"identity_id": str(e.identity_id) if e.identity_id else None,
|
||||||
|
"detection_confidence": e.detection_confidence,
|
||||||
|
"similarity_score": e.similarity_score,
|
||||||
|
"face_snapshot_path": e.face_snapshot_path,
|
||||||
|
"captured_at": e.captured_at,
|
||||||
|
}
|
||||||
|
for e in events
|
||||||
|
]
|
||||||
139
backend/app/api/vehicles.py
Normal file
139
backend/app/api/vehicles.py
Normal file
@@ -0,0 +1,139 @@
|
|||||||
|
"""
|
||||||
|
Vehicle API — Query vehicle events and identities.
|
||||||
|
See where a plate has been seen across all cameras and when.
|
||||||
|
"""
|
||||||
|
from fastapi import APIRouter, Depends, Query
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
from sqlalchemy import select, desc
|
||||||
|
from pydantic import BaseModel
|
||||||
|
from typing import Optional
|
||||||
|
from uuid import UUID
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from app.core.database import get_db
|
||||||
|
from app.models.vehicle import VehicleEvent, VehicleIdentity
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/vehicles", tags=["vehicles"])
|
||||||
|
|
||||||
|
|
||||||
|
class VehicleIdentityResponse(BaseModel):
|
||||||
|
id: UUID
|
||||||
|
plate_number: Optional[str]
|
||||||
|
vehicle_type: str
|
||||||
|
color: Optional[str]
|
||||||
|
label: Optional[str]
|
||||||
|
is_whitelisted: int
|
||||||
|
is_blacklisted: int
|
||||||
|
first_seen_at: Optional[datetime]
|
||||||
|
last_seen_at: Optional[datetime]
|
||||||
|
total_sightings: int
|
||||||
|
|
||||||
|
class Config:
|
||||||
|
from_attributes = True
|
||||||
|
|
||||||
|
|
||||||
|
class VehicleEventResponse(BaseModel):
|
||||||
|
id: UUID
|
||||||
|
camera_id: UUID
|
||||||
|
plate_number: Optional[str]
|
||||||
|
vehicle_type: str
|
||||||
|
color: Optional[str]
|
||||||
|
person_count: Optional[int]
|
||||||
|
detection_confidence: Optional[float]
|
||||||
|
snapshot_path: Optional[str]
|
||||||
|
plate_snapshot_path: Optional[str]
|
||||||
|
captured_at: datetime
|
||||||
|
|
||||||
|
class Config:
|
||||||
|
from_attributes = True
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/identities", response_model=list[VehicleIdentityResponse])
|
||||||
|
async def list_vehicle_identities(
|
||||||
|
plate: Optional[str] = Query(None),
|
||||||
|
vehicle_type: Optional[str] = Query(None),
|
||||||
|
limit: int = Query(50, le=200),
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
):
|
||||||
|
q = select(VehicleIdentity).order_by(desc(VehicleIdentity.last_seen_at)).limit(limit)
|
||||||
|
if plate:
|
||||||
|
q = q.where(VehicleIdentity.plate_number.ilike(f"%{plate}%"))
|
||||||
|
if vehicle_type:
|
||||||
|
q = q.where(VehicleIdentity.vehicle_type == vehicle_type)
|
||||||
|
result = await db.execute(q)
|
||||||
|
return result.scalars().all()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/identities/{identity_id}/trail")
|
||||||
|
async def vehicle_trail(identity_id: UUID, db: AsyncSession = Depends(get_db)):
|
||||||
|
"""
|
||||||
|
Get full sighting trail for a vehicle — every camera it was seen on and when.
|
||||||
|
"""
|
||||||
|
result = await db.execute(
|
||||||
|
select(VehicleEvent)
|
||||||
|
.where(VehicleEvent.identity_id == identity_id)
|
||||||
|
.order_by(desc(VehicleEvent.captured_at))
|
||||||
|
.limit(500)
|
||||||
|
)
|
||||||
|
events = result.scalars().all()
|
||||||
|
return {
|
||||||
|
"identity_id": str(identity_id),
|
||||||
|
"total_sightings": len(events),
|
||||||
|
"trail": [
|
||||||
|
{
|
||||||
|
"camera_id": str(e.camera_id),
|
||||||
|
"captured_at": e.captured_at,
|
||||||
|
"plate_number": e.plate_number,
|
||||||
|
"color": e.color,
|
||||||
|
"vehicle_type": e.vehicle_type,
|
||||||
|
"person_count": e.person_count,
|
||||||
|
"snapshot_path": e.snapshot_path,
|
||||||
|
}
|
||||||
|
for e in events
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/events", response_model=list[VehicleEventResponse])
|
||||||
|
async def list_vehicle_events(
|
||||||
|
camera_id: Optional[UUID] = Query(None),
|
||||||
|
plate: Optional[str] = Query(None),
|
||||||
|
limit: int = Query(100, le=500),
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
):
|
||||||
|
q = select(VehicleEvent).order_by(desc(VehicleEvent.captured_at)).limit(limit)
|
||||||
|
if camera_id:
|
||||||
|
q = q.where(VehicleEvent.camera_id == camera_id)
|
||||||
|
if plate:
|
||||||
|
q = q.where(VehicleEvent.plate_number.ilike(f"%{plate}%"))
|
||||||
|
result = await db.execute(q)
|
||||||
|
return result.scalars().all()
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch("/identities/{identity_id}")
|
||||||
|
async def update_vehicle_identity(
|
||||||
|
identity_id: UUID,
|
||||||
|
label: Optional[str] = None,
|
||||||
|
is_whitelisted: Optional[int] = None,
|
||||||
|
is_blacklisted: Optional[int] = None,
|
||||||
|
notes: Optional[str] = None,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
):
|
||||||
|
"""Label a vehicle — mark as resident, delivery, blacklisted, etc."""
|
||||||
|
result = await db.execute(select(VehicleIdentity).where(VehicleIdentity.id == identity_id))
|
||||||
|
identity = result.scalar_one_or_none()
|
||||||
|
if not identity:
|
||||||
|
from fastapi import HTTPException
|
||||||
|
raise HTTPException(status_code=404, detail="Vehicle identity not found")
|
||||||
|
|
||||||
|
if label is not None:
|
||||||
|
identity.label = label
|
||||||
|
if is_whitelisted is not None:
|
||||||
|
identity.is_whitelisted = is_whitelisted
|
||||||
|
if is_blacklisted is not None:
|
||||||
|
identity.is_blacklisted = is_blacklisted
|
||||||
|
if notes is not None:
|
||||||
|
identity.notes = notes
|
||||||
|
|
||||||
|
await db.commit()
|
||||||
|
return {"message": "Updated"}
|
||||||
0
backend/app/core/__init__.py
Normal file
0
backend/app/core/__init__.py
Normal file
42
backend/app/core/config.py
Normal file
42
backend/app/core/config.py
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
from pydantic_settings import BaseSettings
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
APP_NAME: str = "BantayCam"
|
||||||
|
APP_VERSION: str = "0.1.0"
|
||||||
|
DEBUG: bool = False
|
||||||
|
|
||||||
|
# Database
|
||||||
|
DATABASE_URL: str = "postgresql+asyncpg://bantaycam:bantaycam@localhost:5432/bantaycam"
|
||||||
|
|
||||||
|
# Storage (MinIO / local)
|
||||||
|
STORAGE_TYPE: str = "local" # "local" or "minio"
|
||||||
|
STORAGE_PATH: str = "./snapshots"
|
||||||
|
MINIO_ENDPOINT: str = ""
|
||||||
|
MINIO_ACCESS_KEY: str = ""
|
||||||
|
MINIO_SECRET_KEY: str = ""
|
||||||
|
MINIO_BUCKET: str = "bantaycam"
|
||||||
|
|
||||||
|
# AI Models
|
||||||
|
YOLO_MODEL: str = "yolov8n.pt" # Vehicle + person detection
|
||||||
|
PLATE_MODEL: str = "yolov8n.pt" # Plate detection (custom PH)
|
||||||
|
FACE_MODEL: str = "buffalo_l" # InsightFace model
|
||||||
|
GPU_DEVICE: int = 0 # -1 for CPU, 0 for first GPU
|
||||||
|
|
||||||
|
# Processing
|
||||||
|
PROCESS_EVERY_N_FRAMES: int = 5 # Skip frames for performance
|
||||||
|
FACE_SIMILARITY_THRESHOLD: float = 0.5 # Cosine distance threshold
|
||||||
|
PLATE_CONFIDENCE_THRESHOLD: float = 0.6
|
||||||
|
DETECTION_CONFIDENCE_THRESHOLD: float = 0.5
|
||||||
|
|
||||||
|
# Alerts
|
||||||
|
TELEGRAM_BOT_TOKEN: Optional[str] = None
|
||||||
|
TELEGRAM_CHAT_ID: Optional[str] = None
|
||||||
|
|
||||||
|
class Config:
|
||||||
|
env_file = ".env"
|
||||||
|
case_sensitive = True
|
||||||
|
|
||||||
|
|
||||||
|
settings = Settings()
|
||||||
34
backend/app/core/database.py
Normal file
34
backend/app/core/database.py
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession, async_sessionmaker
|
||||||
|
from sqlalchemy.orm import DeclarativeBase
|
||||||
|
from app.core.config import settings
|
||||||
|
|
||||||
|
|
||||||
|
engine = create_async_engine(
|
||||||
|
settings.DATABASE_URL,
|
||||||
|
echo=settings.DEBUG,
|
||||||
|
pool_pre_ping=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
AsyncSessionLocal = async_sessionmaker(
|
||||||
|
engine,
|
||||||
|
class_=AsyncSession,
|
||||||
|
expire_on_commit=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Base(DeclarativeBase):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
async def get_db():
|
||||||
|
async with AsyncSessionLocal() as session:
|
||||||
|
try:
|
||||||
|
yield session
|
||||||
|
finally:
|
||||||
|
await session.close()
|
||||||
|
|
||||||
|
|
||||||
|
async def init_db():
|
||||||
|
"""Create all tables on startup."""
|
||||||
|
async with engine.begin() as conn:
|
||||||
|
await conn.run_sync(Base.metadata.create_all)
|
||||||
91
backend/app/main.py
Normal file
91
backend/app/main.py
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
"""
|
||||||
|
BantayCam — Integrated License Plate & Face Recognition for Philippine CCTV
|
||||||
|
FastAPI Application Entry Point
|
||||||
|
"""
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
from fastapi.staticfiles import StaticFiles
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from app.core.config import settings
|
||||||
|
from app.core.database import init_db
|
||||||
|
from app.services.vehicle_detector import vehicle_detector
|
||||||
|
from app.services.face_service import face_service
|
||||||
|
from app.services.stream_manager import StreamManager
|
||||||
|
from app.api import cameras, vehicles, persons
|
||||||
|
|
||||||
|
logging.basicConfig(
|
||||||
|
level=logging.INFO,
|
||||||
|
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s"
|
||||||
|
)
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(app: FastAPI):
|
||||||
|
"""Startup and shutdown events."""
|
||||||
|
logger.info("🚀 BantayCam starting up...")
|
||||||
|
|
||||||
|
# Init database
|
||||||
|
await init_db()
|
||||||
|
logger.info("✅ Database initialized")
|
||||||
|
|
||||||
|
# Load AI models
|
||||||
|
logger.info("Loading AI models (this may take a moment)...")
|
||||||
|
vehicle_detector.load()
|
||||||
|
face_service.load()
|
||||||
|
logger.info("✅ AI models loaded")
|
||||||
|
|
||||||
|
# Ensure snapshot storage exists
|
||||||
|
Path(settings.STORAGE_PATH).mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
yield
|
||||||
|
|
||||||
|
# Shutdown
|
||||||
|
logger.info("Shutting down streams...")
|
||||||
|
await StreamManager.get().stop_all()
|
||||||
|
logger.info("👋 BantayCam shut down")
|
||||||
|
|
||||||
|
|
||||||
|
app = FastAPI(
|
||||||
|
title="BantayCam",
|
||||||
|
description="Integrated License Plate & Face Recognition for Philippine CCTV Infrastructure",
|
||||||
|
version=settings.APP_VERSION,
|
||||||
|
lifespan=lifespan,
|
||||||
|
)
|
||||||
|
|
||||||
|
app.add_middleware(
|
||||||
|
CORSMiddleware,
|
||||||
|
allow_origins=["*"], # Tighten in production
|
||||||
|
allow_credentials=True,
|
||||||
|
allow_methods=["*"],
|
||||||
|
allow_headers=["*"],
|
||||||
|
)
|
||||||
|
|
||||||
|
# API routes
|
||||||
|
app.include_router(cameras.router, prefix="/api/v1")
|
||||||
|
app.include_router(vehicles.router, prefix="/api/v1")
|
||||||
|
app.include_router(persons.router, prefix="/api/v1")
|
||||||
|
|
||||||
|
# Serve snapshots as static files
|
||||||
|
snapshot_dir = Path(settings.STORAGE_PATH)
|
||||||
|
snapshot_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
app.mount("/snapshots", StaticFiles(directory=str(snapshot_dir)), name="snapshots")
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/")
|
||||||
|
async def root():
|
||||||
|
return {
|
||||||
|
"app": settings.APP_NAME,
|
||||||
|
"version": settings.APP_VERSION,
|
||||||
|
"status": "running",
|
||||||
|
"docs": "/docs",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/health")
|
||||||
|
async def health():
|
||||||
|
return {"status": "ok"}
|
||||||
0
backend/app/models/__init__.py
Normal file
0
backend/app/models/__init__.py
Normal file
30
backend/app/models/camera.py
Normal file
30
backend/app/models/camera.py
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
from sqlalchemy import Column, String, Boolean, DateTime, Text, Enum
|
||||||
|
from sqlalchemy.dialects.postgresql import UUID
|
||||||
|
from sqlalchemy.sql import func
|
||||||
|
import uuid
|
||||||
|
import enum
|
||||||
|
|
||||||
|
from app.core.database import Base
|
||||||
|
|
||||||
|
|
||||||
|
class CameraStatus(str, enum.Enum):
|
||||||
|
active = "active"
|
||||||
|
inactive = "inactive"
|
||||||
|
error = "error"
|
||||||
|
|
||||||
|
|
||||||
|
class Camera(Base):
|
||||||
|
__tablename__ = "cameras"
|
||||||
|
|
||||||
|
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||||
|
name = Column(String(255), nullable=False, unique=True) # e.g. "Gate 1 - Main Entrance"
|
||||||
|
rtsp_url = Column(Text, nullable=False) # rtsp://admin:pass@192.168.1.64:554/...
|
||||||
|
location = Column(String(255), nullable=True) # e.g. "North Gate, Building A"
|
||||||
|
description = Column(Text, nullable=True)
|
||||||
|
status = Column(Enum(CameraStatus), default=CameraStatus.inactive)
|
||||||
|
is_enabled = Column(Boolean, default=True)
|
||||||
|
created_at = Column(DateTime(timezone=True), server_default=func.now())
|
||||||
|
updated_at = Column(DateTime(timezone=True), onupdate=func.now())
|
||||||
|
|
||||||
|
def __repr__(self):
|
||||||
|
return f"<Camera {self.name} ({self.status})>"
|
||||||
65
backend/app/models/person.py
Normal file
65
backend/app/models/person.py
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
from sqlalchemy import Column, String, Integer, Float, DateTime, Text, ForeignKey, JSON
|
||||||
|
from sqlalchemy.dialects.postgresql import UUID, ARRAY
|
||||||
|
from sqlalchemy.sql import func
|
||||||
|
from pgvector.sqlalchemy import Vector
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
from app.core.database import Base
|
||||||
|
|
||||||
|
|
||||||
|
class PersonIdentity(Base):
|
||||||
|
"""
|
||||||
|
Unique person identity — grouped by face embedding similarity.
|
||||||
|
All face events linked to this identity let us trace where this person has been.
|
||||||
|
"""
|
||||||
|
__tablename__ = "person_identities"
|
||||||
|
|
||||||
|
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||||
|
name = Column(String(255), nullable=True) # e.g. "Juan Dela Cruz" if registered
|
||||||
|
label = Column(String(100), nullable=True) # e.g. "Unit 3A Resident", "Delivery Rider"
|
||||||
|
notes = Column(Text, nullable=True)
|
||||||
|
is_registered = Column(Integer, default=0) # 1 = manually registered with name
|
||||||
|
is_watchlisted = Column(Integer, default=0) # 1 = flagged for alerts
|
||||||
|
thumbnail_path = Column(Text, nullable=True) # Best face shot
|
||||||
|
|
||||||
|
# Face embedding (512-dim for InsightFace buffalo_l)
|
||||||
|
embedding = Column(Vector(512), nullable=True)
|
||||||
|
|
||||||
|
# Stats
|
||||||
|
first_seen_at = Column(DateTime(timezone=True), nullable=True)
|
||||||
|
last_seen_at = Column(DateTime(timezone=True), nullable=True)
|
||||||
|
total_sightings = Column(Integer, default=0)
|
||||||
|
created_at = Column(DateTime(timezone=True), server_default=func.now())
|
||||||
|
|
||||||
|
def __repr__(self):
|
||||||
|
return f"<PersonIdentity {self.name or 'Unknown'} ({self.total_sightings} sightings)>"
|
||||||
|
|
||||||
|
|
||||||
|
class FaceEvent(Base):
|
||||||
|
"""
|
||||||
|
Every face detection event — one row per detected face per frame.
|
||||||
|
"""
|
||||||
|
__tablename__ = "face_events"
|
||||||
|
|
||||||
|
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||||
|
camera_id = Column(UUID(as_uuid=True), ForeignKey("cameras.id"), nullable=False, index=True)
|
||||||
|
identity_id = Column(UUID(as_uuid=True), ForeignKey("person_identities.id"), nullable=True, index=True)
|
||||||
|
|
||||||
|
# Detection
|
||||||
|
detection_confidence = Column(Float, nullable=True)
|
||||||
|
face_bbox = Column(JSON, nullable=True) # {"x","y","w","h"}
|
||||||
|
face_embedding = Column(Vector(512), nullable=True) # Per-event embedding for re-clustering
|
||||||
|
similarity_score = Column(Float, nullable=True) # Match score to identity
|
||||||
|
|
||||||
|
# Context — if detected alongside a vehicle event
|
||||||
|
vehicle_event_id = Column(UUID(as_uuid=True), ForeignKey("vehicle_events.id"), nullable=True)
|
||||||
|
|
||||||
|
# Storage
|
||||||
|
snapshot_path = Column(Text, nullable=True) # Full frame
|
||||||
|
face_snapshot_path = Column(Text, nullable=True) # Cropped face chip
|
||||||
|
|
||||||
|
# Meta
|
||||||
|
captured_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
|
||||||
|
|
||||||
|
def __repr__(self):
|
||||||
|
return f"<FaceEvent identity={self.identity_id} cam={self.camera_id} at={self.captured_at}>"
|
||||||
73
backend/app/models/vehicle.py
Normal file
73
backend/app/models/vehicle.py
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
from sqlalchemy import Column, String, Integer, Float, DateTime, Text, ForeignKey, Enum, JSON
|
||||||
|
from sqlalchemy.dialects.postgresql import UUID
|
||||||
|
from sqlalchemy.sql import func
|
||||||
|
import uuid
|
||||||
|
import enum
|
||||||
|
|
||||||
|
from app.core.database import Base
|
||||||
|
|
||||||
|
|
||||||
|
class VehicleType(str, enum.Enum):
|
||||||
|
car = "car"
|
||||||
|
motorcycle = "motorcycle"
|
||||||
|
truck = "truck"
|
||||||
|
van = "van"
|
||||||
|
jeepney = "jeepney"
|
||||||
|
tricycle = "tricycle"
|
||||||
|
unknown = "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
class VehicleIdentity(Base):
|
||||||
|
"""
|
||||||
|
Unique vehicle identity — grouped by license plate.
|
||||||
|
Tracks every time this vehicle was seen across any camera.
|
||||||
|
"""
|
||||||
|
__tablename__ = "vehicle_identities"
|
||||||
|
|
||||||
|
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||||
|
plate_number = Column(String(20), nullable=True, index=True, unique=True)
|
||||||
|
vehicle_type = Column(Enum(VehicleType), default=VehicleType.unknown)
|
||||||
|
color = Column(String(50), nullable=True)
|
||||||
|
make_model = Column(String(100), nullable=True) # e.g. "Toyota Vios"
|
||||||
|
notes = Column(Text, nullable=True)
|
||||||
|
label = Column(String(100), nullable=True) # e.g. "Unit 4B Owner", "Delivery Van"
|
||||||
|
is_whitelisted = Column(Integer, default=0) # 1 = resident/approved
|
||||||
|
is_blacklisted = Column(Integer, default=0) # 1 = flagged/banned
|
||||||
|
thumbnail_path = Column(Text, nullable=True) # Best shot saved
|
||||||
|
first_seen_at = Column(DateTime(timezone=True), nullable=True)
|
||||||
|
last_seen_at = Column(DateTime(timezone=True), nullable=True)
|
||||||
|
total_sightings = Column(Integer, default=0)
|
||||||
|
created_at = Column(DateTime(timezone=True), server_default=func.now())
|
||||||
|
|
||||||
|
|
||||||
|
class VehicleEvent(Base):
|
||||||
|
"""
|
||||||
|
Every detection event — one row per camera capture.
|
||||||
|
"""
|
||||||
|
__tablename__ = "vehicle_events"
|
||||||
|
|
||||||
|
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||||
|
camera_id = Column(UUID(as_uuid=True), ForeignKey("cameras.id"), nullable=False, index=True)
|
||||||
|
identity_id = Column(UUID(as_uuid=True), ForeignKey("vehicle_identities.id"), nullable=True, index=True)
|
||||||
|
|
||||||
|
# Detection data
|
||||||
|
plate_number = Column(String(20), nullable=True)
|
||||||
|
plate_confidence = Column(Float, nullable=True)
|
||||||
|
vehicle_type = Column(Enum(VehicleType), default=VehicleType.unknown)
|
||||||
|
color = Column(String(50), nullable=True)
|
||||||
|
person_count = Column(Integer, nullable=True) # For motorcycle: how many riders
|
||||||
|
detection_confidence = Column(Float, nullable=True)
|
||||||
|
|
||||||
|
# Bounding boxes (stored as JSON: {"x":0,"y":0,"w":100,"h":100})
|
||||||
|
vehicle_bbox = Column(JSON, nullable=True)
|
||||||
|
plate_bbox = Column(JSON, nullable=True)
|
||||||
|
|
||||||
|
# Storage
|
||||||
|
snapshot_path = Column(Text, nullable=True) # Full frame snapshot
|
||||||
|
plate_snapshot_path = Column(Text, nullable=True) # Cropped plate image
|
||||||
|
|
||||||
|
# Meta
|
||||||
|
captured_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
|
||||||
|
|
||||||
|
def __repr__(self):
|
||||||
|
return f"<VehicleEvent plate={self.plate_number} type={self.vehicle_type} at={self.captured_at}>"
|
||||||
0
backend/app/services/__init__.py
Normal file
0
backend/app/services/__init__.py
Normal file
88
backend/app/services/face_service.py
Normal file
88
backend/app/services/face_service.py
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
"""
|
||||||
|
FaceService — InsightFace-based face detection, embedding, and identity matching.
|
||||||
|
Uses pgvector cosine similarity for fast 1:N identity search.
|
||||||
|
"""
|
||||||
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
import logging
|
||||||
|
from typing import Optional
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from app.core.config import settings
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class FaceDetection:
|
||||||
|
bbox: dict = field(default_factory=dict) # {"x","y","w","h"}
|
||||||
|
embedding: Optional[np.ndarray] = None # 512-dim vector
|
||||||
|
detection_confidence: float = 0.0
|
||||||
|
face_crop: Optional[np.ndarray] = None
|
||||||
|
matched_identity_id: Optional[UUID] = None
|
||||||
|
similarity_score: float = 0.0
|
||||||
|
|
||||||
|
|
||||||
|
class FaceService:
|
||||||
|
def __init__(self):
|
||||||
|
self._app = None
|
||||||
|
self._ready = False
|
||||||
|
|
||||||
|
def load(self):
|
||||||
|
"""Load InsightFace model — call once at startup."""
|
||||||
|
try:
|
||||||
|
from insightface.app import FaceAnalysis
|
||||||
|
self._app = FaceAnalysis(
|
||||||
|
name=settings.FACE_MODEL,
|
||||||
|
allowed_modules=["detection", "recognition"],
|
||||||
|
)
|
||||||
|
self._app.prepare(
|
||||||
|
ctx_id=settings.GPU_DEVICE,
|
||||||
|
det_size=(640, 640),
|
||||||
|
)
|
||||||
|
self._ready = True
|
||||||
|
logger.info(f"InsightFace loaded (model={settings.FACE_MODEL})")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Failed to load InsightFace: {e}")
|
||||||
|
|
||||||
|
def detect(self, frame: np.ndarray) -> list[FaceDetection]:
|
||||||
|
"""
|
||||||
|
Detect all faces in a frame and extract embeddings.
|
||||||
|
Returns list of FaceDetection objects.
|
||||||
|
"""
|
||||||
|
if not self._ready or self._app is None:
|
||||||
|
return []
|
||||||
|
|
||||||
|
try:
|
||||||
|
faces = self._app.get(frame)
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(f"Face detection error: {e}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
detections = []
|
||||||
|
for face in faces:
|
||||||
|
bbox = face.bbox.astype(int)
|
||||||
|
x1, y1, x2, y2 = bbox[0], bbox[1], bbox[2], bbox[3]
|
||||||
|
|
||||||
|
det = FaceDetection(
|
||||||
|
bbox={"x": x1, "y": y1, "w": x2 - x1, "h": y2 - y1},
|
||||||
|
embedding=face.normed_embedding, # Already L2-normalized
|
||||||
|
detection_confidence=float(face.det_score),
|
||||||
|
face_crop=frame[max(0, y1):y2, max(0, x1):x2],
|
||||||
|
)
|
||||||
|
detections.append(det)
|
||||||
|
|
||||||
|
return detections
|
||||||
|
|
||||||
|
def cosine_similarity(self, emb1: np.ndarray, emb2: np.ndarray) -> float:
|
||||||
|
"""Cosine similarity between two normalized embeddings."""
|
||||||
|
return float(np.dot(emb1, emb2))
|
||||||
|
|
||||||
|
def embeddings_to_list(self, embedding: np.ndarray) -> list[float]:
|
||||||
|
"""Convert numpy embedding to list for pgvector storage."""
|
||||||
|
return embedding.tolist()
|
||||||
|
|
||||||
|
|
||||||
|
# Singleton
|
||||||
|
face_service = FaceService()
|
||||||
153
backend/app/services/stream_manager.py
Normal file
153
backend/app/services/stream_manager.py
Normal file
@@ -0,0 +1,153 @@
|
|||||||
|
"""
|
||||||
|
StreamManager — Manages RTSP stream connections and dispatches frames to workers.
|
||||||
|
Each camera runs in its own asyncio task with a frame queue.
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import cv2
|
||||||
|
import logging
|
||||||
|
from typing import Dict, Optional
|
||||||
|
from uuid import UUID
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from app.core.config import settings
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class StreamWorker:
|
||||||
|
"""
|
||||||
|
Handles one RTSP camera stream.
|
||||||
|
Reads frames and puts them into a queue for the AI pipeline to consume.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, camera_id: UUID, camera_name: str, rtsp_url: str):
|
||||||
|
self.camera_id = camera_id
|
||||||
|
self.camera_name = camera_name
|
||||||
|
self.rtsp_url = rtsp_url
|
||||||
|
self.frame_queue: asyncio.Queue = asyncio.Queue(maxsize=10)
|
||||||
|
self.is_running = False
|
||||||
|
self._task: Optional[asyncio.Task] = None
|
||||||
|
self.frame_count = 0
|
||||||
|
self.last_frame_at: Optional[datetime] = None
|
||||||
|
self.error: Optional[str] = None
|
||||||
|
|
||||||
|
async def start(self):
|
||||||
|
self.is_running = True
|
||||||
|
self._task = asyncio.create_task(self._read_loop())
|
||||||
|
logger.info(f"[{self.camera_name}] Stream started")
|
||||||
|
|
||||||
|
async def stop(self):
|
||||||
|
self.is_running = False
|
||||||
|
if self._task:
|
||||||
|
self._task.cancel()
|
||||||
|
try:
|
||||||
|
await self._task
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
pass
|
||||||
|
logger.info(f"[{self.camera_name}] Stream stopped")
|
||||||
|
|
||||||
|
async def _read_loop(self):
|
||||||
|
"""Read frames from RTSP in a thread pool (OpenCV is blocking)."""
|
||||||
|
loop = asyncio.get_event_loop()
|
||||||
|
|
||||||
|
while self.is_running:
|
||||||
|
try:
|
||||||
|
cap = await loop.run_in_executor(
|
||||||
|
None,
|
||||||
|
lambda: cv2.VideoCapture(self.rtsp_url)
|
||||||
|
)
|
||||||
|
|
||||||
|
if not cap.isOpened():
|
||||||
|
self.error = "Cannot open RTSP stream"
|
||||||
|
logger.error(f"[{self.camera_name}] {self.error}")
|
||||||
|
await asyncio.sleep(5)
|
||||||
|
continue
|
||||||
|
|
||||||
|
self.error = None
|
||||||
|
frame_idx = 0
|
||||||
|
|
||||||
|
while self.is_running:
|
||||||
|
ret, frame = await loop.run_in_executor(None, cap.read)
|
||||||
|
|
||||||
|
if not ret:
|
||||||
|
logger.warning(f"[{self.camera_name}] Frame read failed, reconnecting...")
|
||||||
|
break
|
||||||
|
|
||||||
|
frame_idx += 1
|
||||||
|
self.frame_count += 1
|
||||||
|
self.last_frame_at = datetime.utcnow()
|
||||||
|
|
||||||
|
# Skip frames for performance
|
||||||
|
if frame_idx % settings.PROCESS_EVERY_N_FRAMES != 0:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Non-blocking put — drop frame if queue is full
|
||||||
|
try:
|
||||||
|
self.frame_queue.put_nowait({
|
||||||
|
"frame": frame,
|
||||||
|
"frame_idx": frame_idx,
|
||||||
|
"captured_at": self.last_frame_at,
|
||||||
|
})
|
||||||
|
except asyncio.QueueFull:
|
||||||
|
pass # AI pipeline is slow — drop frame
|
||||||
|
|
||||||
|
cap.release()
|
||||||
|
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
self.error = str(e)
|
||||||
|
logger.exception(f"[{self.camera_name}] Stream error: {e}")
|
||||||
|
await asyncio.sleep(5)
|
||||||
|
|
||||||
|
|
||||||
|
class StreamManager:
|
||||||
|
"""
|
||||||
|
Singleton that manages all active camera streams.
|
||||||
|
"""
|
||||||
|
_instance = None
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._workers: Dict[str, StreamWorker] = {}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def get(cls) -> "StreamManager":
|
||||||
|
if cls._instance is None:
|
||||||
|
cls._instance = StreamManager()
|
||||||
|
return cls._instance
|
||||||
|
|
||||||
|
async def start_stream(self, camera_id: UUID, camera_name: str, rtsp_url: str):
|
||||||
|
key = str(camera_id)
|
||||||
|
if key in self._workers:
|
||||||
|
await self.stop_stream(camera_id)
|
||||||
|
|
||||||
|
worker = StreamWorker(camera_id, camera_name, rtsp_url)
|
||||||
|
self._workers[key] = worker
|
||||||
|
await worker.start()
|
||||||
|
return worker
|
||||||
|
|
||||||
|
async def stop_stream(self, camera_id: UUID):
|
||||||
|
key = str(camera_id)
|
||||||
|
if key in self._workers:
|
||||||
|
await self._workers[key].stop()
|
||||||
|
del self._workers[key]
|
||||||
|
|
||||||
|
def get_worker(self, camera_id: UUID) -> Optional[StreamWorker]:
|
||||||
|
return self._workers.get(str(camera_id))
|
||||||
|
|
||||||
|
def get_all_status(self) -> list:
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"camera_id": str(k),
|
||||||
|
"is_running": w.is_running,
|
||||||
|
"frame_count": w.frame_count,
|
||||||
|
"last_frame_at": w.last_frame_at,
|
||||||
|
"error": w.error,
|
||||||
|
}
|
||||||
|
for k, w in self._workers.items()
|
||||||
|
]
|
||||||
|
|
||||||
|
async def stop_all(self):
|
||||||
|
for worker in list(self._workers.values()):
|
||||||
|
await worker.stop()
|
||||||
|
self._workers.clear()
|
||||||
227
backend/app/services/vehicle_detector.py
Normal file
227
backend/app/services/vehicle_detector.py
Normal file
@@ -0,0 +1,227 @@
|
|||||||
|
"""
|
||||||
|
VehicleDetector — YOLOv8-based vehicle detection, classification, and person counting.
|
||||||
|
Also handles license plate OCR via fast-alpr.
|
||||||
|
"""
|
||||||
|
import cv2
|
||||||
|
import numpy as np
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
|
from app.core.config import settings
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
VEHICLE_CLASSES = {
|
||||||
|
2: "car",
|
||||||
|
3: "motorcycle",
|
||||||
|
5: "bus",
|
||||||
|
7: "truck",
|
||||||
|
}
|
||||||
|
|
||||||
|
# COCO person class
|
||||||
|
PERSON_CLASS = 0
|
||||||
|
|
||||||
|
# Color detection ranges (HSV)
|
||||||
|
COLOR_RANGES = {
|
||||||
|
"red": [(0, 70, 50), (10, 255, 255)],
|
||||||
|
"red2": [(170, 70, 50), (180, 255, 255)],
|
||||||
|
"blue": [(100, 70, 50), (130, 255, 255)],
|
||||||
|
"white": [(0, 0, 180), (180, 30, 255)],
|
||||||
|
"black": [(0, 0, 0), (180, 255, 50)],
|
||||||
|
"silver": [(0, 0, 100), (180, 30, 180)],
|
||||||
|
"yellow": [(20, 70, 50), (35, 255, 255)],
|
||||||
|
"green": [(35, 70, 50), (85, 255, 255)],
|
||||||
|
"orange": [(10, 70, 50), (20, 255, 255)],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class VehicleDetection:
|
||||||
|
vehicle_type: str = "unknown"
|
||||||
|
color: Optional[str] = None
|
||||||
|
plate_number: Optional[str] = None
|
||||||
|
plate_confidence: float = 0.0
|
||||||
|
person_count: int = 1
|
||||||
|
detection_confidence: float = 0.0
|
||||||
|
vehicle_bbox: dict = field(default_factory=dict)
|
||||||
|
plate_bbox: dict = field(default_factory=dict)
|
||||||
|
vehicle_crop: Optional[np.ndarray] = None
|
||||||
|
plate_crop: Optional[np.ndarray] = None
|
||||||
|
|
||||||
|
|
||||||
|
class VehicleDetector:
|
||||||
|
def __init__(self):
|
||||||
|
self._yolo = None
|
||||||
|
self._alpr = None
|
||||||
|
self._ready = False
|
||||||
|
|
||||||
|
def load(self):
|
||||||
|
"""Load models — call once at startup."""
|
||||||
|
try:
|
||||||
|
from ultralytics import YOLO
|
||||||
|
self._yolo = YOLO(settings.YOLO_MODEL)
|
||||||
|
logger.info("YOLO model loaded")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Failed to load YOLO: {e}")
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
from fast_alpr import ALPR
|
||||||
|
self._alpr = ALPR(
|
||||||
|
detector_model="plate-detection-v1-large",
|
||||||
|
ocr_model="global-plates-mobile-vit-v2-model",
|
||||||
|
device="cuda" if settings.GPU_DEVICE >= 0 else "cpu",
|
||||||
|
)
|
||||||
|
logger.info("ALPR model loaded")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"ALPR not loaded (will skip plate reading): {e}")
|
||||||
|
|
||||||
|
self._ready = True
|
||||||
|
|
||||||
|
def detect(self, frame: np.ndarray) -> list[VehicleDetection]:
|
||||||
|
"""
|
||||||
|
Run vehicle detection on a frame.
|
||||||
|
Returns list of VehicleDetection objects.
|
||||||
|
"""
|
||||||
|
if not self._ready or self._yolo is None:
|
||||||
|
return []
|
||||||
|
|
||||||
|
results = self._yolo(
|
||||||
|
frame,
|
||||||
|
conf=settings.DETECTION_CONFIDENCE_THRESHOLD,
|
||||||
|
verbose=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
detections = []
|
||||||
|
h, w = frame.shape[:2]
|
||||||
|
|
||||||
|
for result in results:
|
||||||
|
boxes = result.boxes
|
||||||
|
if boxes is None:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Group: find all vehicles and all persons
|
||||||
|
vehicles = []
|
||||||
|
persons = []
|
||||||
|
|
||||||
|
for box in boxes:
|
||||||
|
cls_id = int(box.cls[0])
|
||||||
|
conf = float(box.conf[0])
|
||||||
|
x1, y1, x2, y2 = map(int, box.xyxy[0])
|
||||||
|
|
||||||
|
if cls_id in VEHICLE_CLASSES:
|
||||||
|
vehicles.append({
|
||||||
|
"type": VEHICLE_CLASSES[cls_id],
|
||||||
|
"conf": conf,
|
||||||
|
"bbox": {"x": x1, "y": y1, "w": x2 - x1, "h": y2 - y1},
|
||||||
|
"crop": frame[y1:y2, x1:x2],
|
||||||
|
})
|
||||||
|
elif cls_id == PERSON_CLASS:
|
||||||
|
persons.append({"bbox": {"x": x1, "y": y1, "w": x2 - x1, "h": y2 - y1}})
|
||||||
|
|
||||||
|
for vehicle in vehicles:
|
||||||
|
det = VehicleDetection(
|
||||||
|
vehicle_type=vehicle["type"],
|
||||||
|
detection_confidence=vehicle["conf"],
|
||||||
|
vehicle_bbox=vehicle["bbox"],
|
||||||
|
vehicle_crop=vehicle["crop"],
|
||||||
|
)
|
||||||
|
|
||||||
|
# Detect color
|
||||||
|
det.color = self._detect_color(vehicle["crop"])
|
||||||
|
|
||||||
|
# Count persons on/near motorcycle
|
||||||
|
if vehicle["type"] == "motorcycle":
|
||||||
|
det.person_count = self._count_persons_on_motorcycle(
|
||||||
|
vehicle["bbox"], persons
|
||||||
|
)
|
||||||
|
|
||||||
|
# Read plate
|
||||||
|
if self._alpr is not None:
|
||||||
|
plate_result = self._read_plate(frame, vehicle["bbox"])
|
||||||
|
if plate_result:
|
||||||
|
det.plate_number = plate_result["text"]
|
||||||
|
det.plate_confidence = plate_result["confidence"]
|
||||||
|
det.plate_bbox = plate_result["bbox"]
|
||||||
|
bx = plate_result["bbox"]
|
||||||
|
det.plate_crop = frame[
|
||||||
|
bx["y"]:bx["y"] + bx["h"],
|
||||||
|
bx["x"]:bx["x"] + bx["w"]
|
||||||
|
]
|
||||||
|
|
||||||
|
detections.append(det)
|
||||||
|
|
||||||
|
return detections
|
||||||
|
|
||||||
|
def _read_plate(self, frame: np.ndarray, vehicle_bbox: dict) -> Optional[dict]:
|
||||||
|
"""Run ALPR on vehicle region."""
|
||||||
|
if self._alpr is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
# Expand vehicle crop slightly for plate detection
|
||||||
|
x, y, w, h = vehicle_bbox["x"], vehicle_bbox["y"], vehicle_bbox["w"], vehicle_bbox["h"]
|
||||||
|
crop = frame[max(0, y):y + h, max(0, x):x + w]
|
||||||
|
results = self._alpr.run(crop)
|
||||||
|
if results:
|
||||||
|
best = results[0]
|
||||||
|
ocr = best.ocr
|
||||||
|
if ocr and ocr.text:
|
||||||
|
# Offset bbox back to full frame coordinates
|
||||||
|
pb = best.detection.bounding_box
|
||||||
|
return {
|
||||||
|
"text": ocr.text.upper().replace(" ", ""),
|
||||||
|
"confidence": float(ocr.confidence),
|
||||||
|
"bbox": {
|
||||||
|
"x": x + int(pb.x1),
|
||||||
|
"y": y + int(pb.y1),
|
||||||
|
"w": int(pb.x2 - pb.x1),
|
||||||
|
"h": int(pb.y2 - pb.y1),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(f"ALPR error: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _detect_color(self, crop: np.ndarray) -> str:
|
||||||
|
"""Detect dominant vehicle color using HSV histogram."""
|
||||||
|
if crop is None or crop.size == 0:
|
||||||
|
return "unknown"
|
||||||
|
try:
|
||||||
|
hsv = cv2.cvtColor(crop, cv2.COLOR_BGR2HSV)
|
||||||
|
max_pixels = 0
|
||||||
|
detected_color = "unknown"
|
||||||
|
|
||||||
|
for color_name, (lower, upper) in COLOR_RANGES.items():
|
||||||
|
mask = cv2.inRange(hsv, np.array(lower), np.array(upper))
|
||||||
|
pixel_count = cv2.countNonZero(mask)
|
||||||
|
if pixel_count > max_pixels:
|
||||||
|
max_pixels = pixel_count
|
||||||
|
detected_color = color_name.replace("2", "") # red2 → red
|
||||||
|
|
||||||
|
return detected_color
|
||||||
|
except Exception:
|
||||||
|
return "unknown"
|
||||||
|
|
||||||
|
def _count_persons_on_motorcycle(self, moto_bbox: dict, persons: list) -> int:
|
||||||
|
"""Count persons whose center falls within or near the motorcycle bbox."""
|
||||||
|
count = 0
|
||||||
|
mx, my, mw, mh = moto_bbox["x"], moto_bbox["y"], moto_bbox["w"], moto_bbox["h"]
|
||||||
|
|
||||||
|
for person in persons:
|
||||||
|
px, py, pw, ph = person["bbox"]["x"], person["bbox"]["y"], person["bbox"]["w"], person["bbox"]["h"]
|
||||||
|
# Person center
|
||||||
|
cx = px + pw // 2
|
||||||
|
cy = py + ph // 2
|
||||||
|
# Check if center is within motorcycle bounding box (with some margin)
|
||||||
|
margin = 30
|
||||||
|
if (mx - margin <= cx <= mx + mw + margin and
|
||||||
|
my - margin <= cy <= my + mh + margin):
|
||||||
|
count += 1
|
||||||
|
|
||||||
|
return max(count, 1) # At least 1 rider assumed
|
||||||
|
|
||||||
|
|
||||||
|
# Singleton
|
||||||
|
vehicle_detector = VehicleDetector()
|
||||||
0
backend/app/workers/__init__.py
Normal file
0
backend/app/workers/__init__.py
Normal file
250
backend/app/workers/pipeline.py
Normal file
250
backend/app/workers/pipeline.py
Normal file
@@ -0,0 +1,250 @@
|
|||||||
|
"""
|
||||||
|
Detection Pipeline Worker
|
||||||
|
Pulls frames from a StreamWorker queue and runs:
|
||||||
|
1. Vehicle detection (type, color, plate, person count)
|
||||||
|
2. Face detection (embedding, identity match)
|
||||||
|
3. Saves snapshots and events to DB
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import cv2
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
from sqlalchemy import select, text
|
||||||
|
|
||||||
|
from app.core.config import settings
|
||||||
|
from app.core.database import AsyncSessionLocal
|
||||||
|
from app.models.camera import Camera
|
||||||
|
from app.models.vehicle import VehicleEvent, VehicleIdentity, VehicleType
|
||||||
|
from app.models.person import FaceEvent, PersonIdentity
|
||||||
|
from app.services.vehicle_detector import vehicle_detector
|
||||||
|
from app.services.face_service import face_service
|
||||||
|
from app.services.stream_manager import StreamWorker
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class DetectionPipeline:
|
||||||
|
"""
|
||||||
|
Runs detection on frames for a single camera stream.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, camera_id: UUID, camera_name: str, worker: StreamWorker):
|
||||||
|
self.camera_id = camera_id
|
||||||
|
self.camera_name = camera_name
|
||||||
|
self.worker = worker
|
||||||
|
self.snapshot_dir = Path(settings.STORAGE_PATH) / str(camera_id)
|
||||||
|
self.snapshot_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._running = False
|
||||||
|
self._task = None
|
||||||
|
|
||||||
|
async def start(self):
|
||||||
|
self._running = True
|
||||||
|
self._task = asyncio.create_task(self._loop())
|
||||||
|
logger.info(f"[{self.camera_name}] Pipeline started")
|
||||||
|
|
||||||
|
async def stop(self):
|
||||||
|
self._running = False
|
||||||
|
if self._task:
|
||||||
|
self._task.cancel()
|
||||||
|
try:
|
||||||
|
await self._task
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def _loop(self):
|
||||||
|
loop = asyncio.get_event_loop()
|
||||||
|
|
||||||
|
while self._running:
|
||||||
|
try:
|
||||||
|
frame_data = await asyncio.wait_for(
|
||||||
|
self.worker.frame_queue.get(), timeout=2.0
|
||||||
|
)
|
||||||
|
except asyncio.TimeoutError:
|
||||||
|
continue
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
raise
|
||||||
|
|
||||||
|
frame = frame_data["frame"]
|
||||||
|
captured_at = frame_data["captured_at"]
|
||||||
|
|
||||||
|
try:
|
||||||
|
await loop.run_in_executor(
|
||||||
|
None, self._process_frame, frame, captured_at
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.exception(f"[{self.camera_name}] Pipeline error: {e}")
|
||||||
|
|
||||||
|
def _process_frame(self, frame, captured_at: datetime):
|
||||||
|
"""Synchronous processing — runs in thread pool."""
|
||||||
|
import asyncio
|
||||||
|
loop = asyncio.new_event_loop()
|
||||||
|
asyncio.set_event_loop(loop)
|
||||||
|
try:
|
||||||
|
loop.run_until_complete(self._async_process(frame, captured_at))
|
||||||
|
finally:
|
||||||
|
loop.close()
|
||||||
|
|
||||||
|
async def _async_process(self, frame, captured_at: datetime):
|
||||||
|
"""Run detections and save to DB."""
|
||||||
|
# --- Vehicle Detection ---
|
||||||
|
vehicle_detections = vehicle_detector.detect(frame)
|
||||||
|
|
||||||
|
# --- Face Detection ---
|
||||||
|
face_detections = face_service.detect(frame)
|
||||||
|
|
||||||
|
if not vehicle_detections and not face_detections:
|
||||||
|
return
|
||||||
|
|
||||||
|
# Save snapshot
|
||||||
|
ts = captured_at.strftime("%Y%m%d_%H%M%S_%f")
|
||||||
|
snapshot_filename = f"{ts}.jpg"
|
||||||
|
snapshot_path = str(self.snapshot_dir / snapshot_filename)
|
||||||
|
cv2.imwrite(snapshot_path, frame)
|
||||||
|
|
||||||
|
async with AsyncSessionLocal() as db:
|
||||||
|
vehicle_event_ids = []
|
||||||
|
|
||||||
|
# Save vehicle events
|
||||||
|
for det in vehicle_detections:
|
||||||
|
identity_id = await self._get_or_create_vehicle_identity(
|
||||||
|
db, det.plate_number, det.vehicle_type, det.color
|
||||||
|
)
|
||||||
|
|
||||||
|
# Save plate crop
|
||||||
|
plate_snap = None
|
||||||
|
if det.plate_crop is not None and det.plate_crop.size > 0:
|
||||||
|
plate_snap = str(self.snapshot_dir / f"{ts}_plate.jpg")
|
||||||
|
cv2.imwrite(plate_snap, det.plate_crop)
|
||||||
|
|
||||||
|
event = VehicleEvent(
|
||||||
|
camera_id=self.camera_id,
|
||||||
|
identity_id=identity_id,
|
||||||
|
plate_number=det.plate_number,
|
||||||
|
plate_confidence=det.plate_confidence,
|
||||||
|
vehicle_type=det.vehicle_type,
|
||||||
|
color=det.color,
|
||||||
|
person_count=det.person_count,
|
||||||
|
detection_confidence=det.detection_confidence,
|
||||||
|
vehicle_bbox=det.vehicle_bbox,
|
||||||
|
plate_bbox=det.plate_bbox,
|
||||||
|
snapshot_path=snapshot_path,
|
||||||
|
plate_snapshot_path=plate_snap,
|
||||||
|
captured_at=captured_at,
|
||||||
|
)
|
||||||
|
db.add(event)
|
||||||
|
await db.flush()
|
||||||
|
vehicle_event_ids.append(event.id)
|
||||||
|
|
||||||
|
# Save face events
|
||||||
|
for i, det in enumerate(face_detections):
|
||||||
|
identity_id, similarity = await self._match_or_create_person(
|
||||||
|
db, det.embedding
|
||||||
|
)
|
||||||
|
|
||||||
|
# Save face crop
|
||||||
|
face_snap = None
|
||||||
|
if det.face_crop is not None and det.face_crop.size > 0:
|
||||||
|
face_snap = str(self.snapshot_dir / f"{ts}_face{i}.jpg")
|
||||||
|
cv2.imwrite(face_snap, det.face_crop)
|
||||||
|
|
||||||
|
embedding_list = face_service.embeddings_to_list(det.embedding) if det.embedding is not None else None
|
||||||
|
|
||||||
|
event = FaceEvent(
|
||||||
|
camera_id=self.camera_id,
|
||||||
|
identity_id=identity_id,
|
||||||
|
detection_confidence=det.detection_confidence,
|
||||||
|
face_bbox=det.bbox,
|
||||||
|
face_embedding=embedding_list,
|
||||||
|
similarity_score=similarity,
|
||||||
|
vehicle_event_id=vehicle_event_ids[0] if vehicle_event_ids else None,
|
||||||
|
snapshot_path=snapshot_path,
|
||||||
|
face_snapshot_path=face_snap,
|
||||||
|
captured_at=captured_at,
|
||||||
|
)
|
||||||
|
db.add(event)
|
||||||
|
|
||||||
|
await db.commit()
|
||||||
|
|
||||||
|
async def _get_or_create_vehicle_identity(
|
||||||
|
self, db: AsyncSession, plate_number, vehicle_type, color
|
||||||
|
):
|
||||||
|
"""Find existing vehicle identity by plate or create new one."""
|
||||||
|
if not plate_number:
|
||||||
|
return None
|
||||||
|
|
||||||
|
result = await db.execute(
|
||||||
|
select(VehicleIdentity).where(VehicleIdentity.plate_number == plate_number)
|
||||||
|
)
|
||||||
|
identity = result.scalar_one_or_none()
|
||||||
|
|
||||||
|
if identity:
|
||||||
|
identity.last_seen_at = datetime.now(timezone.utc)
|
||||||
|
identity.total_sightings += 1
|
||||||
|
if color and not identity.color:
|
||||||
|
identity.color = color
|
||||||
|
else:
|
||||||
|
identity = VehicleIdentity(
|
||||||
|
plate_number=plate_number,
|
||||||
|
vehicle_type=vehicle_type or VehicleType.unknown,
|
||||||
|
color=color,
|
||||||
|
first_seen_at=datetime.now(timezone.utc),
|
||||||
|
last_seen_at=datetime.now(timezone.utc),
|
||||||
|
total_sightings=1,
|
||||||
|
)
|
||||||
|
db.add(identity)
|
||||||
|
await db.flush()
|
||||||
|
|
||||||
|
return identity.id
|
||||||
|
|
||||||
|
async def _match_or_create_person(self, db: AsyncSession, embedding):
|
||||||
|
"""
|
||||||
|
Find closest matching person identity using pgvector cosine similarity.
|
||||||
|
If no match above threshold, create new identity.
|
||||||
|
Returns (identity_id, similarity_score).
|
||||||
|
"""
|
||||||
|
if embedding is None:
|
||||||
|
return None, 0.0
|
||||||
|
|
||||||
|
emb_list = face_service.embeddings_to_list(embedding)
|
||||||
|
emb_str = f"[{','.join(str(x) for x in emb_list)}]"
|
||||||
|
|
||||||
|
# pgvector cosine distance (1 - cosine_similarity)
|
||||||
|
threshold = settings.FACE_SIMILARITY_THRESHOLD # distance threshold
|
||||||
|
|
||||||
|
result = await db.execute(
|
||||||
|
text(f"""
|
||||||
|
SELECT id, 1 - (embedding <=> '{emb_str}'::vector) AS similarity
|
||||||
|
FROM person_identities
|
||||||
|
WHERE embedding IS NOT NULL
|
||||||
|
AND (embedding <=> '{emb_str}'::vector) < :threshold
|
||||||
|
ORDER BY embedding <=> '{emb_str}'::vector
|
||||||
|
LIMIT 1
|
||||||
|
"""),
|
||||||
|
{"threshold": threshold}
|
||||||
|
)
|
||||||
|
row = result.fetchone()
|
||||||
|
|
||||||
|
if row:
|
||||||
|
identity_id, similarity = row
|
||||||
|
# Update last seen
|
||||||
|
await db.execute(
|
||||||
|
text("UPDATE person_identities SET last_seen_at=NOW(), total_sightings=total_sightings+1 WHERE id=:id"),
|
||||||
|
{"id": identity_id}
|
||||||
|
)
|
||||||
|
return identity_id, float(similarity)
|
||||||
|
else:
|
||||||
|
# New unknown person
|
||||||
|
identity = PersonIdentity(
|
||||||
|
first_seen_at=datetime.now(timezone.utc),
|
||||||
|
last_seen_at=datetime.now(timezone.utc),
|
||||||
|
total_sightings=1,
|
||||||
|
embedding=emb_list,
|
||||||
|
)
|
||||||
|
db.add(identity)
|
||||||
|
await db.flush()
|
||||||
|
return identity.id, 0.0
|
||||||
27
backend/requirements.txt
Normal file
27
backend/requirements.txt
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
# Web framework
|
||||||
|
fastapi==0.115.0
|
||||||
|
uvicorn[standard]==0.30.6
|
||||||
|
pydantic==2.9.2
|
||||||
|
pydantic-settings==2.5.2
|
||||||
|
|
||||||
|
# Database
|
||||||
|
sqlalchemy[asyncio]==2.0.35
|
||||||
|
asyncpg==0.29.0
|
||||||
|
pgvector==0.3.2
|
||||||
|
alembic==1.13.3
|
||||||
|
|
||||||
|
# AI / Computer Vision
|
||||||
|
ultralytics==8.3.0 # YOLOv8/v11 vehicle + person detection
|
||||||
|
fast-alpr==0.2.0 # License plate recognition
|
||||||
|
insightface==0.7.3 # Face detection + recognition (ArcFace)
|
||||||
|
opencv-python-headless==4.10.0.84
|
||||||
|
numpy==1.26.4
|
||||||
|
onnxruntime-gpu==1.19.2 # GPU inference (use onnxruntime for CPU-only)
|
||||||
|
|
||||||
|
# Optional: GPU acceleration
|
||||||
|
# torch>=2.0 (install separately with CUDA support)
|
||||||
|
|
||||||
|
# Utilities
|
||||||
|
python-multipart==0.0.9 # File uploads
|
||||||
|
aiofiles==24.1.0
|
||||||
|
httpx==0.27.2
|
||||||
37
docker-compose.yml
Normal file
37
docker-compose.yml
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
version: "3.9"
|
||||||
|
|
||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
image: pgvector/pgvector:pg16
|
||||||
|
container_name: bantaycam-db
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: bantaycam
|
||||||
|
POSTGRES_PASSWORD: bantaycam
|
||||||
|
POSTGRES_DB: bantaycam
|
||||||
|
ports:
|
||||||
|
- "5432:5432"
|
||||||
|
volumes:
|
||||||
|
- pgdata:/var/lib/postgresql/data
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
backend:
|
||||||
|
build:
|
||||||
|
context: ./backend
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
container_name: bantaycam-api
|
||||||
|
depends_on:
|
||||||
|
- postgres
|
||||||
|
environment:
|
||||||
|
DATABASE_URL: postgresql+asyncpg://bantaycam:bantaycam@postgres:5432/bantaycam
|
||||||
|
STORAGE_PATH: /app/snapshots
|
||||||
|
GPU_DEVICE: -1 # Change to 0 for GPU
|
||||||
|
ports:
|
||||||
|
- "8000:8000"
|
||||||
|
volumes:
|
||||||
|
- ./snapshots:/app/snapshots
|
||||||
|
- ./backend:/app
|
||||||
|
restart: unless-stopped
|
||||||
|
command: uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
pgdata:
|
||||||
Reference in New Issue
Block a user