Skip to content

Per-Org Resource Usage (admin dashboard)

How the super-admin dashboard shows each organisation's share of the shared VDS — CPU, RAM, disk, live calls — with 1/3/7-day averages, peak times, and a recorded host history graph. Shipped via astradial-platform PRs #508–#510 + #512 (2026-07-07), promoted to prod 2026-07-08 (PR #513) and prod-verified: endpoint live on 147.93.168.216 (20 orgs, sampler Up in services-health, samples recording). Per-org rows populate on the first live call.

v1.1 fixes (PR #512, from staging review): - Live-call counts dedup by Linkedid — one call = 2+ channels (caller leg + answering leg), so channel counting showed "2" for one call. The tile, sampled calls column, and share proportions now mean calls. - History graph — the card renders host.series (CPU %, RAM %, calls) as a recharts LineChart with time labels and a tooltip like "14:00 · CPU 4% · RAM 5.5 GB (47%) · 1 call". - RAM = MemAvailable (PR #510) — os.freemem() is MemFree on Linux and counts reclaimable page cache as used (tile drifted to 85% after a big file walk while free -h said 47%). Sampler reads MemAvailable from /proc/meminfo; used = total − MemAvailable matches free -h.

What it shows (and what the numbers mean)

  • Host CPU / RAM / Disk / Live calls — real, whole-box readings. The Disk tile is the entire root filesystem (statfs on the recordings path, which lives on /), not recordings — recordings have their own per-org column. See the log-bloat entry in the troubleshooting doc for the day-one lesson (63% disk on staging was 68 GB of dead Asterisk debug logs).
  • Per-org CPU / RAM shareestimates, labeled as such in the UI: host usage split proportionally by each org's live channels (host CPU × org channels ÷ total channels). Asterisk is ONE process on a shared box; true per-org CPU/RAM does not exist without per-org containers/cgroups (deliberately out of scope — phase 2 "cell architecture"). RAM share uses the same channel-proportional basis, so CPU-share and RAM-share differ only via the CPU weighting per sample.
  • Avg / max calls, peak hour — real, from the per-minute samples. Peak hour = the hour-of-day with the highest summed org channel-minutes in the window.
  • Recordings — real bytes currently on local disk per org (files are swept to Firebase by rclone, so this is local footprint, not lifetime total).

Architecture

os.cpus()/freemem/statfs ─┐
liveCallsService.channels ─┤  (in-memory AMI mirror — see live-calls.md)
        jobs/resourceUsageSampler.js  (60s tick, serviceHealth-registered)
                           │ bulkCreate
        resource_usage_samples  (MySQL; 1 host row/min + sparse org rows;
                                 32-day retention, hourly prune)
                           │ grouped range scans
        GET /api/v1/admin/resource-usage?days=1|3|7   (X-Internal-Key)
        editor /api/admin/resource-usage proxy  (gateway_admin_key →
                                                 INTERNAL_API_KEY laundering)
        dashboard: ResourceShareCard + inline org-row bars

Key files (astradial-platform):

Piece Path
Sampler job api/src/jobs/resourceUsageSampler.js
Channel counting api/src/services/liveCallsService.jsorgChannelCounts()
Model / migration api/src/models/ResourceUsageSample.js, api/database/migrations/20260707100000-*
API route api/src/routes/admin-resource-usage.js (mounted router — no route-snapshot churn)
Editor proxy editor/app/api/admin/resource-usage/route.ts
UI card + hook editor/components/admin/ResourceShareCard.tsx (useResourceUsage)
Dashboard wiring editor/app/dashboard/page.tsx (1:1 resizable split; inline bars in org rows)

Sampling semantics (the non-obvious bits)

  • Sparse org rows: an org gets a row only for minutes it has ≥1 live channel. Averages therefore divide by the host-row count for the window, not the org-row count — otherwise an org with one busy minute per day would show a huge "average".
  • First tick writes nothing — it only seeds the os.cpus() baseline for the CPU delta. Host rows appear ~2 minutes after a pm2 restart.
  • Org attribution order: rec.AccountCode === org.id first (dialplan stamps CDR(accountcode); attributes shared-gateway legs), else boundary-aware context_prefix regex on Channel/Context (same semantics as Live Calls; bare includes() would leak org_mp3 into org_mp3t4g5m). Each channel counts for at most one org.
  • Recording disk per org runs hourly: readdir+stat of the flat monitor dir, then chunked asterisk_cdr lookups (recordingfile IN (...), 500/chunk, 50ms yield between chunks) because the org is NOT in the recording filename (YYYYMMDD-HHMMSS-<cid>-<ext>.wav). Unmatched bytes land in _unattributed.
  • Services & Schedulers shows the sampler as Resource Usage Sampler (heartbeat pattern, stale after 300s).

Passive-by-design (why this can't affect calls)

Same contract as Live Calls: reads the in-memory channel map, OS counters, and the filesystem. No AMI/ARI actions, no Asterisk config or dialplan writes, no changes to existing tables (new table only, CASCADE on org delete), no existing route modified (routes.snapshot.txt byte-identical). Sampler start and every tick are try/catch-isolated — worst-case failure is a gap in the usage graph. Existing live-calls-service tests pass unchanged; the new method is purely additive.

Operational notes

  • New deploys need the migration: npx sequelize-cli db:migrate (creates resource_usage_samples).
  • Table growth ≈ (1 + orgs-with-calls) rows/minute; 32-day prune runs hourly inside the sampler tick. At current scale this is a few MB/month.
  • The 1d/3d/7d toggle changes bucket size (10 min / 30 min / 60 min).
  • Dashboard layout: left half = org list (with inline CPU/RAM share bars + live count per org, hidden below lg breakpoint), right half = onboarding chart · compact Services & Schedulers (full table behind "View all" dialog) · Resource Usage by Org card. The divider is drag-resizable; 50/50 is the default (PR #509).

Phase 2 (deliberately not built)

Multi-cell fleet (the "Fleet Management System" design handoff: cells table, add/move/disable-cell wizards with drain orchestration), true per-org CPU isolation, recording-disk history, usage alerting. The design prototype lives in ~/Downloads/Fleet Management System-handoff.zip.