Industry-standard test suite covering the features currently shipped in prod: org management, users/extensions, DIDs, outbound routes, SIP trunks, queues, IVR, workflows, CRM, call recording, auth & admin impersonation. Use this document as the UAT/regression reference before every prod release.
Conventions
| Field | Value |
| Severity | P0 (blocker) / P1 (critical) / P2 (major) / P3 (minor) |
| Type | F (functional) / S (smoke) / R (regression) / N (negative) / E (e2e-call) |
| Env | Prod / Staging / Both |
| Result | PASS / FAIL / BLOCKED / N/A |
Pass criteria — test passes only if: 1. Expected result matches actual result. 2. No new errors in /var/log/asterisk/full.log or pm2 logs <svc>. 3. No regression in unrelated features (spot-check smoke suite).
Test data — always use the designated test org (AstraPrivate on staging, a dedicated QA DID on prod). Never run destructive tests against live customer orgs.
Environment & access
| Env | VPS | Editor URL | PBX API URL | DB |
| Prod | 89.116.31.109 | https://editor.astradial.com | https://devpbx.astradial.com | mariadb -upbx_api -p<pw> pbx_api_db |
| Staging | 94.136.188.221 | https://stageeditor.astradial.com | https://stagepbx.astradial.com | mariadb -upbx_user -p<pw> pbx_api_db |
All stage* hosts are behind Cloudflare Access. Get team email whitelisted before attempting staging tests.
Prod orgs: Om Chamber (org_mo8vbv60), GrandEstancia (org_mnd5khym), Zauto AI (org_mo47r5n8). Staging org: AstraPrivate (org_mna9x47k).
Smoke suite (run before every release)
| # | Severity | Type | Test | Expected |
| S1 | P0 | S | curl -sfI https://editor.astradial.com/dashboard | HTTP 200 |
| S2 | P0 | S | pm2 status on prod | editor + astrapbx + workflow-engine all online |
| S3 | P0 | S | asterisk -rx "core show channels count" on prod | No stuck channels; total calls processed increasing |
| S4 | P0 | S | Admin Firebase login → see org list | Login succeeds, list renders |
| S5 | P0 | S | Admin impersonates Om Chamber → overview loads | Users count, call stats, recent calls render |
| S6 | P1 | S | Call existing Om Chamber DID from external phone | Rings + lands on configured destination |
1. Authentication & authorization
| # | Severity | Type | Scenario | Steps | Expected |
| A1 | P0 | F | Admin Firebase login | /dashboard → admin tab → valid creds | Redirect to org list, gateway_admin_key set, admin_session_start set in localStorage |
| A2 | P0 | F | Admin login — wrong password | 3x invalid | Generic "invalid credentials" error, no lockout escalation |
| A3 | P1 | F | Org user sign-up → email verify → login | Full flow | Verified user reaches dashboard, pbx_org_token_exp set |
| A4 | P1 | F | Org-user login without email verified | Login before clicking link | "Verify your email first", new verification sent |
| A5 | P1 | F | Admin impersonates org → 24h JWT expiry fires | localStorage.setItem('pbx_org_token_exp', String(Date.now()-1)) + reload | Redirect to /dashboard (org picker). gateway_admin_key preserved. Firebase admin session intact. |
| A6 | P1 | F | 24h admin session expiry | localStorage.setItem('admin_session_start', String(Date.now()-24*3600_000-1)) + reload | Redirect to /dashboard login form. Firebase signed out. All auth keys cleared. |
| A7 | P1 | R | Normal org user — 401 from PBX | Manually expire pbx_org_token, trigger any API call | Full logout: Firebase signOut + redirect to login form |
| A8 | P2 | F | Manual logout from Sidebar (impersonating) | Click Logout | Returns to /dashboard org picker. Admin Firebase + gateway key intact. |
| A9 | P2 | F | Manual logout from Sidebar (normal user) | Click Logout | Firebase signed out, login form rendered. |
| A10 | P1 | N | Cross-org access attempt | JWT for org A → hit /api/v1/... with org_id=B | 403 Forbidden, audit-logged |
2. Organization management
| # | Severity | Type | Scenario | Steps | Expected |
| O1 | P1 | F | New org request from signed-up user | Login → no org → submit org request | Row in DB with status=pending, admin sees in Pending Approvals |
| O2 | P1 | F | Admin approves pending org | Admin dashboard → Approve | status=active, config deployed (asterisk files generated per org), org owner can login |
| O3 | P1 | F | Admin rejects pending org | Admin dashboard → reject flow | Org not activated, user sees "awaiting approval" |
| O4 | P0 | R | Existing prod orgs still load | Admin impersonates Om Chamber / GrandEstancia / Zauto AI | Overview, users, calls, DIDs all render |
3. Users / Extensions
| # | Severity | Type | Scenario | Steps | Expected |
| U1 | P1 | F | Create SIP extension | Users page → New → ext 1099, name, role | Row created, pjsip endpoint auto-registered, dialplan rebuilt for org |
| U2 | P1 | F | Extension uniqueness per org | Try to create a duplicate extension | 409 Conflict, clear error message |
| U3 | P2 | F | Edit user ring target (SIP → phone) | User detail → ring_target=phone, phone number | Dialplan regenerated, Dial goes to trunk with correct CID |
| U4 | P1 | F | Delete user | Users page → Delete | Endpoint removed from pjsip, dialplan rebuilt, CDR of past calls preserved |
| U5 | P2 | F | Register Zoiper with new extension | Use generated SIP creds | Zoiper shows Registered, pjsip show aor contact listed |
| U6 | P0 | E | SIP-to-SIP internal call | Ext 1001 → ext 1002 | Rings, two-way audio, CDR logged |
| U7 | P1 | N | SIP registration with wrong password | Zoiper with bad pw | 401 Unauthorized, no contact registered, no account lockout |
4. DID (phone number) management
| # | Severity | Type | Scenario | Steps | Expected |
| D1 | P1 | F | Admin releases a DID to pool | /admin/dids → add available DID | Row with status=available, visible to orgs |
| D2 | P1 | F | Org requests a DID | DIDs page → Buy → Request | status=pending, admin sees it |
| D3 | P1 | F | Admin approves DID request | Approve | Org sees it under My Numbers, can configure routing |
| D4 | P1 | F | Route DID to extension | Edit → routing_type=extension, pick ext | Saved, dialplan includes Goto(..._internal,<ext>,1) for DID |
| D5 | P1 | F | Route DID to queue | Edit → routing_type=queue, pick queue | Dialplan Goto(..._queue,<num>,1) |
| D6 | P1 | F | Route DID to IVR | Edit → routing_type=ivr → dropdown lists IVRs by <ext> — <name> | routing_destination stored as IVR UUID (NOT extension), dialplan Goto(..._ivr,<ivr_ext>,1) |
| D7 | P1 | R | Route DID to external number | Edit → routing_type=external, type +919944421125 | Cursor stays in input across all keystrokes. Stored verbatim. Dialplan dials via trunk. |
| D8 | P2 | F | DIDs table shows IVR by name | Return to DIDs list after routing=ivr | Table column shows <ext> — <name>, NOT raw UUID |
| D9 | P0 | E | External call → DID → extension | Call DID from PSTN phone | Destination extension rings |
| D10 | P0 | E | External call → DID → IVR | Call DID from PSTN phone | IVR greeting plays, DTMF routes correctly |
| D11 | P1 | N | Delete a DID with active routing | Admin deletes DID | Confirmation required; if deleted, calls to it go to "number not in service" |
5. Outbound routes & SIP trunks
| # | Severity | Type | Scenario | Steps | Expected |
| T1 | P0 | S | Prod SIP trunks load | pjsip show endpoints | All per-org _trunk* endpoints listed, Not in use status |
| T2 | P0 | E | Prod outbound PSTN from SIP phone | Ext dials 9XXXXXXXXXX | Rings out, two-way audio, CDR records |
| T3 | P1 | E | E.164 outbound (with +) | Ext dials +919944421125 | _+X. strips +, re-enters dialplan, rings out |
| T4 | P1 | F | Outbound route: strip_digits + prepend_digits | Route with strip=1, prepend=91, dial 09944421125 | Dials 919944421125 |
| T5 | P2 | F | SIP trunk metadata not splatted | sip_trunks.configuration has {system_trunk:true} | pjsip endpoint loads (no "Could not find option" error) |
| T6 | P1 | R | Concurrent call limit | Set org limit=1, make 2nd call | 2nd call hits limit_reached branch, plays all-circuits-busy-now |
| T7 | P2 | F | Per-route caller ID override | Set caller_id_override=+91XXX on route | Outbound CID matches |
6. Queues
| # | Severity | Type | Scenario | Steps | Expected |
| Q1 | P1 | F | Create queue | Queues page → new, number=5099 | Row created, dialplan includes queue context |
| Q2 | P1 | F | Add member extension to queue | Queue detail → Add member 1001 | Agent reachable when queue rings |
| Q3 | P0 | R | Internal dial of queue number hits queue (not PSTN) | Ext 1001 dials 5001 in Om Chamber | dialplan show 5001@..._internal matches org_mo8vbv60__queue, not _X.. Queue MOH plays, agent rings. |
| Q4 | P0 | E | External call → DID → queue | Call DID with queue routing | MOH plays, agent rings |
| Q5 | P2 | F | Queue timeout fallback | Set timeout=5s, no agent answers | Falls to configured timeout destination (extension/external/hangup) |
| Q6 | P2 | F | Queue greeting plays | Upload greeting, call queue | Greeting plays before MOH |
| Q7 | P1 | R | Timeout destination picker — kind selector + per-kind dropdown | Edit queue → "On timeout, route caller to" section | See 4 buttons [ No routing \| User \| Queue \| Phone ]. Clicking each reveals the right widget below: User/Queue → SearchableSelect, Phone → free input, No routing → helper text. Searching by name or extension filters the list. Queue list excludes the queue being edited. Picker writes both timeout_destination_type and timeout_destination atomically. |
| Q8 | P1 | N | Timeout destination validator rejects bad combos | Edit queue → Phone tab → destination 5004 → Save | 400 with "Phone number 5004 does not look like a valid number (need 7-15 digits)" toast. Save rejected. (This is the regression for the May 20 Om Chambers incident — see Error 59.) |
| Q9 | P1 | R | Timeout destination persists across save+reopen | Pick Queue: supervisors (5004) → Save → close dialog → re-open | Picker opens with the Queue button highlighted and supervisors still selected. Pre-PR #251 the type was silently dropped by the API and the dialog reverted to stale state. |
| Q10 | P0 | R | Concurrent reload smoke test (deadlock regression) | On prod or staging VPS: fire two POST /api/v1/admin/regenerate-gateway ~300 ms apart with X-Internal-Key header | Both return 200 with {success:true, didCount:N, orgCount:M}. pm2 logs astrapbx shows 2 × 🔄 Reloading… followed by 2 × ✅ Asterisk configuration reloaded (dialplan + res_pjsip + app_queue + devstate seed) lines, serial not interleaved. grep "previous reload command" /var/log/asterisk/full.log returns nothing. See Error 60. |
| Q11 | P2 | F | Queue "Timeout" column NOT in list (removed PR #251) | Open Departments / Queues page | Columns are: Ext, Name, Strategy, MOH, Max Wait, Timeout Routes To, Status. No standalone "Timeout (s)" column. (See queue-architecture.md → Operator-facing knobs for why.) |
7. IVR (visual builder)
IVR is the newly-shipped feature from PR #63. Test suite is deeper here.
| # | Severity | Type | Scenario | Steps | Expected |
| I1 | P1 | F | Create IVR via UI | IVR section → New, ext=7099, name | Row in ivrs table, greeting_language=en-IN, greeting_voice=en-IN-Wavenet-D by default |
| I2 | P1 | F | Extension uniqueness per org | Create two IVRs with same ext | 2nd one 409 Conflict |
| I3 | P1 | F | Edit IVR settings: timeout / max_retries | Settings panel → Timeout=15, Max retries dropdown → 2 attempts | Saved. Helper text "Greeting repeats on no-response..." visible. |
| I4 | P1 | F | Max retries is a dropdown with exactly 1/2/3 | Settings panel inspect | Options: "1 attempt", "2 attempts", "3 attempts" only |
| I5 | P1 | F | Generate greeting via TTS | Settings → enter text → Generate | .wav created under /var/lib/asterisk/sounds/greetings/greeting_ivr_<uuid>.wav. greeting_prompt set in DB. |
| I6 | P2 | F | Preview voice | Settings → pick voice → Preview | Audio plays in browser without writing to disk |
| I7 | P1 | F | Add menu option (visual builder) | Node editor → add "Press 1 → extension 1001" | Row in ivr_menus, digit=1, action_type=extension, action_destination=1001 |
| I8 | P1 | F | Menu option: action_type=ai_agent | Add an AI agent option | Saves (enum now supports ai_agent post-migration) |
| I9 | P1 | F | Publish IVR | Click Publish | configDeploymentService.deployOrganizationConfiguration runs, dialplan regenerated to include IVR context |
| I10 | P0 | R | Generated dialplan uses greetings/ prefix | grep Background /etc/asterisk/ext_<org>.conf | Background(greetings/greeting_ivr_<uuid>) — NOT bare filename |
| I11 | P0 | R | Internal includes put _ivr first | grep include => /etc/asterisk/ext_<org>.conf | Order: _ivr → _queue → _outbound |
| I12 | P0 | E | Call DID routed to IVR from PSTN | Call IVR-routed DID from external phone | Greeting plays, DTMF routes to menu option |
| I13 | P0 | E | SIP phone dials IVR extension | Zoiper ext 1001 → 7002 | dialplan show 7002@..._internal hits IVR context (exact match wins over _X.). Greeting plays. |
| I14 | P1 | E | IVR no-response retry | Call IVR, stay silent | Greeting repeats up to max_retries, then plays goodbye + hangup |
| I15 | P1 | E | IVR invalid DTMF retry | Press 9 when only 1/2 configured | Plays invalid prompt, retries up to max_retries |
| I16 | P1 | F | Direct extension dial toggle | Enable "Direct extension dial" | 4-digit dial from IVR goes to _internal ext |
| I17 | P1 | F | Delete IVR | Delete button | Row + menu options cascade-deleted, greeting .wav removed |
| I18 | P1 | F | Route DID to deleted IVR (negative) | Delete IVR referenced by a DID | UI warns / prevents, OR DID falls back to number-not-in-service on call |
8. Call recording
| # | Severity | Type | Scenario | Steps | Expected |
| R1 | P0 | E | Org-wide recording on → call records | Make/receive a call | .wav in /var/spool/asterisk/monitor/, CDR recordingfile set, GCS upload eventually |
| R2 | P1 | F | Per-user opt-out | User.call_recording=false | That user's calls NOT recorded, others still are |
| R3 | P1 | F | Org-wide recording off | Org.settings.recording_enabled=false | No calls recorded regardless of per-user |
| R4 | P2 | F | Recording playback in UI | Calls page → play | Streams audio, no 404 |
9. Workflows & CRM
| # | Severity | Type | Scenario | Steps | Expected |
| W1 | P2 | F | Create workflow | Workflows → new | Saves, appears in list |
| W2 | P2 | F | Trigger workflow on call event | Run workflow | Executes without unhandled errors |
| C1 | P2 | F | Create CRM lead | CRM → Leads → New | Row in Firestore, appears in list |
| C2 | P2 | F | Move lead → deal | Kanban drag | Status updates |
| C3 | P2 | F | Custom pipeline field | Customize → add field | Field renders in deal detail |
10. Admin dashboard
| # | Severity | Type | Scenario | Steps | Expected |
| AD1 | P1 | F | Org list pagination | 30+ orgs | Paginates cleanly, no console errors |
| AD2 | P1 | F | DID pool management | /admin/dids | Add/edit/remove DIDs; orgs see availability change |
| AD3 | P1 | F | Regenerate gateway config | /api/admin/regenerate-gateway | Runs without error, NUC dialplan updated |
| AD4 | P2 | F | Admin switch organization | Sidebar → Switch Org | Lands back on admin dashboard with org list |
11. Browser / UI regression
| # | Severity | Type | Scenario | Expected |
| B1 | P2 | R | Page load on Chrome, Firefox, Safari | No console errors, correct render |
| B2 | P1 | R | Mobile viewport on dashboard | Sidebar collapses to hamburger, content readable |
| B3 | P2 | R | Dark/light theme toggle | Persists across reload |
| B4 | P1 | R | DID dialog: type in free-text input | Cursor stays in input for all characters (regression test for remount bug) |
| B5 | P2 | R | Network latency: slow 3G on Chrome DevTools | Pages render loading states, don't crash |
12. API contract
| # | Severity | Type | Scenario | Expected |
| API1 | P0 | F | POST /api/v1/auth/user-login with valid Firebase token | Returns JWT + user payload, JWT has exp claim |
| API2 | P1 | F | POST /api/v1/config/deploy (admin or owner role) | Regenerates all per-org files, reloads asterisk |
| API3 | P1 | F | POST /api/v1/ivrs/:id/publish | Triggers dialplan regen for the org |
| API4 | P1 | F | POST /api/v1/admin/impersonate/:orgId (admin) | Returns 24h user JWT |
| API5 | P1 | N | All /api/v1/... without token | 401 Unauthorized |
| API6 | P1 | N | Org A JWT hitting Org B resource | 403 Forbidden |
13. Negative & security
| # | Severity | Type | Scenario | Expected |
| N1 | P1 | N | SIP INVITE from unknown source IP | 401/404, no dialplan execution |
| N2 | P1 | N | SIP scanner scanning extensions | fail2ban blocks after N attempts |
| N3 | P1 | N | SQL injection attempt in org name | Stored as-is (no SQL execution), no 500 |
| N4 | P1 | N | XSS in user display name | Escaped in all UI surfaces |
| N5 | P1 | N | CSRF on state-changing endpoint | Requires Bearer token or session cookie; plain POST rejected |
| N6 | P0 | N | Cross-org data leak | Impersonation or fuzzed org_id never returns another org's data (see incident-apr18 for context) |
14. Infrastructure / runbook
| # | Severity | Type | Scenario | Expected |
| INF1 | P0 | S | WireGuard tunnel up prod ↔ NUC | wg show wg0 shows recent handshake, ping 10.10.10.2 |
| INF2 | P0 | S | WG tunnel prod ↔ staging | ping 10.10.10.3 from prod |
| INF3 | P1 | S | Netdata alerts clear on both VPS | No critical alerts |
| INF4 | P1 | S | Cloudflare Tunnel nuc.astradial.com reachable | SSH works via Cloudflare |
Run procedure
- Execute smoke suite first (6 tests, ~5 min). Any FAIL = abort.
- Run regression subset (Q3, D6, D7, I10, I11, I13, B4) covering known-previously-broken paths. ~15 min.
- Run full suite by area as needed for the release. New features warrant extended suite in that area.
- Every FAIL gets a ticket linked to the incident + commit SHA that caused it.
- PASS requires three green: UI, API/dialplan, live-call. Never declare release complete on UI-only PASS.
Known gaps (can't automate today)
- Live PSTN calls require a human with a real phone. Listed as
Type=E above. - Audio quality assessment (jitter, MOS) requires an external tool — not in this suite.
- Failover testing (kill NUC, kill prod VPS) needs chaos infra, flagged as future work.
Incident-linked regression tests
See the operations runbooks for each past incident and pair with the TC above: - Apr 14 monorepo incident — pair with O4, S2. - Apr 18 cross-org leak — pair with N6, A10. - Session freeze / auto-logout (today's session) — pair with A5, A6, A7, A8. - IVR dropdown + include reorder (today's session) — pair with D6, D7, I10, I11, I13, Q3.