Skip to content

Cloud Operations Runbook

SSH Access

ssh root@89.116.31.109

Check NUC / Tata Gateway Status

Endpoint Status

asterisk -rx "pjsip show endpoint tata_gateway"

AOR and Contact

asterisk -rx "pjsip show aor tata_gateway"

Endpoint Shows Unavailable

The tata_gateway endpoint will always show "Unavailable" because Tata does not respond to OPTIONS. This is expected. Verify trunk health with a test call.

WireGuard Tunnel

wg show wg0
ping -c 3 10.10.10.2   # Ping NUC end of tunnel

DID Routing

View Current Routes

asterisk -rx "dialplan show tata-did-route"

Current DID Routing Table

DID Target Context Target Extension
+918065978000 acme_inbound s (queue:support)
+918065978001 org_mna9x47k_inbound 1001 (Hari Surya)
+918065978002 tech_inbound s (queue:tech_support)
+918065978003 org_mna9x47k_inbound 0986 (Manivel)

For adding new DIDs, see DID Management.

Reload Commands

# Reload PJSIP (endpoints, transports, AORs)
asterisk -rx "module reload res_pjsip.so"

# Reload dialplan only
asterisk -rx "dialplan reload"

# Full restart (drops all active calls)
asterisk -rx "core restart now"

Warning

core restart now drops all active calls. Use module and dialplan reloads whenever possible.

Regenerating org configs (preferred: GitHub Actions workflow)

When a queueService / dialplanGenerator emission-shape change has merged and deployed but the on-disk /etc/asterisk/queues_*.conf and ext_*.conf haven't been rewritten yet (Deploy API only rsyncs the Node source; it does not re-run the regen), use the Regen org configs workflow:

  1. GitHub → Actions → Regen org configs → Run workflow → pick staging or production.
  2. The workflow pre-flight-greps the deployed queueService.js for a fix signature, snapshots every queues_*.conf to /root/queues-bak-<ts>/, runs regen-all-org-configs.js, audits non-zero penalty members before and after, and prints the rollback recipe.
  3. Single AMI reload at the end (core + app_queue + devstate seed) — no core restart, no dropped active calls.

Source: .github/workflows/regen-org-configs.yml in astradial-platform. See Queue Routing Architecture → Regen org configs workflow for the design rationale.

The previous manual path — SSH in and run cd /opt/astrapbx && node scripts/regen-all-org-configs.js — still works, but the workflow is preferred because: - Audit trail in Actions log (who, when, env). - Identical runner as Deploy API (no SSH key rotation needed). - Reusable button for future emission-shape changes.

GitHub Actions staging-sync trap

When a PR merges to main, the sync-staging workflow auto-fast-forwards the staging branch — but it pushes using the default GITHUB_TOKEN, which does not trigger downstream workflows (GitHub's loop-prevention rule). So Deploy API to staging and Deploy Editor to staging do not run, and the staging VPS keeps the old code even though the branch shows the new commits.

To recover after a direct-to-main merge that skipped staging:

gh workflow run "Deploy API to staging" --ref staging
gh workflow run "Deploy Editor to staging" --ref staging

Both have workflow_dispatch: enabled. Re-run on the staging self-hosted runner. After they succeed, pm2 list | grep astrapbx on the staging VPS should show a recent uptime; sentinel-grep the deployed source to confirm new code landed.

The long-term fix is to swap sync-staging.yml's GITHUB_TOKEN for a PAT or GitHub App token so downstream workflows DO fire on the auto-sync push.

API Endpoints

Method Endpoint Description
GET /api/v1/organizations List all organizations
POST /api/v1/organizations Create a new organization
POST /api/v1/organizations/:id/regenerate Regenerate org config files
GET /api/v1/trunks List all trunks
POST /api/v1/trunks Create a new trunk
GET /api/v1/dids List all DIDs
POST /api/v1/dids Create a new DID
POST /api/v1/config/deploy Deploy generated configuration
POST /api/v1/config/reload Reload Asterisk configuration

Base URL: https://devpbx.astradial.com

Swagger documentation: https://devpbx.astradial.com/docs

Organization Reference

Name Context Prefix UUID Type
Acme Corp acme_ (see database) Test
TechStart tech_ (see database) Test
AstraPrivate org_mna9x47k_ (see database) Production

To look up UUIDs:

curl -s https://devpbx.astradial.com/api/v1/organizations | jq '.[] | {name, id, prefix}'

Or via database:

USE pbx_api_db;
SELECT id, name, context_prefix, type FROM organizations;

Restart Services

# Asterisk PBX
systemctl restart asterisk

# AstraPBX API (Node.js)
systemctl restart astrapbx

# Nginx reverse proxy
systemctl restart nginx

# MariaDB
systemctl restart mariadb

# WireGuard tunnel
systemctl restart wg-quick@wg0

# Netdata monitoring
systemctl restart netdata

Log Locations

Service Log Location
Asterisk /var/log/asterisk/messages, /var/log/asterisk/full
AstraPBX API journalctl -u astrapbx
Nginx /var/log/nginx/access.log, /var/log/nginx/error.log
MariaDB /var/log/mysql/error.log
WireGuard journalctl -u wg-quick@wg0
System journalctl -xe

Tailing Logs

# Asterisk messages (most useful for SIP debugging)
tail -f /var/log/asterisk/messages

# Asterisk full log (verbose, all output)
tail -f /var/log/asterisk/full

# AstraPBX API
journalctl -u astrapbx -f

# Nginx errors
tail -f /var/log/nginx/error.log