Prod Direct-Edit Workflow¶

When a user asks to fix something in prod and tells you to "use the VPS to resolve directly" — SSH to the prod VPS, treat VPS code as the source of truth, fix there, then sync the change back to git. This page is the procedure.

Environment map¶

App layout is identical on both hosts:

Monorepo: https://github.com/astradial/astradial-platform.

The procedure¶

1. VPS is the source of truth — read it first¶

Before editing anything, compare VPS to local git:

# Example: the editor's auth watcher
scp root@89.116.31.109:/opt/pipecat-flow-editor/components/auth/AuthExpiryWatcher.tsx /tmp/prod-file
diff /tmp/prod-file ~/AstradialDevelopment/astradial-platform/editor/components/auth/AuthExpiryWatcher.tsx

If they differ, the VPS has an untracked change. Do not overwrite it — investigate why first, then either commit the VPS version back to git or consciously discard it.

A handy bulk diff for editor:

ssh root@89.116.31.109
cd /opt/pipecat-flow-editor
# spot-check a subtree against your local clone
# (assumes your laptop is reachable; otherwise, scp individual files)

2. Fix on the VPS directly¶

ssh root@89.116.31.109
cd /opt/pipecat-flow-editor                   # or /opt/astrapbx, etc.
vim components/auth/AuthExpiryWatcher.tsx     # make the change

# Rebuild & restart (editor only — other apps have no build step)
npm run build
pm2 restart editor --update-env

# Verify
pm2 status editor
pm2 logs editor --lines 50 --nostream

For astrapbx / workflow-engine:

pm2 restart astrapbx --update-env
pm2 restart workflow-engine --update-env

3. Sync the fix back to git — always¶

The live fix MUST land in git the same session. Otherwise the next CI/CD deploy (or manual rsync --delete) will overwrite it.

# On your laptop:
scp root@89.116.31.109:/opt/pipecat-flow-editor/components/auth/AuthExpiryWatcher.tsx \
    ~/AstradialDevelopment/astradial-platform/editor/components/auth/AuthExpiryWatcher.tsx

cd ~/AstradialDevelopment/astradial-platform

# CRITICAL — always branch off a FRESH origin/main, not the current branch.
# See "Branching footgun" below for why.
git fetch origin main
git checkout main
git reset --hard origin/main                   # discard local main drift
git checkout -b fix/auth-expiry-watcher-prod   # now branched off clean main

git diff                                       # sanity-check the scp'd diff
git add editor/components/auth/AuthExpiryWatcher.tsx
git commit -m "fix(auth): <what the prod hotfix did>"

# Verify the branch contains ONLY your intended changes before pushing.
# This command should list only your one commit:
git log origin/main..HEAD --oneline

git push -u origin fix/auth-expiry-watcher-prod
gh pr create --base main --title "..." --body "..."

Branch conventions:

• Prod hotfix → branch off origin/main → PR to main.
• Staging hotfix → branch off origin/staging → PR to staging.

Branching footgun (learned the hard way)¶

Always branch new work off a freshly-fetched origin/<base>, never off whatever branch git status shows you're currently on.

When you run git checkout -b new-branch without first checkouting the base, git branches from your CURRENT branch. If you have an earlier feature branch still checked out (one with unmerged commits, maybe an open PR), those commits silently ride along into the new branch.

Consequence: when the new branch's PR is squash-merged, the squash pulls in the earlier branch's commits too — because they're part of the PR's history. A follow-up merge of the earlier PR then shows as empty in git log (nothing left to apply), and the commit history makes it look like two separate features were one combined change. Audit trail suffers. Prod ends up correct, but from the wrong commit.

Happened 2026-04-23 during the Scalar/IVR ship — PR #64 (asterisk-configs) and PR #65 (scalar-spec-branding) squash-merged as one combined commit because #65 was branched off #64 instead of origin/main. Prod got everything correctly, but git log main shows #64 as a zero-line commit.

Pre-branch sanity checklist:

# Confirm which branch you're on — should be main or staging
git branch --show-current

# Confirm it's clean
git status

# Pull latest base
git fetch origin

# Hard-reset to origin/<base> so no local divergence leaks in
git checkout main
git reset --hard origin/main

# NOW create the feature branch
git checkout -b <new-branch-name>

# Final sanity — nothing ahead yet
git log origin/main..HEAD --oneline   # expect empty

Run that block every time. Takes 5 seconds, prevents 15 minutes of confused git archaeology later.

4. Double-check the loop closed¶

After merging the PR:

• CI/CD (prod: GitHub Actions self-hosted runner on 89.116.31.109) redeploys /opt/<app>/.
• Re-run diff between VPS and local main — should be empty except for .env, node_modules/, .next/, .bak*, .backup*.

When NOT to use direct-edit¶

• Anything touching auth / billing / SIP routing that needs review — go through a normal feature-branch PR instead.
• Changes you can't finish and commit back the same session — the fix will rot.
• When the change is big enough that a 3-minute Next.js build hurts — build locally first, then deploy via the normal CI/CD path.

What to NEVER do without explicit user instruction¶

• Restart Asterisk or the NUC gateway.
• Modify /etc/asterisk/*, /etc/wireguard/*, or /etc/nginx/* on prod.
• Delete .env, firebase-sa-key.json, google-credentials.json, or any *.bak* / *.backup* files.
• Run rsync --delete from laptop → VPS without diffing first. VPS has code not in git; a blind rsync can wipe hotfixes.
• Push to main without a PR, even for "tiny" prod fixes. The PR creates the audit trail.

Per-host Asterisk configs NOT in the monorepo¶

Some /etc/asterisk/*.conf files are hand-maintained on each VPS and are NOT tracked in astradial/astradial-platform. Changes here don't round-trip through git and won't be reverted by CI/CD (CI only rsyncs editor/ and api/ directories). But they're also invisible to anyone reading the repo.

Known hand-maintained files (document here when you touch one):

When editing any of these:

1. Back up first: cp <file> <file>.bak-$(date +%Y-%m-%d)
2. Edit with a cp-backed file under the same dir (never swap files from a different directory to avoid permission drift).
3. asterisk -rx "dialplan reload" or pjsip reload as appropriate.
4. Document the change in docs/operations/session-<date>-<summary>.md so it's discoverable — the file isn't in git, the change context must live somewhere. Future agents will not find the change by git log alone.

Related docs¶

• Staging Direct-Edit — same idea, lower stakes.
• CI/CD Pipeline — what the self-hosted runners do.
• Cloud Runbook — prod ops, service restarts.
• Prod Monorepo Cutover — how prod moved to the monorepo on 2026-04-17.
• Session Apr 23 — IVR Prod Ship — the most recent prod deploy + troubleshooting reference.