▶Stream4US

Running in production

For administrators: install, secure, monitor, back up and update a castgate server.

A checklist for administrators: install, secure, monitor, back up and update a castgate server. Each item says why it matters. Admin → System checks many of these for you and gives the exact fix.

Before you start

  • [ ] Server: Linux (Ubuntu 22.04 or 24.04 recommended), 4+ CPU cores, 8+ GB memory, an SSD for the data folder. Recordings take about 1 GB per hour at 2,500 kbps.
  • [ ] Bandwidth: outgoing traffic is viewers × bitrate (500 viewers × 2.5 Mbps = 1.25 Gbps). For large audiences put a CDN in front (Admin → Settings → Streaming address).
  • [ ] DNS: an A record (and AAAA for IPv6) for the site name pointing to the server, before the first start.
  • [ ] Firewall: open TCP 80 and 443, UDP 443 (HTTP/3), TCP 1935 (RTMP), UDP 9000 (SRT). Nothing else needs to be reachable.
  • [ ] Install with sudo sh deploy/install.sh --domain ... --email ... (docs/GO-LIVE.md), or by hand, and run castgate -preflight with the service's flags: no FAIL lines.
  • [ ] HTTPS (-domain): browser and phone Go live need it for the camera.
  • [ ] More viewers than one machine's bandwidth: add edge servers (castgate -edge https://main -domain edgeN.example.com) and list them in the main server's -trusted-proxies; see README → Edge servers.
  • [ ] Desktop encoder downloads: automatic when the server program runs from the release (dist/castgate-linux-amd64 next to dist/encoder/). If you install the program elsewhere, put an encoder/ folder with the packages next to it, or copy them to <data>/downloads/. Admin → System → "Desktop encoder downloads" should list Windows, both Macs and Linux.
  • [ ] ffmpeg installed (apt install ffmpeg): uploads, MP4 downloads, thumbnails, browser Go live, server qualities, captions and radio need it.
  • [ ] Time synchronization on (timedatectl shows "synchronized: yes"): schedules, certificates and two-step sign-in depend on it.

Install

sudo useradd --system --home /var/lib/castgate --shell /usr/sbin/nologin castgate
sudo install -m 755 castgate-linux-amd64 /usr/local/bin/castgate
sudo cp castgate.service /etc/systemd/system/
sudo nano /etc/systemd/system/castgate.service   # -domain and -acme-email
sudo systemctl daemon-reload && sudo systemctl enable --now castgate
journalctl -u castgate -f                        # shows the /setup link on the first start

The unit runs castgate as its own user with only the rights it needs (no new privileges, read-only system folders, private /tmp), restarts it if it stops, and allows a million open connections.

Important options

OptionDefaultUse
-domain live.example.orgnoneAutomatic Let's Encrypt certificates; several names separated by commas
-acme-email ops@example.orgnoneLet's Encrypt contact for expiry notices
-data /var/lib/castgate./dataSettings, recordings, certificates, backups
-trusted-proxies 10.0.0.0/8,...noneYour CDN or proxy addresses, so viewer addresses and limits are right
-max-conns-per-ip 10001000Web connections one address may keep open; 0 = no limit
-rtmp :1935, -srt :9000as shownIngest ports
-http3=falseonTurn off HTTP/3 if UDP 443 cannot be opened
-playout-resume 30s30 sWait after a live stream before 24/7 playout returns
-record-grace 90s90 sHow long a recording waits for an encoder to reconnect
-reset-admin email:passwordCreate or reset an administrator, then exit

Secure

  • [ ] HTTPS with a real certificate (-domain or -tls-cert/-tls-key). castgate then sends HSTS so browsers always use HTTPS.
  • [ ] Two-step sign-in for every administrator (Security), and Admin → Settings → "Require two-step sign-in for administrators".
  • [ ] Automatic blocking stays on (Admin → Blocking): addresses that guess passwords or stream keys are blocked for an hour, a day on repeat.
  • [ ] Trusted proxies set if a CDN or reverse proxy is in front; otherwise every viewer looks like the proxy (one country, one address) and limits would hit the proxy.
  • [ ] Backups downloaded from Admin → System are kept safe: they contain password hashes and stream keys.
  • [ ] Keep the server updated (unattended-upgrades on Ubuntu).

Built in: CSRF protection on every form, sign-in throttling, request size limits, security headers (including a Content-Security-Policy that allows no inline script, 1.7.2), per-address connection limit, files written privately (mode 0600/0700), atomic saves. A password change or reset signs the login out everywhere, and Security → "Sign out of all other devices" does it on demand (1.7.2). Restreaming and importing from a link refuse every internal address range, the same list as the relay (1.7.2). The full audit with results: docs/READINESS.md.

Geofencing and peer-to-peer

  • [ ] GeoIP databases installed and refreshed weekly (geoipupdate -d /dev/shm from cron): the geofence and the location reports need them. Without them the location rules are not applied.
  • [ ] Server-wide blocks (Admin → Blocking → Countries, networks and cities) for countries you may not serve.
  • [ ] Built-in TURN relay (Admin → Peer-to-peer → Built-in relay): open 3478 UDP and TCP, the relay range (49160-49360 UDP by default) and 5349 TCP for TLS; press Test STUN and TURN, and test once from a machine outside your network.
  • [ ] Peer-to-peer (Admin → Peer-to-peer): your own STUN server if viewers are in countries that block Google, and a TURN relay (coturn with use-auth-secret) for strict networks. Open the TURN ports (3478 UDP/TCP, 5349 TCP for TLS, and its relay range).
  • [ ] A reverse proxy in front of castgate passes WebSocket upgrades for /p2p/ws.
  • [ ] Run the Field test (Admin → Peer-to-peer) from a phone on mobile data and from a typical church Wi-Fi before you announce peer-to-peer; keep the reports as a baseline.
  • [ ] Never start a public server with -turn-allow-private (it lets the relay reach your private network; it is for lab tests only).
  • [ ] Live captions inside the stream add about 30 seconds of delay for viewers while they are on; tell churches that use them.

Billing

  • [ ] If you sell plans online: Admin → Billing connected to the billing site, checkout links entered, a test event logged and one test purchase made in the billing site's test mode. See billing integration.
  • [ ] Email works (new customers receive their sign-up link by email).

Email

  • [ ] Email configured and tested (Admin → Settings → Send a test email). Without it there are no password resets, prayer emails, weekly reports or disk alerts. See email setup.

Capacity and tuning

  • [ ] Admin → System → Setup checks all OK or understood: kernel network buffers for SRT, connection limits, BBR congestion control, open-file limits, swap, disk space, clock, ffmpeg, GeoIP, HTTPS.
  • [ ] GeoLite2 City and ASN databases in /dev/shm with a weekly geoipupdate (locations, maps, providers, country restrictions).
  • [ ] Re-encoding: choose the graphics card (NVIDIA, Intel, AMD) under Admin → Settings if you offer server-made qualities or many uploads.
  • [ ] A load test before a big event: many viewers on one channel from another machine (see "Load test" below).

Monitor

  • [ ] Prometheus (or another tool) reads /metrics (token on Admin → System; not needed from the server itself or trusted proxies): castgate_up, castgate_viewers, castgate_connections{kind}, castgate_live_streams, castgate_ingest_kbps, CPU, memory, traffic, per channel.
  • [ ] Alerts: on castgate_up missing, CPU over 85 %, memory over 90 %, outgoing traffic near the link speed, castgate_connections_refused_total rising.
  • [ ] Health check for a load balancer or uptime monitor: GET /healthz answers ok.
  • [ ] The log is in the journal: journalctl -u castgate --since today. Disk alerts start with ALERT:.
  • [ ] Admin → Audit log shows every administrative change.

Back up

  • [ ] castgate copies store.json (accounts, plans, settings, schedules) to backups/ in the data folder every day and keeps 14 days.
  • [ ] Copy the whole data folder to another machine regularly (recordings are the bulk): for example a nightly rsync -a /var/lib/castgate/ backup-host:/backups/castgate/.
  • [ ] Restore: stop castgate, put the backup store.json (and the recordings) in the data folder, start castgate.

Disk space

  • [ ] Below 2 GB free (or 2 % of the disk), castgate pauses new recordings and uploads, keeps live streams running, and emails administrators every six hours until space is freed.
  • [ ] Plans' archive days and storage limits keep each church within bounds; old recordings past the archive days are removed automatically.

Update

  1. Read the CHANGELOG for the new version.
  2. Download the backup (Admin → System).
  3. Replace the program and restart:
sudo install -m 755 castgate-linux-amd64 /usr/local/bin/castgate
sudo systemctl restart castgate

Encoders reconnect by themselves within seconds and recordings continue; viewers' players retry. Do it outside service times anyway. Settings are upgraded automatically; nothing else to migrate.

Load test

The package includes tools/loadtest, a small program that plays a channel like many viewers at once (it fetches the playlist and every segment as a player does):

go run ./tools/loadtest -url https://live.example.org/c/grace-church -viewers 300 -duration 2m

Run it from another machine against a live stream. It reports how many viewers kept up, segment download times and errors. Watch Admin → System meanwhile. Each test viewer counts against the channel's viewer limit, so raise the limit for the test channel first (Admin → Accounts). All test viewers come from one address: for more than a few hundred, start castgate with a higher -max-conns-per-ip during the test, or add the test machine to -trusted-proxies.

TV apps for the stores

  • Roku: the church's package from TV apps (or Admin → TV apps); developer settings, then the Roku Channel Store submission. Deep links: contentId = channel name, channel/live, or channel/RECORDING-ID; mediaType live, episode or movie.
  • Fire TV and Android TV: the Android source kit; build a release with your own key (-Pcastgate.keystore=..., see its README) and submit to the Amazon Appstore and Google Play. The banner, TV launcher entry and no-touch-screen settings are already in place.
  • Fire TV on Vega: build with the Vega SDK (npm run build:release), set SERVER_URL in src/config.ts for a church's own app, or keep the example address for the universal app.

Production readiness (1.6.0; audited again in 1.7.2, see docs/READINESS.md)

Ready and tested on this release's test machine: accounts, plans and billing hooks; RTMP/SRT ingest; HLS playback with DVR, adaptive qualities, recordings, editing and archive; 24/7 playout and the weekly grid; restreaming; analytics; live captions; chat, prayer, podcast and radio; P2P with the built-in relay; phone and browser Go live; the desktop encoder (Linux run, Windows/Mac command lines checked); edge servers; TV app contract tests; HTTPS, two-step sign-in, rate limits, daily settings backups, /healthz and /metrics.

Not yet proven, so do these before charging churches:

  • [ ] Field tests: the encoder on real Windows PCs and Macs with real USB cameras and capture cards; the phone sender on a real iPhone and Android phone; every TV app on a real device; P2P and edges across real networks.
  • [ ] A load test from several machines against production hardware (tools/loadtest), for your bandwidth and the number of churches per server.
  • [ ] Signed apps: Windows code signing (no SmartScreen warning), a notarized Mac app; store listings for Roku, Amazon (Fire TV, Vega), Google Play (Android TV, the Sender) and the App Store (the iPhone Sender).
  • [ ] Real outside services connected and tried: your mail server, billing at encrypted.com, a transcription server for captions, Drive/Dropbox import.
  • [ ] Recording storage set up and "Test storage" passing, for a second copy of every recording.
  • [ ] Monitoring: point your monitoring at /healthz and /metrics on the main server and each edge, and watch disk space in the data folder.

Limits of this design (fine for a first few hundred churches on one main server; plan ahead beyond that):

  • One main server: it is the single point of failure for sign-in, ingest and recordings. Edges carry viewers, not ingest. Keep a standby machine and the nightly copy of the data folder (see Back up).
  • Recordings live on the main server's disk unless recording storage is set up (Admin → Settings → Recording storage, 1.7): then every recording has a second copy in an S3-compatible bucket, and old ones can live only there.
  • Accounts and settings are one file (store.json), written on each change: simple and fast for hundreds of accounts, not meant for tens of thousands.

This guide is also in the castgate package as docs/PRODUCTION.md.