Compare commits

..

No commits in common. "a0514f08232f1103512dbe6cb628c62b103c004c" and "52af2a755c9a20b625cfc21f14b3566b221d866a" have entirely different histories.

4 changed files with 41 additions and 79 deletions

View File

@ -85,45 +85,14 @@ jobs:
ssh "${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }}" \ ssh "${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }}" \
"cd '$DEPLOY_PATH' && IMAGE_TAG='$IMAGE_TAG' docker compose pull && IMAGE_TAG='$IMAGE_TAG' docker compose up -d --remove-orphans && docker image prune -f" "cd '$DEPLOY_PATH' && IMAGE_TAG='$IMAGE_TAG' docker compose pull && IMAGE_TAG='$IMAGE_TAG' docker compose up -d --remove-orphans && docker image prune -f"
- name: Verify Web and Scheduler Are Running
run: |
set -euo pipefail
ssh "${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }}" "DEPLOY_PATH='$DEPLOY_PATH' bash -s" << 'EOF'
set -euo pipefail
cd "$DEPLOY_PATH"
web_id="$(docker compose ps -q web)"
scheduler_id="$(docker compose ps -q scheduler)"
if [ -z "$web_id" ]; then
echo "web service container not found"
exit 1
fi
if [ -z "$scheduler_id" ]; then
echo "scheduler service container not found"
exit 1
fi
web_running="$(docker inspect -f '{{.State.Running}}' "$web_id")"
scheduler_running="$(docker inspect -f '{{.State.Running}}' "$scheduler_id")"
if [ "$web_running" != "true" ]; then
echo "web service is not running"
exit 1
fi
if [ "$scheduler_running" != "true" ]; then
echo "scheduler service is not running"
exit 1
fi
EOF
- name: Wait for Ready Health Check - name: Wait for Ready Health Check
env: env:
HEALTH_URL: ${{ secrets.DEPLOY_HEALTHCHECK_URL }} HEALTH_URL: ${{ secrets.DOKPLOY_HEALTHCHECK_URL }}
MAX_ATTEMPTS: "30" MAX_ATTEMPTS: "30"
SLEEP_SECONDS: "10" SLEEP_SECONDS: "10"
run: | run: |
if [ -z "$HEALTH_URL" ]; then if [ -z "$HEALTH_URL" ]; then
echo "Missing DEPLOY_HEALTHCHECK_URL secret" echo "Missing DOKPLOY_HEALTHCHECK_URL secret"
exit 1 exit 1
fi fi
bash scripts/wait-for-health.sh bash scripts/wait-for-health.sh

View File

@ -8,23 +8,19 @@
- [ ] Postgres port `5432` is not public. - [ ] Postgres port `5432` is not public.
## B) App and Deployment ## B) App and Deployment
- [ ] SSH deployment host is prepared (`/opt/fiddy`, Docker Engine, Compose plugin). - [ ] Dokploy project connected to Gitea repo.
- [ ] Deploy host runtime env is configured in `/opt/fiddy/.env`: - [ ] Secrets configured:
- [ ] `DATABASE_URL` - [ ] `DATABASE_URL`
- [ ] `DATABASE_SSL` - [ ] `DATABASE_SSL`
- [ ] `ALLOWED_DB_NAMES` - [ ] `ALLOWED_DB_NAMES`
- [ ] `SESSION_COOKIE_NAME` - [ ] `SESSION_COOKIE_NAME`
- [ ] `SESSION_TTL_DAYS` - [ ] `SESSION_TTL_DAYS`
- [ ] `DEBUG_API` - [ ] `DEBUG_API=0`
- [ ] Gitea Actions secrets configured: - [ ] `DOKPLOY_DEPLOY_HOOK`
- [ ] `REGISTRY_USER` - [ ] `DOKPLOY_SCHEDULER_DEPLOY_HOOK`
- [ ] `REGISTRY_PASS` - [ ] `DOKPLOY_HEALTHCHECK_URL`
- [ ] `DEPLOY_KEY`
- [ ] `DEPLOY_HOST`
- [ ] `DEPLOY_USER`
- [ ] `DEPLOY_HEALTHCHECK_URL`
- [ ] Deploy workflow passes build/test/push/deploy. - [ ] Deploy workflow passes build/test/push/deploy.
- [ ] Deploy guard confirms `web` and `scheduler` are running. - [ ] Scheduler deploy workflow step passes.
- [ ] Post-deploy health gate passes (`scripts/wait-for-health.sh`). - [ ] Post-deploy health gate passes (`scripts/wait-for-health.sh`).
- [ ] Manual smoke passes (`scripts/smoke-public-launch.sh`). - [ ] Manual smoke passes (`scripts/smoke-public-launch.sh`).
@ -51,6 +47,6 @@
- [ ] Measured RTO is acceptable. - [ ] Measured RTO is acceptable.
## F) Rollback Readiness ## F) Rollback Readiness
- [ ] Previous stable image tags retained in registry (for rollback). - [ ] Previous stable release retained in Dokploy.
- [ ] Rollback runbook tested once in staging or low-risk window (SSH Compose deploy by older image tag). - [ ] Rollback runbook tested once in staging or low-risk window.
- [ ] Rollback smoke check verified. - [ ] Rollback smoke check verified.

View File

@ -25,30 +25,28 @@ Use these in execution updates for fast scanning:
- [ ] `docs/08_NGINX_PROXY_MANAGER_SETUP.md` - [ ] `docs/08_NGINX_PROXY_MANAGER_SETUP.md`
- [ ] `docs/06_SECURITY_REVIEW.md` - [ ] `docs/06_SECURITY_REVIEW.md`
## Phase 1: Registry + SSH Compose Wiring (Operator Needed) ## Phase 1: Registry + Dokploy Wiring (Operator Needed)
First-time reference: `docs/11_DOKPLOY_FIRST_TIME_WALKTHROUGH.md`.
Hands-on checkpoints: Hands-on checkpoints:
1. Create/verify secrets in Gitea: 1. Create/verify secrets in Gitea:
- `REGISTRY_USER` - `REGISTRY_USER`
- `REGISTRY_PASS` - `REGISTRY_PASS`
- `DEPLOY_KEY` - `DOKPLOY_DEPLOY_HOOK`
- `DEPLOY_HOST` - `DOKPLOY_SCHEDULER_DEPLOY_HOOK`
- `DEPLOY_USER` - `DOKPLOY_HEALTHCHECK_URL`
- `DEPLOY_HEALTHCHECK_URL` 2. In Dokploy app settings (Web):
2. Prepare deploy host for SSH Compose: - image source points to `git.nicosaya.com/nalalangan/fiddy/web`
- install Docker Engine + Compose plugin - health endpoint is `/api/health/ready`
- create `/opt/fiddy/.env` with production variables - release history retention is enabled
- run `docker login git.nicosaya.com` as deploy user 3. In Dokploy app settings (Scheduler):
3. Confirm production compose contract: - image source points to `git.nicosaya.com/nalalangan/fiddy/scheduler`
- web image source: `git.nicosaya.com/nalalangan/fiddy/web` - no public port exposed
- scheduler image source: `git.nicosaya.com/nalalangan/fiddy/scheduler` - env vars set: `DATABASE_URL`, `DATABASE_SSL`, `ALLOWED_DB_NAMES`
- web publishes `3010:3000`
- scheduler has no public port
Validation: Validation:
- [ ] Push-to-main triggers `.gitea/workflows/deploy-ssh-compose.yml` - [ ] Push-to-main triggers `.gitea/workflows/deploy-dokploy.yml`
- [ ] SSH deploy updates both web and scheduler containers - [ ] Web and Scheduler deploy hooks fire successfully
- [ ] Deploy guard confirms web and scheduler are running
- [ ] Health gate completes via `scripts/wait-for-health.sh` - [ ] Health gate completes via `scripts/wait-for-health.sh`
## Phase 2: NPM Edge Setup (Operator Needed) ## Phase 2: NPM Edge Setup (Operator Needed)

View File

@ -1,4 +1,4 @@
# Public Launch Runbook (Self-Hosted + SSH Compose) # Public Launch Runbook (Self-Hosted + Dokploy)
## 1) Goals ## 1) Goals
- Deploy Fiddy publicly without stack rewrite. - Deploy Fiddy publicly without stack rewrite.
@ -6,14 +6,14 @@
- Enable fast rollback and basic operational visibility. - Enable fast rollback and basic operational visibility.
- Keep security baseline enforceable for direct home-IP exposure. - Keep security baseline enforceable for direct home-IP exposure.
## 2) Deploy Host (SSH Compose) ## 2) Deploy Control Plane (Dokploy)
1. Prepare Linux deploy host with Docker Engine + Compose plugin. 1. Install Dokploy on your Proxmox Docker host.
2. Ensure deploy target directory exists (`/opt/fiddy`). 2. Add project in Dokploy and connect Gitea repository.
3. Configure web image source: `git.nicosaya.com/nalalangan/fiddy/web`. 3. Configure web image source: `git.nicosaya.com/nalalangan/fiddy/web`.
4. Configure scheduler image source: `git.nicosaya.com/nalalangan/fiddy/scheduler`. 4. Configure scheduler image source: `git.nicosaya.com/nalalangan/fiddy/scheduler`.
5. Deploy by immutable tag (`github.sha`) and keep `main` as convenience tag. 4. Deploy by immutable tag (`github.sha`) and keep `main` as convenience tag.
6. Configure health check endpoint: `/api/health/ready`. 5. Configure health check endpoint: `/api/health/ready`.
7. Keep previous image tags for rollback. 6. Keep previous releases for rollback and verify rollback button path.
### Required secrets/variables ### Required secrets/variables
- `DATABASE_URL` - `DATABASE_URL`
@ -26,16 +26,15 @@
- `SCHEDULER_BATCH_SIZE` (scheduler app, optional) - `SCHEDULER_BATCH_SIZE` (scheduler app, optional)
## 3) CI/CD (Gitea Actions) ## 3) CI/CD (Gitea Actions)
- Use `.gitea/workflows/deploy-ssh-compose.yml`. - Use `.gitea/workflows/deploy-dokploy.yml`.
- Required secrets: - Required secrets:
- `REGISTRY_USER` - `REGISTRY_USER`
- `REGISTRY_PASS` - `REGISTRY_PASS`
- `DEPLOY_KEY` - `DOKPLOY_DEPLOY_HOOK`
- `DEPLOY_HOST` - `DOKPLOY_SCHEDULER_DEPLOY_HOOK`
- `DEPLOY_USER` - `DOKPLOY_HEALTHCHECK_URL`
- `DEPLOY_HEALTHCHECK_URL`
- Health gate: - Health gate:
- workflow calls `scripts/wait-for-health.sh` against `DEPLOY_HEALTHCHECK_URL` - workflow calls `scripts/wait-for-health.sh` against `DOKPLOY_HEALTHCHECK_URL`
- default retry window: 5 minutes (30 attempts x 10s) - default retry window: 5 minutes (30 attempts x 10s)
## 4) Reverse Proxy + Network Hardening ## 4) Reverse Proxy + Network Hardening
@ -101,12 +100,12 @@
1. Identify failing request and `request_id`. 1. Identify failing request and `request_id`.
2. Correlate application logs (Loki) by `request_id`. 2. Correlate application logs (Loki) by `request_id`.
3. Check `/api/health/ready` status and DB connectivity. 3. Check `/api/health/ready` status and DB connectivity.
4. Roll back to previous known-good image tag via SSH Compose if needed. 4. Roll back to previous known-good Dokploy release if needed.
5. Capture root cause and update this runbook/checklist. 5. Capture root cause and update this runbook/checklist.
## 8) Rollback Checklist ## 8) Rollback Checklist
1. Select previous healthy image tag for both `web` and `scheduler`. 1. Select previous healthy image in Dokploy release history.
2. Trigger rollback deploy and wait for completion. 2. Trigger rollback and wait for deployment completion.
3. Run `scripts/smoke-public-launch.sh https://your-domain`. 3. Run `scripts/smoke-public-launch.sh https://your-domain`.
4. Verify error-rate drop in Grafana/Loki and confirm no DB migration mismatch. 4. Verify error-rate drop in Grafana/Loki and confirm no DB migration mismatch.
5. Log the rolled back version, timestamp, and reason. 5. Log the rolled back version, timestamp, and reason.