PupuniBackup, te whakaora ki tētahi wā, me te whakamātau whakahoki

Pupuria a Quire, whakahokia ia ki tētahi wā, ā, whakaaturia tēnei ma te whakamātau whakahoki.

Ko te hoahoa te docs/architecture/23-ops.md wāhanga 8. Ko tēnei te aratohu mahi mō te produk Docker Compose. Ka tuhia ia kia whāia ia e tētahi tāngata kāore ia e tuhi i a ia; mēnā he kōrero kāore mārama tētahi kaupapa, ko ia te hapa o tēnei tuhinga.

He aha te e ārahihia ana, ā, te āhua

Taonga	Āhua	Wāhi
Te pūtaka	Te WAL e kohia ana i ngā wā katoa, ia 60 hēkona iti rawa, mai i te whakatīmata tuatahi	te volume `pgwal`
Te pūtaka	Ngā pupuniBackup tūmatanui me te `pg_basebackup`, ia rā i te tūtohu (`backup-scheduler`)	te volume `pgbackup`
Te pūtaka	Ngā tārua kua uhipu o ngā pupuniBackup tūmatanui me te WAL, ia 5 meneti (`backup-offsite`)	Tētahi kohinga motuhake e kōwhiria ana e koe
Ngā kōnae	Te volume `files`. Tāruatia ia ma te utu pupuniBackup o tō pūrere, rānei whakamahia he kohinga āhuatanga mā ngā āhua tuhia	te volume `files`
Ngā kupu tūmataiti	`docker/.env`, i runga rawa te `QUIRE_MASTER_KEY` (me ēhea `QUIRE_MASTER_KEY_RETIRED` kua whakamahia tonu), te `QUIRE_BACKUP_ENCRYPTION_KEY`, me te `docker/secrets/audit-signing-key.pem`	Pupuritia tētahi tārua i waho o tēnei pūrere
Ngā tohu whakarārangi rapu, ngā kohinga, ngā āhua ataata	Kāore e pupuria; ka hanga anō

Ngā tūmanako: tētahi wāhi whakaora i roto i te 60 hēkona o te whakararu, me tētahi whakahoki i roto i te 60 meneti mō tētahi pūtaka 500 GB.

He hē rua te tūpono. Ka whakaatu tētahi pūtaka kua whakahoki me ōna kōnae kāore ngā whārangi kua pāngia. Kāore e taea e tētahi pūtaka kua whakahoki me te kore te QUIRE_MASTER_KEY te wehenga i ngā tohu whakamanatanga SSO, webhook me te hononga e pupuri ana ia; tae noa ki te wā ka oti te hurihuri kī matua me te kore mea kāore kua whakataunga (key-rotation.md), ko ēnei hoki ngā kī kua wehea. Ērua he wāhanga o te pupuniBackup.

Te tuku pupuniBackup

He pupuniBackup tūmatanui mō te rōpū pūtaka katoa:

docker compose -f docker/compose.yaml --profile backup run --rm backup

Ka pupuria ngā QUIRE_BACKUP_KEEP pupuniBackup tūmatanui hou rawa (tūtohu 5), ā, ka tukua te WAL e kore tonu e hiahiatia e te tawhito rawa, nō reira kāore e taea te tipu whakarara o te kōpaki. Whakarite ia ia rā ma te cron rānei tētahi tākiri systemd i te pūrere:

15 2 * * * cd /srv/quire && docker compose -f docker/compose.yaml --profile backup run --rm backup >> /var/log/quire-backup.log 2>&1

Rānei e whakarite ana te kōpae i a ia: ka mahi te profile backup i te backup-scheduler, e tukuna ana tētahi pupuniBackup tūmatanui ia QUIRE_BACKUP_INTERVAL_HOURS (tūtohu 24), me te backup-offsite, e ārahitia ana i muri mai.

docker compose -f docker/compose.yaml --profile backup up -d

Ngā tārua kua uhipu i waho o te pūrere

Kei te tahi pūrere ngā ērua volume me te pūtaka, ā, ehara i te pupuniBackup tētahi kei te pūrere kua raru. Ka tukua te backup-offsite ia pupuniBackup tūmatanui me ia wāhanga WAL kua kohia ki tētahi kohinga motuhake ma te ara kohinga, kua uhipu, ā, ka noho rātou i reira i raro i te pupuri:

Te uhipu. AES-256-GCM me te QUIRE_BACKUP_ENCRYPTION_KEY (rānei te kōnae e kīia ana e te QUIRE_BACKUP_ENCRYPTION_KEY_FILE): 32 byte, mai i te openssl rand -hex 32. He ake nonce tō ia kōnae me tētahi tohu whakamanatanga, nō reira ka kore e pānui tētahi tārua me te kore te kī, ā, ka kitea ia huringa. Pupuritia te kī me te QUIRE_MASTER_KEY, i waho o tēnei pūrere, ā, i waho o te kohinga pupuniBackup. Me te kore te kī, kāore he whakahoki.
Te wāhi. Ko te QUIRE_BACKUP_STORAGE_DRIVER ko te s3, te azure rānei te local (tētahi papa waho kua tukuhia i te QUIRE_BACKUP_STORAGE_ROOT). Ko ngā tautuhinga ē tautuhinga kohinga kōnae me te tohu tāpiri QUIRE_BACKUP_: te QUIRE_BACKUP_S3_ENDPOINT, te QUIRE_BACKUP_S3_BUCKET, te QUIRE_BACKUP_S3_ACCESS_KEY_ID, me ērā atu. Whakamahia tētahi pākete rerekē, ā, i tōna tino pai tētahi pūkete rerekē i ngā kōnae, me ē tohu whakamanatanga e āea ana te tuhi engari kāore e mukua mēnā ka āea te kaiwhakarato.
Te pupuri. Ngā QUIRE_BACKUP_OFFSITE_KEEP pupuniBackup tūmatanui hou rawa (tūtohu te QUIRE_BACKUP_KEEP, mēnā kāore, ko te 7) me te WAL e hiahiatia ana e te tawhito rawa o rātou; ka mukua ngā kōpui me ngā wāhanga tawhito i te kohinga.
Te wā. Ia QUIRE_BACKUP_SHIP_INTERVAL_SECONDS (tūtohu 300). He ōrite te tukunga: ka wehea te e kua kohia kē, ā, ka noho tētahi pupuniBackup tūmatanui hei kua kohia anake i te wā kua tuhia tana rārangi tuhinga, whakamutunga.

Ka mahia te tahi whakahau i te ringa:

docker compose -f docker/compose.yaml run --rm backup-offsite bun apps/worker/src/backups/main.ts ship
docker compose -f docker/compose.yaml run --rm backup-offsite bun apps/worker/src/backups/main.ts verify

Mō te whakahoki ki tētahi pūrere hou, tāmoa tuatahi tētahi kōpui, kātia ka whai ngā kaupapa raro me te rārangi kua tikina i te wāhi o te volume pgbackup, me te wal-archive kua tikina i te wāhi o te pgwal:

bun apps/worker/src/backups/main.ts fetch base-20260924T021500Z /srv/restore

Te whakaora ki tētahi wā

Whakamahia tēnei i muri i te raru raraunga: tētahi whakaurunga kāore tika, tētahi akoranga kua mukua, tētahi whakawhiti whakaiti e hiahiatia ana te whakakore. Ka whakakapi ia te pūtaka ora, nō reira whakamātauria ia me te whakamātau raro tuatahi.

Kōwhiria te wā tūmanako, i te UTC, i te tata rawa ki mua i te pānga: 2026-09-24 09:30:00+00. Ka whakaatu i ngā wā katoa te rārangi arotake (/admin/audit) i te wā tika.
Tūhia ngā mea katoa e tuhi ana: docker compose -f docker/compose.yaml stop web content worker scheduler collab

Pupuria te rōpū kua pāngia tae noa ki te whakaaetia te whakahoki:

docker compose -f docker/compose.yaml stop postgres
docker run --rm -v quire_postgres18-data:/from -v quire_postgres-damaged:/to alpine cp -a /from/. /to/

Whakapakia te pupuniBackup tūmatanui hou rawa o ngā tawhito ake i te wā tūmanako ki te volume raraunga, ā, tono te whakaora ki te tūmanako:

docker run --rm -v quire_pgbackup:/backups:ro -v quire_postgres18-data:/var/lib/postgresql postgres:18-alpine sh -euc '
  base="$(ls -1d /backups/base-* | sort | tail -n 1)"   # or the one before the target
  rm -rf /var/lib/postgresql/18/docker && mkdir -p /var/lib/postgresql/18/docker
  tar -xzf "$base/base.tar.gz" -C /var/lib/postgresql/18/docker
  touch /var/lib/postgresql/18/docker/recovery.signal
  chown -R postgres:postgres /var/lib/postgresql/18/docker && chmod 700 /var/lib/postgresql/18/docker'

Whakaora: tīmata te Postgres kotahi wā me ngā tautuhinga whakaora, mai i tētahi whakakapi Compose kia kore e pāngia te kōnae tūtohu:

# docker/compose.recover.yaml
services:
  postgres:
    command: [postgres, -c, "restore_command=cp /var/lib/postgresql/wal-archive/%f %p",
              -c, "recovery_target_time=2026-09-24 09:30:00+00",
              -c, recovery_target_action=promote, -c, archive_mode=off,
              -c, max_connections=200, -c, hba_file=/etc/postgresql/pg_hba.conf]

Ka whakakapi te whakakapi te whakahau katoa, nō reira ka tāhaina anō ngā tautuhinga e rua e whino ana te whakaora: te max_connections kāore iti rawa i te o te matua (mēnā kāore, ka wehea te whakaora me te “insufficient parameter settings”) me te pg_hba.conf kua tukuhia.

docker compose -f docker/compose.yaml -f docker/compose.recover.yaml up -d postgres
docker compose -f docker/compose.yaml logs -f postgres   # wait for "database system is ready"

Tātaria i mua i te whakawhinga i te tāngata kotahi: te hono arotake (docker compose -f docker/compose.yaml run --rm worker bun tooling/audit-verify/run.ts), me te mea kua hoki mai te raraunga kua ngaro.
Hoki ki te tūtohu: docker compose -f docker/compose.yaml up -d. Ka tīmata anō tēnei te Postgres me te kohia tonu, ā, ka tīmata tētahi ara wā WAL hou. Tangohia tētahi pupuniBackup tūmatanui hou i te wā tonu.

Ko te ingoa whakaturanga ko te quire te tohu tāpiri o ia volume; ka whakaatuhia te docker volume ls i ngā ingoa tino tika.

Te whakamātau tautara whakamātau

Ka mahi hoki te backup-offsite i tētahi whakamātau ia QUIRE_BACKUP_DRILL_INTERVAL_HOURS (tūtohu 168, ia wiki), ā, anō i te tukunga nō muri i te wā ka rahua tētahi. Ka tikina te pupuniBackup tūmatanui kua tuku i waho hou rawa me ia wāhanga WAL i muri i a ia, ka wehenga ia (e whakaatu tēnei ko te kī ka noho tonu e tuwhera i a rātou, ā, kāore kua hurihia tētahi), ka tātaritia ia kōnae me tana rārangi tuhinga, ka tātaritia me he te kōpaki he whārangi raraunga Postgres, ā, ka tātaritia me he kāore wehenga te WAL mai i te pupuniBackup ki mua. Ka tuhia te pūrongo ki te kohinga ma te reports/drill-<time>.json me te rārangi pūrongo o te ratonga; ka whakaatuhia e tētahi whakamātau kua rahua te kōnae rānei te wāhanga tuatahi kāore e kitea ana.

Te whakamātau whakahoki

Ehara i te pupuniBackup tētahi kāore kua whakahoki i ngā wā katoa. Ka whakahoki te whakamātau te pupuniBackup tūmatanui hou rawa kua mau i mua i te wā tūmanako, me te kōpaki WAL, ki tētahi Postgres whakawhiti kāore he mea e tū tahi me te ora, ā, ka whakaatu te hua:

docker/scripts/restore-drill.sh                                  # to ninety minutes ago
docker/scripts/restore-drill.sh --target "2026-09-24 09:30:00+00"

Ko te UTC te wā tūmanako i tēnei āhua tino tika. Ka hiahiatia ia tētahi pupuniBackup tūmatanui tawhito ake i a ia, me te WAL kua kohia i tua atu i a ia: i tētahi whakaturanga hou, tangohia tētahi pupuniBackup tūmatanui, ā, tatari i te wāhanga kua kohia nō muri (te iti rawa meneti me ngā tuhi) i mua i te kōwhiri i tētahi wā tūmanako i muri i te pupuniBackup. Ka hiahiatia e te whakamātau te Docker me te bash i te pūrere, kāore tētahi atu.

Ka rahua ia whakamātau ia kaupapa:

Te whakaoratanga: ka whakahoki te rōpū whakawhiti ki te wā tūmanako, ā, ka tuwhera.
Te whakaotitanga: te tatauranga rārangi o ia pūrataka ki te pūtaka ora (tooling/restore-drill). Kua nekehia te pūtaka ora i mua i te wā tūmanako, nō reira ka āea te pūrataka te wehenga i te rahi rawa o te 500 rārangi me te wāhanga tahi o te tekau o tana rahinga, i tēhea aronga e rua (e whakatoro ana ngā tuhi i a ia, ka pupuri te whakahoki i tētahi nui ake); ehara i te pūrataka kua ngaro tētahi wāhanga kua hangaia i mua i te wā tūmanako. Whakanuia te āhei i tētahi whakaturanga nui ake ma te QUIRE_DRILL_MAX_BEHIND me te QUIRE_DRILL_MAX_DRIFT_RATIO. Ka rahua tētahi pūrataka kua ngaro rānei kua whakakorehia.
Te motuhaka: ka tautara te hono wehenga arotake i te tārua kua whakahoki.
Te whakamahinga: ka pānui te tūwāhi taupānga ma te haumarutanga ā-rārangi.
Te wā: mai i te tīmata ki te kākāriki, ki te QUIRE_DRILL_RTO_SECONDS (tūtohu 3600).

Kāore e tuhi ia i ngā wā katoa i te pūtaka ora rānei i ōna volume: ka tukuhia ngā volume pupuniBackup me te WAL hei pānui-anake, ā, ka tangohia te rōpū whakawhiti i te mutunga, ka pāngia rānei kāore.

Tautuhia te QUIRE_DRILL_REPORT ki tētahi ara kia tuhia tētahi pūrongo JSON, ka pāngia rānei kāore, ā, whakatematema ia i te takotahi mai i te pūrere Docker:

30 3 1 * * cd /srv/quire && QUIRE_DRILL_REPORT=/var/log/quire-drill.json docker/scripts/restore-drill.sh >> /var/log/quire-drill.log 2>&1

Whakatematemaina ia ia maramataka, ā, i mua i ia whakahou. Ka ārahi tētahi whakamātau kua rahua i te whakahou. I ia toru maramataka, kia mahia e tētahi tāngata kāore ia e tuhi i tēnei aratohu tētahi whakaora tūturu ki tētahi wā i tētahi pūrere hereke, me te whakamahi i tēnei tuhinga anake.

Ngā kōnae

Kei te volume files ngā kōnae ā-rohe. Pupuria ia me te pūtaka, i te tahi wā, ā, whakahokia ērua ngātahi:

docker run --rm -v quire_files:/files:ro -v "$PWD":/out alpine tar -czf /out/files-$(date -u +%Y%m%d).tar.gz -C /files .

Me te kohinga āhuatanga, whakawāhia te whakaturanga pākete me te pupuri i ngā 35 rā o ngā āhua kāore tūturu; ko te whakaora ki tētahi wā mō ngā kōnae te ake o te pākete i muri i tēnei.