---
title: "Serep, pamulihan point-in-time lan latihan restore"
description: "Serep Quire, balekake menyang titik wektu, lan buktekake kanthi latihan restore."
image: "https://docs.quirelms.com/og.png"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.quirelms.com/jv/llms.txt
> Use this file to discover all available pages before exploring further.

# Serep, pamulihan point-in-time lan latihan restore

<span id="backup-point-in-time-recovery-and-the-restore-drill"></span>

Desaine iku `docs/architecture/23-ops.md` bagean 8. Iki runbook kanggo produk Docker Compose. Ditulis supaya ditindakake dening wong sing ora nulis; yen langkah ora cetha, iku cacat ing dokumen iki.

## Apa sing dilindhungi, lan carane <!--quire:what-is-protected-and-how-->

| Aset | Carane | Ing ngendi |
| --- | --- | --- |
| Database | WAL diarsipake terus, paling suwe saben 60 detik, wiwit boot kapisan | volume `pgwal` |
| Database | Serep dasar nganggo `pg_basebackup`, saben dina kanthi gawan (`backup-scheduler`) | volume `pgbackup` |
| Database | Salinan dienkripsi saka serep dasar lan WAL, saben limang menit (`backup-offsite`) | Toko kapisah sing dijenengi |
| File | Volume `files`. Salin nganggo piranti serep host, utawa gunakake panyimpenan objek berversi | volume `files` |
| Rahasia | `docker/.env`, utamane `QUIRE_MASTER_KEY` (lan `QUIRE_MASTER_KEY_RETIRED` apa wae sing isih digunakake), `QUIRE_BACKUP_ENCRYPTION_KEY`, lan `docker/secrets/audit-signing-key.pem` | Simpen salinan ing njaba host iki |
| Indeks panelusuran, cache, rendisi | Ora diserep; dibangun maneh | |

Target: titik pamulihan sajrone 60 detik saka gagal, lan restore sajrone 60 menit kanggo database 500 GB.

Rong kaluputan umum. Database sing dibalekake **tanpa file-e** ngrender kaca rusak. Database sing dibalekake **tanpa `QUIRE_MASTER_KEY`** ora bisa dekripsi kredensial SSO, webhook lan integrasi sing dicekel; nganti rotasi kunci master rampung tanpa sing ora dirampungake ([key-rotation.md](/jv/ops/key-rotation/)), iku kalebu kunci sing dipensiunake. Kalorone iku bagean saka serep.

## Njupuk serep <!--quire:taking-backups-->

Serep dasar kabeh cluster:

```sh
docker compose -f docker/compose.yaml --profile backup run --rm backup
```

Iki njaga serep dasar `QUIRE_BACKUP_KEEP` paling anyar (gawan 5) lan motong WAL sing ora dibutuhake sing paling lawas maneh, supaya arsip ora bisa tuwuh tanpa wates. Jadwalake saben dina nganggo cron utawa timer systemd ing host:

```cron
15 2 * * * cd /srv/quire && docker compose -f docker/compose.yaml --profile backup run --rm backup >> /var/log/quire-backup.log 2>&1
```

Utawa jarake stack jadwalake: profil `backup` mbukak `backup-scheduler`, sing njupuk serep dasar saben `QUIRE_BACKUP_INTERVAL_HOURS` (gawan 24), lan `backup-offsite`, sing diterangake sabanjure.

```sh
docker compose -f docker/compose.yaml --profile backup up -d
```

## Salinan off-host dienkripsi <!--quire:encrypted-off-host-copies-->

Kalorone volume manggon ing host sing padha karo database, lan serep ing mesin sing gagal dudu serep. `backup-offsite` nyalin saben serep dasar lan saben segmen WAL sing diarsipake menyang toko kapisah liwat port panyimpenan, dienkripsi, lan njaga ing kana ing sangisore retensi:

- **Enkripsi.** AES-256-GCM nganggo `QUIRE_BACKUP_ENCRYPTION_KEY` (utawa file sing dijenengi dening `QUIRE_BACKUP_ENCRYPTION_KEY_FILE`): 32 byte, saka `openssl rand -hex 32`. Saben file duwe nonce dhewe lan tag autentikasi, supaya salinan ora bisa diwaca tanpa kunci lan owahan apa wae dideteksi. Simpen kunci karo `QUIRE_MASTER_KEY`, adoh saka host iki lan adoh saka toko serep. Tanpa kunci ora ana restore.
- **Ing ngendi.** `QUIRE_BACKUP_STORAGE_DRIVER` iku `s3`, `azure` utawa `local` (disk remote sing dipasang ing `QUIRE_BACKUP_STORAGE_ROOT`). Setelane iku setelan panyimpenan file kanthi prefiks `QUIRE_BACKUP_`: `QUIRE_BACKUP_S3_ENDPOINT`, `QUIRE_BACKUP_S3_BUCKET`, `QUIRE_BACKUP_S3_ACCESS_KEY_ID`, lan sapanunggalane. Gunakake bucket beda lan, idealnya, akun beda saka file, kanthi kredensial sing bisa nulis nanging ora mbusak yen panyedhiya ngidinake.
- **Retensi.** Serep dasar `QUIRE_BACKUP_OFFSITE_KEEP` paling anyar (gawan `QUIRE_BACKUP_KEEP`, yen ora 7) lan WAL sing dibutuhake sing paling lawas; set lan segmen luwih lawas dibusak saka toko.
- **Kapan.** Saben `QUIRE_BACKUP_SHIP_INTERVAL_SECONDS` (gawan 300). Pangiriman idempoten: apa sing wis disimpen dilewati, lan serep dasar dietung disimpen mung yen manifeste ditulis, pungkasan.

Prentah sing padha mlaku kanthi tangan:

```sh
docker compose -f docker/compose.yaml run --rm backup-offsite bun apps/worker/src/backups/main.ts ship
docker compose -f docker/compose.yaml run --rm backup-offsite bun apps/worker/src/backups/main.ts verify
```

Kanggo mbalekake ing host anyar, gawa set bali dhisik, banjur tindakake langkah ing ngisor kanthi direktori sing dijupuk ngganti volume `pgbackup` lan `wal-archive` sing dijupuk ngganti `pgwal`:

```sh
bun apps/worker/src/backups/main.ts fetch base-20260924T021500Z /srv/restore
```

## Mbalekake menyang titik wektu <!--quire:restoring-to-a-point-in-time-->

Gunakake iki sawise data ilang: impor ala, kursus sing dibusak, migrasi kontrak sing perlu dibatalake. Iki ngganti database live, dadi latih dhisik nganggo latihan ing ngisor.

1. **Pilih wektu target**, ing UTC, sakdurunge rusak: `2026-09-24 09:30:00+00`. Log audit (`/admin/audit`) biasane nuduhake wektune.
2. **Mungkasi kabeh sing nulis**: `docker compose -f docker/compose.yaml stop web content worker scheduler collab`
3. **Njaga cluster rusak** nganti restore diverifikasi:
   ```sh
   docker compose -f docker/compose.yaml stop postgres
   docker run --rm -v quire_postgres18-data:/from -v quire_postgres-damaged:/to alpine cp -a /from/. /to/
   ```
4. **Bongkar serep dasar paling anyar sing luwih lawas saka target** menyang volume data, lan njaluk pamulihan target:
   ```sh
   docker run --rm -v quire_pgbackup:/backups:ro -v quire_postgres18-data:/var/lib/postgresql postgres:18-alpine sh -euc '
     base="$(ls -1d /backups/base-* | sort | tail -n 1)"   # or the one before the target
     rm -rf /var/lib/postgresql/18/docker && mkdir -p /var/lib/postgresql/18/docker
     tar -xzf "$base/base.tar.gz" -C /var/lib/postgresql/18/docker
     touch /var/lib/postgresql/18/docker/recovery.signal
     chown -R postgres:postgres /var/lib/postgresql/18/docker && chmod 700 /var/lib/postgresql/18/docker'
   ```
5. **Pulihake**: miwiti Postgres sapisan kanthi setelan pamulihan, saka override Compose supaya file normal ora didemek:
   ```yaml
   # docker/compose.recover.yaml
   services:
     postgres:
       command: [postgres, -c, "restore_command=cp /var/lib/postgresql/wal-archive/%f %p",
                 -c, "recovery_target_time=2026-09-24 09:30:00+00",
                 -c, recovery_target_action=promote, -c, archive_mode=off,
                 -c, max_connections=200, -c, hba_file=/etc/postgresql/pg_hba.conf]
   ```
   Override ngganti kabeh prentah, dadi mbaleni rong setelan sing gumantung pamulihan: `max_connections` ora luwih endhek saka primary (yen ora pamulihan dibatalake kanthi "insufficient parameter settings") lan `pg_hba.conf` sing dipasang.
   ```sh
   docker compose -f docker/compose.yaml -f docker/compose.recover.yaml up -d postgres
   docker compose -f docker/compose.yaml logs -f postgres   # wait for "database system is ready"
   ```
6. **Priksa** sadurunge ngidinake sapa wae mlebu: rante audit (`docker compose -f docker/compose.yaml run --rm worker bun tooling/audit-verify/run.ts`), lan manawa data sing ilang bali.
7. **Bali menyang normal**: `docker compose -f docker/compose.yaml up -d`. Iki miwiti maneh Postgres kanthi arsip urip, lan timeline WAL anyar diwiwiti. Jupuk serep dasar anyar langsung.

Jeneng proyek `quire` ngawali saben volume; `docker volume ls` nuduhake jeneng persis.

## Latihan verifikasi terjadwal <!--quire:the-scheduled-verification-drill-->

`backup-offsite` uga mbukak latihan saben `QUIRE_BACKUP_DRILL_INTERVAL_HOURS` (gawan 168, mingguan), lan maneh ing pass sabanjure sawise gagal siji. Iki njupuk serep dasar off-host paling anyar lan saben segmen WAL sawise iku, dekripsi saben (sing mbuktekake kunci isih mbukak lan ora ana sing diowahi), mbandhingake saben file karo manifeste, mriksa arsip iku direktori data Postgres, lan mriksa WAL saka serep terus ora ana jeda. Laporan ditulis menyang toko minangka `reports/drill-<time>.json` lan menyang log layanane; latihan gagal nyebutake file utawa segmen ilang kapisan.

## Latihan restore <!--quire:the-restore-drill-->

Serep sing ora tau dibalekake dudu serep. Latihan mbalekake serep dasar lengkap paling anyar sing dijupuk sadurunge target, ditambah arsip WAL, menyang Postgres scratch sing ora nuduhake apa-apa karo sing live, lan mbuktekake asile:

```sh
docker/scripts/restore-drill.sh                                  # to ninety minutes ago
docker/scripts/restore-drill.sh --target "2026-09-24 09:30:00+00"
```

Target iku UTC ing wujud persis kasebut. Butuh serep dasar sing luwih lawas saka iku lan WAL sing diarsipake ngluwihi iku: ing instalasi anyar, jupuk serep dasar lan ngenteni segmen sing diarsipake sabanjure (paling suwe semenit kanthi tulisan) sadurunge milih target sawise serep. Latihan butuh Docker lan bash ing host, ora ana liyane.

Saben langkah nggagalake latihan:

1. **Bisa dipulihake**: cluster scratch muter maneh menyang target lan mbukak.
2. **Lengkap**: cacah baris saben tabel nglawan database live (`tooling/restore-drill`). Database live wis maju wiwit target, dadi tabel bisa beda kanthi sing luwih gedhe saka 500 baris lan sepersepuluh ukurane, ing arah endi wae (tulisan nggawe ketinggalan, pambusakan nggawe restore nyekel luwih akeh); partisi sing digawe sawise target dudu tabel ilang. Ambakake kelonggaran ing instalasi luwih rame nganggo `QUIRE_DRILL_MAX_BEHIND` lan `QUIRE_DRILL_MAX_DRIFT_RATIO`. Tabel ilang utawa kosong gagal.
3. **Integritas**: rante hash audit verifikasi ing salinan sing dibalekake.
4. **Bisa digunakake**: peran aplikasi maca liwat keamanan tingkat baris.
5. **Wektu**: miwiti nganti ijo, nglawan `QUIRE_DRILL_RTO_SECONDS` (gawan 3600).

Ora tau nulis menyang database live utawa volume-e: volume serep lan WAL dipasang read-only lan cluster scratch dibusak ing pungkasan, lulus utawa gagal.

Setel `QUIRE_DRILL_REPORT` menyang path supaya laporan JSON ditulis, lulus utawa gagal, lan mbukak ing jadwal saka host Docker:

```cron
30 3 1 * * cd /srv/quire && QUIRE_DRILL_REPORT=/var/log/quire-drill.json docker/scripts/restore-drill.sh >> /var/log/quire-drill.log 2>&1
```

Mbukak saben sasi lan sadurunge saben upgrade. Latihan gagal mblokir upgrade. Sapisan seperapat, njaluk wong sing ora nulis runbook iki nindakake restore point-in-time nyata ing host serep, nggunakake mung dokumen iki.

## File <!--quire:files-->

File lokal manggon ing volume `files`. Serep bebarengan karo database, ing wektu sing padha, lan balekake kalorone bebarengan:

```sh
docker run --rm -v quire_files:/files:ro -v "$PWD":/out alpine tar -czf /out/files-$(date -u +%Y%m%d).tar.gz -C /files .
```

Kanthi panyimpenan objek, uripake versioning bucket lan njaga 35 dina versi noncurrent; pamulihan point-in-time kanggo file banjur duweke bucket dhewe.

Source: https://docs.quirelms.com/jv/ops/backup-restore/index.mdx