---
title: "Papaʻa, hoʻihoʻi manawa kikoʻī, a me ka hoʻāʻo hoʻihoʻi"
description: "Hana i papaʻa no Quire, hoʻihoʻi iā ia i kekahi manawa i hala, a hōʻoia me ka hoʻāʻo hoʻihoʻi."
image: "https://docs.quirelms.com/og.png"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.quirelms.com/haw/llms.txt
> Use this file to discover all available pages before exploring further.

# Papaʻa, hoʻihoʻi manawa kikoʻī, a me ka hoʻāʻo hoʻihoʻi

<span id="backup-point-in-time-recovery-and-the-restore-drill"></span>

Aia ka hoʻolālā ma ka māhele 8 o `docs/architecture/23-ops.md`. ʻO kēia ke alakaʻi hana no ka huahana Docker Compose. Ua kākau ʻia i hiki i kekahi kanaka ʻaʻole nāna i kākau ke hahai; inā pohihihi kekahi ʻanuʻu, he hewa ia o kēia palapala.

## Nā mea i pale ʻia a me ke ʻano <!--quire:what-is-protected-and-how-->

| Waiwai | Pehea | Kahi |
| --- | --- | --- |
| Ka waihona ʻikepili | Waihona WAL mau, ʻaʻole i ʻoi aku ma mua o kēlā me kēia 60 kekona, mai ka hoʻomaka mua | Volume `pgwal` |
| Ka waihona ʻikepili | Nā papaʻa kumu me `pg_basebackup`, i kēlā me kēia lā ma ke ʻano paʻamau (`backup-scheduler`) | Volume `pgbackup` |
| Ka waihona ʻikepili | Nā kope hoʻopāʻālua o nā papaʻa kumu a me WAL, i kēlā me kēia ʻelima minuke (`backup-offsite`) | Waihona kaʻawale āu e kapa ai |
| Nā faila | Volume `files`. E kope me ka mea hana papaʻa o kāu host, a i ʻole hoʻohana i waihona mea i mana ʻia | Volume `files` |
| Nā ʻōlelo huna malu | `docker/.env`, ʻoi loa `QUIRE_MASTER_KEY` (a me kekahi `QUIRE_MASTER_KEY_RETIRED` e hoʻohana mau ʻia ana), `QUIRE_BACKUP_ENCRYPTION_KEY`, a me `docker/secrets/audit-signing-key.pem` | E mālama i kope ma waho o kēia host |
| Nā helu kuhikuhi huli, cache, a me nā ʻano i hana ʻia | ʻAʻole i hoʻokomo ʻia i loko o ka papaʻa; kūkulu hou ʻia | |

Nā pahuhopu: hoʻihoʻi i kahi kūlana ma loko o 60 kekona ma mua o ka pilikia, a hoʻopau i ka hoʻihoʻi ma loko o 60 minuke no ka waihona 500 GB.

ʻElua hewa maʻamau. Hōʻike ka waihona ʻikepili i hoʻihoʻi ʻia **me ka ʻole o nā faila** i nā ʻaoʻao haki. ʻAʻole hiki i ka waihona ʻikepili i hoʻihoʻi ʻia **me ka ʻole o `QUIRE_MASTER_KEY`** ke wehe i nā palapala hōʻoia SSO, webhook, a hoʻohui; a pau ka hoʻololi kī haku me ka ʻole o kekahi mea i koe ([key-rotation.md](/haw/ops/key-rotation/)), pili kēia i nā kī hoʻomaha pū. Māhele lāua o ka papaʻa.

## Hana i nā papaʻa <!--quire:taking-backups-->

Papaʻa kumu o ke kāʻei holoʻokoʻa:

```sh
docker compose -f docker/compose.yaml --profile backup run --rm backup
```

Mālama ia i nā papaʻa kumu hou loa e like me `QUIRE_BACKUP_KEEP` (5 ma ke ʻano paʻamau), a wehe i ka WAL i pono ʻole i ka papaʻa kahiko loa, i ulu palena ʻole ʻole ai ka waihona. Hoʻonohonoho i ia mea i kēlā me kēia lā me cron a i ʻole timer systemd ma ka host:

```cron
15 2 * * * cd /srv/quire && docker compose -f docker/compose.yaml --profile backup run --rm backup >> /var/log/quire-backup.log 2>&1
```

A i ʻole e ʻae i ka stack e hoʻonohonoho: holo ka profile `backup` iā `backup-scheduler`, nāna e hana i papaʻa kumu i kēlā me kēia `QUIRE_BACKUP_INTERVAL_HOURS` (24 ma ke ʻano paʻamau), a me `backup-offsite` e wehewehe ʻia aʻe.

```sh
docker compose -f docker/compose.yaml --profile backup up -d
```

## Nā papaʻa hoʻopāʻālua ma waho o ka host <!--quire:encrypted-off-host-copies-->

Noho nā volume ʻelua ma ka host like me ka waihona ʻikepili; ʻaʻole he papaʻa ka papaʻa ma ka mīkini i hāʻule. Kope ʻo `backup-offsite` i kēlā me kēia papaʻa kumu a me kēlā me kēia māhele WAL i waihona kaʻawale ma o ka puka waiho ʻikepili; hoʻopāʻālua, a mālama ma lalo o ka lula mālama:

- **Hoʻopāʻālua.** AES-256-GCM me `QUIRE_BACKUP_ENCRYPTION_KEY` (a i ʻole ka faila i kapa ʻia e `QUIRE_BACKUP_ENCRYPTION_KEY_FILE`): 32 byte mai `openssl rand -hex 32`. Loaʻa i kēlā me kēia faila kona nonce a me ka hōʻailona hōʻoia ponoʻī, no laila ʻaʻole hiki ke heluhelu i kope me ka ʻole o ke kī a ʻike ʻia kekahi hoʻololi. E mālama i ke kī me `QUIRE_MASTER_KEY`, ma kahi mamao o kēia host a me ka waihona papaʻa. ʻAʻohe hoʻihoʻi me ka ʻole o ke kī.
- **Kahi.** ʻO `QUIRE_BACKUP_STORAGE_DRIVER` kekahi o `s3`, `azure`, a i ʻole `local` (disk mamao i kau ʻia ma `QUIRE_BACKUP_STORAGE_ROOT`). ʻO nā hoʻonohonoho mālama faila me ka prefix `QUIRE_BACKUP_`: `QUIRE_BACKUP_S3_ENDPOINT`, `QUIRE_BACKUP_S3_BUCKET`, `QUIRE_BACKUP_S3_ACCESS_KEY_ID`, a pēlā aku. E hoʻohana i bucket ʻokoʻa a, ʻoi aku ka maikaʻi, i moʻokāki ʻokoʻa mai nā faila, me nā palapala hōʻoia hiki ke kākau akā ʻaʻole holoi inā ʻae ka mea hoʻolako.
- **Mālama.** Nā papaʻa kumu hou loa e like me `QUIRE_BACKUP_OFFSITE_KEEP` (paʻamau `QUIRE_BACKUP_KEEP`, a i ʻole 7) a me ka WAL e pono ai ka papaʻa kahiko loa; holoi ʻia nā pūʻulu a me nā māhele kahiko mai ka waihona.
- **Ka wā.** I kēlā me kēia `QUIRE_BACKUP_SHIP_INTERVAL_SECONDS` (300 ma ke ʻano paʻamau). Hana hoʻokahi wale nō ka hoʻouna: kāpae ʻia nā mea i mālama ʻia, a helu ʻia kahi papaʻa kumu he mālama ʻia ma hope wale nō o ke kākau ʻia ʻana o kāna hōʻike hope.

Holo lima pū kēia kauoha:

```sh
docker compose -f docker/compose.yaml run --rm backup-offsite bun apps/worker/src/backups/main.ts ship
docker compose -f docker/compose.yaml run --rm backup-offsite bun apps/worker/src/backups/main.ts verify
```

No ka hoʻihoʻi ma host hou, e kiʻi mua i pūʻulu, a laila hahai i nā ʻanuʻu ma lalo me ka waihona i kiʻi ʻia ma kahi o ka volume `pgbackup`, a me ka `wal-archive` i kiʻi ʻia ma kahi o `pgwal`:

```sh
bun apps/worker/src/backups/main.ts fetch base-20260924T021500Z /srv/restore
```

## Hoʻihoʻi i kekahi manawa kikoʻī <!--quire:restoring-to-a-point-in-time-->

E hoʻohana ma hope o ka nalowale ʻikepili: hoʻokomo hewa, holoi papa, a i ʻole neʻe pani e pono ai ke kāpae. Hoʻololi kēia i ka waihona ʻikepili ola, no laila e hoʻāʻo mua me ka hana ma lalo.

1. **Koho i ka manawa pahuhopu**, ma UTC, ma mua koke o ka pōʻino:
   `2026-09-24 09:30:00+00`. Hōʻike maʻamau ka moʻolelo hōʻoia (`/admin/audit`) i ia manawa.
2. **Hoʻōki i nā mea a pau e kākau ana**:
   `docker compose -f docker/compose.yaml stop web content worker scheduler collab`
3. **Mālama i ke kāʻei i pohō** a hōʻoia ʻia ka hoʻihoʻi:
   ```sh
   docker compose -f docker/compose.yaml stop postgres
   docker run --rm -v quire_postgres18-data:/from -v quire_postgres-damaged:/to alpine cp -a /from/. /to/
   ```
4. **Wehe i ka papaʻa kumu hou loa ma mua o ka pahuhopu** i loko o ka volume ʻikepili, a noi i ka hoʻihoʻi kikoʻī:
   ```sh
   docker run --rm -v quire_pgbackup:/backups:ro -v quire_postgres18-data:/var/lib/postgresql postgres:18-alpine sh -euc '
     base="$(ls -1d /backups/base-* | sort | tail -n 1)"   # or the one before the target
     rm -rf /var/lib/postgresql/18/docker && mkdir -p /var/lib/postgresql/18/docker
     tar -xzf "$base/base.tar.gz" -C /var/lib/postgresql/18/docker
     touch /var/lib/postgresql/18/docker/recovery.signal
     chown -R postgres:postgres /var/lib/postgresql/18/docker && chmod 700 /var/lib/postgresql/18/docker'
   ```
5. **Hoʻihoʻi**: hoʻomaka hoʻokahi manawa iā Postgres me nā hoʻonohonoho hoʻihoʻi, mai ka pani Compose i ʻole e hoʻopā i ka faila maʻamau:
   ```yaml
   # docker/compose.recover.yaml
   services:
     postgres:
       command: [postgres, -c, "restore_command=cp /var/lib/postgresql/wal-archive/%f %p",
                 -c, "recovery_target_time=2026-09-24 09:30:00+00",
                 -c, recovery_target_action=promote, -c, archive_mode=off,
                 -c, max_connections=200, -c, hba_file=/etc/postgresql/pg_hba.conf]
   ```
   Hoʻololi ka pani i ke kauoha holoʻokoʻa, no laila hana hou i nā hoʻonohonoho
   ʻelua e pono ai ka hoʻihoʻi: ʻaʻole emi ʻo `max_connections` ma mua o ka
   hoʻonohonoho nui (i ʻole pau ka hoʻihoʻi me ka memo «insufficient parameter
   settings») a me ka `pg_hba.conf` i kau ʻia.
   ```sh
   docker compose -f docker/compose.yaml -f docker/compose.recover.yaml up -d postgres
   docker compose -f docker/compose.yaml logs -f postgres   # wait for "database system is ready"
   ```
6. **Nānā iā ia** ma mua o ka ʻae ʻana i kekahi: ke kaulahao hōʻoia
   (`docker compose -f docker/compose.yaml run --rm worker bun tooling/audit-verify/run.ts`),
   a hōʻoia ua hoʻi ka ʻikepili i nalowale.
7. **Hoʻi i ka maʻamau**: `docker compose -f docker/compose.yaml up -d`. Hoʻomaka hou
   ʻo Postgres me ka waihona WAL i hoʻā ʻia, a hoʻomaka ka laina manawa WAL hou.
   Hana koke i papaʻa kumu hou.

Hoʻokomo ka inoa papahana `quire` i prefix ma kēlā me kēia volume; hōʻike ʻo `docker volume ls` i nā inoa pololei.

## Ka hoʻāʻo hōʻoia i hoʻonohonoho ʻia <!--quire:the-scheduled-verification-drill-->

Holo pū ʻo `backup-offsite` i hoʻāʻo i kēlā me kēia `QUIRE_BACKUP_DRILL_INTERVAL_HOURS` (168, i kēlā me kēia pule ma ke ʻano paʻamau), a i ka holo aʻe ma hope o kekahi hāʻule. Kiʻi ia i ka papaʻa kumu waho hou loa a me kēlā me kēia māhele WAL ma hope ona, wehe i kēlā me kēia hoʻopāʻālua (hōʻoia kēia e wehe mau ke kī a ʻaʻohe mea i hoʻololi), hoʻohālikelike i kēlā me kēia faila me kona hōʻike hope, nānā he papa kuhikuhi ʻikepili Postgres ka waihona, a hōʻoia ʻaʻohe hakahaka ma ka WAL mai ka papaʻa. Kākau ʻia ka hōʻike ma ka waihona ma ke ʻano he `reports/drill-<time>.json` a ma ka moʻolelo lawelawe; kapa ka hoʻāʻo hāʻule i ka faila a i ʻole ka māhele mua i nalowale.

## Ka hoʻāʻo hoʻihoʻi <!--quire:the-restore-drill-->

ʻAʻole he papaʻa ka papaʻa i hoʻihoʻi ʻole ʻia. Hoʻihoʻi ka hoʻāʻo i ka papaʻa kumu piha hou loa ma mua o ka pahuhopu, me ka waihona WAL, i Postgres hoʻāʻo kaʻawale loa mai ka mea ola, a hōʻoia i ka hopena:

```sh
docker/scripts/restore-drill.sh                                  # to ninety minutes ago
docker/scripts/restore-drill.sh --target "2026-09-24 09:30:00+00"
```

Pono ka pahuhopu ma UTC a ma kēlā ʻano kikoʻī. Pono kekahi papaʻa kumu ma mua ona a me WAL i waiho ʻia ma hope: ma kahi hoʻouka hou, e hana i papaʻa kumu a kali i ka māhele waihona aʻe (ʻaʻole i ʻoi aku ma mua o hoʻokahi minuke ke kākau ʻia nā mea), ma mua o ke koho ʻana i pahuhopu ma hope o ka papaʻa. Pono ka hoʻāʻo i Docker a me bash ma ka host; ʻaʻohe mea ʻē aʻe.

Hāʻule ka hoʻāʻo ke hāʻule kekahi ʻanuʻu:

1. **Hiki ke hoʻihoʻi**: hoʻi ka cluster hoʻāʻo i ka pahuhopu a wehe ʻia.
2. **Piha**: helu lālani o kēlā me kēia papaʻaina e hoʻohālikelike ʻia me ka waihona ola (`tooling/restore-drill`). Ua neʻe ka waihona ola ma hope o ka pahuhopu, no laila hiki i kekahi papaʻaina ke ʻokoʻa ma ka waiwai nui aʻe o 500 lālani a i ʻole ka hapaʻumi o kona nui, ma kēlā me kēia ʻaoʻao (hoʻemi nā kākau i nā mea i hoʻihoʻi ʻia; hoʻonui nā holoi ʻana); ʻaʻole helu ʻia kahi māhele i hana ʻia ma hope o ka pahuhopu he papaʻaina nalowale. E hoʻonui i ka ʻae ma kahi hoʻouka hana nui me `QUIRE_DRILL_MAX_BEHIND` a me `QUIRE_DRILL_MAX_DRIFT_RATIO`. Hāʻule ka papaʻaina i nele a i hoʻokaʻawale ʻia.
3. **Pono**: hōʻoia ʻia ke kaulahao hash hōʻoia ma ka kope i hoʻihoʻi ʻia.
4. **Hiki ke hoʻohana**: heluhelu ke kuleana polokalamu ma lalo o ka palekana pae-lālani.
5. **Manawa**: mai ka hoʻomaka a hiki i ka holomua, hoʻohālikelike ʻia me `QUIRE_DRILL_RTO_SECONDS` (3600 ma ke ʻano paʻamau).

ʻAʻole loa ia e kākau i ka waihona ola a me kona mau volume: kau ʻia nā volume papaʻa a WAL ma ke ʻano heluhelu wale nō, a holoi ʻia ke cluster hoʻāʻo i ka hopena, hala a hāʻule paha.

E hoʻonoho iā `QUIRE_DRILL_REPORT` i helu wahi e kākau ai i hōʻike JSON, inā hala a hāʻule paha; a holo ma ka papa manawa mai ka host Docker:

```cron
30 3 1 * * cd /srv/quire && QUIRE_DRILL_REPORT=/var/log/quire-drill.json docker/scripts/restore-drill.sh >> /var/log/quire-drill.log 2>&1
```

E holo i kēlā me kēia mahina a ma mua o kēlā me kēia hōʻano hou. Kāohi ka hoʻāʻo hāʻule i ka hōʻano hou. Hoʻokahi manawa i ka hapahā makahiki, e noi i kekahi mea ʻaʻole i kākau i kēia alakaʻi e hana i hoʻihoʻi manawa kikoʻī ma host pani me kēia palapala wale nō.

## Nā faila <!--quire:files-->

Noho nā faila kūloko ma ka volume `files`. E hana pū i papaʻa me ka waihona ʻikepili i ka manawa like, a hoʻihoʻi pū iā lāua:

```sh
docker run --rm -v quire_files:/files:ro -v "$PWD":/out alpine tar -czf /out/files-$(date -u +%Y%m%d).tar.gz -C /files .
```

Me ka waihona mea, hoʻā i ka mālama mana bucket a mālama i nā mana kahiko ʻole no 35 lā; na ka bucket ponoʻī ka hoʻihoʻi manawa kikoʻī o nā faila ma ia manawa.

Source: https://docs.quirelms.com/haw/ops/backup-restore/index.mdx