Backup and Recovery

In single-node deployment, all of FinClaw's stateful data lives in Docker named volumes (the chatkit_* prefix). This page gives the simplest viable backup procedure: stop services -> archive each volume -> start services. For stricter RPO/RTO requirements in production, layer pg_dump logical backups or volume snapshots on top.

1. What to Back Up

1.1 Configuration Files

Path	Notes
deploy/.env	LLM credentials, JWT keys, internal tokens, concurrency settings
deploy/config/	services.yaml, ai-infra.yaml, idp.yaml, and other YAML configs
deploy/docker-compose.override.yml (if any)	Custom port overrides, etc.

1.2 Docker Volumes

The chatkit_* family of volumes, all declared external: true in docker-compose.yml:

Volume name	Contents
chatkit_postgres_data	PostgreSQL business data (users / tenants / personas / skills / inbox / document metadata, plus Feishu, DingTalk, and WeCom connector data)
chatkit_timescaledb_data	TimescaleDB data (conversation-store session history and audit data)
chatkit_ai_infra_data	AI runtime persistent data (audit logs, memory index, session state, MCP servers config, fleet runtime state)
chatkit_workspaces	User workspaces (files, memory files, installed skills)
chatkit_skills_data	Platform skills data (builtin + catalog)
chatkit_persona_uploads	Persona service upload files
chatkit_identity_uploads	Identity service upload files (avatars, etc.)
chatkit_doc_toolkit_cache	Document toolkit cache (pure cache, theoretically skippable; archiving it along with the rest is the safest)
chatkit_nats_data	NATS JetStream (event streams, short-lived messages)
chatkit_redis_data	Redis cache and coordination state
chatkit_restate_data	Restate scheduled tasks / cron state
chatkit_tei_models	TEI embedding model files (cannot be re-downloaded in offline environments - must be backed up)

Simple rule for single-node: archive every chatkit_* volume - do not try to cherry-pick; missing one may mean data loss.

2. Backup

Step 1: Stop services

Backing up volumes while containers are running yields an inconsistent snapshot (especially for PostgreSQL). Stop services first.

cd /data/chatkit-offline-release-1.1.20-linux-amd64/deploy   # Replace with your actual version
./scripts/fleet.sh down

down stops all containers (including the per-tenant claw containers) but does not delete any volumes.

Step 2: Archive config + data volumes

⚠️ Offline environments: the one-shot container used for archiving/extraction must use an image bundled with the offline release package (see manifests/basic-images.list); do not use alpine, busybox, or other external images that require a network pull. This page uses postgres:16-alpine throughout - always present on machines that have deployed this stack.

# Prepare backup directory
BACKUP_DIR=/data/backups/finclaw-$(date +%Y%m%d-%H%M%S)
mkdir -p "$BACKUP_DIR"

# 1) Config: tar the key files in the deploy directory
tar -czf "$BACKUP_DIR/deploy-config.tar.gz" -C /data/chatkit-offline-release-1.1.20-linux-amd64/deploy .env config docker-compose.override.yml 2>/dev/null || true

# 2) Confirm the helper image exists (always present on machines that have deployed this stack; no output means OK)
docker image inspect postgres:16-alpine >/dev/null && echo "helper image OK"

# 3) Data volumes: use a one-shot container to archive each volume (tar preserves file ownership)
for vol in $(docker volume ls -q | grep '^chatkit_'); do
  echo ">> backing up volume: $vol"
  docker run --rm \
    -v "$vol":/source:ro \
    -v "$BACKUP_DIR":/backup \
    postgres:16-alpine \
    tar -czf "/backup/${vol}.tar.gz" -C /source .
done

# 4) Record volume list + version info
docker volume ls --filter "name=chatkit_" --format '{{.Name}}' > "$BACKUP_DIR/volumes.txt"
echo "release_dir=$(realpath /data/chatkit-offline-release-1.1.20-linux-amd64)" > "$BACKUP_DIR/meta.txt"
date -u +"backup_time=%Y-%m-%dT%H:%M:%SZ" >> "$BACKUP_DIR/meta.txt"

# 5) Verify and inspect the backup directory (archive count must match the volume list)
ls "$BACKUP_DIR"/chatkit_*.tar.gz | wc -l
cat "$BACKUP_DIR/volumes.txt" | wc -l
ls -lh "$BACKUP_DIR"
du -sh "$BACKUP_DIR"

A fresh deployment has 12 volumes; machines upgraded from older versions may retain a few retired legacy volumes (mounted by no container) - archiving them too is harmless.

Alternative (no container): you can also tar the volume directories directly on the host as root, without any image. First get the real path of the volume with docker volume inspect (independent of the Docker data-root setting), then archive the volume directory:

# Backup: resolve the volume path and archive it
tar -czf <archive> -C $(docker volume inspect -f '{{.Mountpoint}}' <volume>) .

# Restore (symmetric): create the volume first, then extract
docker volume create <volume>
tar -xzf <archive> -C $(docker volume inspect -f '{{.Mountpoint}}' <volume>)

The downside: requires host root permissions (/var/lib/docker is root-only).

Step 3: Start services

./scripts/fleet.sh up -d
./scripts/doctor.sh

Backup Layout Example

/data/backups/finclaw-20260625-030000/
├── deploy-config.tar.gz           # .env + config/ + override
├── chatkit_postgres_data.tar.gz
├── chatkit_timescaledb_data.tar.gz
├── chatkit_ai_infra_data.tar.gz
├── chatkit_workspaces.tar.gz
├── chatkit_skills_data.tar.gz
├── chatkit_persona_uploads.tar.gz
├── chatkit_identity_uploads.tar.gz
├── chatkit_doc_toolkit_cache.tar.gz
├── chatkit_nats_data.tar.gz
├── chatkit_redis_data.tar.gz
├── chatkit_restate_data.tar.gz
├── chatkit_tei_models.tar.gz
├── volumes.txt                    # volume name list
└── meta.txt                       # version and backup timestamp

3. Restore

Step 1: Stop services and clean old volumes

cd /data/chatkit-offline-release-1.1.20-linux-amd64/deploy   # Replace with your actual version
./scripts/fleet.sh down

# Delete all chatkit_* volumes (only execute after confirming you want to roll back to the backup!)
docker volume ls -q | grep '^chatkit_' | xargs -r docker volume rm

⚠️ This step loses any current un-backed-up data; confirm before executing.

Step 2: Restore configuration

BACKUP_DIR=/data/backups/finclaw-20260625-030000   # Replace with your backup directory

tar -xzf "$BACKUP_DIR/deploy-config.tar.gz" -C /data/chatkit-offline-release-1.1.20-linux-amd64/deploy

⚠️ .env and data volumes must be restored as a pair: USER_SECRETS_KEK (the user-secrets encryption key) is bound to the encrypted user data, and ADMIN_EMAILS decides which accounts keep admin privileges. Always use the .env that was backed up together with the data volumes; pairing an .env freshly generated by init-env.sh with old data volumes makes the encrypted user data undecryptable.

Step 3: Restore data volumes

# Restore each volume: create an empty volume first, then extract the tar.gz into it
# (one-shot container uses postgres:16-alpine - no network needed in offline environments)
for tarball in "$BACKUP_DIR"/chatkit_*.tar.gz; do
  vol=$(basename "$tarball" .tar.gz)
  echo "<< restoring volume: $vol"
  docker volume create "$vol" >/dev/null
  docker run --rm \
    -v "$vol":/target \
    -v "$BACKUP_DIR":/backup:ro \
    postgres:16-alpine \
    tar -xzf "/backup/${vol}.tar.gz" -C /target
done

# Check: the volume count must match the backup list
docker volume ls --filter "name=chatkit_" --format '{{.Name}}' | wc -l

Step 4: Start services + verify

./scripts/fleet.sh up -d
./scripts/doctor.sh
curl http://127.0.0.1:26100/health

Expected: doctor.sh returns all ✅ / /health returns 200 OK. If individual services are slow on first start, wait 2–5 minutes and run again. up -d automatically rebuilds the per-tenant claw containers after the platform starts; if they are not rebuilt, run ./scripts/reconcile-claw-deployments.sh --force manually.