Operations Inspection

1. Overview

This manual performs finclaw system inspection via ansible and generates an HTML report, covering:

• Base services (PostgreSQL / TimescaleDB / Restate / Redis / TEI / NATS / MinIO)
• K8s cluster (RKE2 nodes, Pods, Deployments, PVCs)
• finclaw application layer (health of 21 microservices)
• Resource watermarks (CPU / memory / disk / network)

Applicable to clusters deployed via ansible.

2. Preparation

The inspection playbook shares the same source as the deployment playbook and ships with the deployment package; no separate download of check-master.tar.gz is required.

On the control node:

cd /data/finclip-installation/finclip-deploy-ansible-master

# Reuse the inventory from deployment (or create a new one)
# Skip if inventory already exists
ls inventory

3. Steps

cd /data/finclip-installation/finclip-deploy-ansible-master

# Run the inspection (reads .secret.yaml for each service password)
ansible-playbook check.yaml

When execution completes, no error messages means the inspection passed.

4. Inspection Coverage

Corresponds to 6 scripts under roles/check/files/:

Script	Coverage
check_linux.sh	Host layer: CPU / memory / disk / network / processes
check_docker.sh	Single-node mode: container status, Docker daemon
check_k8s.sh	K8s cluster: nodes, Pods, Deployments, StatefulSets, PVCs
check_mysql.sh	MySQL (legacy only; finclaw does not depend on MySQL; auto-skipped when the inventory has no mysql group)
check_redis.sh	Redis Cluster: node status, master-replica replication
check_minio.sh	MinIO: node status, buckets

5. Report Generation

Corresponds to 5 HTML templates under roles/check/templates/:

Template	Content
report-system.html	System resources (CPU / memory / disk / network)
report-docker.html	Container status
report-k8s.html	K8s cluster status
report-data.html	Data services (PG / Redis / MinIO)
report-cssinline.html	Merged report with inlined CSS

6. Viewing the Report

The report file is generated on the control node at /tmp/report-YYYY-MM-DD.html.

Download the file to your local computer and open it in a browser.

Items to Watch

• Server resource usage Normal indicator: ok Total = Total
• K8s cluster and service status Normal indicator: cluster is healthy, service status is normal, no restarted Pods compared with the previous inspection
• Data services
  • PostgreSQL master-replica sync is normal
  • All Restate cluster nodes are Member
  • Redis Cluster has 3 masters and 3 replicas
• finclaw application layer
  • 21 Deployments are Available
  • No CrashLoopBackOff

7. finclaw-Specific Inspection Add-ons

The ansible roles/check/ does not cover finclaw application-layer details; manually supplement:

Contract Version

curl -s http://<host>:26450/health | jq .contract_version

A non-unknown value is sufficient.

License Status

curl -s http://<host>:26101/health | jq .deployment_license

status being active is sufficient; expired means the license has expired (see expires_at for the expiry time). Under K8s, run kubectl -n <ns> port-forward svc/identity-service 26101:26101 first, then hit the local port.

LLM Connectivity

curl -s -H "Authorization: Bearer $LLM_API_KEY" $LLM_BASE_URL/v1/models | jq .

Returning a model list means it passes.

Channel Connectivity

Send a test message to each Feishu / WeCom / DingTalk bot.

Intranet Proxy (If Applicable)

# No 4xx / 5xx spike in proxy logs (replace <proxy-container> with the actual proxy container name)
docker logs <proxy-container> 2>&1 | tail -100 | grep -E "40[0-9]|50[0-9]"

8. Inspection Frequency Recommendations

Frequency	Content
Daily (every day)	ansible inspection + Grafana dashboards
Weekly	Supplement finclaw-specific items + Loki error log trends
Monthly	Backup/restore drill + pre-license-expiry reminders

9. Reference Documentation

• Routine Operations
• Service Health Checks