Host-Related Metrics

1. CPU

• CPU Utilization: Monitor total CPU usage and per-core usage. High utilization may indicate excessive load.
• System CPU Usage: Indicates CPU time occupied by the operating system kernel. Sustained high levels may require attention to system processes and configuration.
• User CPU Usage: Indicates CPU time occupied by user processes, helping evaluate application CPU requirements.
• IO Wait: Time CPU waits for disk or network I/O operations to complete. High I/O wait may indicate disk or network bottlenecks.

2. Memory

• Memory Usage: Current memory consumption, helps identify memory usage patterns. High usage may lead to performance degradation.
• Available Memory: System remaining memory. Low available memory may indicate the system is approaching memory limits.
• Cached Memory: Memory used by the operating system as cache, helps improve access speed for files and block devices.

3. Disk

• Disk Usage: Disk partition usage status, helps monitor whether disk space is sufficient.
• Disk Read/Write Rate: Disk read and write bytes per second, indicates I/O activity level.
• Disk IO Wait Time: Time waiting for disk I/O operations to complete. High I/O wait time may indicate insufficient disk performance.
• Disk IO Operations: Number of read/write operations per second, helps identify I/O bottlenecks.

4. Network

• Network Throughput: Data transmitted per second, including inbound (received) and outbound (sent), helps understand network bandwidth usage.
• Network Latency: Data transmission time in the network. Increased latency may indicate network congestion or insufficient bandwidth.
• Packet Loss: Proportion of data packets lost during transmission. High packet loss usually indicates network problems.
• Network Connections: Number of active TCP/UDP connections, helps identify network connection usage.

5. Process

• Critical Process CPU and Memory Usage: Monitor CPU and memory usage of critical processes, promptly detect processes consuming abnormal resources.
• Process Uptime: Records continuous running time of processes, helps discover process restarts or abnormal termination.

K8S Related Metrics

1. Pod and Container

• container_cpu_usage_seconds_total: Container CPU usage time (seconds).

• container_memory_usage_bytes: Container memory usage (bytes).

• container_memory_working_set_bytes: Container working memory, excluding cache.

• container_fs_usage_bytes: Container disk space usage (bytes).

• container_last_seen: Last time the container was checked.

• container_start_time_seconds: Container startup timestamp (seconds).

• kube_pod_status_phase: Current Pod status (Running, Pending, Succeeded, Failed, Unknown).

• kube_pod_container_status_restarts_total: Number of Pod container restarts.

• kube_pod_container_status_running: Whether Pod container is running.

• kube_pod_container_status_terminated: Whether Pod container is terminated.

• kube_pod_container_status_waiting: Whether Pod container is in waiting state.

2. Scheduling and Resource Allocation

• kube_pod_container_resource_requests_cpu_cores: CPU resources requested by Pod container (cores).
• kube_pod_container_resource_requests_memory_bytes: Memory resources requested by Pod container (bytes).
• kube_pod_container_resource_limits_cpu_cores: CPU limit for Pod container (cores).
• kube_pod_container_resource_limits_memory_bytes: Memory limit for Pod container (bytes).
• kube_node_status_capacity_cpu_cores: Node total CPU capacity (cores).
• kube_node_status_capacity_memory_bytes: Node total memory capacity (bytes).
• kube_node_status_allocatable_cpu_cores: Node allocatable CPU capacity (cores).
• kube_node_status_allocatable_memory_bytes: Node allocatable memory capacity (bytes).

3. Controller and Replica

• kube_deployment_status_replicas: Current number of Deployment replicas.
• kube_deployment_status_replicas_updated: Number of updated Deployment replicas.
• kube_replica_set_status_replicas: Number of ReplicaSet replicas.
• kube_replica_set_status_replicas_available: Number of available ReplicaSet replicas.
• kube_statefulset_status_replicas: Number of StatefulSet replicas.
• kube_statefulset_status_replicas_available: Number of available StatefulSet replicas.

4. API Service

• apiserver_request_duration_seconds: API server request processing time (seconds).
• apiserver_request_count: Total number of requests processed by API server.
• apiserver_request_latencies_seconds: API server request latency distribution.
• controller_manager_reconcile_duration_seconds: Controller manager reconcile duration (seconds).
• controller_manager_reconcile_count: Controller manager reconcile count.

5. Network and Load

• kube_service_info: Kubernetes service metadata.
• kube_endpoint_address_count: Number of service addresses.
• kube_service_spec_type: Service type (ClusterIP, NodePort, LoadBalancer, etc.).

6. Storage

• kube_persistentvolumeclaim_resource_requests_storage_bytes: Storage space requested by persistent volume claim (bytes).
• kube_persistentvolumeclaim_resource_requests_storage_bytes: Storage space actually used by persistent volume claim (bytes).
• kube_persistentvolume_status_phase: Persistent volume status (Available, Bound, Released, Failed).

Redis Related Metrics

1. Resource Usage

• used_cpu_sys: CPU time used by Redis process in system (seconds).

• used_cpu_user: CPU time used by Redis process in user space (seconds).

• used_cpu_sys_children: CPU time used by all child processes in system (seconds).

• used_cpu_user_children: CPU time used by all child processes in user space (seconds).

• used_memory_rss: Memory allocated to Redis process by the operating system (RSS).

• used_memory_peak: Maximum memory usage of Redis instance (bytes).

• maxmemory: Maximum memory limit configured for Redis (bytes).

• mem_fragmentation_ratio: Memory fragmentation ratio. Calculated as used_memory_rss / used_memory, indicates memory waste.

• used_memory: Total memory used by Redis instance (bytes).

2. Key-Value Operations

• total_connections_received: Total number of connections received by Redis.
• total_commands_processed: Total number of commands processed by Redis.
• keyspace_hits: Number of cache query hits.
• keyspace_misses: Number of cache query misses.
• evicted_keys: Number of keys evicted (due to memory limit).
• expired_keys: Number of expired keys.
• persistent_connections: Number of persistent connections.

3. Persistence

• rdb_changes_since_last_save: Number of key changes since last RDB persistence.
• rdb_last_bgsave_status: Status of last RDB save operation (OK or ERROR).
• rdb_last_save_time: Timestamp of last RDB persistence.
• aof_current_size: Current size of AOF file (bytes).
• aof_last_bgrewrite_status: Status of last AOF rewrite.
• aof_last_write_status: Status of last AOF write operation.

4. Cluster and Replication

• role: Redis instance role (master/slave).
• connected_slaves: Number of currently connected slave nodes.
• master_last_io_seconds_ago: Last interaction time between master and slave nodes (seconds).
• replication_backlog_active: Whether replication backlog is enabled (1 for enabled, 0 for disabled).

5. Network

• total_net_input_bytes: Total bytes received by Redis.
• total_net_output_bytes: Total bytes sent by Redis.
• blocked_clients: Number of currently blocked clients.

6. Slow Query

• slowlog_len: Number of entries in Redis slow query log.
• slowlog_get (N): Retrieve N slow query log entries.

MySQL Related Metrics

1. Resource Usage

• Cpu_usage: CPU usage rate.
• Memory_usage: Memory usage rate.
• Disk_space_usage: Disk usage status.

2. Connections

• Threads_connected: Number of currently connected threads.
• Threads_running: Number of currently running threads.
• Threads_created: Number of threads created since startup.
• Max_used_connections: Maximum concurrent connections in MySQL history.
• Aborted_connects: Number of failed connection attempts.

3. Query Performance

• Queries: Queries per second.
• Slow_queries: Number of slow queries.
• Select_full_join: Number of queries performing full table joins.
• Select_range_check: Number of range queries used.
• Sort_merge_passes: Number of merge operations needed during sorting.
• Com_select, Com_insert, Com_update, Com_delete: Number of executions for SELECT, INSERT, UPDATE, DELETE commands respectively.

4. Cache Related Metrics

• Key_read_requests: Number of read requests from cache.
• Key_reads: Number of keys read from disk.
• Key_write_requests: Number of write requests to cache.
• Key_writes: Number of keys written to cache.
• InnoDB_buffer_pool_reads: Number of reads from InnoDB buffer pool.
• InnoDB_buffer_pool_pages_dirty: Number of dirty pages in buffer pool.

5. Replication Related Metrics

• Seconds_Behind_Master: Slave server lag behind master server (seconds).
• Slave_IO_Running: Status of replication IO thread.
• Slave_SQL_Running: Status of replication SQL thread.

6. InnoDB

• Innodb_buffer_pool_size: Size of InnoDB buffer pool.
• Innodb_buffer_pool_free: Unused space in buffer pool.
• Innodb_buffer_pool_wait_free: Number of waits for available memory.
• Innodb_row_lock_current_waits: Current number of row lock waits.

MinIO Related Metrics

1. Resource Usage

• minio_memory_used_bytes: Current memory usage (bytes).
• minio_cpu_usage: CPU usage rate.
• minio_cpu_load: CPU load (1-minute, 5-minute, 15-minute average load)

2. Requests and Throughput

• minio_api_requests_total: Total number of API requests.
• minio_api_request_duration_seconds: API request response time (seconds).
• minio_api_requests_rate: API request rate (requests per second).
• minio_api_errors_total: Total number of errors in API requests.
• minio_api_success_total: Total number of successfully processed API requests.
• minio_api_latency_seconds: API request latency time (seconds).

3. Network

• minio_network_received_bytes_total: Total bytes received from clients.
• minio_network_sent_bytes_total: Total bytes sent to clients.
• minio_network_error_total: Total number of network errors.
• minio_network_requests_total: Total number of network requests.

4. Errors and Exceptions

• minio_errors_total: Total number of errors.
• minio_error_rate: Error rate (errors per second).
• minio_object_errs: Number of errors in object storage.
• minio_request_errs: Number of errors in API requests.

5. Health and Status

• minio_health_check: MinIO instance health check status.

• minio_uptime_seconds: MinIO instance uptime (seconds).

• minio_heartbeat: Status for detecting node heartbeat.

• minio_watchdog_timeouts: Number of watchdog timeouts.

• minio_cluster_health: MinIO cluster health status (for cluster deployment).

• minio_cluster_nodes: Number of available nodes in cluster.