Manage Kafka Clusters Using Confluent Platform

• Overview
  • Related content
• Cluster Metadata Management
  • Overview
  • KRaft Overview
    • The controller quorum
    • Scaling Kafka with KRaft
  • Configure KRaft
    • Hardware and JVM requirements
    • Configuration options
    • Settings for other Kafka and Confluent Platform components
    • Generate and format IDs
    • Tools for debugging KRaft mode
    • Monitor KRaft
    • Related content
  • Find ZooKeeper Resources
    • Related content
• Manage Self-Balancing Clusters
  • Overview
    • How Self-Balancing simplifies Kafka operations
    • Self-Balancing vs. Auto Data Balancer
    • How it works
    • Configuration and monitoring
    • Replica placement and rack configurations
    • Security considerations
    • Troubleshooting
    • Related content
  • Tutorial: Adding and Remove Brokers
    • Configuring and starting controllers and brokers in KRaft mode
    • Prerequisites
    • Environment variables
    • Configure Kafka brokers
    • Start Confluent Platform, create topics, and generate test data
    • (Optional) Install and configure Confluent Control Center
    • Use the command line to test rebalancing
    • Use Control Center to test rebalancing
    • Shutdown and cleanup tasks
    • (Optional) Running the other components
    • Related content
  • Configure
    • Self-Balancing configurations on the brokers
    • Self-Balancing internal topics
    • Required Configurations for Control Center
    • Examples: Update broker configurations on the fly
    • Monitoring the balancer with kafka-rebalance-cluster
    • kafka-remove-brokers
    • Related content
  • Performance and Resource Usage
    • Add brokers to expand a small cluster with a high partition count
    • Test scalability of a large cluster with many partitions
    • Repeatedly bounce the controller
• Auto Data Balancing
  • Overview
  • Quick Start
    • Requirements and Limitations
    • Confluent Auto Data Balancer Quick Start
    • Licensing
    • Suggested Reading
  • Tutorial: Add and Remove Brokers
    • Installing and running Docker
    • Use Docker to set up a three Node Kafka cluster
    • Create a topic and generate data
    • Related content
  • Configure
    • confluent-rebalancer command flags
    • Using a command-config file to talk to Kafka
    • Specifying Auto Data Balancer properties in a config-file
    • Example: Run the rebalancer with Security, Metrics, and License Configurations
    • Example: Run the rebalancer with a Separate Metrics Cluster
    • ACLs for Auto Data Balancing
• Tiered Storage
  • Enabling Tiered Storage on a broker
    • AWS
    • GCS
    • Azure
    • Pure Storage FlashBlade
    • Nutanix Objects
    • NetApp Object Storage
    • Dell EMC ECS
    • MinIO
    • Hitachi Content Platform Object Storage
    • Cloudian HyperStore Object Storage
    • CEPH
  • Configuring Tiered Storage to support compacted topics
  • Creating a topic with Tiered Storage
  • Sending test messages to experiment with data storage
  • Best practices and recommendations
    • Tuning
    • Time interval for topic deletes
    • Log segment sizes
    • ACLs on Tiered Storage internal topics
    • TLS settings and troubleshooting certificates
  • Sizing brokers with Tiered Storage
    • Tier archiver metrics
    • Tier fetcher metrics
    • Kafka log of tier size per partition
  • Example performance test
  • Supported platforms and features
  • Known limitations
  • Configuration options
  • Disabling Tiered Storage
  • Related content