Alauda Container Platform

High CPU Utilization in Kafka

TOC

Concurrency Control

Kafka uses a thread pool for concurrency control when processing messages. Proper configuration of the thread pool size can make full use of CPU resources and avoid CPU overload.

ParameterDescription
num.network.threadsThe number of threads for handling network requests, default value is 3. Properly setting this parameter can adjust Kafka's ability to handle network requests.
num.io.threadsThe number of threads for handling disk I/O, default value is 8. Properly setting this parameter can adjust Kafka's ability to handle disk I/O.

Message Batching

Kafka supports message batching, which combines multiple messages into batches to reduce network and disk I/O overhead. Batching significantly improves Kafka performance and reduces CPU utilization.

ParameterDescription
batch.sizeThe message batch size set on the producer side, in bytes, with a default value of 16384. Properly setting this parameter can improve Kafka's throughput.
linger.msThe message wait time set on the producer side, in milliseconds, with a default value of 0. Messages during this time will be merged into a batch.

Consumer Pull Strategy

Properly configuring the consumer pull strategy can reduce CPU load.

ParameterDescription
fetch.min.bytesThe minimum amount of data required when the consumer pulls data from the broker, with a default value of 1.
fetch.max.wait.msThe maximum wait time when the consumer pulls data from the broker, with a default value of 500.

Disable Message Compression

Message compression can reduce the space occupied by messages for storage and network transmission, as well as bandwidth consumption, but it also increases CPU load. If CPU resources are limited, consider disabling message compression to free up more CPU resources.

ParameterDescription
compression.typeOptional values include uncompressed, zstd, lz4, snappy, gzip, and producer.

Avoid Frequent GC

To ensure Kafka's performance and stability, attention should be paid to the JVM's GC (Garbage Collection) performance. Frequent GC can increase CPU load. Therefore, it is recommended to properly configure the JVM parameters of the Kafka broker to optimize the garbage collection strategy, improve performance, and reduce the frequency of GC.

spec:
   kafka:
     ...
     jvmOptions:
      '-Xms': 512m
      '-Xmx': 512m
     ...

Broker Scaling

To improve the cluster's throughput and availability, it is recommended to distribute newly created topics across new brokers to reduce the number of connections to a single broker.