Install Guide
Set Up Apache Kafka Logger
Add a logger like below to receive the Apache Kafka messages.
- Kafka broker address list: IP: A comma-separated list of Kafka broker addresses in the format host:port, for example: host1:9092,host2:9093,host3:9094, for example: IP1:9092, IP2:9092, IP3:9092.
- Kafka topic: Kafka topic name
- Kafka group ID: Enter the Apache Kafka consumer group ID. Enter the same ID for each logger to balance the message load of the topic.
Logstash Integration
Below is an example of a Logstash configuration that reads input data and sends it to the Apache Kafka topic "test" using the plain codec, keeping the message in its original form:
/usr/share/logstash/bin/logstash -e 'input { stdin { } } output { kafka { codec => plain { format => "%{message}" } topic_id => "test" } }'
Below is an example of a Logstash configuration that reads input data and sends it to the Apache Kafka topic "test" using the json codec, formatting the message as JSON:
/usr/share/logstash/bin/logstash -e 'input { stdin { } } output { kafka { codec => json { format => "%{message}" } topic_id => "test" } }'
By loading messages into an Apache Kafka stream with the json codec and using the parsejson command, you can easily create a custom logger model to receive structured messages.
