Kafka from the Command Line

Get fluent with the Kafka command-line tools — create and inspect topics, produce and consume messages, track consumer-group lag, and tune topic configuration, all from the bundled kafka-*.sh scripts.

Every Kafka install ships with a set of small shell tools — kafka-topics.sh, kafka-console-producer.sh, kafka-console-consumer.sh, kafka-consumer-groups.sh, kafka-configs.sh. They are the fastest way to look inside a cluster, push a test message, or find out why a consumer is falling behind, and they are always there on the broker you just connected to. We start with an empty cluster and build real fluency one command at a time: creating a topic and reading its partition layout, producing JSON orders at an interactive prompt, consuming history versus only-new, tracking lag for a consumer group, and tuning retention. Each idea lands on its own command so nothing blurs together.

What you'll build

• Create, list, describe and delete topics
• Produce messages at the interactive console producer
• Consume from the beginning versus only new messages
• Use consumer groups and read their lag
• Tune topic configuration like retention
• Drive the tools non-interactively for scripts

Contents

1. Create a topic
2. List the topics
3. Describe a topic
4. Open the console producer
5. Send a few messages
6. Stop producing
7. Read the topic from the start
8. Read only new messages
9. Print keys alongside values
10. Take a fixed number of messages
11. Consume as a group
12. Describe the consumer group
13. List the consumer groups
14. Change a topic's retention
15. Read the topic's config
16. Add more partitions
17. Confirm the new partitions
18. Delete a topic
19. The shape of every Kafka command