wtf is kafka-go?

kafka-go is a Go programming library for working with Kafka, a system used to pass large streams of messages between different parts of a software application. Think of Kafka as a high-throughput message queue: one part of your system sends messages in, other parts read them out. This library is the connector that Go programs use to talk to a Kafka server. The library was created by Segment because the existing Go options for Kafka had problems. The most popular one at the time had a confusing low-level API, was poorly documented, and did not support modern Go features. Another option required bundling a C library into your Go code, which complicates builds and deployment. kafka-go was written to avoid these issues by following the conventions of Go's standard library, making it feel familiar to Go developers. It offers two levels of control. The lower-level connection type lets you write and read messages directly, with full control over exactly how you connect and what you send. The higher-level reader and writer types handle more things automatically, including reconnecting if the connection drops and tracking where you left off in the message stream so you do not re-read or skip messages. The higher-level API also supports cancellation, meaning you can stop a read operation cleanly if your program needs to shut down. The library is tested against several versions of Kafka and requires Go 1.15 or later to use. It is written entirely in Go with no external C dependencies, which makes it straightforward to include in a Go project without special build steps. Segment built this for their own production infrastructure and released it publicly. It is a developer tool, not an end-user application.