Kafka for people who write Go
An Apache Kafka course in Go — from the first topics and partitions to production-grade patterns: transactions and exactly-once, outbox, Debezium CDC, gRPC + Kafka hybrid. All code runs on a local sandbox (KRaft, Schema Registry, Connect, Kafka UI) and leaves an observable effect — these are working scenarios, not whiteboard lectures.
Course outline
- 01
Foundations
The Kafka base model: broker, controller, topic, partition, replication, offset, retention. The first producer and consumer on franz-go — after this module you have a working mental model that every other topic builds on.
not startedin progresscomplete0/66 lessons4 h 10 min - 02
Producer
Where the one-key-one-partition guarantee lives, how acks affects durability, why you need idempotency, how batching and compression shift throughput, and which error classes the producer sees on retries.
not startedin progresscomplete0/55 lessons3 h 15 min - 03
Consumer
Consumer groups and rebalances (including cooperative-sticky), offset commits, processing guarantees on the consumer side, error handling via retry topics and DLQ, concurrency and lag.
not startedin progresscomplete0/55 lessons3 h 30 min - 04
Reliability
Transactions and exactly-once semantics, the consume-process-produce pattern, transactional outbox, retry/DLQ deep dive, delivery to external systems (an HTTP courier with circuit breaker and backpressure via PauseFetchPartitions).
not startedin progresscomplete0/55 lessons3 h 55 min - 05
Contracts
Why schemas matter and a comparison of wire formats (JSON / Avro / Protobuf), Protobuf in Go via buf, Schema Registry with magic byte and schema_id, schema evolution (BACKWARD / FORWARD / FULL), and what counts as a breaking change in protobuf.
not startedin progresscomplete0/44 lessons2 h 45 min - 06
Communication patterns
For those who design systems: gRPC basics and streaming, sync vs async (a decision matrix), a gRPC + Kafka hybrid in one service, saga (choreography vs orchestration).
not startedin progresscomplete0/55 lessons3 h 50 min - 07
Streams and Connect
Stream processing concepts (event-time, windowing, watermark, late events), a stateful top-drinks counter on franz-go + Pebble (a Kafka Streams replacement for Go), Kafka Connect, and Debezium CDC.
not startedin progresscomplete0/44 lessons3 h 15 min - 08
Operations
Monitoring and metrics (kminion + Grafana), retention vs compaction in practice, topic sizing and tuning for a load profile, a troubleshooting runbook for common problems.
not startedin progresscomplete0/44 lessons2 h 55 min - 09
Use cases
End-to-end scenarios — they tie everything above into working applications with integration tests. Use cases are larger than lectures: several services, proto schemas, sometimes a docker-compose.override.yml for Postgres / ClickHouse / Elasticsearch.
not startedin progresscomplete0/44 lessons6 h