Highlights
- Covers the full Kafka stack: producers, consumers, storage, security, Connect, and Streams
- Hands-on labs every module, including Avro producers/consumers and JDBC connectors
- Up to date with Kafka 4.0, including KRaft and the new consumer protocol
- Practical comparisons: Kafka vs. database, RabbitMQ, and Pulsar
- Covers reliability patterns including exactly-once processing and multi-cluster architectures
Course Details
Day 1 — Kafka Fundamentals and Client APIs
- Kafka overview: ecosystem, use cases, messaging concepts, topics, partitions, replication, and KRaft
- Producers: architecture, partition selection, keys and ordering, acknowledgements, retries, batching and compression
- Consumers: consumer groups, offsets, rebalancing, group coordinator, the new Kafka 4.0 consumer protocol
- Storage: retention, segment files, compaction, state vs. event topics, and GDPR considerations
- Lab: Create topics, produce and consume records, observe partition assignment, manage offsets
Day 1 — Kafka Architecture, Schema Registry and Security
- Kafka in context: when Kafka fits, anti-patterns, and comparisons with databases, RabbitMQ, Pulsar, and Kafka Queues
- Schema Registry: why schemas matter, Avro fundamentals, compatibility modes, and schema evolution
- Cluster architecture: ZooKeeper, controller election, KRaft architecture, and the KRaft migration process
- Security: TLS/SSL, SASL mechanisms, ACL authorization, and end-to-end encryption
- Lab: Avro producer/consumer and a schema evolution exercise
Day 2 — Reliability, Performance and Kafka Connect
- Reliability: replication internals, ISR, acknowledgement strategies, idempotent producers, transactions, and exactly-once processing
- Performance tuning: optimizing for throughput, latency, durability, and availability
- Kafka Connect: architecture, workers and tasks, converters, SMTs, JDBC connectors, and Debezium CDC
- REST Proxy: architecture, use cases, and trade-offs
- Lab: Build a JDBC source connector and a JDBC sink connector
Day 2 — Multi-Cluster Architectures, Monitoring and Stream Processing
- Multi-cluster Kafka: active-passive, active-active, stretched cluster, and RPO/RTO concepts
- Monitoring: broker, producer, and consumer metrics, JMX, Prometheus, Cruise Control, Kafka Monitor, and Trogdor
- Stream processing concepts: event-driven analytics, stateless and stateful transformations, event time, and windowing
- Kafka Streams: architecture, KStream, KTable, state stores, joins, aggregations, and scaling
Who should attend
This course is designed for developers, engineers, and architects who need to build, operate, or secure Kafka-based systems in production. Basic programming knowledge and familiarity with distributed systems concepts are recommended; no prior Kafka experience is required.
Feedback
4.8 out of 5 average
"Good introduction to Apache Spark. The trainer was great at talking us through the information, specifically optimisation methods. He spoke slowly and concisely which really got his points across. He effectively tailored the course to our specifications which we also appreciated."
RL, Financial Crime Technologist, Apache Spark, April 2021
“JBI did a great job of customizing their syllabus to suit our business needs and also bringing our team up to speed on the current best practices. Our teams varied widely in terms of experience and the Instructor handled this particularly well - very impressive”
Brian F, Team Lead, RBS, Data Analysis Course, 20 April 2022