Kafka Complete Guide - Core
From the core of Kafka to in-depth content on internal mechanisms, the course is structured so that even beginners can quickly reach the expert level through detailed theoretical explanations, hands-on practice, and practical Kafka application development practice.
This course is prepared for Intermediate Learners.
What you will learn!
Understanding Kafka's core architecture and key components
Understanding Kafka's internal core mechanisms
Practice operating Kafka with various Kafka commands based on CLI
Implement various functions of Producer and Consumer directly using Java
Understanding Kafka's internal mechanisms and key environment parameters through hands-on practice
Implementing a practical Kafka application using Producer and Consumer
Hands-on and understanding of replication and availability architecture in a multi-broker Kafka cluster
Practice and understand the startup mechanism of Zookeeper and Kafka and the leader election process of the controller
Monitoring Kafka with UI using Confluent Control Center
Serialization/Deserialization of custom objects
Distributed streaming platform, the trend is Kafka!
Learn it thoroughly, step by step, starting from the core.
For processing large amounts of data
Representative solution, Kafka!
Kafka, a messaging system, has become an indispensable choice for collecting large amounts of streaming data, building large-scale data pipelines, and building event-based microservice architectures (MSAs). As a result, many companies have been rushing to adopt Kafka in recent years, and the demand for specialized personnel is also continuously increasing.
Why Kafka?
great
Performance & Availability
forcible
Scalability
formidable
Open source ecosystem
distinguished
Mission Stability
Wide
User Community
rich
Online Resources
This course will not only provide a theoretical explanation of Kafka, but will also provide detailed explanations and practical training on the internal workings of core mechanisms and implementation of real-world Kafka application development to help you become a Kafka operation and development expert .
Of course the essentials
High level skills are OK.
'Kafka Complete Guide - Core'
Kafka
First time
Kafka skills
For those who want to improve
Theory and Practice
Anyone who wants to take care of everything
This course is designed for those who are learning Kafka for the first time, as well as those who are applying Kafka to their work but want to build a more solid foundation and improve their skills at a higher level.
- Core mechanisms of Kafka's major components
- Hands-on Kafka using CLI commands
- Implementing various functions of Producer and Consumer using Java-based Kafka client library
- Understanding and practicing high-level internal mechanisms of Kafka core elements
- Implementing a production-grade Kafka application
- Kafka Operation and Monitoring UI Practice Using Confluent Control Center
- Serialization/Deserialization of custom objects
It consists of contents such as etc., and you will be able to learn everything from the essential elements required for operating and developing Kafka to the more advanced core mechanisms through this lecture.
Core concepts and practice all in one
The course consists of 30% theory and 70% practice, and the curriculum is filled with various practical classes so that you can learn the theory more naturally through practice.
Most of the practical classes are conducted through live coding . If you follow the videos step by step, you will soon find yourself at a level where you can confidently handle Kafka from development to operation.
💡 Please note before taking the class!
- Kafka Connector and Kafka Streams/KSQ are not covered separately in this lecture. These contents will be released as a series of lectures in the future.
Unique features of this course
Check it out.
Intuitive understanding of core mechanisms through illustrations and animations
To help you intuitively understand the core mechanisms of Kafka Broker, Topic, Partition, Consumer, Replication, Zookeeper, etc., we have organized the theoretical lectures with numerous pictures and PPT animations.
Detailed explanation and hands-on practice of Kafka's internal deep learning mechanism & various environment parameters
In theory, theory and practice are the same. In practice, they are not.
In principle, theory and reality are the same, but in practice they are quite different. (Albert Einstein)
This lecture goes beyond the conceptual understanding of the main components of Kafka and explains in detail the essential elements of Kafka's internal mechanisms as well as more advanced mechanisms along with the core environment parameters of Broker, Producer, and Consumer at a level not seen in any other lectures so far.
In addition, the lectures are structured in the order of explaining the internal mechanism theory and then conducting practical training, so that you can more firmly master what you have learned in theory.
From basic to practical level, various Java-based practice codes are explained in detail line by line.
Most of the practical lectures are conducted through live coding. I will explain each line in detail for any code that may be a little difficult. You will be able to further improve your skills by following along with me to implement Kafka applications at a difficult level, beyond the basic Kafka Producer and Consumer client codes.
Practice Environment 💾
Server OS
We use Ubuntu Linux 20.04 on Oracle VirtualBox VM as the Kafka server OS. Although it uses Linux, it is run on a virtual machine basis, so it can be configured on both Windows/macOS environments.
VirtualBox can be installed in almost all Windows/macOS environments. However, in the case of Mac, VirtualBox is not installed in the latest M1 model, so you must install Ubuntu using a virtual environment such as UTM. For M1 models, please make sure that Ubuntu is installed in a virtual environment before selecting a lecture.
Confluent Kafka
Community Edition
Kafka uses Confluent Kafka Community Edition version 7.1.2, not Apache Kafka.
Confluent is a company founded by the core people who created Kafka, and provides enterprise Kafka that is more advanced in terms of performance and convenience for corporate customers. It is 100% compatible with Apache Kafka, but you can use more diverse Kafka modules and integrated binaries. Use the powerful distributed system Kafka in a more elastic and scalable form with Confluent. It will help you reduce the burden of infrastructure construction and maintenance, and help you develop faster.
Client Development
Kafka client development is based on Java and uses IntelliJ IDEA . Kafka provides client libraries based on various languages, but among them, Java-based clients are the first to be supported in terms of performance, stability, and implementation of the latest features.
Recommended PC Specifications
A full lab environment configuration may require a PC environment with 20-30 GB of storage capacity and 4 GB or more of RAM .
Check out the Q&A 💬
Q. Why should I learn Kafka?
Kafka has established itself as a representative solution for real-time, large-capacity Event Streams Platforms. Thousands of companies around the world use Kafka due to its high message processing performance and stability, support for various functions, and large-scale ecosystem and community. It is used for various purposes, from building data pipelines to microservices.
That's why many companies are currently seeing an increase in demand for people who can handle Kafka. This demand will continue to grow in the future, so learning Kafka will be a huge advantage for your career.
Q. Can I take the course even if I don’t know Kafka?
Yes, this course is structured step-by-step from basic to advanced content so that even beginners can follow along. If you slowly follow the many practical classes consisting of live coding, even beginners will soon be able to handle Kafka proficiently.
Q. Can I take the course if I don't have any Java development experience?
You don't need to know Kafka, but you need basic Java programming skills to understand Threads to perform the practical training. You can check the Java code used in the practical training on GitHub .
If you do not have basic Java programming experience , you may have difficulty with the practical training. Therefore, please review the practical Java code in advance before taking the class and make sure that it is appropriate for your level.
Q. Is prior knowledge of Linux and IntelliJ required?
Linux and IntelliJ are explained in detail from installation to environment configuration, so you don't need to know them, but it may be more helpful for the practice if you know them.
Q. Are Kafka Connect or Kafka Streams/KSQL also covered in the course?
This lecture is the first lecture in the Kafka series and covers the essential architecture of Kafka. The entire series consists of three lectures. The second lecture will be released sequentially on Kafka Connect and Schema Registry, and the third lecture will be released sequentially on Kafka Streams/KSQL. (The detailed lecture release schedule has not been set yet.)
Kafka series!! Great lectures to listen to together 📖
Recommended for
these people!
Who is this course right for?
For those who want to learn Kafka for the first time
Anyone who is using Kafka but wants to gain a more solid foundation of knowledge
Anyone who needs a solid understanding of Kafka's Advanced Internal mechanism and key environment parameters
For those who want to implement Producer and Consumer functions using Java
For those who want to develop Kafka applications closer to real-world situations
Data engineers considering adopting Kafka for large-scale data collection and data pipeline construction
Solution architects considering implementing a messaging system in their enterprise environment
Anyone interested in Kafka's distributed messaging system architecture
Need to know before starting?
You don't need to know Kafka, but you need basic Java programming knowledge to understand Threads.
Hello
This is 권 철민
Students
23,084
Reviews
1,060
Rating
4.9
Courses
12
(전) 엔코아 컨설팅
(전) 한국 오라클
AI 프리랜서 컨설턴트
파이썬 머신러닝 완벽 가이드 저자
Curriculum
All
149 lectures ∙ (22hr 53min)
are provided.