인프런 영문 브랜드 로고
인프런 영문 브랜드 로고
BEST

Kafka Complete Guide - ksqlDB

This course is designed to help you learn the use of ksqlDB and its core mechanisms through various hands-on exercises. After completing the course, you will be able to easily and quickly build a real-time streaming data analysis system based on Kafka.

(4.8) 16 reviews

331 students

Kafka
ksqlDB
Data Engineering

This course is prepared for Intermediate Learners.

What you will learn!

  • From KSQLDB's basic concepts to advanced architecture

  • Difference between Stream and Table, and Stateful Streaming Processing Mechanism

  • Information on the creation and management of KSQLDB's main objects, understanding of various data types

  • RocksDB operation mechanism in KSQLDB

  • Understanding various query syntax and functions of KSQLDB

  • Understanding and using Group by and Mview, and its specificity and limitations

  • Understanding and utilizing various types of KSQLDB joins, as well as their specificities and limitations

  • Understanding of various types of Windows and the operation mechanism of Time-based Window Aggregation and Window Join

  • Using Connect in KSQLDB

  • Integration of KSQLDB and Elasticsearch and visualization of analysis results through Kibana

A large-scale real-time streaming analytics system,
Easy and powerful with Kafka + ksqlDB!

Real-time streaming data analysis
Want to achieve efficiency and scalability?

If you are using Kafka , the easiest and fastest way to implement a large-scale real-time streaming analytics system is to use ksqlDB .

ksqlDB, which is installed and operated integrated with Kafka, can easily process/transform/analyze real-time streaming data with just a few lines of SQL code without using complex streaming APIs.

Excerpt from the official Confluent page (link)

Leading domestic and international companies are already facing the need to analyze streaming data in real time with incomparable large volumes and fast latency compared to the past and to immediately reflect the results, and are actively introducing ksqlDB to this end.

In the past, the complex Kafka Streams API was used to process/transform/analyze real-time streaming data based on Kafka, but now ksqlDB, which allows you to build a streaming analysis system easily and quickly with just simple queries, is becoming the trend for building real-time streaming data analysis systems.

ksqlDB is growing fast, and there is a shortage of experts

ksqlDB is rapidly replacing the existing Kafka Streams API due to its many advantages, including its easy and convenient SQL-based implementation.

However, it is very difficult to find field personnel with the skills to use ksqlDB in practical work . This is because ksqlDB is a solution that has emerged relatively recently, and most of the materials and lectures covering ksqlDB are composed of superficial concepts, which are insufficient for building the skills required in practical work.


Real-time, high-volume streaming data processing applications made easy and fast.

'Kafka Complete Guide - ksqlDB' is

This course is designed to be a practical course that will help you grow into a ksqlDB expert. Our goal is to help you grow into a ksqlDB expert that companies want .

Those who are repeatedly blocked by the high wall of ksqlDB

Anyone who wants to understand the core mechanism of ksqlDB

✅ Those who want to immediately use ksqlDB for work

Accordingly, this lecture is filled with the contents that students must acquire in order to utilize ksqlDB in practical work.

Unique features of this course
Check it out.

Learn the core operation mechanisms of key ksqlDB components through detailed visual aids and hands-on exercises.

ksqlDB has some similarities to general RDBMS, but there are many differences. Therefore, in order to handle ksqlDB well, you must have a detailed understanding of the operation mechanisms of the main components, such as Stream, Table, Query, Mview, and RocksDB. This lecture will help you learn the core mechanisms of ksqlDB through detailed visual materials and practice.

Various functions unique to ksqlDB, joins, Group by, Window utilization, and practical training for practical use

In order to effectively utilize ksqlDB in practice, you must be familiar with the various functions provided by ksqlDB, such as Join, Group by, and Window usage. In particular, ksqlDB's Join, Group by, and Window have different restrictions than SQL, and you cannot properly utilize ksqlDB without understanding them. In this lecture, you will be able to clearly understand the difference by directly performing these elements through many practical exercises.

In addition, through the separately provided 'Online Shoe Shop' practical training section, we will guide you to a level where you can efficiently apply various real-time analyses in ksqlDB in practice.

Using Connect in ksqlDB, and then linking with Elasticsearch and visualizing with Kibana

We will explain how to integrate and utilize Connect in ksqlDB. You will also learn through practice how to save the analysis results of ksqlDB to Elasticsearch through Connect and visualize them through Kibana.

We will guide you from beginner to expert level in ksqlDB.

⚙️

Core mechanisms of ksqlDB's main components

🔎

Differences between Stream and Table and how to use them, creation and management of major objects

🧰

Understanding various query syntax and main functions of ksqlDB through practice

📊

Understanding and utilizing ksqlDB's unique Group by, MView, and Join, differences and limitations from RDBMS

🔐

Understanding of various types of Windows and practicing the operation mechanism of Time-based Window Aggregation and Window Join

💾

Integration with Connect and collection and visualization of analysis results through Elasticsearch and Kibana

ksqlDB We have put a lot of effort into covering the core content and usage methods that cannot be found in any lectures or online materials so far, rather than just skimming the surface. We have also filled the curriculum with various practical classes so that you can learn the theory more naturally through practice.

After completing this course, you will find yourself becoming a ksqlDB expert who can compete with anyone else.

We've organized the content so that you can fully understand it with detailed explanations and various practical exercises.
(We provide students with a PDF of lecture materials of more than 100 pages.)

💡 Please note before taking the class!

  • This lecture only covers ksqlDB and does not cover Kafka Streams.
  • Virtualbox VM compatibility practice may be difficult on Macbook M1/M2 model PCs.

Practice environment

Server OS

We use Ubuntu Linux 20.04 on Oracle VirtualBox VM as the Kafka server OS. Although it uses Linux, it is run on a virtual machine basis, so it can be configured on both Windows/macOS environments.

VirtualBox can be installed in almost all Windows/macOS environments. However, in the case of Mac, VirtualBox is not installed in the latest M1 model, so you must install Ubuntu using a virtual environment such as UTM. For M1 models, please make sure that Ubuntu is installed in a virtual environment before selecting a lecture.

Confluent Kafka
Community Edition

Kafka uses Confluent Kafka Community Edition version 7.1.2, not Apache Kafka.

Confluent is a company founded by the core people who created Kafka, and provides enterprise Kafka that is more advanced in terms of performance and convenience for corporate customers. It is 100% compatible with Apache Kafka, but you can use more diverse Kafka modules and integrated binaries. Use the powerful distributed system Kafka in a more elastic and scalable form with Confluent. It will help you reduce the burden of infrastructure construction and maintenance, and help you develop faster.

Recommended PC Specifications

A full lab environment configuration may require a PC environment with 20-30 GB of storage capacity and 4 GB or more of RAM .


Check out the Q&A 💬

Q. Should I take the previous lecture, Kafka Complete Guide - Core or Connect?

Two sections of this lecture cover the integration of ksqlDB and Connect. Even if you have not taken the Complete Guide to Kafka - Connect, you should have a basic understanding of Connect and practical experience to understand the practical exercises in this section.

Understanding Kafka Core is essential. It is recommended that you take the previous lecture, Kafka Complete Guide - Core. However, even if you did not take the lecture, if you have experience using Kafka's basic Broker, Producer, and Consumer and have a good grasp of the core concepts, you can sufficiently take this lecture.

Q. Do I need to have RDBMS SQL experience to take this course?

Many of the exercises in this course are query-based, so you should have experience using basic RDBMS SQL syntax and Group by and Join.

Recommended for
these people!

Who is this course right for?

  • Anyone who wants to understand the main components of KSQLDB easily and deeply

  • Data engineer who wants to quickly and effectively build a large-scale real-time streaming data processing/transformation analysis system based on Kafka

  • Analysts and data scientists who need to leverage real-time streaming data analytics

  • Developers who want to migrate applications from existing Producer/Consumer-based or Kafka Streams-based to KSQLDB-based

Need to know before starting?

  • (It would be best if you took the Kafka Complete Guide - Core, but if not) you need a solid basic knowledge of topics/producers/consumers.

  • Basic knowledge of Kafka Connect

  • Many of the exercises are query-based. Basic SQL knowledge is required to understand joins and Group By.

Hello
This is dooleyz3525

25,162

Students

1,192

Reviews

3,920

Answers

4.9

Rating

13

Courses

(전) 엔코아 컨설팅

(전) 한국 오라클

AI 프리랜서 컨설턴트

파이썬 머신러닝 완벽 가이드 저자

Curriculum

All

139 lectures ∙ (21hr 31min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

16 reviews

4.8

16 reviews

  • logt님의 프로필 이미지
    logt

    Reviews 10

    Average Rating 5.0

    Edited

    5

    100% enrolled

    선생님이 마지막에 보너스로 ElasticSearch와 Kibana를 이용해서 그래프 만드는 것까지 모든 실습을 가르쳐 주셔서 Kafka 로드맵의 피날레를 장식한 것 같습니다. 직접 그래프를 눈으로 확인하니 뿌듯하네요. 좋은 강의 제공해주셔서 감사합니다.

    • 권 철민
      Instructor

      오, Kafka 로드맵 피날레 강의까지 다 들으셨다니, 축하드립니다. 좋은 수강평도 감사 드려요 ^^

  • 정진묵님의 프로필 이미지
    정진묵

    Reviews 1

    Average Rating 4.0

    4

    30% enrolled

    • jsggo2001님의 프로필 이미지
      jsggo2001

      Reviews 8

      Average Rating 5.0

      5

      100% enrolled

      정말 많은 도움이 되었습니다 카프카에 대해서 1도 몰랐는데 카프카 코어편부터 커넥트 ksqldb 까지 어디가서 카프카에 대한 업무를 해야할때나 주제가 나오면 주저 없이 참여할 수 있을것 같습니다 물론 반복을 통해서 진정 내것이 되도록 해야하겠지만 그래도 정말 많은 도움이 된것 같습니다.

      • 권 철민
        Instructor

        Kafka 강의 시리즈를 만든 보람을 느꼈습니다. 좋은 수강평 감사합니다 ^^

    • Matthew님의 프로필 이미지
      Matthew

      Reviews 9

      Average Rating 4.4

      5

      33% enrolled

      나만 듣고 싶은 강의👏👏👏 강의자료와 설명이 매우 좋습니다. ksqlDB만 들으려고 했는데, 다른 강의도 듣고 싶어졌어요.

      • 권 철민
        Instructor

        좋은 칭찬에 저도 기분이 좋아지는 군요. 훌륭한 수강평 감사드립니다.

    • sjhappy님의 프로필 이미지
      sjhappy

      Reviews 2

      Average Rating 5.0

      5

      6% enrolled

    dooleyz3525's other courses

    Check out other courses by the instructor!

    Similar courses

    Explore other courses in the same field!

    $77.00