Real-Time Streaming Data Pipelines (Kafka, Flink & Spark Streaming)

Real-Time Streaming Data Pipelines (Kafka, Flink & Spark Streaming)

Real-Time Streaming Data Pipelines (Kafka, Flink & Spark Streaming)

Category: Data Engineering

Instructor: Perizona Industry Trainer

Lessons: 56

Price: $23000

Start Free Trial

Course Overview

Process event streams with Kafka and compare real-time transformations using Spark Streaming and Flink.
1. Who can join
    Suitable for: Data engineers, platform developers and distributed-systems learners
    Mandatory prerequisites: Python or Java, SQL, Docker and basic distributed-data concepts
2. Duration
    56 live hours • Approximately 10 weeks at 6 hours per week; actual timetable to be confirmed.
3. Mode
     Classroom - Thaiyur / Kelambakkam, OMR Chennai (subject to schedule)
     Live Online (subject to schedule)


4. Syllabus — five modules


Module 1: Event-Driven Architecture: Batch vs. Real-Time Stream Processing
                  Learning outcome: Explain event streams, latency and delivery guarantees.
Module 2: Apache Kafka Fundamentals: Producers, Consumers, Topics & Schema Registry
                  Learning outcome: Create Kafka producers, consumers and topics.
Module 3: Real-Time Stream Analytics using Structured Spark Streaming
                  Learning outcome: Transform streaming records using Spark Structured Streaming.
Module 4: Stateful Event Processing with Apache Flink
                  Learning outcome: Demonstrate stateful processing and windows using Flink.
Module 5: End-to-End Stream Ingestion into Real-Time Dashboards ; Storage
                  Learning outcome: Deliver an end-to-end alert stream and evaluate failure recovery.
5. Practical project
     Source capstone title: Build a Real-Time Financial Fraud Detection Pipeline using Kafka & Spark
Student deliverable: A live synthetic fraud-event pipeline publishing events to Kafka, processing flags and
displaying aggregate alerts in a demonstration dashboard.

6. What is included
     Live trainer-led sessions
     Practical assignments
     Sample code or templates, as applicable
     Project feedback
     Learning assessment
     Perizona course-completion certificate subject to assessment
7. Upcoming batch
 Start date: To be announced. Students can register interest; dates are shared only after confirmation.

go to top