Data Pipelines
Streaming Data Mesh Lab
Backpressure-aware ingestion, schema contracts, and incremental feature stores for teams shipping near-real-time ML features.
- Duration
- 5 weeks · online
- Format
- Live labs
- Level
- Advanced
- Tuition (informational)
- ₩1,250,000
Program narrative
We build a reference mesh with bounded queues, dead-letter handling, and schema evolution playbooks. Labs stress failure injection so you can narrate what happens when a broker partition pauses mid-day.
What is included
- · Contract testing for Avro/JSON schemas with CI snippets
- · Lag dashboards that separate consumer drift from upstream stalls
- · Feature materialization windows with explicit freshness SLOs
- · Replay harness for debugging ordering assumptions
- · Cost guardrails worksheet for bursty traffic
- · Pair reviews on your own topic definitions
- · Lightweight governance checklist for cross-team ownership
Outcomes you can demo
- · Stand up a replayable ingestion branch with documented RPO/RTO assumptions
- · Draft a schema migration note your data platform team can approve
- · Facilitate a blameless review using the provided incident template
Mentor of record
Sora Kwon
Built event pipelines for mobility telemetry; teaches with an emphasis on operable documentation.
Participant questions
Will we touch managed cloud services?
Yes, with vendor-agnostic patterns called out. You may mirror exercises in your employer cloud with instructor guidance.
Is there certification?
You receive a completion certificate describing projects submitted. It is not an industry credential exam.
Limitations?
We do not tune hardware-level NIC offload or bespoke kernel modules. Those belong to vendor-specific advanced courses.
Recent participant notes
“The replay harness in Streaming Data Mesh Lab mirrored a production bug we had filed as “ghost latency.” Finally reproducible.”
“Clearer than internal docs. Still wish we had 30 more minutes on Iceberg compaction trade-offs.”
“Client in fintech — cohort policies on PII were respected; templates were easy to scrub for our compliance pack.”