🧱 Free course
Foundations
Know how data moves through a company, and why the pieces are separate.
What you will learn
- What Data Engineering Actually Does
Apps create data. Data engineers make that data trustworthy, timely and usable at scale. - OLTP vs OLAP
One system answers 'what is true right now for this one ride?'. The other answers 'what happened across ten million rides?' - ETL vs ELT
Same three letters, different order — and the order decides what you can recover from. - Batch vs streaming — how to choose
Streaming is not the grown-up version of batch. It is a more expensive answer to a question most dashboards never ask: how stale can this number be before someone loses money? - File formats: CSV, JSON, Parquet, Avro
The same 1 crore rides can be a 9 GB CSV that takes four minutes to query, or a 900 MB Parquet file that answers in four seconds. The data didn't change. The layout did. - Compression and encoding: why Parquet is 10x smaller
Writing Shillong one crore times costs 80 MB as text. Parquet stores it in a few kilobytes — not by compressing harder, but by noticing it's the same word. - Data lake vs warehouse vs lakehouse
A warehouse is a restaurant: fast, clean, expensive, you eat what's on the menu. A lake is a wholesale market: cheap, anything goes, bring your own knives. The lakehouse is the attempt to get restaurant service at market prices. - Table formats: Delta, Iceberg, Hudi
A folder of Parquet files is not a table. It has no idea which files are current, who is writing, or what it looked like yesterday. A table format is the small ledger that answers all three.
How the course works
- Each chapter is a short book: the basics, the explanation in several ways, the words to know, and a two-minute summary.
- Then you test yourself — quiz, puzzles — and recall it on two different days, so it stays.
- Blinks (one idea per screen), games and a daily plan keep you coming back. It is free.