◐ ai-teacher

🐍 Free course

Python for Data Engineers

Write Python that survives 50 million rows, a flaky API and a code review.

Start the course free →

What you will learn

  1. Values, names and types
    A variable is not a box you put a value in. It is a label you stick on a value. Everything else follows from that.
  2. Control flow: if, for, while
    Three shapes — ask a question, repeat over things, repeat until something changes. Every program you will ever write is these three arranged.
  3. Functions: the unit of everything
    A function is a name you give to a piece of thinking, so you never have to think it again.
  4. Lists, dicts, sets, tuples — choosing the right one
    Four containers. Pick by the question you'll ask of the data — order, lookup, membership, or 'never change this'.
  5. Errors, files and imports
    Read the last line of the traceback first. Then read the line above it. That habit alone is worth a month of guessing.
  6. Idiomatic Python for pipelines
    The Python that passes code review is a small dialect. Learn the dozen idioms that make up 90% of pipeline code.
  7. Generators and memory — 50M rows without dying
    The difference between a pipeline that runs and one that gets OOM-killed is usually one character: [ versus (.
  8. Files, APIs, retries and backoff
    Every ingestion job is a negotiation with something unreliable. The code that handles the failure is bigger than the code that handles success.
  9. Concurrency: threads, processes, asyncio
    Ask one question — is this job waiting, or is it computing? — and the right concurrency model falls out immediately.
  10. Python: validating events at the gate
    Every pipeline has a front door. Validation is the lock — and where the rejected records go matters more than the check itself.
  11. Pandas: the 15 operations that cover 90%
    Pandas is a small language hiding inside Python. Learn its fifteen verbs and stop writing loops.
  12. Pandas to PySpark: what changes and what breaks
    The API looks familiar on purpose. The execution model underneath is nothing like Pandas, and that gap is where jobs die.
  13. Testing pipelines with pytest
    Application bugs return a 500. Data bugs return a plausible wrong number that nobody questions for three weeks.
  14. Packaging: venv, dependencies, Docker
    "It works on my machine" is not a joke — it is a description of an unpinned dependency tree.

How the course works