🛡 Quality, Testing & Ops · Chapter 8

Git for data teams: branching, PRs, review, CI/CD

In application code, a bad merge breaks a feature. In a data repo, a bad merge rewrites history — every table downstream recomputes with the wrong logic. Data teams need the same Git discipline plus one extra question on every PR: what does this do to the data that already exists?

Study this lesson free in the app →

The basics

Git for data teams is the workflow a team uses to change pipeline code safely together: short-lived branches, pull requests that are reviewed, and automated checks (CI) that build and test the change before it reaches production (CD).

Why it exists: five engineers editing the same dbt project and Airflow DAGs will overwrite each other, ship untested SQL and break dashboards. A shared workflow means every change is small, visible, tested on real data shapes and reversible.

What's different for data: code changes are cheap to revert, data changes aren't. A changed fare formula alters every row it rebuilds. So a data PR also needs: which tables change, how many rows differ from production, and whether a backfill is needed.

The mental model:

branch → commit small → open PR → CI builds changed models in a
throwaway schema + tests + data diff → review → merge to main
→ CD deploys to prod → (backfill if needed)

Words to know: feature branch, pull request, branch protection, CI, CD, slim CI, backfill, CODEOWNERS.

Explained

Friday 6 PM. A PR titled “small cleanup” changes fct_rides from JOIN payments to LEFT JOIN payments. 14 lines. Approved with “LGTM 👍” in 40 seconds.

Saturday's run: rides with two payment attempts now appear twice. Driver payouts on Monday are ₹2.8 lakh too high.

The code review looked at code. Nobody looked at what it did to the data. A CI data diff would have printed: fct_rides rows: prod 95,214 → PR 97,031 (+1.9%).

The shared recipe book

A restaurant chain has one master recipe book used by 40 kitchens.

A chef can't just cross out ‘2 tsp salt’ and write ‘2 tbsp’. He writes the change on a slip (branch), cooks a test batch in the test kitchen (CI), and the head chef tastes it next to the current dish (review with a data diff). Only then is the page replaced (merge), and the new page is sent to all kitchens (deploy).

And one extra question: what about the 300 jars of sauce already made with the old recipe? Throw them out and remake (backfill), or label them and let them run out? Software teams rarely ask that. Data teams must, every time.

Slim CI for a dbt repo (GitHub Actions)

# .github/workflows/ci.yml
on: pull_request
jobs:
  dbt-ci:
    runs-on: ubuntu-latest
    env:
      DBT_SCHEMA: pr_${{ github.event.number }}      # throwaway schema per PR
      SNOWFLAKE_PASSWORD: ${{ secrets.SNOWFLAKE_CI_PASSWORD }}
    steps:
      - uses: actions/checkout@v4
      - run: pip install dbt-snowflake sqlfluff
      - run: sqlfluff lint models/ --dialect snowflake
      - run: ./scripts/fetch_prod_manifest.sh prod_artifacts/   # last prod manifest.json
      - run: dbt deps
      - run: >
          dbt build --target ci
          --select state:modified+
          --defer --state prod_artifacts
      - run: python scripts/data_diff.py --schema $DBT_SCHEMA   # posts diff as PR comment

state:modified+ = models changed in this PR and everything downstream. --defer makes unchanged parents read from production instead of rebuilding them. A 400-model project builds 6 models in CI instead of 400.

What a reviewer actually checks on a data PR

Style is the linter's job. The human checks:

• Grain — is it still one row per ride? Any new join that could fan out?
• Join type — INNER silently drops rows with no match; LEFT keeps them with nulls. Which is intended?
• Nulls — WHERE city != 'Tura' also drops null cities. Intended?
• Incremental logic — does the filter miss late-arriving rows? Is unique_key set so reruns merge?
• Data diff — rows, distinct keys, sum(fare_inr) vs prod. Is the change the size the author claims?
• Cost — a new cross join or a full scan of a 2 B-row table every hour?
• Backfill and consumers — historic rows need rebuilding? Who must be told? Renamed columns kept as aliases?
• Tests — did the PR add tests for the new behaviour?

A good PR description answers all of these before the reviewer asks.

The merge conflict that wasn't a conflict

Asha's branch (5 days old) adds a surge_multiplier column to fct_rides. Meanwhile Rohan merged a change renaming fare to fare_inr in the same model.

Git merges cleanly — different lines. CI on Asha's PR fails: her new line computes fare * surge_multiplier, and fare no longer exists.

Two lessons:

1. Git conflicts are textual; data conflicts are semantic. Only CI that actually builds the model catches the second kind.
2. Long-lived branches are the cause. Rebasing on main daily (git fetch && git rebase origin/main) would have shown it on day 1, not day 5.

Gotchas

• CI that only lints. SQL can be beautifully formatted and still double every ride. Build and test on real engine.
• Building every model in CI. 90 minutes per push; people stop waiting for it. Use slim CI.
• Shared CI schema. Two PRs overwrite each other's tables and each other's test results. One schema per PR, dropped on close.
• ‘LGTM’ reviews. Approve only after reading the data diff and the backfill plan.
• Merging = backfilling. Deploying new logic changes only future runs. Historic partitions keep the old logic until you deliberately backfill — so for a while the table mixes both.
• Secrets in profiles.yml. Once pushed, it's in history forever; rotate the key, don't just delete the file.
• Friday-evening merges to money models. No one's around when Saturday's run breaks.

Small branch, CI that builds what changed, a review that reads the data diff, and a backfill plan.

Code is cheap to revert. Data isn't — so every PR answers: what happens to the rows that already exist?

How to answer “what does your team's CI/CD look like?”

“Trunk-based with short-lived branches; main is protected, so every change is a PR with green CI and an approval, and CODEOWNERS routes finance models to the finance-data reviewers. CI lints SQL with sqlfluff, then runs dbt build on state:modified-plus with defer, in a schema per PR, so we only build changed models and their children. A script posts a data diff to the PR — row counts, distinct keys and fare totals against prod. On merge, CD deploys to production. Anything that changes history ships with a backfill plan in the PR description, and we run the backfill as a separate step.”

Follow-up: “How do you avoid CI being slow?” → slim CI with state selection and defer, plus where windows on tests.

Words to know

In two minutes

• Use short-lived feature branches off main; merge within a day or two to avoid painful conflicts.
• Protect main: no direct pushes, PRs need a passing CI run and at least one approving review.
• CI for a data repo lints SQL, builds only the changed models and their children in a per-PR schema, and runs their tests.
• Slim CI (dbt state:modified+ with --defer) keeps CI minutes small by reusing production for unchanged parents.
• A data diff — row counts and key metrics, PR build vs production — is the most useful thing a reviewer can see.
• Reviewers check grain, joins, nulls, incremental logic, cost and the backfill plan, not just style.
• CD deploys main to production automatically; schema-changing PRs ship with a migration or backfill plan.
• Secrets and credentials never go in the repo; use the CI's secret store and environment variables.

Test yourself

There are 8 questions, 3 puzzles and 5 recall cards for this lesson in the app — the recall cards come back on later days, just before you would forget.