25 Data Engineer Interview Questions & Pipeline Answers

Data engineering interviews have a reputation for being the most practical in the data family — and it's deserved. Nobody asks you to derive backpropagation. They ask how you'd move a hundred million rows a day reliably, what happens when a job fails halfway, and why your pipeline is slow. Reliability thinking, not algorithm trivia, is the whole game.

The core idea: A data engineer's product is trustworthy data. Interviews test whether you build pipelines that are correct (no duplicates, no silent loss), recoverable (rerunnable without disaster), and observable (you know when they break before your users do).

The question taxonomy

• SQL, but harder. Window functions, deduplication, incremental queries. The SQL guide is essential groundwork.
• Data modelling. Star schemas, slowly changing dimensions, denormalisation trade-offs, partitioning strategy.
• Pipeline design. Batch vs streaming, idempotency, backfills, late-arriving data.
• Distributed processing. Spark partitions and shuffles, why a job skews, broadcast joins.
• Operations. Orchestration, data quality checks, alerting, lineage.

Worked example: the idempotent backfill

"Your daily job failed on Tuesday. It's now Thursday. Walk me through the fix." The junior answer is "rerun it." The senior answer starts with a question: is the job idempotent? Then show what idempotency looks like:

-- Idempotent write: delete the partition, then rewrite it.
-- Rerunning for the same date never creates duplicates.
DELETE FROM events
WHERE event_date = '2026-08-25';

INSERT INTO events
SELECT *
FROM staging_events
WHERE event_date = '2026-08-25';

(Many warehouses express this as a MERGE or a partition swap — the principle is identical.) Then narrate the operational story: fix the root cause, backfill Tuesday and Wednesday in order, verify row counts against the source, and add a data-quality check so the next failure pages you instead of corrupting dashboards silently. That last sentence — and here's how we catch it next time — is what interviewers are waiting to hear.

How answers get scored

Rubrics reward failure-mode thinking above all: what breaks, how you detect it, how you recover without duplicates or gaps. In Spark questions, knowing that a shuffle is expensive — and that skewed keys cause one executor to do all the work — matters more than API trivia. In modelling questions, justifying your grain ("one row per what?") is the fastest credibility signal there is.

Common mistakes

• Designing pipelines that can't be rerun safely — append-only writes with no dedupe strategy.
• Ignoring late-arriving and out-of-order data until it corrupts aggregates in production.
• Choosing a partition key with high skew (like a status column with two values).
• Describing tools — Airflow, dbt, Spark — without being able to explain the design choices underneath them.

The modelling question: "design a schema for X"

Expect a modelling design round: "design tables for an e-commerce order history" or "model ride-sharing trips." Start with the grain — one row per what? Everything follows from that sentence. Then separate facts (events with measures: orders, clicks, payments) from dimensions (the who/what/where: customers, products, dates). Discuss slowly changing dimensions when attributes mutate: does the business need the customer's address at time of order or the current one? That single question marks experience. Finish with physical design: partition large fact tables by date, cluster or index by the most common filter column, and justify each choice in terms of the queries it serves. The rubric rewards candidates who design from query patterns backward rather than normal forms forward — analytics schemas exist to be read, and saying so frames every decision correctly.

FAQ

Spark or SQL — which matters more? Strong SQL first, always; it's assumed. Spark depth matters for roles processing large volumes — understand partitions, shuffles, and joins conceptually even if you mostly write SQL.

Is streaming required? For most roles, conceptual fluency is enough: event time vs processing time, watermarks, exactly-once semantics. Deep Kafka/Flink internals are for streaming-specialist positions.

Compare this path with the analytics-leaning data scientist guide, and drill your SQL with Aissence's coding copilot.