Data Pipeline Review
Checks idempotency and late-arriving data, both of which are otherwise discovered in production.
Free — no card needed
Create a free account
to open Data Pipeline Review — and the other 364 prompts across 21 categories.
Already have an account?
Running it, start to finish
- Verify the pipeline is idempotent on rerun.
- Establish what happens to late-arriving data.
- Write the rerun runbook before you need it.
What you get back
The output this produces, every time.
- Checks whether a rerun duplicates data, which happens whenever a pipeline is retried.
- Establishes what happens to late-arriving records, where silent dropping is the common default.
- Verifies consumers can tell a partially-written output from a complete one.
Getting better results
Where this usually goes wrong, and how to avoid it.
- Prefer partition overwrite or upsert. Append-based writes are where rerun duplication comes from, and reruns are routine rather than exceptional.
- Fail loudly on schema change. Silently ignoring a renamed upstream column produces wrong output that looks entirely normal.
- Handle the zero-row case. A pipeline that succeeds on empty input and writes an empty output can wipe a downstream table.
More Data Analysis prompts
All Data Analysis →- Data Analysis
Segmentation Analysis
Produces segments that are actionable and stable, since statistically clean but unusable is the standard failure.
ClaudeChatGPT002Open → - Data Analysis
Metric Definition Review
Treats two teams reporting one metric differently as a governance problem rather than a query bug.
ClaudeChatGPT000Open → - Data Analysis
A/B Test Design and Reading
Designs a test that can actually detect the effect you care about, and states the decision rule before the data arrives.
ClaudeChatGPT000Open →
Written for The AI University. Every prompt in this library is original work — authored, tested and revised here, not collected from elsewhere. 365 of them, free with an account.