The Data Foundation for AI Checklist

Most AI projects don't fail at the model. They fail at the data underneath. This checklist is the same one we run in every paid AI Enablement assessment — twenty-two questions across the six layers that decide whether RAG, copilots, and agentic workflows ship to production or quietly stall.

How to use it

Score every question 1–5. Total each section, then convert to a percentage of the maximum.

• 80%+ — green. The dimension is unlikely to block your AI roadmap.
• 60–79% — yellow. Fix this in parallel with the AI build, not after.
• Under 60% — red. Don't start the AI build until this is addressed; you will hit it in week six.

If any single section lands in the red, the whole project is at risk. AI is unforgiving about weak foundations because the model is the thing that surfaces every inconsistency in your data, in front of the people who care most.

Section 1 — Identity & Entity Resolution

The model is only as confident as the entities you can hand it. If "one customer" means three records in three systems, every downstream answer inherits that ambiguity.

1. Can you produce a single, durable identifier for each customer / vendor / product across every source system you intend to feed into the model?
2. Do you have documented matching, merging, and survivorship rules — and are they versioned in code, not in someone's notebook?
3. When a duplicate or merge happens upstream, does the resolved entity propagate to the warehouse / lakehouse on a defined SLA?
4. Can a non-engineer (a product owner, a compliance lead) inspect why two records were merged or kept separate?

Section 2 — Lineage & Trust

Every model output a customer or executive sees needs a defensible answer to "where did this come from?" Without lineage, the first wrong answer ends the project.

1. For your top 20 reports / metrics, can you produce column-level lineage from raw source to consumed value?
2. Is freshness tracked and surfaced — i.e., can the model (or its prompt) know that a number is N hours stale?
3. When a transformation changes, are downstream consumers notified before the change ships?
4. Do you have automated data tests (row counts, referential integrity, null thresholds) running on every pipeline run, with failures alerted?

Section 3 — Semantic Layer & Definitions

If "active customer" means three different things in three tools, the copilot will tell three different stories. AI raises the cost of disagreement on definitions from "annoying" to "trust-destroying."

1. Is there a single, governed definition for your top 20 business metrics that BI tools, notebooks, and AI tools all read from?
2. Are dimensional definitions ("active," "churned," "deal stage," "qualified lead") owned by a named human with a documented change process?
3. Can a new AI feature consume those definitions without re-implementing the SQL?

Section 4 — Document & Unstructured Data Readiness

RAG over a SharePoint dump is not RAG. It's a search engine with a chatbot taped to it, and it will hallucinate confidently. (We go deeper on what production retrieval-augmented generation and document automation actually require in the deep-dive pages.)

1. Do you have a defined corpus per use case — i.e., a known, bounded set of documents the model should retrieve from, not "everything we've ever stored"?
2. Are documents extracted to clean text with sensible chunking, version control, and a record of which version was retrieved at inference time?
3. Is each document tagged with the metadata the model needs to filter intelligently — effective date, owner, jurisdiction, document type, supersession status?
4. Are superseded documents actually marked superseded, or do they still surface in retrieval?

Section 5 — Vector & Retrieval Infrastructure

The retrieval layer is the model. Most "the LLM hallucinates" complaints are actually retrieval problems. The platform-specific tradeoffs — Snowflake Cortex Search vs Databricks Vector Search and Mosaic AI vs Pinecone or pgvector — are covered in the platform deep-dives.

1. Do you have a chosen vector store with documented embedding model(s), and a plan for re-embedding when models change?
2. Are retrieval results evaluated against a labeled gold set on a regular cadence — not just at launch?
3. Is the retrieval pipeline observable — can you replay any production query and see which chunks came back, with what scores?

Section 6 — Governance, PII & Access

The week-six demo dies the moment somebody realizes the model can answer questions about a customer who has invoked their right to deletion.

1. Do you have a current, accurate inventory of where PII / PHI / regulated data lives across the systems the model will touch?
2. Are access controls on training, embedding, and retrieval pipelines aligned with the access controls on the source systems — i.e., the model cannot answer questions a user wouldn't be allowed to ask of the database directly?
3. Do you have a documented process for honoring deletion / right-to-be-forgotten requests across embeddings, fine-tuned weights, and retrieval indexes — not just the source tables?
4. Is there a named accountable executive for AI risk, with a budget for evaluation, monitoring, and red-teaming that does not get raided when other priorities arrive?

What to do with the score

Anything in the red, fix first. Anything in the yellow, fix in parallel. Anything in the green, you can build on.

If you want a second pair of eyes on your scores — or you want us to actually run this assessment alongside your team — book a call. We've used some version of this checklist on every AI Enablement engagement since 2024, and the scoring almost always tells us something the kickoff conversation didn't. (For predictive ML rather than GenAI, the parallel write-up is the ML productionization deep-dive.)

Want this as a printable PDF? Drop your email above and we'll send it.