Why do so many production AI use cases fail?
Your AI demos look brilliant, and POCs work great; then why do your AI projects fail in production?
The answer is simple: production data is messy, and the model doesn’t have enough context about your business and your tacit knowledge.
But the real answer is more nuanced. I discovered a pattern where my AI model kept repeating the same mistakes, and codifying the business context and design standards into two simple markdown files quickly made it more efficient and more accurate. You can implement this in your own messy production environment without hand-writing all the rules yourself; just ask your AI what’s missing.
Production data is messy
LLMs have come a long way and can achieve long-horizon tasks with minimal prompting. I set out to test this in data analytics, build an end-to-end data pipeline from a single prompt, and quickly observed that Data Engineering is not the same as Software Engineering. Data Engineering involves much more than standard algorithms and design patterns; pipelines must encode real-world business rules and operating procedures, and handle ambiguity around messy data, varying data formats, vendor format changes, and nuances in KPI interpretation that vary by department within the same org.
AI needs your tacit knowledge
During my initial runs, my model kept making the same mistakes over and over again. Telling it how the data behaved, what the business rule was, or how I would do things quickly set the model right, and that gave me the idea to codify those rules as CONVENTIONS.md and STANDARDS.md.
So I ran two experiments: one where I provided my agent with the basic required information, and another where I provided my model with the two documents mentioned above.
Run A, just a bare-bones repo
Run A is a bare-bones repo where I give it sample data, some basic requirements, and then ask it to build the source and staging models. It only has two files, AGENTS.md and README.md, two standard files, basic requirements, nothing more than that.
I tell Claude Code here to read AGENTS.md and then create the source and staging models only for the CRM accounts. I expected it to use a `UNION ALL`, but it didn’t. A seasoned data engineer would also introduce a SURROGATE KEY and a few other audit columns, which Claude Code did not.
Then there was the biggest miss. Storing history is a data warehouse’s cardinal principle; however, Claude Code overwrote old data with new incoming data, wiping out any history.
In all fairness, I didn't give my AI agent any ground rules or business knowledge, so that’s what I'll do in my next experiment.
In Run B, I specifically give it my business rules and design standards
The two experiments are almost identical, except the new repo has CONVENTIONS.md and STANDARDS.md.
CONVENTIONS has my business rules. If data arrives late, how late is it, etc. All the tacit knowledge is codified in conventions. STANDARDS.md codifies business standards, naming conventions, how we define a surrogate key, audit-column requirements, etc.
Last time the agent skipped the history entirely, but this time, after the standards were clearly conveyed, it knew this was a data warehouse; we’re getting daily snapshots; and if a row is missing, it means it’s deleted. It understood, as a data engineer would, that we are storing history; it did a diff operation, marked anything deleted as isDeleted, stored the full history, and added the audit columns, the surrogate key, pretty much everything that was missed in the previous experiment.
What are my takeaways from this experiment?
Demos and POCs run on a clean subset of data; however, your production environment is messy.
Your AI model doesn’t know your specific context or your business rules.
AI models are smart and trained on a wide corpus of code and best practices, but they tend to forget the specifics and predict the most probable outcome. They fill in the gap when they see it.
When you ask the model to generate code or fix a bug, it tends to produce generic results without specific context about your environment.
The model needs to know your specific business rules and standards; it needs an SOP for your business.
An AI model already knows its blind spots.
But how do I create these SOPs for my business because they are tribal knowledge?
Just ask the model what’s missing from your environment that the model can not determine without external input, because it knows its own blind spots.
List the facts about this business that you cannot determine from this repository.
How does the model already know its blind spots?
The model doesn’t really know its blind spots, not in the truest sense. The model was trained on tens of thousands of data engineering and ETL projects, standards, and best practices. Thus, it knows that in a typical data engineering project, artifacts like data dictionaries, ETL mapping documents, specifications and runbooks, schema descriptions, and data model docs exist. The model knows its Kimball and Inmon, understands grain, SCDs, and late-arriving data, and public style guides and handbooks.
So when we ask it what’s missing from my repo, it can compare against its own training distribution and tell us exactly what’s missing.
Why doesn’t it produce slop in the first place? Why wait until we actually ask it?
The AI model we use is generative and probabilistic, so when we ask it to build and it hits a gap, it naturally fills it in with the most probable answer. The model has no way to know a fact is missing unless we tell it what the facts or rules are.
But we are not just using a model; we are using an Agent with a harness, with security and guardrails. Why don’t they prevent the model from producing an incorrect answer?
None of these guardrails are about code correctness; the harness sets the blast radius and the agent loop if the build was green. No error message said the model discarded the entire history.
Preserving history is a cardinal rule of data warehousing. However, it’s not a universal software engineering or data-modeling rule, so a general-purpose agentic harness doesn’t necessarily implement that guardrail.
Unlike software code, data engineering code isn’t openly available, so the training data available for frontier models on data engineering is sparse and may not encompass every implicit rule we data engineers have learned over our careers.
However, the fix is simple. Just tell the model what you already know, the way you would tell a fellow data engineer who is new to your specific environment. Even better, give it the rules and standards and let it discover what’s missing.
The two-document format one for business rules and the other for design standards is completely my own invention in lieu of having to point the model to a bunch of Confluence pages with all the standards and rules, only for the model to gloss over them. This two-document setup distills all the SPOs for the model into a starting point, and it can still refer to other docs, with the two Markdown files as a guide.
Frustrating as it is, there is a name for this phenomenon: Knowledge Elicitation. Getting the tacit knowledge out of experts is a hard problem- what we call tribal knowledge- and it seems that the same problem exists with AI models too.
The model already knew all the facts; it just didn’t know they were applicable in this case. It under-elicited. Just asking the AI model what’s missing helps.
Everything in this post is in the repo: github.com/snudurupati/agentic-data-analytics. Clone it, start from ep03-baseline for the run without the two documents and ep03-baseline-context for the run with them, and paste the same prompts I used.
I opened this post complaining about demos and POCs, and then showed you a toy example myself. So next week we build the real thing. The full pipeline, ingestion all the way through to the data marts, end-to-end in a single shot. We’ll see how far CONVENTIONS.md and STANDARDS.md take us, and where they need to evolve. Come back and check this space.