From prototype to production: why AI demos stall

The short version

• A prototype answers "is there something here?" Production asks whether it can be trusted, supported and measured.
• AI work stalls in four predictable places: ownership, workflow change, evaluation and success criteria.
• Before each step forward (prototype, pilot, production), ask four plain questions about ownership, measurement, failure and daily use.

Prototypes answer a useful question: is there something here? Production asks different ones. Can this be trusted? Can it integrate into real work? Can teams support it? Can we measure whether it creates durable value?

Moving AI from prototype to production is part of my day job at Al Jazeera, where I operationalize LLMs and open-source model stacks and bring AI prototypes into newsroom workflows. In my experience, the hard parts are rarely the model.

Where it stalls

Ambiguous ownership. Product, data and engineering each assume another team owns the outcome. Name one accountable owner for the workflow, and make clear what the other teams are responsible for.

Underestimated workflow change. A model that works in a notebook still has to fit into how people actually work: the tools they use, the approvals they need, the deadlines they face. The change to the workflow is often the larger task.

Weak evaluation and feedback loops. If you can't tell whether quality is improving or degrading, you can't operate the system. This is where structured evaluation and prompt governance earn their keep.

Success criteria that celebrate demos. "It impressed the leadership team" is not a measure. Choose an outcome tied to the work, such as less manual effort or better publishing efficiency.

What "production-ready" means

Beyond the model itself, I look for:

• Reliability: a known quality level, and a known fallback when it isn't met.
• Evaluation: a representative test set, reviewed by domain experts, re-run on every change.
• Governance: versioned prompts and models, clear access rules, and a record of what changed.
• Monitoring: quality, cost and latency are visible, with someone watching.
• Support: a named owner and a path for users to report problems.

Run it as a lifecycle

Production AI is an operating model, not a launch. The model lifecycle runs on: deploy, monitor, update, roll back. That is the MLOps side of the work, and it needs owners and routines like any other service. Decide early who handles model updates, who reviews quality, and how a change gets promoted.

Make adoption part of the plan

Even a reliable system fails if people don't use it. I plan adoption alongside the build: training, champions inside the teams, and a short feedback loop so users see their input change the product. In newsroom workflows, I aim for AI that reduces manual work and fits into how editors already publish, rather than asking them to change tools.

A simple promotion test

Before moving from prototype to pilot, and from pilot to production, I ask four questions:

1. Who owns it? One name, not a committee.
2. How do we measure it? One metric, with a baseline.
3. What happens when it fails? A fallback, and someone who will notice.
4. Who uses it on Monday? A real person, in real work.

If the answers are clear, the move is usually safe. If they're not, the gap is rarely technical, and it is the work worth doing next.