A closed-loop AI system checks its own output against a target and corrects itself, without waiting for a person to notice something went wrong. An open-loop system acts, and finds out weeks later whether it worked.
Almost everything sold as "autonomous AI" today is open-loop. That gap explains most of the failure rate.
The numbers are worth stating plainly. MIT's NANDA research, built on 150 interviews, a 350-employee survey and 300 public deployments, found that 95% of generative AI pilots deliver no measurable financial return. About 12% reach production at all. In a March 2026 survey of 650 enterprise technology leaders, only 14% had scaled an agent organisation-wide, and Gartner expects more than 40% of agentic AI projects to be cancelled by 2027.
Those figures usually get read as evidence that the models aren't ready. The models are the part that works. What fails is everything around them: nothing watches the output, nothing records what happened, and nothing changes as a result.
Open loop, closed loop
Y Combinator's Diana Hu has given this idea the vocabulary it was missing. Her argument is that the best AI-native companies have made the whole organisation queryable. Every meeting recorded, every ticket tracked, every customer interaction captured, so an intelligence layer has something to learn from. The company stops being a series of decisions reviewed weeks later and becomes a system that watches, compares against targets, and adjusts as it goes.
Her companion slogan is "burn tokens, not headcount". An uncomfortably high API bill is cheap measured against the coordination work it absorbs, and most of that work is what middle management does now.
The part people skip is the caveat she attaches. Building this takes extensive integration work, stitching together Slack, Linear, GitHub, Notion, call recordings and a great deal of custom glue. The concept spreads for free. The plumbing does not, and the plumbing is the job.
A closed loop needs five things. Miss one and it collapses back to open:
- Sense. The system can observe the state it is meant to act on.
- Decide. It picks an action against an explicit target.
- Act. It executes with nobody in the path.
- Record. The action leaves a durable artifact, not a log line.
- Adjust. That record changes the next decision.
Most AI automation projects implement two and three, skip four, and ship. What they have built is an open loop with a language model inside it.
Step four gets cut first because it is the least interesting to build. Without a durable record there is nothing to learn from, nothing to audit, and, as we found out, nothing stopping the system doing the same thing twice.
A worked example
Architecture diagrams are easy to agree with and impossible to check. So here is a real closed loop, running in production, described precisely enough to argue with.
We run a system that publishes articles to customer websites every day with nobody in the path. Its loop:
- Sense. Read the active content beats for every customer, and the ledger of what has already published this month.
- Decide. Pick a topic nobody has used, check the monthly quota, check the destination site is reachable.
- Act. Research, draft, score the draft against the brief, revise, publish through the CMS.
- Record. Write one row per beat per day: published, held, failed or skipped, with the reason and the cost.
- Adjust. Tomorrow's selection reads that ledger. A spent keyword is never reused. A held article does not consume the month's quota.
Two decisions in there carry the weight, and both concern step four rather than the AI.
The ledger has a uniqueness constraint: one row per beat per day, enforced by the database rather than by application code. That constraint is what makes the loop safe to retry. A cron that fires twice, an overlapping run, a container restart mid-flight, none of them can publish the same article to a customer's live site twice. Double-publishing is the worst outcome available in this domain, considerably worse than missing a day, and the thing standing between us and it is a database index rather than a prompt.
The quality gate is a publish condition rather than a metric. The system scores every draft against the brief it was written from. Below the threshold, or with any critical brief component unmet, the article is held: stored as a draft, never published, reason recorded. Most content tools generate, ship, and let the customer find the problem.
That second decision separates a loop that improves from one that decays. A system that publishes whatever it produces generates the signal "we published 30 articles", which is true, useless, and entirely compatible with 30 bad articles. A system that refuses below a bar generates "we published 22 and held 8, here is why". The second is actionable, and it turns the held articles into a training set rather than an embarrassment.
The three ways it failed, all of them silently
We built that system, tested it, and shipped it. It then failed three times in production. No errors. No alerts. No log line indicating a problem. Two of the three reported success.
This is the part that generalises.
The system had nothing to write about
The component that proposes content topics inserted its recommendations with empty keyword arrays, a placeholder deferred to a configuration step nobody ever built. Nothing downstream filled them in. Accepting a recommendation copied it verbatim; activating it only changed a status field.
So every topic reached the scheduler with no subject attached. The scheduler looked at it, found nothing to write about, and skipped it. It did that correctly, and without complaint, every single day. We checked against production and found all four topics empty, including all three active ones. The publishing pipeline was structurally complete and had never been capable of producing a single article.
No monitor would have caught this. Every component did exactly what it was written to do.
The ledger did not exist
We deployed the code that writes to the run ledger before the migration that creates the table.
The ordering mistake is ordinary. What the absence did to the loop is the interesting part. Both safety guards read that table. "Has this topic already run today" queries it. "How many have published this month" queries it. When the table is unreadable both queries come back empty, and both guards answer go ahead, on every single run.
A missing safety mechanism did not cause the system to stop. It caused the system to publish repeatedly to live customer sites. The guarantee we had designed so carefully inverted into the exact failure it existed to prevent, and it would have done that while reporting success.
The fix was to make the loop refuse to run at all when it cannot read its own ledger. Publishing nothing beats publishing without deduplication, and a preflight check now enforces that rather than leaving it to assumption.
Abandoned checkouts granted full access
Different system, same disease. Our billing webhook mapped every subscription status that was not explicitly active or past_due onto active. The payment provider creates subscriptions in an incomplete state before payment clears, so every signup passed through that branch. Anyone who opened the payment form and walked away was provisioned a full paid plan with a month of credits.
Cancelled and unpaid subscriptions landed in the same branch. A cancelled customer read as active until a separate event happened to arrive.
Again, no error. Every function returned successfully. The system was confidently wrong and completely quiet about it.
What the three have in common
None of these were model failures. No hallucination, no bad prompt, no capability limit. All three were loops that looked closed and were open, and all three reported success while delivering nothing, or in the second case while doing active harm.
That is the failure signature of AI systems running without supervision, and it is why the 95% number is not really a statement about AI quality. An open-loop system with a language model in it produces confident output nobody checks. It will run for months looking healthy.
The uncomfortable part follows: you cannot find these by monitoring for errors, because they generate none. You find them by making the system prove it did the work, and by checking that proof against production data rather than against what the code intended. We found all three by querying the live database and comparing what was actually there against what the code assumed.
Five questions for your own system
If you are running something described as autonomous, these separate a closed loop from an open one.
What durable artifact does each run leave behind? If the answer is "logs", it is open-loop. Logs are for a human reading after a failure. A closed loop needs a record its own next decision reads.
What happens if it runs twice? If that depends on timing, there is no idempotency guarantee, and the database rather than the application is where that guarantee belongs.
What does it do when it cannot verify its own state? Carrying on is nearly always wrong. A system that cannot check whether it already acted should refuse to act.
Can it decline? Something with no ability to hold or reject its own output has a volume signal and no quality signal.
When nothing happened, can it say why? "It ran and produced nothing" and "it ran, produced something, and rejected it" are different states. A system that cannot tell them apart cannot be improved.
The last one is the tell. All three failures above stayed invisible for the same reason: from outside, working correctly and doing nothing looked identical to broken and doing nothing. Each fix had less to do with correcting logic than with making the system's silence legible.
Closed-loop systems are less an AI problem than a recording problem, an idempotency problem and a refusal problem. Three pieces of unglamorous engineering decide whether the model in the middle is worth anything.
We build closed-loop systems for companies that already tried AI once and have a stalled pilot to show for it. If any of those five questions landed badly, that is usually where we start.