Big appetite, thin governance — the state of agentic AI
of IT and business leaders expect their organizations to use AI agents at least moderately by 2027 (Deloitte survey of 3,235 leaders across 24 countries).
Source: Deloitte
of those same organizations have a mature governance model for agentic AI — meaning roughly 80% are deploying agents faster than they can oversee them (Deloitte).
Source: Deloitte
of agentic-AI projects will be canceled by end of 2027 — escalating cost, unclear value, weak risk controls, not the technology itself (Gartner).
Source: Gartner / MarTech
of organizations are experimenting with AI agents, but only 23% are actively scaling them — few pilots reach production (McKinsey, via CX Today).
Source: McKinsey / CX Today
forecast AI-agent market growth from 2025 to 2030 (~41.5% CAGR) — heavy investment and a crowded, uneven vendor field (MarkNtel, via Yahoo Finance).
Source: MarkNtel / Yahoo Finance
Agentic AI — the key terms
- Agentic AI system
- Software built around a large language model that pursues a goal over multiple steps: it plans, calls tools and systems (APIs, databases, search), observes the results, and adapts its next action, rather than producing a single one-shot response. The defining trait is autonomy over a workflow, bounded by rules and approvals you set.
- RPA (Robotic Process Automation)
- Classic automation that follows fixed, pre-scripted rules and screen or API steps. RPA excels at high-volume, deterministic tasks but breaks when a form, layout, or exception changes. Agentic AI differs by reasoning over unstructured inputs and handling variation and edge cases — at the cost of being probabilistic and needing evaluation and guardrails.
- Tool calling (function calling)
- The mechanism that lets an agent do things instead of just talk: the model is given a set of defined tools (look up an invoice, post a ledger entry, create a ticket) and decides which to call, with what arguments, at each step. Well-scoped tools with permission limits are the primary place you constrain what an agent can and cannot do.
- Multi-agent system
- An architecture where several specialized agents (for example, a planner, a retriever, a validator, and an executor) collaborate under an orchestrator, each handling part of a workflow. It can improve reliability and separation of duties but adds coordination cost and more surfaces to monitor, so it is worth it only when a single agent genuinely cannot do the job.
- Human-in-the-loop (HITL)
- A designed checkpoint where a person reviews or approves an agent’s proposed action before it takes effect, especially for irreversible or high-value steps like paying an invoice or closing an account. HITL is how autonomy is made safe: the agent drafts and recommends, and a human authorizes.
How to buy agentic automation safely — a checklist
Autonomy is only safe with approval gates, evaluation, and audit trails. Insist on all three.
- Human-in-the-loop by default: the builder designs explicit approval gates for irreversible or high-value actions (payments, ledger writes, account changes) rather than defaulting to full autonomy.
- Evaluation is built in: they can show how they measure accuracy, task success, and failure modes on your real data, and report accuracy honestly instead of guaranteeing perfection.
- Monitoring and audit trails: every agent action is logged with inputs, tool calls, and outputs, so you can trace, review, and roll back decisions.
- Clear decision boundaries: the system defines which decisions an agent can make independently versus which require human approval, in writing.
- Grounded in your systems (RAG and tools): answers and actions are tied to your source data and permissioned APIs, reducing hallucination, with source citations where facts are asserted.
- Security, access control, and data handling fit your regulatory context (financial services, healthcare, etc.): least-privilege tool access, data residency, and PII handling addressed up front.
- Realistic scoping and timeline: a bounded feasibility phase before full build, a defined path to production, and a start on lower-risk, high-volume use cases rather than a big-bang rollout.
- Cost and value controls: token and run-cost budgeting, a measurable business case, and an exit or rollback plan if ROI does not materialize.
Frequently asked questions
How is an AI agent different from RPA or a chatbot?
A chatbot answers a single question. RPA follows a fixed script and breaks on any variation. An AI agent sits between goal and outcome: it plans a multi-step task, calls tools and systems to act, checks the results, and adapts — handling exceptions and unstructured inputs that would stop a rule-based bot. The trade-off is that agents are probabilistic, so they require evaluation, monitoring, and human approval on consequential steps.
Which back-office workflows are the best candidates to start with?
Start where volume is high, inputs vary, and a wrong step is recoverable or reviewable: invoice and accounts-payable processing, purchase-order and procurement matching, IT service-desk tickets (password resets, access requests), reconciliation, data entry and enrichment, and routine HR-ops requests. Begin with a lower-risk, well-scoped workflow, prove accuracy, then expand — rather than automating your riskiest process first.
Why do so many agentic AI projects get canceled?
Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, largely due to escalating costs, unclear business value, and inadequate risk controls — not the technology itself. Deloitte finds only about 21% of organizations have mature governance for agents. The common thread is projects driven by hype rather than a scoped use case with defined ROI, approvals, and monitoring.
What governance and guardrails does an agent actually need?
At minimum: clear decision boundaries (what the agent can do alone vs. what needs human approval), human-in-the-loop checkpoints for irreversible actions, ongoing evaluation of accuracy and failure modes, real-time monitoring that flags anomalies, and complete audit trails of every action. No auto-execution of high-value steps without a person authorizing them. Autonomy without these is the fastest route to a canceled or harmful deployment.
What are the red flags when choosing a builder?
Walk away from anyone who proposes fully autonomous agents with no human approval on consequential actions, guarantees a specific outcome or 100% accuracy (reputable builders measure accuracy, they don’t promise it), has no evaluation or monitoring story, cannot produce audit logs, or disparages every alternative to make the sale. “Agent washing” — relabeling a simple chatbot or RPA script as an autonomous agent — is another warning sign.
Should we build in-house or hire a specialist firm?
It depends on your engineering depth and risk profile. In-house gives you control but requires scarce LLM, evaluation, and MLOps skills. A specialist firm can move faster and bring governance patterns from prior deployments. As one example, CONE RED builds agentic and multi-agent systems, RAG, and LLM deployments for regulated and enterprise operators in financial services, healthcare, industrial, and logistics — typically running a feasibility phase in about six weeks and reaching production in roughly 90 days, and measuring accuracy rather than guaranteeing it. Whichever route you choose, hold the builder to the checklist above.
CONE RED’s ~6-week feasibility and ~90-day production timeline is a first-party positioning claim, not a guarantee. Agent outputs are probabilistic; accuracy is measured per engagement, and consequential actions should always sit behind a human approval gate.