Source: Be Datable. Autonomy amplifies both capability and risk.

Evaluation

Three-Layer Stack

Agents, Assistants, and Automations: What vendors call them vs. what they actually do

Why This Matters for Your Business

When vendors demo their "AI agent," they're often showing you a workflow automation with better branding. Most agents in the market are really just automations: repeated steps you would do often, now wrapped in AI terminology.

The confusion comes from treating these three categories as interchangeable when they represent fundamentally different capabilities with different cost structures and risk profiles. Like handing your teenager the car keys: with autonomy comes additional risk.

The terminology matters less than understanding what you're actually getting. This framework helps you cut through vendor BS and evaluate AI systems based on what they can actually do.

The Penalty for Being Wrong

The diagram above shows the critical relationship: as autonomy increases, so does the penalty when things go wrong.

A competitor used an agent to change the primary category across a business's Google profile and other distribution sites. They systematically ruined the company's digital visibility and triggered a re-verification. This is a massive penalty for being wrong.

High-agency, low-intelligence systems make confident, wrong decisions at scale. Before increasing autonomy, ask: what's the penalty if this system makes a mistake?

The Three Layers

BASE

Automations

Simple triggers that move data, push alerts, and keep routine tasks moving. Fixed rules, predictable failures.

Present day: Predefined workflows triggered by specific conditions.

Examples: Zapier, Make, IFTTT, basic HubSpot workflows

Risk level: Low. When they fail, they fail predictably.
MIDDLE

AI Assistants

The conversation surface that gathers information, answers questions, and figures out what you're trying to do. Human oversight catches errors before impact.

Present day: Interactive interfaces that require human prompts and approval.

Examples: ChatGPT, Claude, Gemini, Perplexity

Risk level: Medium. You review output before it goes anywhere.
TOP

AI Agents

Narrow capability units that stay in their lane so their decisions stay focused. Autonomous actions at scale without supervision.

Present day: Systems that can take multi-step actions independently.

Examples: Claude Code, Replit Agents, CrewAI

Risk level: High. Autonomous actions at scale without supervision.

How to Use This Framework

When Evaluating Vendors

Ask specific questions about autonomy:

  1. Does it set its own goals or follow predefined workflows?
  2. Can it adapt its approach without human reprogramming?
  3. What happens if it makes a mistake? Is it reversible?
  4. What oversight mechanisms exist?

When Building AI Systems

  1. Start with automations for routine, low-risk tasks
  2. Add assistants for complex tasks requiring human judgment
  3. Deploy agents only where the penalty for being wrong is acceptable
  4. Always maintain human oversight for high-stakes decisions

When Scoping AI Projects

Match the layer to the risk profile:

  1. Low Risk: Use automations (Data sync, notifications)
  2. Medium Risk: Use assistants (Content drafts, analysis)
  3. High Risk: Human + assistant (Customer comms, finance)

Quick Comparison

AspectAutomationAssistantAgent
TriggerPredefined conditionHuman promptGoal assignment
Decision-makingFixed logicSuggests optionsAutonomous
Human oversightSetup onlyEvery interactionPeriodic review
Failure modePredictableCaught in reviewCascading
Best forRoutine tasksComplex analysisMulti-step workflows

Self-Assessment

Answer these questions to evaluate your position.

What layer are your current AI initiatives actually operating at?

Are you calling something an "agent" that's really an automation?

What's the penalty for being wrong in your AI deployments?

Do you have appropriate oversight for each layer?

Vendors call everything an agent because it sounds more advanced. The terminology matters less than understanding what you're actually getting.

We want AI that is smart enough to be told what to do, to go do it reasonably, and to not make mistakes along the way. We're not there yet. Pretending we are by calling automations "agents" just creates misaligned expectations and budgets.

Sources & Further Reading

Get more frameworks like this

Weekly frameworks for becoming more data-able in the age of AI.

Related Frameworks