You don't need another explainer about agents changing everything. You need to know how to build an AI agent: the components, the loop, the code, and a path from an empty file to something running in production. That's this guide.
Everything below is framework-agnostic first, with notes on how LangGraph, the OpenAI Agents SDK, and Rasa handle each piece. The code runs.
How To Create an AI Agent: TL;DR
- An AI agent is an LLM plus tools, memory, and an orchestration loop that runs until the goal is met.
- Tool and function calling is what turns an LLM into an agent: the model decides which function to call, your code executes it, and the result feeds back into the loop.
- Start with a single-tool agent on a narrow task, then add planning, memory, and guardrails.
- Most production failures are reliability and observability problems, not model quality problems.
- Build vs. buy comes down to ownership: control, cost at scale, and where your data and tool execution live.
What Is an AI Agent?
An AI agent is a system that perceives input, reasons with an LLM, chooses and calls tools, updates its state, and loops until it reaches a goal. That loop is the defining feature.
A plain LLM call generates one response and stops. A rule-based chatbot follows a script.
In the AI agent vs chatbot distinction, the agent decides what to do next, acts on external systems, and adapts when a step fails.
Concretely: ask an agent “Where's my order?”, and it calls your order API, reads the result, and answers, or notices the API returned an error, retries, and escalates.
The four components that make this work are what you'll build in this AI agent tutorial: the model and its reasoning loop, tools, memory, and orchestration.
Core Components of an AI Agent
Four building blocks, always the same regardless of framework: a model that reasons, tools that act, memory that persists, and orchestration that keeps the loop under control.
Get these right, and the framework choice becomes an implementation detail.
The Model (LLM) and Reasoning Loop
The model interprets input, plans, and decides the next action. The standard pattern is a ReAct-style loop: the model reasons about the task, acts by calling a tool, observes the result, and repeats until it can answer.
The system prompt defines the agent's job, its constraints, and what to do on failure; treat it as code, versioned and tested.
Model selection is a three-way trade between capability, latency, and cost. A frontier model plans better across multi-step tasks; a smaller model answers in a fraction of the time and cost.
Many production agents route: a fast model for classification and routine turns, a stronger one for complex reasoning.
Add explicit planning (asking the model to write a plan before acting) only when tasks span more than three or four steps; for simple tasks, it adds latency without adding reliability.
Tools and Function Calling
Tools are what make it an agent. You define functions with typed schemas, expose them to the model, and the model responds with a structured tool call instead of prose.
Your code executes the function, appends the result to the conversation, and the loop continues.
Two rules from production experience:
- First, write tool descriptions like documentation for a new hire; the model chooses tools based entirely on them, and vague descriptions produce wrong calls.
- Second, design for failure: every tool call can time out, return garbage, or 500, and the agent needs a defined behavior for each case.
Good tools also follow four engineering principles:
- Least privilege: each tool gets the narrowest permissions that do the job, so a read-status tool holds no write credentials.
- Idempotency where possible: a retried call shouldn't create a second refund.
- Hard timeouts: a hanging API should fail fast into the loop, not stall the conversation.
- Structured errors: return 'order not found' as data the model can reason about, not a stack trace.
Memory and State
Short-term state is the conversation itself: the message history, tool results, and any scratchpad the model writes to. It lives in the context window, which means it has a hard size limit; past a threshold, summarize older turns or drop resolved tool outputs.
In practice, there are three layers:
- Working memory is the current loop: this task, these tool results.
- Conversation memory is the session across turns, persisted in your database, and reloaded per message.
- Entity memory sits between them: durable facts about this user (their plan, their open ticket) injected as context rather than rediscovered every session.
Most agents need the first two; add the third when repetition starts annoying users.
Long-term memory is retrieval: past interactions, user preferences, or knowledge stored outside the context window and pulled in when relevant, typically with a vector store.
When the agent reasons about what to retrieve and re-queries until it has enough grounding, you've moved into agentic RAG territory, which pays off for knowledge-heavy tasks and adds latency everywhere else.
Orchestration, Planning, and Control Flow
Orchestration is everything around the model call:
- Step limits so the loop can't run forever
- Retries with backoff
- Branching between deterministic paths and model-driven ones
- Deciding when to plan versus react
This is where agent projects succeed or die, because an unbounded loop with production credentials is an incident waiting to happen.
Two architectures dominate:
- Graph or state-machine orchestration (LangGraph's model) makes every state and transition explicit, which buys debuggability.
- Declarative business-logic orchestration (Rasa's CALM) separates concerns: the LLM interprets what the user wants, and deterministic flows decide what happens, which is how you keep hallucinations out of business rules.
Here's how dialogue management works in Rasa if you want the deeper mechanics. Either way, the principle is the same: the model proposes, your control flow disposes.
Multi-Agent Systems and Handoffs
At scale, one agent doing everything becomes one prompt trying to do everything. Multi-agent designs split the work: specialized agents (billing, scheduling, research) with a coordinator that routes between them and hands off state.
The pattern shows up in agentic workflows across every framework: sub-agents with scoped tools, explicit handoff points, shared memory.
Honest framing: multi-agent is more advanced and less proven than single-agent builds, and it multiplies your observability problem. Build one agent well first. Add a second when the first one's prompt or tool list is clearly doing two jobs.
How to Build an AI Agent Step by Step
Here's how to build an AI agent step by step, from scoping to deployment.
If you're wondering how to create an AI from scratch, this is the honest version: you're not training a model, you're engineering a system around one, and that system is where all the real work lives.
Step 1: Define the Task, Scope, and Success Criteria
Before code, answer four questions. What exactly does the agent do (one task, narrowly scoped)? What are its inputs and outputs? Which tools and systems does it need to touch? And how will you measure success (task completion rate, escalation rate, latency budget)?
Scope brutally. 'Answer order-status questions using the orders API' is buildable in a day and measurable. 'Handle customer support' is six months of scope creep. The best first agents do one thing against one or two tools.
Step 2: Choose Your Model, Framework, and Tools
Pick the LLM last week's benchmarks favor the least, and your latency budget favors the most; you can swap it later if your framework isn't provider-locked. The framework decision matters more:
- Plain SDK (no framework): maximum control, ~50 lines for a working loop (Step 3 shows it). You own state, retries, and observability. Right for learning and for teams with strong infra opinions.
- LangGraph: explicit state graphs with checkpointing and durable execution. Strongest for complex, branching, multi-step workflows. You still build governance and evaluation yourself.
- OpenAI Agents SDK: fastest path if you're committed to OpenAI models; handoffs and tracing built in, portability limited by design.
- Rasa: purpose-built for conversational agents in production, where deterministic business logic, self-hosted deployment, and voice/chat channels matter. More platform than you need for a background task runner; the right fit when the agent talks to customers.
For a full comparison of the platforms and frameworks in this space, see our ranking of the best ai agents for enterprise.
Step 3: Implement the Core Loop and Tool Calling
The core of every agent, in every framework, is the same five-move loop:
- You send the model the conversation so far, plus your tool definitions.
- The model responds one of two ways: a final answer, or a request to call a specific tool with specific arguments.
- Your code executes that tool, appends the result to the conversation, and sends everything back.
The loop repeats until the model answers or hits your step limit.
You write four things to make that work:
- A system prompt that defines the agent's job, its constraints, and what to do when something fails.
- Tool definitions: for each tool, a name, a plain-language description the model uses to decide when to call it, and a typed schema for its arguments.
- A registry that maps tool names to your real functions.
- And the loop itself, capped at a maximum number of steps.
In plain Python against a model API, the whole thing is about 50 lines.
Three design decisions matter more than the rest:
- First, execution ownership: the model never runs anything. It only requests; your application executes, which is exactly where you enforce permissions and validation.
- Second, completion signaling: the loop's exit condition is 'the model responded without a tool call,' so your prompt should tell the model to answer directly once it has what it needs, or it will keep calling tools out of habit.
- Third, failure behavior: wrap every tool execution so a timeout or exception becomes a readable message fed back to the model, typically 'retry once, then escalate,' rather than a crashed process.
The step cap deserves emphasis because it's the difference between an agent and a runaway process. Five steps cover almost any single task. If the agent regularly hits the cap, the task is scoped too broadly, or a tool is failing silently; raise the observability, not the limit.
Frameworks wrap this same loop in different clothes. LangGraph makes each move an explicit node in a graph, so you can checkpoint and replay it. The OpenAI Agents SDK gives you the loop with tracing attached. Rasa replaces the free-form loop with declared flows, so the model interprets and your business logic sequences.
Understanding the raw loop first means none of them are magic to you later.
Step 4: Add Memory, State, and Guardrails
State first: persist the messages list per conversation (Redis or your database keyed by session), and cap context growth by summarizing turns older than the last ten or dropping resolved tool outputs.
Add long-term memory only when the task needs it, as retrieval over a vector store queried before the model call.
Guardrails make the agent shippable, and they layer:
- Input side: validate and sanitize what enters the loop, including redacting PII you don't need before it reaches the model.
- Action side: a tool allow-list per agent, so a support agent can never call a payments function, argument validation before execution, and confirmation steps for anything irreversible.
- Output side: checks on user-facing text for policy, tone, and leaked internal detail.
- Resource side: hard limits on steps, tokens, and spend per conversation, with alerts when any conversation approaches them.
In flow-based frameworks like Rasa, the highest-stakes guardrail is architectural rather than bolted on: sensitive actions run inside deterministic flows the model can trigger but never rewrite, so 'the LLM got creative' is structurally impossible in your business rules.
Step 5: Test, Evaluate, and Deploy
Evaluation is what separates “how to create AI agents that ship” from prototypes that stall.
Build a golden set of 30 to 50 real task examples before launch: typical requests, edge cases, adversarial phrasings, and cases where the correct behavior is to refuse or escalate.
Score four things on every change: task completion rate, tool-call correctness (right tool, right arguments), escalation correctness (escalated when it should, didn't when it shouldn't), and latency at P95, not average.
Automate the scoring where you can. Exact checks work for tool calls and structured outputs; for judging answer quality, an LLM-as-judge setup works if you calibrate it against human ratings on a sample first and re-check that calibration periodically.
Run the full set on every prompt or model change, because agents regress sideways: the change that fixes billing questions quietly breaks returns.
Budget for runtime cost the same way. Cost per resolved conversation is the metric that matters, not cost per token, and the levers are model routing (small model for routine turns, large for hard ones), prompt and tool-result trimming, and caching repeated context. Instrument it from day one; discovering unit economics in month three is how agent projects get canceled.
Then deploy it like any service: containerized, behind an API, with rate limits and cost alarms.
Decide deliberately where it runs, because model calls, tool execution, and conversation data all carry your customers' information; self-hosting the stack keeps that in your environment, which, for regulated teams, settles the question.
Where AI Agent Builds Fail (and How to Avoid It)
Most agent projects don't fail on model quality. They fail on the engineering around the model, in predictable ways.
If you're learning how to make an AI agent that survives production, these are the five failure modes to design against from the start.
Scope creep into one giant agent
The support agent picks up billing, then returns, then sales questions. The prompt balloons, tool selection accuracy drops, and every fix breaks something else.
The remedy is boundaries: one agent per job, and a router in front once you genuinely have several jobs.
Demo-grade tools in production
Tools without timeouts, retries, or structured errors work fine until the CRM has a slow day, and then containment collapses silently.
Treat every tool as an unreliable dependency, because it is one.
No evaluation harness
Without a golden set, every prompt tweak is a gamble you can't measure, and teams either freeze (afraid to change anything) or thrash (changing things blind).
The harness costs a day to build and pays for itself the first week.
Unbounded autonomy
An agent with production credentials, no step cap, and no spend limit is an incident report with a timestamp still to be determined.
Caps, allow-lists, and confirmation gates are cheap; the alternative isn't.
Deferring the data question
Where model calls run, where tool execution happens, and where conversation logs live will decide whether your security review passes, and retrofitting a vendor-cloud prototype into a self-hosted deployment is usually a rebuild.
Teams in regulated industries should answer the deployment question in week one, not at review time.
Should You Build Your Own AI Agent or Use a Platform?
Bear these points in mind if you’re wondering if you should build your own AI agent or use a platform.
Start With the Task, Not the Framework
The smallest useful agent for your task determines everything downstream.
A background research agent, a coding assistant, and a customer-facing support agent have different reliability bars, latency budgets, and governance needs, and different right answers on build vs. buy.
Scope the task, then pick the lightest stack that meets its bar.
Build-and-Own vs. Buy-a-Packaged-Tool
If you want to build your own AI agent, the plain-SDK path above works and teaches you the mechanics.
The gap between that and production is real, though: state persistence, evaluation, observability, guardrails, channels, and multi-agent coordination are each their own project.
Packaged tools close that gap for narrow use cases fast, at the cost of control, per-usage pricing that compounds, and your conversation data living in someone else's cloud.
Rasa sits deliberately between those poles: a platform you own rather than rent. CALM gives you the architectural guardrail (the LLM understands, your deterministic flows decide), deployment is self-hosted, private cloud, or air-gapped, and the same agent runs chat and voice.
The LLM handles everything conversational (understanding 'where's my stuff from last week?', topic changes, corrections), while the flow guarantees the business logic runs exactly as written. Deutsche Telekom's internal IT agent runs this architecture across 10,000+ employees, resolving 50% of service desk inquiries autonomously.
→ Read the Deutsche Telekom case study here.
Building an agent your compliance team has to sign off on? Book a Rasa demo and pressure-test the architecture against your use case.
What to Check Before You Commit
For any framework or platform, check before you commit:
- Deployment options (self-hosted or vendor cloud only)
- Where model calls and tool execution happen
- How tools and integrations are defined and what happens when they fail
- Built-in evaluation and tracing versus build-your-own
- Auditability of every agent decision
- Cost model at your projected volume
The framework that demos best and the one that's still maintainable in month nine are rarely the same one.
How to Build AI Agents: Key Takeaways
Learning how to make an AI agent comes down to four components (model, tools, memory, orchestration) and one loop: reason, act, observe, repeat, bounded and instrumented.
Start with a single-tool agent scoped to one task, add guardrails and evaluation before you add capabilities, and treat reliability as the actual engineering problem, because it is.
The build-vs-own decision deserves as much thought as the architecture. Hand-rolled agents teach you the mechanics, frameworks accelerate the middle, and owning the platform matters when the agent faces customers, touches regulated data, or needs to survive years of production.
Choose for month nine, not day one.
FAQs
What's the best framework to build an AI agent?
It depends on the agent. Rasa leads for customer-facing conversational agents needing deterministic business logic and self-hosted deployment, LangGraph leads for complex stateful workflows, the OpenAI Agents SDK is fastest for OpenAI-committed teams.
For simple single-tool agents, a plain SDK loop in 50 lines beats every framework.
How do you give an AI agent access to tools?
Define each tool as a function with a typed JSON schema, pass the schemas to the model, and when the model responds with a tool call, execute the function and append the result to the conversation.
The model never runs code; your application executes every call, which is where you enforce allow-lists and validation.
How do you add memory to an AI agent?
Short-term memory is the persisted message history, capped by summarizing old turns.
Long-term memory is retrieval: store past interactions or knowledge in a vector database and query it before the model call.
Add long-term memory only when the task needs it; most first agents don't.
Should I build an AI agent from scratch or use a framework?
Build from scratch to learn: the core loop is 50 lines and teaches you exactly how agents work.
Use a framework once you need state persistence, branching workflows, evaluation, or multiple channels.
Move to a platform when the agent faces customers or regulated data and needs governance you'd otherwise build yourself.
How do you deploy an AI agent to production?
Containerize it behind an API with per-conversation state in a database, then add the production layer: tracing on every model and tool call, a regression test set, step and spend limits, and alerting.
Decide where it runs deliberately, since conversation data and tool execution carry customer information; regulated teams typically self-host.
Do I need to know Python to build an AI agent?
Python is the default: every major framework (LangGraph, the OpenAI Agents SDK, Rasa) is Python-first, and most examples and tooling assume it. TypeScript support is solid for LangGraph and the OpenAI SDK.
You need working programming skills either way; no-code tools cap out well before production complexity.
How long does it take to build an AI agent?
A working single-tool prototype takes a day. A reliable production agent takes weeks to a few months, depending on integrations, evaluation, and governance requirements; the gap is reliability engineering, not model work.
As a production reference point, Swisscom took a customer-facing agent from prototype to production in 20 weeks.
