If you’re a developer building AI agents, chances are you’re looking for one thing: a prediction. What's the next model going to do? Which framework wins? Where should we place our bets?
While I can’t claim to see the future, there is one prediction I'll actually stake my name on: your AI stack has a half-life of 18 months.
It’s a tough truth. Language models have been eating the AI stack for ten years, and they’re only getting hungrier. They’ll continue to reach up into the pile of code we've built around them and swallow another layer whole. That thing you spent a quarter engineering? In the next release, it'll just be a line in a prompt.
The models won’t stop eating the stack. But, your engineering team can stop welding your product to the capabilities of a single model generation. Instead of being affected by every bite the model takes, it’s time to take accountability: assume the AI stack will keep getting absorbed and build accordingly.
Here’s how.
One token at a time
Let’s take a step back to understand exactly which layers have been swallowed by the model so far.
In 2016, we shipped agents on models that read exactly one token at a time. The model had a single job: turn a message into numbers and match it to one of your fifty intents. Developers still wrote everything by hand: the flow, logic, repair, responses. It mostly worked… until transformers took the first real bite.
The model stopped counting words and started reading them in order, in context. These were better inputs, but the output was still just labels, and deciding what to do with those labels was still on engineers.
The next bite was the whole conversation. In 2017, Google and others tried sequence-to-sequence models; at Rasa, we built TED, the transformer embedding dialogue policy. Both hit the same wall: the only way to steer the agent was to hand-curate thousands of training dialogues… forever.
In 2020, GPT-3 arrived, and with it in-context learning. No dataset to label and maintain; instead, engineers and conversation designers could steer the model the way they'd edit a set of instructions. After years of curating dialogues, it felt like magic.
Now, take a deeper look at that timeline. The models started with a token. Then a sentence. Then a whole conversation. Then your tools and your memory. Every time the context window grew, the model swallowed another layer of the stack we'd built around it.
What's left? A model calling tools in a loop. That's an agent.
And the model is coming for the next layer of the stack.
It's history. But it’s also a forecast.
If you're building enterprise agents today, this history lesson also serves as a warning. Every layer you’ve hard-wired now is a layer you'll migrate and re-certify later. If you ship on behalf of others (as an OEM, BPO, or system integrator) you won't do it once; you'll do it for every client you serve.
That’s the problem. With every bite the model takes, there's a huge amount of activity re-engineering for model capability. As a result, there’s not nearly enough actual value reaching customers. Teams are sprinting to ship (new new RAG pipeline, an MCP server, support for the latest model, etc), but the business is still left feeling stuck.
But there’s good news: the model hasn’t eaten everything.
What the model won't eat
Even as the model gets hungrier, there’s one component of your agent stack it can’t eat: the context engineering layer.
The model can’t control what goes into its context window; that’s up to the engineering team, and that’s still built on plenty of code: retrieval, re-ranking, guardrails, evaluations.
The ability to explain why the agent did what it did doesn’t ride on raw model capability. It's built on accountability – and no model is going to build this for you. In your agent platform, accountability is where value compounds.
The 3-layer agent platform
Think of an agent platform as three layers that age at wildly different rates:
- The Velocity layer: This skills library, a visual builder, insights, simulations, A/B tests. This is the accountability layer.
- The Enterprise Readiness layer: PII handling, guardrails, audit, telephony, release ops. These are essential to run in production, and also almost completely undifferentiated.
- The Capabilities layer: The model, retrieval, memory, the harness, the integrations, everything at runtime that turns an LLM into an agent. It's where most teams pour their time, and it’s also what ages the fastest.
The uncomfortable truth is that most of us spend most of our time on the bottom two foundational layers when we should be focused on velocity – and compounding value.
The velocity layer is where the ROI gets built
What does the velocity layer actually do?
For your engineers, it means building from proven, reusable skills you can compose. When you fix something once, it stays fixed. No more “prompt-and-pray” to fix one problem (and introduce three new regressions somewhere else as a result).
For the business, it means a roadmap driven by real conversations. You can start from a top-line metric and drill all the way down to the single conversation moving the needle, then go back the other way and ask whether one odd conversation is a pattern or a one-off. You can go from idea to production without waiting in a ticket queue behind a single central IT team.
In practice, it looks like a system that comes to you.
Here’s a real example: An agent notices that 212 conversations last month asked about return shipping labels, and most ended in a handoff to a human. What does it do?
- It drafts the fix.
- It replays that fix against those same 212 real conversations and reports back: 178 now resolve without a handoff, zero regressions.
- It asks whether the domain expert – not just the engineer – would like to ship it.
A good velocity layer proactively helps your organization learn from what's actually happening in production. It uses a federated model to empower people across the business to ship every use case and every agent.
As a result, your agents are built by the people who own the process, assembled from skills you already have, steered by real conversations, and hardened in production from day one.
Everything you built comes with you
Most importantly, this focus on the velocity layer means you get to keep what you’ve already built in the capability & enterprise readiness layers.
Teams have invested years in models, APIs, guardrails, channels, and business logic they do uniquely well. When we designed the Rasa platform, we wanted to ensure they wouldn’t have to rip all of that out and start from zero.
Instead, everything you’ve built comes with you. Deploy where your users actually are and own your brand instead of renting a vendor's. That’s what you get with a tool like Rasa.
Build for the next bite
I've never treated the model as a threat, and I'm not about to start. Every time the model ate a layer, it freed engineers to go solve a harder problem. The token-matcher became a conversation model, which became an agent. At each step, the interesting work moved somewhere more valuable.
So, build for the next bite instead of defending the last one. Keep the undifferentiated infrastructure as something you can swap out, not something your product is fused to. Spend your engineers on the work only your team can do: the business logic, the customer journeys, and the accountability layer no model will absorb.
The model will eat this stack too, and that’s a good thing. That means we get to go solve the next set of problems. And that’s where I've wanted to spend my time since the token-matcher days.
That's your AI future: build on what compounds.
To learn more about how Rasa’s developer platform can enable your team to build the velocity layer in your AI platform, speak to our team here.
