Agents Gone Wild: How to Prevent AI Agent Sprawl

Wrangling AI agents can feel like herding cats. 

While developers are keen to test a variety of platforms for building AI agents, doing so can become unwieldy for IT teams, leading to agent sprawl. The push to adopt agents has prompted some enterprises to jump in hastily, and amassing disparate agents can lead to a bevy of issues, said Amr Awadallah, CEO of Vectara. 

“Because each one of these agents is being built in a different way, there is no single pane of glass, from a management perspective,” said Awadallah. 

So how does agent sprawl happen in the first place? When developers test and build agents for multiple use cases on a number of different platforms – such as using OpenAI, Google, AWS or open-source options – these agents are then only controllable by the developers who built them:

• If one of those developers leaves the enterprise, “It’s hard to have developers inherit and maintain each other’s code,” Awadallah said. 
• Moreover, it creates a situation in which IT teams are unable to govern these agents, he said, as they’re built on different platforms without one central system for observability. If something goes wrong, he said, a web of sprawled agents makes the problem that much harder to diagnose and fix. 
• That lack of governance can create problems with agent accuracy, security and performance, he noted, as well as rack up costs. 

Preventing this sprawl relies on deploying a uniform standard from the jump, said Awadallah. Picking one standard platform to build and deploy agents allows developers the freedom of creativity while giving IT teams the ability to manage them, he said. Additionally, to avoid being too locked in to one platform if it jacks up prices, figure out how much migration to another will cost if necessary, he added. 

“If you’re building AI agents en masse … then spend the time and the exercise to come up with a common standard that all of your developers are building with, versus letting the developers go pick whatever they want to pick,” he said. “Otherwise, you’ll end up in a mess a year from now.” 

If you’re already experiencing agent sprawl, now is the time to get a handle on it, Awadallah said. Pick a standard platform for building agents going forward, and start to untangle yourself from the “legacy” ones that were built on others, he said. 

“It only gets worse over time,” he said. “Try to enforce the standard right away, so you don’t have to pay the cost of it later on.”