The more trust we give AI, the more opportunity it has to go haywire.

As agents take center stage in enterprise conversations around AI adoption, so has the idea of connecting autonomous assistants to the core of a business. Last November, Anthropic introduced a way to do so with Model Context Protocol, otherwise known as MCP, an open source framework that allows for just that: Connecting agents to “the systems where data lives.”

Since then, OpenAI has followed suit, launching its own connector offerings between its AI models and enterprises’ internal systems.

Creating a connective tissue between agents and internal systems has significant potential to help enterprises get the most out of their AI investments and deployments, said Greg Anderson, CEO and founder of vulnerability management firm DefectDojo. “They essentially make AI as smart as the tools they’re connected to,” said Anderson. “I think it really helps to bridge the gap where AI has struggled previously.”

The problem, however, is that AI still comes with tons of fundamental risks – one of which is data security:

• Even the major, most commonly used models face data security threats. One study by Cybernews from late May found that OpenAI has suffered 1,140 data breaches.
• By giving these agents access to an enterprise’s data and systems, businesses may create an “increased attack surface area,” said Anderson. That makes things like prompt injection attacks or manipulation of models by attackers all the more risky, he said.

“It’s about exposure,” said Anderson. “With MCP, we’re essentially saying now we can connect these AIs to anything that supports it. And so by proxy, if you compromise that AI, you compromise everything that it’s connected to.”

But it is possible for enterprises to mitigate such risks, said Anderson. That starts with the data that you let the agent access. Start with small, lower-stakes use cases before allowing agents access to deeper systems, he said.

The next step is making sure agents perform at a sufficiently high standard that the risk is limited once access is increased, he said. “I recommend a crawl, walk, run approach,” said Anderson. “How do we roll these things out in increments to not create additional risk to the enterprise while also accomplishing the goal of actually getting these things out the door?”

In the “mad dash rush” to adopt agents and get value out of them, however, enterprises often aren’t thinking through what they’re doing and why they’re doing it, he said.
“Nobody wants to be late,” said Anderson “No one is stopping to say, ‘What does that actually mean? What are the limitations? What do you want to expose? What makes sense to not expose?”

Just because AI can be in control doesn’t mean it should be.

Though increased automation may be the next frontier of AI, humans should remain in the driver’s seat and steering shouldn’t require a software engineering background, said Binny Gill, CEO of AI automation startup Kognitos.

“There’s only 30 million developers in the world, but I believe that there are a billion people who can figure out, stepwise, what to do in order to achieve something new,” said Gill.

Kognitos offers an AI automation platform with the goal of making these systems more predictable and reliable. The company recently announced a $25 million Series B funding round led by Prosperity7 Ventures, with support from Khosla Ventures, Wipro Ventures, Engineering Capital, Dentsu Ventures and Alumni Ventures.

Kognitos’ goal is to “remove the language barrier” between building AI automation and the enterprise. It does so through a paradigm called “English as code,” said Gill, or defining in plain English a process that you want achieved for your business:

• Rather than focusing on goal-oriented AI agents, Gill said, Kognitos focuses on “process-oriented” agents, defining exactly what steps you want an AI agent to take, rather than what goal you want it to accomplish.
• Kognitos’ tech relies on “neurosymbolic” AI, or a kind of model that combines the gray areas of human reasoning with the black-and-white determinism of machines. A neural engine comes up with a game plan for how a process should be run, while the symbolic engine runs it exactly as it should be.

Tech like this can be helpful for enterprises that are seeking to deploy AI, but don’t have the resources to hire tons of AI talent – something that’s come at a premium as demand for AI skills only grows. By breaking down technical barriers, Gill said, enterprises can remove “the bottleneck of computer developers.”

But with fewer engineers, how can you keep AI from going off the rails? By making the buck stop with humans, said Gill. Before any Kognitos workflow process is put in place, it’s presented to a human to be double-checked. The amount of governance and human oversight can be shifted depending on the task, Gill noted.

“Intelligence and hallucination are two sides of the same coin,” said Gill. “My bicycle cannot hallucinate, but my Tesla might … anybody who says that ‘my AI does not hallucinate’ is dead wrong.”

Plus, even as automation picks up the pace, automating humans out entirely is something that no enterprise needs, said Gill – or even wants. “People are realizing they don’t want, even if AI is smarter than humans, to give up control,” said Gill. “Humans must be steering AI, even if it becomes superhuman.”

Coinbase has found a way to regulate AI development using the technology behind cryptocurrency.

The company is seeking to patent a system for “tracking machine learning data provenance via a blockchain,” essentially recording all data that goes in and out of an AI model throughout its lifecycle.

Coinbase’s tech takes note of any data that contributed to a model, including training information and user input prompts and their corresponding outputs. It does so using a “middleware component,” or a system in place between the model and the user, which automatically logs every interaction on the blockchain.

Using blockchain for this purpose provides an immutable and transparent record of who has contributed what to a model, helping to establish ownership and govern usage of AI. The decentralized nature of blockchain also allows no single part to claim ownership over the model fraudulently. This is particularly useful for open source models that often involve many contributors.

“One or more individuals, such as creators, developers, data scientists, engineers, or other stakeholders may contribute to the development of a machine learning model,” Coinbase said in the filing. “However, contributions to the machine learning model may not be captured, and, in some cases, the output of the model may not be tied to the contributors.”

Though blockchain hype has declined from its cryptocurrency and NFT-fueled peak four years ago, the underlying technology still has a number of uses. For example, JPMorgan Chase has sought patents for tech that uses blockchain for data lineage and fact-checking, Intel previously filed a patent application for blockchain-powered security audits, and a Sony patent detailed a means of using the tech to detect deepfakes.

And as copyright infringement lawsuits pile up against companies like OpenAI and Anthropic, playing to blockchain’s strengths could be incredibly useful in tracking down who owns what.