Why ‘Go Big or Go Home’ Doesn’t Work for AI

Sometimes, you need to go back to basics. 

While developers continue to push the boundaries of AI’s capabilities, the best use cases may be the ones that are a little less exciting. Finding value in your enterprise’s AI investments may require looking at the “bottom line efficiency drivers” that create friction in a business, rather than searching for radical change, Jared Coyle, chief AI officer at SAP Americas, told CIO Upside. 

“Companies and leaders and, quite frankly, technologists, need to learn what to look for in terms of what good looks like,” said Coyle. “A lot of C-levels want to jump right to the final stage, but you really have to build up that muscle before you’re ready to handle that.” 

Coyle sat down with CIO Upside to discuss the benefits of niche AI implementation and make the case for “boring” AI within enterprises. This interview has been edited for brevity and clarity. 

Generally, what do you think enterprises are getting right – or getting wrong – in their approaches to AI? 

Where I think most enterprises are getting it right is we have moved past the stage where people just want to throw AI at every problem. Originally, we were getting a ton of requests from organizations just saying, “We need AI; we want to apply AI to this problem,” when AI didn’t reasonably apply. Leadership teams, technology teams have learned that AI, generative AI in particular, is very good at certain things, creating natural user experiences, creating synthetic data sets, etc., but it is not good at everything.

Where I think organizations are struggling today is in how to navigate … AI everywhere. Every single company they work with, every single software company that they procure from, they are all providing AI. Which one of those is valuable, which one of those is actually going to move the needle to improve a business process and/or change the way you do business?

What kinds of AI deployments do you think are the most impactful? 

What we’ve learned across the 34,000 or so organizations that are using our AI today is that the best AI is boring. It’s not those really cool use cases that you make videos about. The No. 1 use case across our entire AI portfolio is receipt-scanning. Why? Because it saves you time. It allows you to have better audits, because you automatically have all the data that you need. It simply makes people’s lives easier and more efficient. 

Is it going to blow your mind that you took a picture of a receipt and it showed up on your expense report? No, it is not. But if you multiply that by an organization of 100,000 people, is it going to radically reduce the amount of unnecessary repetitive cycles that occur? Yes. 

Where do agents fit into this strategy? 

Agents work very well in the right orchestrated framework, and I think that’s something that is often getting ignored today. Everybody just wants to bring agents in because they picture these little James Bonds walking around, just magically working with each other like secret agents. That’s not how it actually works. It’s really key to understand what are the areas of improvement that require that little bit of intelligence to be able to run a process and then you can deploy it. In our world, that means agents for things like receivables and collections, agents for warehouse and inventory optimization, things that you can operate in a way where the agents aren’t going to go rogue.

With these so-called “boring” AI deployments, do enterprises need to leverage the power of large, foundational models? Or can smaller models get the job done? 

There are significant benefits to using smaller, niche models. And quite frankly, I think that’s how organizations need to think going forward. Don’t get me wrong, there is a benefit to large models when it comes to their suavity, or the general ability for them to feel almost human. But when it comes to the ability to understand the HR domain and recruiting, do you really need to swallow the internet whole every night? No. You need to understand the basics of how the process is done. If you do that, it has the same level of intelligence with way less compute, it’s way more energy efficient, and it still does the job just as well and, honestly, sometimes better.

How has SAP considered these boring AI deployments internally? How are employees reacting? 

How employees have reacted to AI has really changed. At first, everyone was just wowed by the user experience of ChatGPT. From there, people started to realize, “I want to play with it. I want to understand.” And so the next phase for us was to provide that playground. That allows them to come in and creatively come up with these new ideas. Now, as we’re in the early stages of actually having some agentic functionality available, the question is, “How has that playground turned into the ability to reimagine what you do day to day?” 

A perfect example is that we have a large event every year called Sapphire. Tens of thousands of people come, and our sales teams spend hours preparing how to optimize and make the best agenda for everybody. Because of the playground, they actually came up with this idea of using AI to understand who’s coming, what they’re interested in, and build the agenda for them. An  agenda builder doesn’t sound like magic, but it is magic when you have tens of thousands of salespeople spending less time making an agenda and more time creating relationships.

Where should the line be drawn with AI deployment? What should and shouldn’t be automated, and how should enterprises balance automation versus augmentation? 

For the past decade or so, well before Gen AI was cool, when it was just a bunch of physicists in a room, we’ve sat down and asked, “What are the key ethical principles that we as a technology community want to commit to?” The basic idea is, what are the aspects wherein humans can provide the greatest creativity and value to an organization? 

Right now, AI is very good at coalescing existing data sets. It doesn’t actually iteratively come up with something wildly new and creative. The true innovation that comes from an organization creating products like SAP really still requires that human. 

What does the future of enterprise AI look like? 

I do think that while agentic AI is somewhat of a buzzword today, the proliferation of smaller models and the ability of them to interact with each other will give way to the interactivity of small AI capabilities over the next 12 to 18 months, and that will take on greater precedence. It is both cheaper to run and just as efficient. 

From there, then, the question is, “Well, if I’m running these models small, why do they need to run anywhere other than on an edge? What does the model look like running on a forklift? What does the model look like running in a hospital room? What does the model look like running on our phones? It starts to introduce a new way of thinking about how a lot of companies operate. They can add intelligence to the physical assets of their existence, not just to the cloud-based office products.