Developers Need More Than Speed to Succeed

In software development, is faster always better? 

With AI tools ramping up speed, striking the balance between risk and reward can be tricky for developers. The solution may be shifting the way you measure success, Catherine McGarvey, senior vice president of developer experience at Capital One, told CIO Upside. 

The primary focus should be on output quality rather than speed or quantity, she said. 

“It’s never just about your lines of code produced,” McGarvey explained. “It is easy to fall into the trap of just looking at that speed of delivery, and it’s really important, you’re actually delivering the thing the customer wants.”

McGarvey sat down with CIO Upside to discuss balancing innovation and safety, preventing AI overreliance and highlights of her first year with Capital One. This interview has been edited and condensed for brevity and clarity. 

How do you measure success within your developer teams? 

One aspect is that you understood what the customer wanted and were able to deliver it to meet what they needed. And the other one is about speed and doing that. To get that, we actually look and do a whole bunch around (Net Promoter Score) and customer satisfaction. When there is a tangible improvement in process, we look at the speed at which they were able to do the activity before versus after. We have real, tangible metrics based on what we’re delivering, so (Net Promoter Score) is always a factor, because how we’re perceived is a really important thing. 

Where does AI fit into development teams at Capital One? How do you strike a balance to avoid verging on overuse? 

We are leveraging generative AI to help our software developers across Capital One. The majority of our 14,000 developers are actually using the tool on a day-in, day-out basis, and just like any tool, it’s a means to help the teams elevate or accelerate what they’re working on. It’s not a replacement for their capability or individual behavior.

One thing we did is conduct a lot of training on how to use it. Then, as the technology is rapidly evolving, we’re continuing to do that regular training or insights on what sort of things you can use it for. We have a ‘Champions’ channel where some devs share with other devs what they discovered. 

You still need your quality checks and your validations of the code that’s generated. You need those in place so you don’t lose the driver in the experience. I think that’s the balance. If you have those quality and behavioral checks in place, I don’t know if there is a concept of overuse. If it can help you solve or do something well, you should be using it. 

Where does it fit within the development cycle?

I think there are multiple places it can fit within the developer lifecycle. The first part we’re really starting to see is as an assistant while coding. We’re exploring lots of other use cases for it as well, anything that can speed up and remove developers from the menial tasks and push them to the creative problem-solving that we think is incredibly valuable for the human to really be engaged in. We look at things from patching and remediation, test coverage, code reviews; everything else within that developer lifecycle is up for grabs. 

How do your developers balance security and safety with innovation?  

This is more how the company approaches this. We have a very focused approach to any tool that we start to introduce. It goes through a big review process that enables us to confirm any gaps or concerns, and then we work really closely with any vendor we’re operating with to turn off brand new capability until it goes through that type of review. 

This means that we don’t get the thing on the day it launches … there is a bit of a time lag between the thing that’s launched versus coming out, but we do encourage certain environments where they can play around with things before they’re widely adopted. That’s how we get the balance between enabling innovation but still making sure it’s meeting our risk profile.

Finance is a heavily regulated industry. How do regulation and compliance play into the use of AI for developers at Capital One? 

What we try to do across the board is make the right thing the easy thing. If there’s a best practice we can instill from the get-go, without templating or without the controls that we have with our standardized pipeline, we will do that. The regulation and control set us up so that it’s easy for the dev team, if they follow them, to then not have to go through additional rigamarole or checks and balances, because they’re already built in. 

As it relates to AI, this means we spend a lot of time deciding what access these models have to what things. We look at what roles should have access to these tools, and we do regular reviews of each of those subgroups to make sure that, when we roll something out, it’s rolled out with the intent of that behavior in mind, so it’s not left up to the individual to try to navigate a really complex, highly regulated industry and all of the nuances around it.

What are some common mistakes enterprises make with their own development teams as far as adopting AI or other new technologies? 

I think there’s sort of the extreme view where you don’t pay attention to the new thing coming out. There are places I’ve sort of done consulting with in the past where you can become very inward-looking, and you’re not looking externally for what’s going on in the industry. You can be developing your own tool and think it’s the best thing, and not be looking outward. So I think one of the biggest mistakes can be just not being on top of what is changing in the industry. That’s where the build versus buy option often comes in. 

I think the other one is you can easily – especially in the AI space – pick the wrong vendor. Things are really changing. Creating a workflow and a process where what you’re learning can be adapted to different approaches or different tools, I think, is really important as well.

What trends are shaping the future of development teams, whether that’s at Capital One or more broadly?

What’s really exciting on the development front is the enabling of a lot more people that haven’t come through traditional software coding backgrounds to play around with things and create things quickly. There’s a whole bunch of different companies out there that enable you to still produce code in the back end with the click of a button and a low-code background. Some of these agentic flows are really exciting for that space. It helps people convey new ideas in a way they weren’t able to before. 

That also means sometimes it’s hard to tell what’s real versus what’s a prototype out there. I think that can be a bit of a challenge. What that means from a future perspective is you’ll have a lot more people able to convey ideas that don’t require a software developer in the mix. But the moment you get to a real product that you want to expand and get use cases, you bring in (engineers).

You’ve been at Capital One for a little over a year now. What are some of your highlights, and what has changed within your unit in that time? 

It has been an incredible experience to learn a lot more about a highly regulated space. One of the things that struck me, and why I joined, is I saw a whole bunch of innovation sitting at Capital One, it being on the forefront of a lot of emerging technologies. It was one of the earliest to adopt a coding assistant that I had seen, and it’s really been big on the cloud. Seeing that made me go, ‘Huh, how do they do it?’ 

From a change perspective, I’ve spent a lot of time with my team doing listening sessions with our developers. We talk about not necessarily making a developer happy, but making their experience better, because a developer’s mindset of always improving things will always help give you feedback on things you could continue to do.