Scaling AI in 2026: From Prototypes to Production Success | Building the AI Era Episode 4

Hi, I'm Pete Johnson. I'm the field CTO of AI at MongoDB, and welcome to Building the AI Era, a look at how organizations can use data to build AI applications smarter, faster, and with purpose. What does scale-out architecture mean for databases, and how does it impact organizations' approach to handling increasing amounts of data? I'm joined by my colleague Shilpa Kolhar, Senior VP of Product Engineering at MongoDB, and Devin Dickerson, Principal Analyst at Forrester, to answer those questions and more, and discuss the need for building distributed architecture that handles any data at scale. We both just were at mongodb.local in San Francisco, and we both had conversations with lots of people about how you get from doing things as toys, as experiments, to full-blown production deployments that are providing actual business value. What are the things that are causing leaders to not quite get past that push and get out of the experimental phase and into something that's actually returning them some value at their bottom line? I actually think that the highest leverage skill set that organizations, the muscle that really needs to be developed, but also continuing to innovate on the vendor side with product features and feature iteration on this, is in context engineering. Getting smart about how, when, and where to infuse powerful business context into agentic processes is where so much of the business value is derived from. And it's tricky. There's too much data. It's not just about loading as much as you can, give it as much context as you can, because sometimes the more context you add actually just confuses the output. There's a little bit of art, a little bit of science, and I think organizations maturing on this while the market matures on it is going to lead to better outcomes for organizations investing in AI. The data that we are seeing clearly says that this scale-up of relational databases is limiting, especially when organizations want to build these AI-native applications, which will need large amounts of data reliably and correctly at scale. What's your take on relational databases scale-up as a limiting factor? I think many different technologies have a place in an architecture. It's not that monoliths don't have a place. Same thing with traditional databases. The problem is the scaling mechanics really can be a limiting factor and don't always apply correctly to certain types of architectures. If we look at applications that have to live in the cloud, vertical scaling with monoliths has imposed severe challenges for both cost and performance optimization. I think the same can be true in this case. When something is able to scale horizontally, it actually allows us to take advantage of the resilience characteristics of cloud computing a little bit more natively than with strict vertical scaling. Yeah, I mean, that's a very good point, Devin. MongoDB supports scale-out, or we call it horizontal scaling, by being able to shard the data. We can distribute the data and compute across commodity machines or commodity computers and hence bringing the cost down and also supporting the needs, AI needs of large-scale data and information for the agents to consume. So let's pivot. One of the most often cited studies in 2025 is the MIT study that showed failure rates for AI projects in the upper 90-95%. I have some thoughts on that and why that is. I'm really curious what your thoughts on that are and how we can avoid that as you're deploying AI-based applications in 2026. I think that it's important to start thinking about production and scale and security as early in the process as possible. I think people will find that when their actual AI workloads make contact with the real world, they're going to discover things, they're going to learn things. Learn those things during the development phase and you won't have those costly failures because the further we push those failures, which are inevitable, to the right, they will cost more money and cost more time. I completely agree with you. When it comes to the stack, you've got to get it off the laptop as quickly as possible and onto something that scales so you can learn those lessons early in the process and not hit them later. The earlier the better. What attributes and capabilities should organizations look for in a data platform, especially when you consider things like scalability and performance that you need for an AI workload? I do a lot of platform evaluations and I tend to focus on developer experience and when I look at features, like in the current offering around scalability and architecture, I like to go in with the recognition that this is a fast-moving space. So I want a track record of extensibility and feature iteration because even if a platform doesn't necessarily have everything today, I want to know that the organization that I'm partnering with can get there. Another thing that I think is important is this idea of friction for the developers. How easy is it to learn this platform? That's part of why there's three flavors of MongoDB, right? Most people use MongoDB Atlas, which is our managed service that you can deploy your actual data in a hyperscaler data center of your choice. We also have the Enterprise Advanced version, which you can run on-prem. Or, if you don't want to deal with us at all, you can use Community Edition, which you can put wherever you want and manage it yourself. We like to meet our customers where they are and give them that exact kind of flexibility when it comes to how they deploy and interact with the base system. What would your advice be to the CTOs and the leaders who are now contemplating to modernize or not to modernize? I think that this is a great opportunity to get modernization initiatives not just in prototyping and MVP phase, but thinking about establishing metrics, thinking about business value. I would also say this is a perfect time not to neglect one of your greatest resources, which is people. I think that part of getting there is also thinking about things more holistically. Instead of thinking about just point solutions, think about platform investments. Instead of thinking about just one project or one individual migration, think about a modernization strategy. I think that should be the difference between how we thought about this in 2025 and 2026. Thanks for tuning in to Building the AI Era. Visit MongoDB.com to learn more.