Fala MongoDB Ep. 26 Bradesco + MongoDB um case de sucesso

Hey, MongoDB! Welcome to another podcast where we will talk about the universe of data and technology. And today, a super special podcast. I am Cami Bonilha. And I am Enzo Taborda, Solution Architecture Manager at MongoDB here in Brazil. And in this episode, we have the honor of welcoming Pércio Bazin, Data Technology and Product Manager at Bradesco. Pércio, thank you so much for accepting the invitation Come record with us today. And this is an episode that certainly so many people who follow us here in this series I was waiting. I wanted to start already inviting you to introduce yourself And already posing our first question, which is the following. I know you're a DBA. very successful here in Brazil And you have successfully worked with relational databases. And today you're on a MongoDB podcast in the documentary cycle Is that right? Tell us a little about this story. That's it. First of all, thank you for the invitation. I thank both of you. Finally worked out, huh? Conflict of agendas. We've rescheduled several times. Good afternoon to all of you here. Thank you for the invitation. That's true. I've always worked with data in some way, right? Throughout my entire career. More than 23 years working with technology and always working with data in some way. I have always worked for large companies. I have worked over 12 years as a DBA, relational databases, right? I worked at Microsiga. I took that whole part of TOTUS when she set up the Joint Inventory. Then I came to work at Bradesco. I had the opportunity and the honor to work where I am today. It's already been 15 years since I started working at Bradesco. And in some way, we've always worked with relational databases. As you yourself said, right? Why do we work with NoSQL today, then? In fact, I've always been curious about technology. What the market is doing, and why it is doing it. So I used to like tracking that Gartner curve, right? Of using the best technologies, primarily focused on databases. And it caught my attention—MongoDB is always right there among the first, among the most used When we talk about document databases. From the BI document, right? And then I started studying a bit more at Bradesco, about this technology. Back then in 2016, 2017. I already wanted to put some things there regarding MongoDB. I thought it made sense the way the data were modeled differently. As collections, right? With sub-document. By reducing the number of tables. Can you create a more embedded data model, right? With subdocuments. That really caught my attention. And we started looking around that time, roughly there About MongoDB, right? We started doing some tests. Some things still needed to evolve in Mongo. In addition to ACID, transaction support, Mongo was still getting better. But we always began to look at it with affection there and understand how that was going to evolve, right? Move from the relational database and start working with NoSQL. Was he capable of handling a critical mission? So it was a question we had we would only be able to find out If we were to actually start implementing something. That was more or less the idea Let's start looking more at NoSQL. A truly fascinating journey. Based on your experience today, Would you recommend it to someone listening? and here we have you on this podcast, for a person who is a data administrator you are the database administrator, who only works with relational databases, Does it make sense for this person to learn MongoDB? Certainly. I think it's a paradigm shift, right? The person does not have to be afraid. Ah, will I lose my job? Am I going to stop being a relational database DBA? Actually, you're adding more knowledge Your skills, huh? A Swiss Army knife. The more you know, The better you position yourself in the market, right? So, instead of knowing only relational, You also know NoSQL databases. There are many excellent NoSQL databases available on the market today. So people can start to better understand how it works, right? You have to stop with that prejudice, right? NoSQL databases are only for intermediate environment Not suitable for critical mission. That's in the past already, right? You need to start looking at high availability, cloud native What does he offer in terms of cloud disaster recovery? I think the flexibility of being able to use it, for you to be able to scale not only vertically but horizontally, I think this is a major difference. for a relational database today. You create multiple shards, right? In this case, using MongoDB, isn't that correct? It gives you a very high write throughput. Then, I guess, yes, Is it worth it for a person to start looking at this market? Without a doubt. Guys, here's a tip from Bazan. And if you are a relational DBA, Tell us here in the comments, What's up, Bazan and Cami, we want to know. Very cool, Percy. It's legal to see that, in this way, You had every probability of failing to adapt to NoSQL. And you brought the opposite. So, this proves to us the extent of innovation you are bringing. And for a bank that's 80 years old, right? Then, I'd also like to hear a bit from you, you know, How was it bringing this inside Bradesco? the challenges of bringing this to the teams And how did this story, this marriage with MongoDB, begin? Hey, well, I think your question was really interesting, right? Truly, Bradesco had a culture heavily centered around mainframes, relational database The initial projects, in fact, We decided to start from scratch already using MongoDB, right? For smaller projects, this was still the case at Monoprime. It was a series of licenses that we purchased back then to use in Monoprime. That's pre-cloud, right? We weren't using the cloud yet at that time. So, we started using the Monoprime environment. For some Brain projects, facial biometrics, agriculture, right? So, we started these projects from scratch, What was easier even for learning what you migrate from a relational database, something more complicated That was the first use. After being there for about 2 or 3 years, practically running without any problems. No availability whatsoever. It was truly a very stable product. They were a bit medium-sized systems, right? For large ones, but based on the size of Bradesco You have systems that are gigantic, right? The size of the database there. So, we started like this just to develop that early efficiency in a technology that was relatively new. Hey, when did Bradesco start that movement? You're changing the bank, right? And we started moving to the cloud, The people started to use it, We submitted the Azure application, I was the first cloud provider there, the Leap project. what we started calling Leap. And the team went ahead with Cosmos DB, using the Mongo API. But it did cause some confusion, didn't it? Because the people who were looking for us there were speaking that project of mine that's running on Mongo But in reality, it wasn't Mongo, was it? I was using an emulated technology. So, it was a Cosmos DB emulating a MongoDB API. And that sometimes caused a bit of confusion among the staff, right? He sometimes wanted to use certain features that they knew was in Mongo, But there wasn't one in the Cosmos, was there? You don't have 100% compatibility there. Basically around 37% compatibility, right? The main cloud operations will function. but some aggregation functions were not working. And what caught my attention, you know? Everything in Open Finance started running on Cosmos. with the Mongo API. That's how the Cosmos works, right? Today, Microsoft is a major partner of yours, So this was something public which we have already presented, indeed, You're a crazy one, I'd say, two years ago. So there's absolutely no problem with saying that. The way they charged there for the database Do NoSQL, do Cosmos, right? It was based on RUs, Request Units. So it was peak billing. Very similar to what we have on the mainframe, right? You have peaks there in MIPS, which we call, right? And that bothered me a little, you know? I thought the projects would end up getting stuck a bit too expensive Hey, back then I met Eduardo Passini, right? Who is your director here today? regional director And then I started asking him, How does it work, huh? MongoDB in the Cloud Then he told me it was Atlas. And I started getting more excited about the topic. We decided to run a benchmark. for both technologies to be fair. So we called Microsoft, We called the team, the engineers, The architects of Mongo. Each one configures their product in the best way possible. And we used the API developed by us there so much, Regarding the market API, to run a benchmark. A fair benchmark, right? With the partnership between the two companies, a Microsoft and a MongoDB. And the results were surprising, right? In most of the cases we tested, in the majority of workloads, No, in 100% of the workloads I can say, Oh Mongo, he had superior performance 100% in some cases. Latency, sometimes 120% smaller in all cases, there, latency. The cost as well It was much smaller, for you paying for the tiers you allocated, and no longer by peak charge of RUs. And that's what opened up The way to Atlas, huh? And then we started the first project No Atlas, which was a project about log ingestion, which we migrated Yes, that was the first migration we made a relational one that ran on Oracle Oh, hack, and it's an environment Limited, we couldn't climb. It was expensive, it was a Primacy, What about? And we started to set up this log management environment for multiple the channels we had to ingest No Atlas, so he went through Databricks, She got some treatment, we even showed it the architecture diagram for Mongo Local, And in this project, he went in there with 6 shards from the beginning, the first time we used shard in the bank, and the project It was a success there, right? Truly demonstrated the capability what Mongo had scalability and environment stable, and offering performance When well modeled, right? This comment is excellent. because I understand that these Two abilities are things technologically innovative that we did not have the possibility of doing it earlier, that is, in the part about the asset modeling, we begin to gain access to data modeling patterns impossible to be made on relational databases. One of these patterns is the embedding, or embed different business entities in the same document That's the sub-document. Other patterns more sophisticated, arrays, they are things that work well for us to put there relational, and MongoDB works in a more natural way with this, in data modeling. But you also said another word Here, what is terror to the listeners of our audience, What is sharding? And sharding, which is growth horizontal of a database It is something that enables the loading of work. Heavy. And I'm understanding that you're running critical and huge You need sharding, right? So I'm understanding the following, These technological possibilities also affect you allowed, in some way, to bring a growth innovation from scale to business an 80-year-old company. Is that right? No, that's right. The concern we had, mainly in those early days projects, it's about taking care so you don't burn the technology. That is a fact. What could it be happen for this to turn into reality, what would be our worst nightmare? The wrong use. And as we had professionals with extensive knowledge relational You must be careful that they don't bring it in. the concept of modeling from relational data to a database The cycle, which changes completely, right? The way you do this is different. You You must think first, How is your system going to work, huh? To model your data according to With that. So back then, I even asked for help. Do Pietro, do Edu, to bring someone Professional service to accompany these projects logo face to face for what they were born certain. And even for Evangelize knowledge within the team itself The one who was working on this first project, right? It was supposed to be a success this project so we can earn confidence and scalability. So, We couldn't afford to take the risk of modeling Wrong, still thinking in terms of relational databases, Né? So, that... That's a true statement, right? What do you read together, You store them together. So, thinking in that case, reduce the amount of the tables, reducing it there in the embedded modeling, where you no longer need to keep doing joins, what was the great a performance issue you sometimes had in the relational database for Large volumes, huh? So, this first one project, today, right? He has around 200 terabytes. This base. That's why you He said the shard. We started with 6, And today, if memory doesn't speak, we're at 15, What? 15 shards. The great... he has potential Absurd, the shard, huh? But I think it's interesting the concern logo of start you choose the key Sure about the partitioning, right? If you set that key, then it also It has to be your lookup key, What about you enjoying it there performing in the best possible way. Because if you delay to redo this shard, and your base grow very large, so that you can do re-shard this so large a base, it might take Dias, huh? It's a bit complicated. So, if you're going to use shard, Here's the tip, right? To think Right on the partition key, for you not having to change it later, which is more Complicated. I think here we already have a couple of tips the most important ones you brought Here. The first, 200 terabytes of data in 15 shards This means a Qualification for the future, right? A potential for scale what you deliver to a business that has a critical load, using This element, right? What's new for those who have never worked in the world in the MongoDB document cycle. So, sharding, that is, scale a database horizontally, It is what allows this type delivery of innovation. And here, folks, we have the explanation of MongoDB What is MongoDB? Database, in Portuguese Database Colossal, for large volumes of information If you still didn't know what What does the name MongoDB mean? Tell me here in the comments, for me. And now you know. The MongoDB Database, or Colossal Database And the other tips you brought, of modeling, are fundamental and very important. What is modeling of embarkation different entities in the same document. And with that, we Free CPU time The time of I.O. You save disk space, junction, It is less performative, because it requires computational resources for recover, reconstruct this information. And you also commented of good choice charging key And there is one more important element. You mentioned MongoDB Professional Services. So, count on MongoDB, with the solution architecture, with the professional services, for what purpose are you the best decisions of adoption of our technology. With this, you have great success these massive loads and work critiques. And You actually brought a journey very significant Not only through the adoption of MongoDB, but how about MongoDB's technology supported a octogenarian organization movement number of the change. What draws my attention is that there are some Interesting movements. I understand that there is a movement of reduction, the famous mainframe offload Maybe there's one more offload from Oracle also in this case journey. And the Your comment on the well-designed one, What is modernization? In the sense of making things that were not possible before in terms of data modeling. As the embedding, which you mentioned here at the beginning of the conversation You are adding multiple business entities in the same document and collect the benefit of this in terms of an I.O.U. in a transaction high-performance critique In this journey, me The following deserves attention. In the Cosmos project, You performed much better. reduced the cost He brought me operational predictability as well as financial performance. Exactly. That's precisely it. So when you have cost reduction allied with performance improvement against these data no argument So there, we truly gained their trust. from the bank's executive, mainly after this first project, to be able to start to gain scale in others new projects. I think this was the turning point of the waters of the decision to begin to use the Atlas more and gain scale. I think that was really what He increased the use of the Atlas at the bank. And I think one point is, I, at the very least, become extremely curious and imagine that you all do too Also. What is the future? The next projects that are coming up After that great success in MongoDB what you had, you managed to leverage. And what else are you all thinking about bringing? Are you already bringing MongoDB along with the database? This first project already has practically One and a half years that we handed him over It's still running. So from there to here We've already gained an absurd scale. We practically have about 189 clusters Already running MongoDB. Then from 1 to 189. This among the staging environments and production. But really We began to use in mission-critical projects. Such as PIX. PIX QR Code that used to run before SQL Server on Azure In SQL Server. He began to become Migrated to Atlas. It was a case. of success as well. And it's running Now also already in the Atlas. A large part of this system. Because the PIX is huge. So there are several areas. There's the QR code, the statement as well. It ran on the mainframe until December. He has to get in. In production running MongoDB. So it's a case of offloading. Check the mainframe and the PIX statement. And we began to actually use it and trust MongoDB for mission-critical projects critique. Even by Mongo architecture. You Works with strong consistency. What we need for financial systems. Ensure consistency. Not occasional consistency. So we have a primary node and two secondary ones. Often We perceive there that The node goes offline because we're monitoring it. And the customer doesn't even notice. He unties the knot in the air to make some update. Something. Stay with two. And keep doing this among the others until finished. The update. We Can you see why we already have access to the console? Do Atlas. More Ultimately, the customer is what truly matters. he didn't realize that this It's happening. It was completely transparent to him. So we started climbing it For critical projects. It also has Safer. What is a bank's anti-fraud project that was running on Hyperscale In Azure, in SQL Hyperscale. And hey, Safer is an interesting case because They came looking for us and they It has been some time now that they need to respond. answer due to Bacena's requirement, the Central Bank's, and the like. And we're going to do a POC. The best way is for you to bring your own workload. Hey, let's see if the technology works. If we don't succeed, we'll think of something else. But it was a success A POC also needs to meet the timeframes they have. They were also deciding to migrate. pro MongoDB. And I think that's It's the best scenario. It's not an indication. mine or the Mongo team saying to use They are testing the load them, their workload I sell the result itself and in practice. The very development team ends up convincing themselves, the business team to use This technology that is truly being disruptive. And the interesting thing is that our area there the area that we to be invited to create this zero area, she works also with automation. So we're creating an entire infrastructure in partnership together with the team also of platforms, Oh Nelsinho, his whole team Ali is supporting us, let's create a portal. No, Bax, we create what we call Tech Products. What is Tech Products? You don't do it, you don't Consume no more products manually. We create an automation there with Terraform In Ansible, where are you eliminate the need for interaction with an engineering department the cloud that needed to open a ticket for you to provision the cluster, then you open another one called pro DBA to create the database, There, I created the user and set the password in Secrets. so this Our department automated all of this. So what we call Tech Products That's it. You handle the provisioning. through an interface we call Bax You choose there the tier you want to use the environment you want to use and when you give a nod there, he's already Provision your cluster Create your Private Endpoint there behind Build your own base Create the user, add to the KB, and return Just the connection string for the user. So we gain an absurd amount of time 5 to 6 days of SLA what you had there among all the areas involved, where you can provision the zero environment around approximately 40 minutes, which is the provisioning time basically from a Atlas database, right? That's really cool, guys. So, like, it was a a delivery Very interesting for business people get free of this bureaucracy and this 6-day SLA waiting time for 40 minutes I'm even moved to tears because one of the questions I hear most often and if we could use MongoDB for workloads what are criticisms of the business which has a high requirement of consistency which has a high requirement high availability and scalability And you just told us that you use MongoDB Atlas prays peaks and for fraud prevention Is that really what I just heard? Why did you just now tell me that you Worked with Relational DBA That's it. I think Relational still has its place in the sun, right? Of course, but It is a disruptive technology. Can you really climb that high vertically? how much horizontal and no relational when we needed to do O shard, we used to do Moreover, it created partitioning in the table through a date or something like that But you couldn't climb that up for other servers. The shard, what do you do That's exactly it, you put in the shard It is a fragment in Portuguese. So you take a fragment from your collection a piece of it and divide it into many clusters, then you win big write throughput, mainly and also for reading. What's interesting is that when we talk From Atlas, we don't speak Just from the primary database, we have a whole ecosystem where we are Still learning, but it's interesting It's an entire ecosystem. of tools there that we can use such as the Online Archive An online archive is something Very interesting where you are not Needs to code, that's the kind of wizard what you configure a rule for you to move your die hot for a storage account This reduces the size Do your backup, then reduce your cost and this also gives you the possibility of you creating a hot area and a cooler area warm or cold access to your data but even more interesting is that you continue with the same private endpoint then the application doesn't need to have this concern about when to access a data or when to access Outro, this is configurable so this entire ecosystem that enables Data federation is something else. These are things we are learning It's the partnership we have with Mongo It is truly making a difference. Pedro, there's another point to consider here as well what I found very interesting was your comment What does it have to do with productivity? the people, so you mentioned that you have a series of automations of Terraform, of Ansible what you deliver to the requesting departments the necessary infrastructure packaged a lovely gift package Plus the tape on top of it dourada, that is, you are empowering the demanding areas and your area is not a gargalo for this work to take place and in the same way the The online archive you mentioned doesn't need to be coded then the developers they feel more productive, because if it means less code you have to write You deliver faster So I think here is a second Very interesting comment one of the great pillars of MongoDB What is productivity? the developers and consequently, manage to deliver to the market more innovation, new features new applications but all of this as well, you have to have a good rapport with The product team, understand thoroughly What are the products, since you mentioned that the MongoDB has an entire ecosystem Tell us about it How is your interaction with the team going? product of MongoDB I think this has really been the the difference of this partnership, this alliance the strategic approach we created with MongoDB In fact, we have the opportunity to directly influence your backlog, and that It's actually interesting, you know It started when we had to use the Online Archive for a First Project and there was no availability in the Brazil region, South Brazil South Azure I only had availability in other regions So we talked with your team Hey guys, Pietro, Edu Filipe, also an architect, helped a lot in this phase to demand that from the team of the United States We had a few remote calls with them. and we can ally show why we needed that and in a few months it was ready for us to use, they made drumbeat, indeed, out of necessity to Bradesco, which is very difficult for a company to do and this interaction with the team of the product, she only gained strength for example, there was the CABE CABE, I just got back from there Actually, it was in September. Now, the CABE, for those who don't know It is the Customer Advisor Board where it takes place every year in New York with the main clients global and Bradesco, I had the honor to represent Bradesco at this event So we get to a big table there, huge with a lot of people And there were people there from Wells Fargo by J.P. Morgan of other major companies in the financial sector Can we talk? about what they are still going to launch They present the roadmap. what they are planning to launch We can give our opinion. and influence the product backlog I found this fantastic. Well, thank you, Pércio, so far I think it was very enriching for everyone. And let's play a quick game here questions for you to answer Short, but to wrap it up Here, for you, Pércio Personally, what does MongoDB mean? I think it's availability The performance I think it defines it well If you had to give one piece of advice for the people who are here accompanying us What would that advice be? I think that to provide an opportunity to try the product build an MVP carry out a proof of concept, give it a chance to discover what the world is in the cycle mainly those who worked for a long time, just like me with the Relational Database Sometimes there is a certain prejudice You fear one can say, to trust a new technology against technologies that have been around for 60 70 years in the market, what is a Relational Database but that's it The tip is, go after it procure, study how to do it differently What benefits will you gain? I think that's it, it's about experimenting. Give innovation a chance This is the tip: give innovation a chance. And how do you see the future? AI, short, medium, and long term? This topic is well, the topic of the moment Everyone talks about AI In my team, we also have an initiative. Quite interesting use of artificial intelligence artificial intelligence agents After that, we can just do only Just talk about this because the topic is lengthy but basically it's an agent that will serve not only to clear up doubts for everyone It will have integration with Teams, so you will have opportunity to speak there using Teams a humanized language as if talking to a person You will have an AI agent behind it He won't just answer questions from your environment technical questions You will also answer database questions relational that people also use the database in the cycle I'm going to give you an overview there of the costs also and will execute actions That's what's cool You'll be able to take some actions there so it won't be a technology that you will use Just for information She's going to carry out the action; I can't say much yet. besides, which is under development but soon we can comment a bit about that, but The use of AI will explode, I think, in the market. last quick question How do I work with you? in a successful company from the Brazilian market with successful technology What about MongoDB? It's interesting that you mention this because We had two positions open for the service team. We have the product team. and the service time that is our father What help is there there to maintain the production environment And I'm going to tell you that was not such an easy task You hire qualified people who had skills primarily in Database NoSQL, not just for Mongo but also for other products, for Redis So you have a difficulty in general, finding professionals qualified to work with NoSQL So here's the tip You who are the DBA Relational Database, takes advantage It also specializes in NoSQL that will broaden your horizons and your possibility of work And here's also the tip. to know automation even better recognize the two worlds, automation and NoSQL It was something we were looking for a lot for our team. We have just closed two positions. There was a person who started today on the team. that I should also welcome him and the other person who is about to enter in the coming weeks So I think this is the tip study these disruptive market technologies and truly you will expand your range of activities Guys, I'm just going to say Here's the tip follow Persio Bazan and make the most of it a MongoDB University that has all the trainings free I'm dealing with this topic; just one more piece of information In that sense, I think it's really cool to share We launched several initiatives. there on the learning bench to train the professional to use MongoDB some of the partnerships we've made with the DS, with the system development team helped leverage this We held an event there at Coco Bambu. The first event that we held with more than 300 developers and after that, some learnings in partnership with DS and the people studying that you mentioned through the platform excluding certifications and badges what you gain when you do Last week we had The news given by Helder What is your VP? that Bradesco is the first in Latin America professional training with the highest number of certifications and badges So the work we did was fantastic. first in Latin America and fourth overall in the world so I think this part about promoting to train our development teams There, I think it was an absolute success. The partnership with you and through the platform Do Mongo University Hey, check out this interesting news, folks Bradesco investing in the training of your professionals So for you who was already looking at Bradesco as a possible next step in your career Check out this cool movement and furthermore, for you who already want take one more step forward In MongoDB, there is the union of the two worlds. MongoDB, University and Bradesco, giving it all this opportunity for us Well, unfortunately we've reached the end of another episode Pércio, I wanted to thank you personally, because you're here today It's very inspiring; I saw a company the Bradesco door investing and continuing to be a reference In technology and we use MongoDB to support you I really appreciate your participation. It's me who thanks you for the invitation, sorry to have to I've pointed it out so many times, but today it finally worked I am very happy to be here and thank you for the invitation. Everyone, we are the ones who thank you, Pércio. It's important that we're together here and mainly sharing This story and this knowledge with the MongoDB community here in Brazil This was the Hi MongoDB, and today we're talking with Percio Bazin Bradesco, which told us A beautiful story of workload criticism, high volume data, running successfully Here in Brazil it's with MongoDB We see you in the upcoming episodes