Software Startup To Success In One Month

If you're a professional software developer, it's not all that often that we get to start something new, untrammeled by past, sometimes bad decisions. So when we do get that chance, how should we start our new project to give ourselves the best possible chance of success, whatever it is that we're doing? That's our topic for today. Hi, I'm Dave Farley of Continuous Delivery and welcome to my channel. If you haven't been here before, please do hit subscribe and if you enjoy the content here today, hit like as well. The cleanest of clean sheets is in a starter. We haven't yet accumulated any of the institutional scar tissue or weight of pre-existing code that generally exists in all pre-existing companies. We have a genuinely blank slate and are free to make our own mistakes and maybe our own successes. This may sound idyllic, but it also comes at a cost. There's always the threat that we all make the same mistakes as everyone else. After all, those slow-moving bureaucratic hidebound institutions that we are so sure that we can improve on were also startups once. Having a blank sheet is a challenge at the same time. So how do we make the best use of that freedom and how do we rise to the challenge and avoid the pitfalls along the way? The real problem with the startup and the freedom that it brings is that we don't yet know anything. We presumably have some cool idea for something new, but we don't know whether these ideas are right or good. We don't know how to do them yet and we don't really know how best to organize ourselves to do all of those things either. So what does the first month look like? What should we hope to achieve in that kind of time frame? In this episode, I want to describe my own experience of a few real startups, how to begin a project well, and what our primary target should be in that first month of development, and what counts as success at this very early stage. Watch the end and I'll give you six key targets to shoot for in your first month. I will also recommend other episodes that explore different aspects of all of this along the way, so do keep an eye out for those too. Let me pause there and thank our sponsors. We're extremely fortunate to be sponsored by Equal Experts, Transfic, Tuple, and Honeycomb. Equal Experts is a multinational consultancy built on applying the kinds of ideas and techniques that we discuss on this channel all of the time to build great software for their clients. Transfic is a financial technology company applying advanced continuous delivery techniques to deliver low latency trade routing services to some of the biggest financial institutions in the world. Tuple builds software to make pair programming easier for people who work remotely, and Honeycomb help engineering teams deeply understand their own production systems through observability with their tools. All these companies offer products and services that are very well aligned indeed with the topics that we discuss on this channel every week, so if you're looking for excellence in continuous delivery and software engineering in general, click on the links in the description below and do check them out. The truth about startups is pretty bleak really. 90% of them fail in the first two years, 34% because they have such a poor market fit, and 22% don't have a sound marketing strategy at all. So what has this really got to do with the tech? Well, for a tech startup, we the technologists are the ones fulfilling the vision, and I'd argue that even more than in more established companies, technology and technologists need to be partners with rather than a service to the business. If our aim is to innovate and differentiate ourselves in the marketplace, which should be the aim of any startup really, and so give AMP startup a better chance of success, then we as technologists are probably better placed than most people to see business opportunities in the technology that we are building. At least we should be in that position. I think that if you're working in a startup, you really should be a believer in the vision. You need to buy into the proposition that the idea that the firm is founded on is a good one, and you want to get involved and make it a success. Because one of the freedoms and challenges of working in startups is that you need to identify problems yourself and fix them yourself, rather than wait for others to tell you exactly what you need to do. Some people love this and some people hate it, so the first steps to making a startup a success is to understand and figure out what you like about the vision personally and really understand that vision in some detail. I wouldn't get involved with a startup without this kind of understanding. I was the first non-founder hire at one of the startups where I worked. I was tasked with running the software development. I was recruited by an old friend who was the founder and CTO. We went out for dinner one evening. I thought that I was selling him some consultancy from my employer, but actually he was selling me a vision, and it turned out that he was right. He said, I want you to be the head of software development. You can organise this however you like. Cool, I thought. We're working on a problem that no one knows how to solve. Oh, very cool, I thought. And we have to figure out how to solve it ourselves because it's not the kind of problem that's written down anywhere. Oh, really cool. And if we can do this, we can probably shake up an entire industry. I'm in, I thought. And maybe if we do this well, we'll make a lot of money. Nice. By the end of his pitch, I was sold. How did we start in that very first month then? First, we needed to explore the problem enough to be able to decide where to start. I was keen to start as soon as possible, but no sooner. We didn't want to start writing code aimlessly. There was a little bit of that going on at the very beginning. The founders had recruited a few contractors as a stopgap to get things started while we were looking to recruit and build a more permanent team. And they'd already begun working on some code, none of which actually made it into the product that we eventually built. So our priorities were to quickly decide where to start and to start recruiting a new development team. We spent several days discussing in some detail the problem that we needed to solve, building a broad picture of that product vision that I was talking about. We were going to build a high-performance financial exchange, and we were going to make it open to the general public over the internet, which was quite novel for that kind of thing. So we had a lot to learn in fields of high-performance computing and in the intricacies of complex trading environments like in derivatives. We had some domain experts, the founders, to help us to explore the problem in reasonable depth. And we had some technologists, me, my friend, and a couple of others who were going to be part of the team, to begin exploring initial guesses at solutions. Part of this exploration was to identify any problems that we foresaw, so that we could check them out, or at least identify areas where we didn't know enough yet, and so we could better target our learning in those areas. Our system was aimed to be ultra high-performance, and so solving the performance problem was certainly at the forefront of much of our thinking. From day one, this was part of the business vision though. This wasn't just technical challenge, this was a challenge for commercial reasons. We needed to outperform our competitors in terms of raw throughput and latency for the business vision to work out. So clearly, we needed to think about the scale of the challenge very early on. We didn't need to design for high performance immediately, but we did need to be careful to avoid any obvious dumb mistakes that we couldn't undo later if we could. We took a very experimental approach, so one of the early experiments that we spent a few days on was measuring the performance limits for a relational database. If you pushed it really hard and really tuned it, how fast can you store information into and retrieve information from a relational database management system? That experience ruled out relational databases as a solution for our exchange. Something that we guessed would be the case, but it was worth trying anyway because if the database had been fast enough, the rest of our job would probably be a little bit easier. So within about a week, maybe a week and a half, we had some hard data on the maximum that you could achieve with database performance, and a much better picture of the vision of our fledgling company. Meanwhile, we started approaching friends who we knew to be very good to seed the core of the development team. By this time, we also had a crude whiteboard sketch of the future of our system, some guesses about how to organize the system and to start building it. At this stage, this was a very rough allocation of behaviors. We'll have a fast bit for hosting the order books for markets, there'll be another fast bit for managing the state of people's accounts, and it was at this early stage that we decided that not everything needed to be ultra fast like that. There were other functions of the system that we could treat as more normal computing problems, things like account registration and administration tools for creating and managing markets in which to trade. This started to structure our thinking about the design of our system so we could start building things to try them out. We made some very early decisions that, with hindsight, were foundational to what came later. Things like building at the high performance core of our system as an asynchronous service-based event-driven thing. These were very good guesses based on our experience for building previous systems, but they were still only guesses, and we didn't assume that any of them would stand up as we learned more. Instead, we assumed that all of our guesses would probably be wrong. In reality, some of them were wrong and others were pretty good choices. We took an experimental evolutionary approach to architecture and design and so we discarded some ideas very quickly and kept revalidating all of our ideas for the rest of the project as we went. We began with a handful of simple user stories and, based on our whiteboard model, we started work on building them, using the first ones to establish continuous integration and build a very simple deployment pipeline, so establishing the core of our development approach, if not from day one, then at least from the end of the week one. This was the walking skeleton approach, using a sketch in code of our system to establish the deployment pipeline. We adopted continuous delivery from day one, so one of the goals of our first story was to build the simplest possible deployable version of our system, and we did. This communicated between our services via our own abstraction for message-based communications. This was based on some work that I'd done previously with a view to open sourcing it, and we had a simple web-based UI based on some other hobby project code from Martin, our CTO. We also started right from the beginning with unit tests and some basic acceptance tests that checked that our basic pieces worked together to do something useful, so we could make a working production quality, whatever that meant at this stage, deployable system and demo it, even though it didn't yet do anything very interesting for anybody else. But we were learning fast, and we were learning what we needed to build, how to build it, what things seemed to work out, and crucially, perhaps most valuable of all, what didn't. As part of our defensive evolutionary approach to design, we were quite diligent in using the techniques for managing complexity, even though I hadn't defined them in the way that I do now, at that time. Check out my book if you'd like to learn more about that. This meant that when we changed our minds or learned something new, the change was usually fairly isolated. So, for example, when after a couple of months we started doing more serious system level performance testing and found that we were nowhere near our target performance in throughput or latency, meaning that our high performance architecture was a bust, we could start trying new things and still keep quite a lot of the code that we'd already written. In particular, because by design we had tended to separate accidental complexity like message transmission or UI interactions from the essential complexity of the logic that encoded the business behavior of our system, it meant that we could change even some of the architectural foundations of our system and keep the core of it intact. Some of this happened by accident, some by design. Because we started using some of my code for inter-service comms, which we knew would never be fast enough, we hid the detail of the actual communications from the services that relied on it. We did this by creating an abstract interface to the communications. So, if you wanted to send a message from one service to another, you called transport send message, which took a message containing an address and a blob of binary data. The transport turned that into a message and sent it to the right place. This meant that when some months later we decided how to solve this problem properly, instead of transmitting the message as xml over HTTP, all we had to do was to recode the message translators and to implement a new transport. All of the services that define the logic of our system remained completely unchanged. This is one of my favorite examples of a core idea of working incrementally using design to defer decision making. In the early stages of a project, we aren't usually ready to solve all the problems, so we need to pick which problems to solve now. The ones we understand are the best ones to work on and hide the rest by design. My advice is to concentrate on making something that works and build that code to production quality, but in a way that you can add to it when you learn more and change when you find out where it is wrong. And every time you get to a point where your answer is we don't know how to do that yet, that's a sign to use abstraction to isolate the bits that you understand well from the bits that you don't. The best way to do this is by always designing things from the perspective of the consumer rather than the producer of any behavior. In my example, we wanted one service to be able to talk to another, but we hadn't decided what technology to use or what the on-the-wire encoding should be like. We didn't know the answer to those questions yet. But from the perspective of the service, what do you need to know to send a message? You need some form of an address to say where you'd like the message to go and you need to be able to put the information in the message and send it to that place. So if we code that abstraction, we can be pretty confident that we can define translators between the addresses that we're using and the content of the message that we're using for any technology that makes that happen. So after about a month, our target and the target that I'd recommend to you is this. Establish a working deployment pipeline that runs all your tests and gives you great feedback on the state of your system so that you can keep your system in a permanently releasable state from then on. Establish a working model for where you want your system to go. Product vision and technical vision both will be wrong at this stage, but if we get the rest of these things right, we can easily change them later when we find out where they're wrong and why they're wrong. Start building the team, but you need to make progress on everything else while you're working on that. Build a working version of your system to validate your models if you can. Get it in front of people to see if your vision still makes sense. Ideally, get it in front of real users if you can, but internal proxies for users if you can't will do. Defer decision-making by hiding more complex stuff that you don't yet understand behind abstractions from the perspective of the code that you do currently understand. You'll be able to implement that stuff later when the time is right if you get these abstractions from that perspective. Sanity check the seriously risky stuff with experiments. Try stuff out in small controlled throwaway explorations. This should be throwaway code so that you can move fast. The learning is the product from these experiments, not the code. This is my approach to starting anything new really. It's based on the idea from Dave Thomas that big design up front is dumb, but doing no design up front is even dumber, which I agree with very strongly. Our real goal here is to get ourselves into a position to be able to try out our ideas for real as soon as we can. We need to give ourselves the freedom to be wrong. To my mind, that's the real target for the first month of a new project, to win that freedom. Thank you very much for watching. I hope you've enjoyed the video today, and if you like our work on the Continuous Delivery channel, do consider supporting us by joining our Patreon community. And to our existing patrons, thank you once again for your help in keeping us going. Thank you, and bye-bye.