How I Review AI-Generated Code

Most developers are using AI agents to write code but how do we know if the code they produce is any good? Research shows that AI generated code has significantly more bugs, more security issues and more logical errors than human written code. So if you're shipping AI code without a review process you're taking a huge risk. When it comes to AI code review the landscape is pretty confusing there are so many different tools and options available to us as developers it can be hard to know where to start. So in this video we're going to walk through the four levels of AI code review and by the end of the video you'll know how to write high quality code with AI agents so let's get into it. I just want to start with the problem statement before we look at any specific code examples and the problem is that bad code in production is really expensive. As an extreme example Amazon loses around 13 million dollars for every hour of downtime and typically downtime is caused by some kind of code change. So obviously code review is one way we can mitigate this problem. The idea is that one developer will write code someone else will take a look at that code so a second pair of eyes will look for potential problems with the code. Often code review results in higher quality code we find issues it leads to a discussion about the right way to implement something and generally everyone is better off by having a code review process. So when you think about code review there are two things to consider the first one is AI. AI can spot things that humans miss so you definitely want some kind of AI agent in your code review process. I found that AI agents can drastically improve the quality of your code because they will just spot issues that you won't see yourself. They don't sleep they can scale to any volume they can scan hundreds of files very very quickly and find issues that you would miss but equally humans can catch things that AI agents will miss as well. So you want both things in the mix humans have a lot more context about the AI agents do so you definitely want some level of human review especially if you've got a difficult change but as we scale with AI agents we want to rely more and more on the automated element of the process so that ideally humans are reviewing very very minimal changes or only reviewing things when they really need to. So I think about four layers when it comes to AI code review the first layer is automating the obvious things so it's not uncommon for software teams to argue about formatting or type checking or code standards and things like that so you want to have some kind of automation in place that deals with all of that stuff straight away. The way you can do that now with AI agents is to use automated hooks. They will run deterministic code to check your types to check linting to check your code formatting to run your tests and even to do security scanning as well. You want to have that automated from the get-go because that will drastically improve the quality of your code. What happens is that the hooks are triggered and then the output is fed back into an agent who can then self-correct. Layer two is to review the code locally this is basically just using an AI agent to review the output so there are all kinds of options here but the general idea is use some kind of AI agent to review your code. The third layer is having some kind of automated check for example on github you could have an automated process that scans the pull requests as soon as it's opened looks for problems so before a human even looks at the code some kind of additional automated check has happened this is like a safety net because often people get tired I do as well so I often forget to run a code review process so having something on your github environment for example is very very valuable and then the final layer is obviously a human review by the time you get to layer four a lot of the issues should have already been caught within layer one two and three. So let's start with automating the obvious so this is basically using deterministic hooks within your AI agents to run basic checks so we can set up checks within Claude Code to verify we can do type checking linting all of that good stuff so I'll quickly show you how that works. So here's an example of the kind of thing you can do with deterministic hooks at this layer basically when Claude Code finishes doing its work you can run a script via a hook and so in this example I'm working on a Ruby on Rails project and what I've got here is a Ruby code linter which runs at the end of the the agent run I've also got a security scanner that you can run as well so these are basically catching fundamental errors in the code and the way you wire this up is you can set up a hook within Claude or your coding agent so what happens is when the coding agent has finished working it will just basically run this script and you can actually run this by hand to kind of see what this looks like so just to show you this in action we can actually go ahead and just run the script so this is basically what Claude Code does after it's finished doing its work and I actually added a typo into the code base so you should be able to see that this will fail with an error you can see here an error where the controller has a basically just like a rogue statement so the syntax is invalid but this will be caught then by your coding agents and so anytime you have some kind of formatting issue or linting issue or type error you can use your language specific tools to catch those issues early on okay so layer two is where the fun happens this is where we're going to use our AI agents to review our code but before you do that just two things to think about it's surprising how many times people push code to review but they haven't run the code locally this is just one of my bugbears in general with development it's always good to run the code locally don't just only rely on the tests an example here would be if the UI is completely broken or the application doesn't even start up for example and the code has been pushed for review it's really really annoying the other thing to kind of think about is just review the diff manually as well just kind of give it a glance over look for any issues but the really interesting thing now is to use an AI agent to review the output of another agent so there are so many different ways to do this there are a bunch of official plugins from Anthropic I will show you in a second I use a custom command because I can tailor it then to my specific needs I started to use this idea of a review.md file so the idea here is you have a markdown file in your repo and so you do a general code review but then also you can have some kind of special process so if you've got some specific things that you want to look at within your project you can put those specific things in this review.md file which is super useful I like this idea Codex from OpenAI is a great experience when it comes to code review there is a built-in review command that you can run and it's really really good out of the box I think they've done a great job here better than Claude Code in my opinion and then the other thing you can do is use a tool like CodeRabbit or some kind of external code review tool this integrates well with Claude Code so this is the documentation for installing CodeRabbit which is the tool that I'm using at the moment but it's really really easy to set up and as you can see here you can basically install it as a CLI and then it plugs into Claude Code so all you need to do then is run this command inside Claude Code and it will do a really deep code review here are a bunch of official Claude plugins you've got the code review plugin here and if you actually go and look into the source code you can see what it's doing I always like to remember that there's no magic here this is just a prompt and honestly you could just say look at the code and find errors and you can say that to Claude and it will probably do a pretty good job but this is obviously a more detailed prompt what I don't like about some of these tools is that they're using like the haiku agent stuff like that I really don't like so I wanted to do my own there's also a code simplifier as well that you can look at so this one again there were some things I didn't like about this so for example it's talking about using JavaScript modules if you're not using JavaScript this is kind of annoying so there's a lot of assumptions built into some of these plugins I found they weren't generic and I didn't really like that so the command I'm using myself is just a really simple prompt it just says to go and spawn a sub-agent to review the code changes we're reviewing the current staged changes and we're also looking for this review file to look at any project specific things we want to look at and then we just evaluate against correctness so does the code do what it's meant to do security you know checking validation and things like that not having secrets in our code simplicity I think this is a really important thing to think about because AI generated code might be quite verbose it might work but it might not be like optimal I think this is always something worth checking could you simplify the code without losing functionality could you make it easier to understand could you make it like cleaner or simpler AI agents will often find a way to improve the code and then things like will it break under load concurrency race conditions and stuff like that and then what I like to do is group the findings and this is a really important part when you're using AI agents to review code they will often identify many many things that you could fix potentially but not all of those things need fixing so you want to focus on what are the most important things to fix so must fix are like critical issues but then we often have minor things that are worth paying attention to but maybe they don't block the the pull requests in general so grouping them is a really really good idea all right so let's spin up Claude code and see if we can find some code review errors so I'm gonna ask Claude to add in an SQL vulnerability to see if we can catch it can you add in an SQL vulnerability I'm testing out code review tool so let's see if we can just go and add in some kind of vulnerability to kind of test okay it looks like I added okay so I was testing this earlier so we already have a vulnerability so that's fine so we can go ahead and run one of our code review tools now so let's go ahead and I have a bunch of different review tools so I have my own but there's also a bunch of built-in review tools here as well so you can see we've got CodeRabbit we've also got the official Anthropic plugin as well so I'm gonna run my own review command and see what we get all right so as you expect the code review picked up the SQL injection vulnerability which is great there's another one that picked up here and then yeah there's just a bunch of other minor things like a race condition as well so this code review did its job it found a bunch of issues so let's also just have a look at the CodeRabbit example as well just to see if we can get a similar result so we're going to run the CodeRabbit review command and see if we can also kind of find other things using this tool all right so we finished the code review with CodeRabbit which is our external code review tool it did find the SQL injection vulnerability as you'd expect it found some nullable things on our database schema yeah so again it did a pretty good job on the code review obviously you can see the value of having something like this installed in your project because there are often so many issues almost every time I run a code review with AI generated code there's something it finds so there's normally something to fix okay so now that we've reviewed our code locally when we push our code up to GitHub we also want to have an additional check so there's a couple of ways you can do that but the one I'm using myself is to use Codex I quite like this OpenAI's Codex integrates well with GitHub this is the page there's really good instructions here on how to do this and the idea here is that you can just say at Codex review within your your code and then Codex will go and do a review of your code and you can also set this to automatically review so anytime you push up a change this should automatically run let me just show you an example of that in action so you can see here I've opened a change on GitHub you can see here the file's changed and if we go to the conversation you can see here that Codex has flagged three issues for this change issue one issue two and then issue three so obviously these are things that you may not have caught locally it's just like an extra layer of defense that you can have to catch potential issues the other one that you could take a look at from Claude or Anthropic is this code review plugin however this is only available in research preview for teams and enterprise this is also very very expensive as well so I think it's like 15 or 20 dollars per code run so this is too expensive for my own use cases but this might be one you want to check out if you work in a big team and then obviously the final layer to all of this is just to have some kind of human review you always want to kind of have some eyes on your code even if it's just to kind of give give you a sense of what's going on like I said humans often have more context than AI agents about the business or the product or whatever environment you're working in and you want to kind of tailor this so the more serious the change if you're making a database migration change or an infrastructure change you want more human review but if you're just making a documentation update or a small bug fix you can probably largely rely on some of the automated tooling especially as we scale but human review is always important so my final takeaway is that there isn't a perfect tool the most important thing is you have a process I wouldn't worry too much personally about the particular prompt you're using or the plugin just remember that when you when you finish writing AI code always be very suspicious always assume there's a problem with the code and get another agent to review the code even just doing that simple thing with just the most basic prompt go and look at the code see if you can find any errors just doing that alone will catch so many things and then the other highest impact thing you can do is probably have something on your CI system so have a automated code review built into github the value of doing that is really really high you set it up once and then every single time you open a change you get a free code review that's a really really valuable thing to do as well I hope you found the video useful if I've missed anything let me know in the comments if you've got any suggestions for tools I'll link all of the resources in the description below thanks for watching and I'll see you in the next video take care