In this first episode of CrewAI: Noob vs Pro, I’m tackling one of the biggest beginner mistakes I see: the God Task. I’ll show you:
1️⃣ What a God Task is and why it’s a common pitfall for new CrewAI developers. 2️⃣ How to refactor your Crews for better results (and avoid scaling headaches). 3️⃣ A comparison of the Noob Crew (with a God Task) vs. the Pro Crew (optimized for success).
Want me to analyze your Crew in an upcoming video? Send it to [email protected]—your project might just get featured!
Whether you’re just starting out or looking to level up your CrewAI skills, this tutorial is packed with practical tips and strategies. Let’s build smarter, more efficient Crews together! 🚀
Timestamps: 00:00 Start 01:11 Call Outs 01:50 God Task Overview 06:47 Noob Crew with God Task Analysis 11:18 Noob Crew with God Task Results 14:56 Pro Crew Refactored Analysis 21:51 Pro Crew Refactored Results 25:34 Outro
Hey guys, and welcome to the first episode of Crayi Noob vs Pro. And in this series,
I'm going to share with you some of the most common mistakes that I see beginner
Crayi developers make, how to avoid those problems and how you can start to build crews like a pro.
Also, if you're new here, welcome. My name is Brandon. I'm a software developer at Crayi,
and I also have a free school community with over 4,000 AI developers, where I've helped
hundreds of them build better crews using everything that I learned as a Crayi developer.
And in this first episode, we're going to cover one of the most common mistakes that I see,
which I call the God Task. And by the end of this episode, you're going to learn what is the God
Task, why it's such a problem, and how you can avoid it and build much better crews.
And to help make everything we're going to learn today crystal clear, I've actually built
two demo crews for you. In the first demo, we're going to show the noob crew that has the God Task,
so you're going to see all the problems it causes in action. And in the second demo,
I'm going to show off the pro crew, where we've completely fixed everything, got rid of the God
Task, so you can see how much better this pro crew performs. I cannot wait to show you guys the
results and show you these demos in actions, but enough talking. Let's go ahead and dive into the
rest of the video. Oh, and before I forget, if you would like me to analyze and improve your crews
right here on this channel, feel free to go ahead and shoot them over to me at Brandon at Crayi.com.
I cannot wait to see what you guys are working on and how I can help improve. And second,
if you would like to meet other like-minded AI developers and get some additional support,
you're going to love the free school community I've put together for you guys called the AI
Developer Accelerator. We have over 4,000 members. We have weekly free coaching calls
where you get to hop on a call with me and other developers where we get to work on and help
improve whatever you're working on. So can't wait to see you over the community. More information
down in the description below. But enough of that, let's go ahead and hop right back to the rest of
the video. All right, so now it's time to dive into the nitty gritty and actually look at what
is a God task, why it's a problem, the key signs of a God task so you can identify them in your own
crews, and then finally, how to go off and fix these. And then after we cover all the basics,
we're going to hop over and actually analyze those two demo crews that I was talking about earlier,
the noob crew where we have the problem, and then the pro crew where we actually
implement everything that we're going to cover in here with the solution and see how you can
actually fix the crew. All right, so let's go ahead and dive into these one at a time
so that we're on the same page. Okay, so first off, what the heck is a God crew? So the key thing of
a God crew is that it's a single task that's just trying to do way too much. At first, it might look
pretty simple, such as like, oh, go off and perform this task. But really, when you dig deeper into
that task, you'll realize that there's actually like 10 things going on underneath. So it's just
doing way too much. And this causes a bunch of problems, such as whenever you're actually trying
to execute the task, you'll notice that the agents get lost, they lose focus, they don't
really follow what you're trying to tell them to do. These agents, they're coming super far,
they're getting way smarter, but they're still not like geniuses yet. So they need things to be
broken down into small tasks so that they can understand exactly what they need to do. They
need clarity and simplicity. Next, what you'll also notice is in these God tasks, because there's
really a bunch of subtasks happening underneath, the agent will often go off and skip certain tasks.
So it might have 10 smaller tasks and it'll skip item four and six, even though you've clearly told
it, do this, it won't do it. And it's just because the agent's trying to juggle too much
and loses context of what actions it needs to be performing in this moment. And then finally,
one of the other things you'll notice is the results of this task, they're not going to be
what you would expect. Because it's accidentally skipped a step or two here and there, it's lost
focus of what it's trying to do. It won't give you the expected results that you would expect
this task to perform. So that's some of the major problems. And because at the end of the day, we
just want our tasks inside of our crews to work as expected. So what the heck are a few key signs
of a God task? So there's really four. And hopefully as you're going off, going and working
on your own crews, you can start to identify, oh yeah, that is a God task. And I need to, you know,
you know, step one is identifying it and step two is going off and fixing it. So here are the four
major identifiers of a God task. So first thing, too many tasks. Ideally, you want to keep your
task, like you want to keep the subtasks within it. And you'll see this all makes sense when you
go look at the demo crew together, but you'll notice that there's too many tasks in a task.
So what's happening is there's 10 subtasks and that's just way too much complexity for our agents
to take care of. Also, it's too broad. Task should be focused on performing one objective. And when
you actually look at some of these God tasks, it's like, oh, research this, analyze this,
make a report and also do this. So it's just, it's doing way too much work. And then finally,
it's actually kind of confusing. That's another one. Like if you actually go through and read
your task and pretend you just handed it off to someone else, you might notice like, yeah,
it's actually not clear what I should be doing or, you know, oh, it does make sense. But if I
was to give it to a fifth grader, a like hyper genius fifth grader, they wouldn't understand
what all to do and they might skip something. And then finally, it's hard to debug. And what I mean
by that is after a task is done running and you look at the results of that task, you might see
like, oh, that's weird. It didn't do what I expected it to do. You know, like the results
I'm getting, it's not what I told it to do. So you're kind of stuck in this weird state of like,
I told it what to do, but I'm not getting the results. So it's hard to debug and figure out
why the heck your task isn't working. And the root issue is just, there's too much going on.
Okay. So now that we've talked about everything else, how the heck do we fix these God tasks?
So it really comes down to three key things that you're going to see later on when we analyze the
pro crew together in just a second. All right. So first things first, what we need to do is
ultimately break up this monster God task into a bunch of small hyper clear tasks where we have
clear inputs and outputs for each task. So that's kind of one and two put together breakup and make
inputs and outputs very clear. And then finally, we're going to use the context property on a task.
And what that's going to allow us to do is go, Hey, we now, instead of having one monster task,
which is going off researching, doing all sorts of stuff, and then pass that result over to the
next group. What we can do is actually break things up into like research, this research,
this research, this into three smaller tasks. And we can combine the output of all three of those
tasks together using context and pass the result of all three of those small tasks over to the
next. And you're going to see exactly how to do this inside of crews in just a second, but this
is ultimately what we have to do. So enough talking about hypotheticals. Let's actually go ahead and
dive in and start looking at the new crew. So you can see the God task in action and all the issues
that it's going to cause. All right, guys. So welcome to our first crew, which is going to be
the noob research crew. And in this crew, I'm going to show you exactly the God task in action.
And we're going to cover, you know, how it covers all the key identifiers that we talked about
earlier. You're going to see an action and see how it just is not performing how we would expect it
to. And also just to help you guys out even more, you're going to be able to download this noob
crew and the pro crew completely for free. I'll have a link to it down in the description below
so that you can go off and use it yourself if you'd like to. Okay. So what are we trying to do?
So for this new crew, we're ultimately trying to go off and do a quick research analysis. And
really it's a comparison. So we want to compare two different things. In this case, we want to
compare the Apple watch ultra to the new Garmix Phoenix nine. These are just smart watches. And
what we're trying to do is like, Hey, tell me what is the best health wearable in 2024 to train for
a marathon. So it's going to compare item one to item two against our goal. And it's going to go,
Oh, actually after researching the internet, item one is better. That's ultimately what we're trying
to do. So pretty straightforward. And this crew is going to be broken into one task and one agent.
So you can see down here, we have one agent who's our researcher. This researcher has three major
tools. Tool one is surfer, which is a web scraping tool to go off and get a brief snippet from a
bunch of different websites. Tavoli is a, another web searching tool that will go off and get a
little bit more information compared to surfer. And then finally, we are going to use a customized
version, a version of this search tool. That's specifically going to look at Reddit. That way
we're looking and exploring a bunch of different sources on the internet to help us find out in
our comparison, which is the better item. Great. So what we have next is we have a single task.
This task is a comparison task, and this is where you're going to see all of the God task in action.
So let's go ahead and dive into this task so you can see it in action. So, all right, you know,
at this point that our whole goal is to compare item one against item two in our goal. So we're
going to say, Hey, go off, conduct a thorough comparison between item one and two with the goal
of blank. So just a quick reminder, these goals right here, we're going to be passed in. So item
one, item two and goal, these are all going to get passed in right here. So it'll actually read
like a real, it'll read like a real sentence, compare it Apple watch ultra against the Garmin
and tell me which one's the better one for running a marathon. Now here is where things start to
kind of unravel and become a God task. So the high level overview is like, Hey, search the internet
with this tool, search the internet with this tool, and then finally search the internet with
this tool. So that's already three tasks. And then finally at the end, after you've researched
everything, using all those tools, go off and put a report. But if you dig deeper, so that's four,
all in all one, two, three, and then task four. So that's four tasks. So we're already kind of
getting like, Oh, this is a little bit, a lot to do for a single task. But if you dive even deeper
inside of each research task, we're doing three additional, you know, actions such as for the
surfer tool, go off and independently look at item one for achieving the goal, look at item two for
achieving the goal, and then eventually do a direct comparison for comparing item one to item
two. So this is actually, if you dig deeper, what's happening, it's like, okay, I'm going to
perform three searches here, another three searches here, another three searches here.
And then finally, I'm going to generate a report. So all in all, we have 10 actions that we're
asking our agent to go off and take, which is a ton of work for our agents to keep up with. And
what you'll see in the end, Hey, just give me a nice looking markdown report that gives me the
final that shows off all your findings. And it also tells me, you know, your final recommendation,
I also want to see links for everything. So there's just a lot that this agent has to keep up
with, as it's going off and executing this comparison task. The agent, it's pretty straightforward. Hey,
you're just a senior researcher. And here's your goal, go off and, you know, do some critical
researching, and actually let us know what you find out based on the specific goals. So what I'm
going to do is I'm going to go ahead and kick off this crew and hop over to agent ops. So you can
see exactly what's going on. And we can start to identify some of the core problems that we've
talked about earlier, such as skipping steps and incomplete results. So let's go ahead and hop over.
I'm going to run the crew for you guys real fast and show you the results over an agent's up. So
just one second. All right, guys. So welcome over to agent ops. This is where we can drill down into
our crew execution and see exactly what happened, what tool calls were made, what LLM calls were
made over and over and over again. So it's super easy to see exact breakdown of what happened. And
just in case you haven't used the agent ops before, super easy to set up inside of crew AI,
just in the side of your base crew, like right in the root class, you're just going to say agent
ops init, pass in your API key, and you're good to go. Okay, so now that we have that out of the way,
let me show you what went down when we executed this crew. So first off, we were using GPT 4.0
mini. So you can see all in all, this costs less than a penny to go off and do our comparison,
which is crazy that it's cheap nowadays. We were using the latest version of crew AI,
and let's scroll down so I can show you exactly some of the symptoms that we talked about earlier
that would arise as of using the God task. So quick breakdown, you should remember that inside
of our task, we were going to search with Serper, then with Tavoli, then we were going to do a
Reddit search. And for each tool, we should do three searches. So let's hop back over to agent
ops. So we can see that it did not perform as we would expect. So right out the gate,
and I'll zoom in for you guys, it's going to go off and make a call saying like, Hey,
you are a senior researcher, your goal is to go off and search the internet, you're trying to
to this watch for finding the best one to use for training a marathon. Cool. So that's exactly what
we'd expect. But now once we've given it the instructions, it's up to the agent to go off
and start making tool calls and start putting together a report. So you can see our first tool
call is going to be one to search the internet. And that's basically Serper. So just know that
is Serper in action, then we're going to have another tool call, search the internet again,
that's another Serper call, then we're going to another Serper call. So it's working so far. But
this is when things start to fall apart. So now that we've kind of searched everything,
we're now going to start going over to using Tavli. So you can see we did one Tavli search here,
we did another Tavli search, and then we hopped over to Reddit. So that means we definitely skipped
out on one tool call. So that's not what was supposed to happen. Like it skipped a major step
for us. And then finally, it did another Reddit search. So it skipped out on a total of two
searches, one Reddit, and one Tavli. So all in all, we're not getting all the data that we requested.
So even though it did do the final output, we can go ahead and see. I'll scroll down and show
to you guys. So in the final output, you can see I'll just scroll down to the very bottom,
you can see in our final output, it goes cool. So here is the final answer when it comes to
comparing these two devices. Here's the results I've gathered. So Apple Watch Ultra, here's
overview of it. When it comes to the Garmin, here's a quick overview of it. And then finally,
when it comes to the comparison, it looks like it says the Garmin watch is better for battery life,
a few other reasons. And here's its final recommendation. So it does also have it's
supposed to have further links, but it didn't do it. So all in all, it's just like, there was too
much of the task, too many things going on in the agent just couldn't handle it all. So now that you
can clearly see that like, it didn't do what we expected it to do. And you can see it actually
skipped out on steps, you can see that that's exactly what are a few of the side effects of
the God task. And all in all, you just don't get what you would expect. So now that you've seen a
noob crew and the problems that come with it, let's hop back over to our pro crew, where I've
completely restructured everything to do the exact same objective, but in a much better structure. So
let's go ahead and hop over to our pro crew to see it in action. All right, guys. So welcome over to
the pro crew demo, where we've restructured the entire crew to be much simpler and work a lot
more effectively. And just as a quick recap of how we go off and fix a God task before showing it to
you, I just want to like quick recap, we need to do three things. First, just break everything up
into smaller tasks. So instead of having one massive task, you know, chunk it up into smaller
tasks. And then finally, we just want to make sure each of the new tasks we create is has a
hyper clear input and super clear output. And then finally, we're going to start using context of a
task strategically. So you're going to see all this in action. So let me show you what we did
quick visual first, and then I'm going to hop you over to the actual crew changes. So it makes sense
when we start looking at the code. So in the past, whenever we were running the crew, it was just
here's the task. It's massive. Good luck. You got it. And as you saw, it did not work. So the way
we've restructure it following our rules. First, we've broken it up into a bunch of smaller tasks.
So you can see we've taken this first task, which is just one, and we've broken it up into four
smaller tasks. Each one of these boxes is a task. So one task is go off and usurper. The next task
is go off and search with tab. The next is go off and search with Reddit. And then finally, once
you've done all the research, now let's put all of that together and generate one report. So as
you can see, four tasks, this is an each task is going to be super clear on what it needs to do.
And finally, there's a few extra things that we're going to do that we can use to level up as a pro
crew AI developer. So the first one is we're going to use context. And the important thing to note
when it comes to context is when it comes to working on tasks, you get to say what inputs
from previous tasks do you want to get passed into this current task? So normally the way task
work inside of Cray is a task, you run a task, it gets an output, that output is passed over to the
next task. So it just gets passed over from one to the next left to right. But using context,
you have the option to overwrite that natural feature and change it. So in this case, what
we're going to say is, hey, instead of just getting the context from the task right before this,
I actually want you to look at the output of all three of these tasks. And that's exactly what we
want to happen in our case, because we want to aggregate the results from all three of these
search tasks into our final report. So hopefully that makes sense. Now, what we're also going to
do to level up is instead of executing each one of these tasks one at a time, we're going to
execute these tasks asynchronously. So all three of them are going to run and the same time and
just going to speed up our crew. And you'll see this later when we actually go off and run it.
It's this crew is going to be way faster. So hopefully that all makes sense. And what we're
going to do next is let's go ahead and hop over to our crew so you can see the code in action.
So let's cover the task. So as you saw back in our diagram, we now have four different tasks,
our surfer, our Tavoli and our Reddit search. So these are our three search tasks. And then we have
our final comparison test. So those are the four new tasks that we've broken it up into. And then
finally, if what you'll also notice at the top of the file is we've actually split up our agents
just to be even more clear. So now we have two agents. As always, we had a research agent in
the previous one, we have a research agent in the new one. The only additional agent is this
reporting agent. And the reporting agent is going to be the one who goes off and writes the report.
And this agent, as you can see, will basically run this task. Cool. So hopefully that makes sense.
So now let's actually dive in to actually see how we've restructured our task to be way more
effective and actually to start using some of the newer features we talked about. So let's go ahead
and look at surfer first, so you can see this in action. So it is your job when using the surfer
tool to go off and compare the items using this developer tool to gather relevant articles. And
then it's the exact same thing that we had in the original task, we've just broken the instructions
out into smaller tasks. So you know, hey, first, independently search it independently independently
search the next one, and then finally compare them. And then hey, your result should be a markdown
file that's easy to read. And the title of it needs to be surfer search report. The reason why
is I just wanted a easy way for us to clearly identify like, oh, yeah, that is the search
results from surfer. And then finally, what we're going to do is you're going to be able to see at
the end with the expected output, you're going to see that it Oh, sorry, my cursor is going a little
crazy on me. But you can see the final expected output. You know, now that we want hyper clear
outputs, it needs to be an aggregated summary of everything you just looked at. And it needs to
have URLs. So what's gonna be nice is the result of this task is just going to be one big, large
markdown file with everything we need. And the result of it is going to get passed over to our
report in just a second. And you'll see how we're gonna do that. The only other important thing that
you need to know is we are going to set up async execution to true. And what that means is this
task is going to run in parallel with other tasks. Alright, so let's keep looking. So Tabli is also
going to be the exact same thing. The only thing different is we specify you're going to use this
tool, everything else is identical. Same for Reddit, go off and use this tool and do the exact
same thing. So as you can see, there's a little bit redundancy, but these tasks are so hyper clear.
And at max, there really is like three sub tests, three or four sub tasks, which is go search,
search, search, and put everything you find into a report. So all in all, this is much more condensed.
And there's agents really can handle three or four tasks, anything past that they kind of blow up. So
this is kind of like rule of thumb. Now finally, what we're going to do is in our last task,
we're going to say great, I'm going to need you to compile all the research information
from these different tools. And what you're going to notice is I want you to make everything into a
markdown final and put the final recommendation with links and resources. So this was exactly
what we mentioned in the God task, except now it's just brought over here. Now this is a part
that is important. So please pay attention when it comes to context. So context is our way of
saying, Hey, I want to overwrite the default features of Cray. And instead of just listening
for the previous tasks that finished, I actually want to not just look at those results. I want to
specifically look at multiple task results. So in this case, I want to look at the result of the
surfer task. I want to look at the result of the Tavoli task, and I want to look at the result of
the Reddit search test. And you'll actually see this in action whenever we hop back over to agent
ops. So hopefully this all makes sense. Now I will just show you the agents as well. You know,
same thing. The researcher basically is like, Hey, you're going to go off and research.
And then the reporting is like, Hey, you're just generating a report on these two items
with this goal. So overall, we're just getting a little bit more structured, a little bit more
clear and breaking things up into smaller subtasks. Okay, cool. So I'm going to go ahead
and run this crew for you guys. And we're going to hop back over to agent ops. So you can see
the result of this crew, and you can see exactly how it hit everything that was supposed to do.
And it's performing a lot better. So let's go ahead and hop back over to agents ops after I
run this task for you guys real fast. All right, guys, so I just kicked off the pro crew. And here
are the results inside of agent ops. So what I want to show you inside of this one is how the
structure of the crew run is differently because this graph definitely looks different than our
previous one. So in our previous one, it was pretty synchronous. So it was like, think about
something, call a tool, think about something, call a tool. And in our new one, because we're
doing asynchronous task calls, we're actually running the Serper, Tavli, and Reddit searches
all in parallel. So that's why things are getting kicked off in threes. So let me show you exactly
what's happening in here. And then at the end, we'll cover the results so you can see everything
in action. So what you'll notice is we do make the official nine tool calls to the proper places. So
three for Serper, three for Tavli, and three for Reddit. So let me just show you. So you can see
this one is with Serper. I'll zoom in just a little bit more. Sorry about that. So you can
see the first one was with Serper. The next one's Tavli. Then we go to another Tavli. Sorry,
it goes Serper, Reddit, then Tavli. And then if you keep going, we now have two for Serper. Now
we have two for Tavli. We have three for Serper. We now have two for Reddit. Then we now have three
for Tavli. And then we have three for Reddit. So all in all, we have three for each, which is
exactly what we were shooting for. Okay, so now that we've covered exactly that it did do what
was expected to do, now that we've refactored everything, now that we're pro crew AI users,
let me show you what happens in our final tool call, where we're actually going off into the
reporting analyst, whose job is to go off and execute the current task, which is compiling all
the research from these different tools. So what I want to show you inside of this is you can see
it starts to go off and actually make basically specific calls. And we're passing it in. And
sometimes agent ops breaks, I have no idea why their tool breaks. So let's just we have to go to
the chat viewer. So you can actually see it in action. So yeah, I apologize, their tool just
kind of breaks sometimes. But you can see current task, we now have the ability to look at the
different contexts that we passed in. So we would expect task one to be Serper, task two to be
Tavli, and task three to be Reddit. And we'd expect the outputs of all those tasks to get
passed in for us to analyze and go off and generate a report. And that's exactly what happens. So
here's the context that you're working with. Here's the Serper search report. So you can see
in here, Apple Watch findings, Garmin findings, comparison findings, we now hop over to Tavli.
And you can see it has the exact same thing. And then finally, we have Reddit, exact same thing.
And then finally, now that it has all of those pieces of information, it can go off and generate
its final response where it generates its comprehensive report. So here's exactly what it
does. Here's my comprehensive report of Apple Watch Ultra versus Garmin for marathon chaining
in 2024. So here's what aggregated from all those outputs of the previous task for the Apple Watch.
Here's the key findings for Garmin. And its final recommendation was the Garmin Phoenix 9. And you
know, basically, they're both great, but you know, this one's the best. And then what's cool is now
it actually goes off and suggest links to us. And it is important to go ahead and mention that in
the previous example, this broke. So as you saw the last time it skipped tasks, it didn't do the
expected output. But now that we've broken everything up into very clear roles and responsibilities
and smaller tasks, things are working properly. So, you know, just a quick recap, if your crews
are not working as expected, you might be putting way too much information into each task. So
definitely please follow everything we've learned here in these God tasks episode so that you will
now be able to break apart your crews and smaller tasks and level up as a crew AI developer. But
yeah, also, if you guys have any questions about anything that I did in this video today,
please drop a comment down below, want to clear this up. And I have a lot more of these videos
planned for you guys. So that's a wrap for this video, guys. I hope you enjoyed the first episode
of crew AI noob versus pro. I have a bunch more of these coming out. And just two quick reminders.
If you would like me to analyze your crews and showcase them right here on this channel,
feel free to go ahead and shoot them over to me at Brandon at cray.com. And second, if you would
like to join a free school community with a bunch of like minded AI developers and get access to the
free weekly coaching calls, you'll definitely want to check out the AI developer accelerator.
I have a link to it down in the description below. But outside of that, if you're looking
for even more AI content, I have a ton of videos for you guys right in front of this channel from
Cray, I Lang chain and a bunch of other full stack tutorials. So definitely want to check
those out and watch whatever video is getting recommended right now. But enough of that,
can't wait to see you guys in the next video. Until then see