Anthropic just dropped Computer Use for Claude Code, which lets it natively control your mouse, keyboard, and take screenshots to interact with any app on your desktop.
In this video I walk through how to enable it, demo it sending files through ClickUp and controlling my Mac remotely from my phone using Dispatch, and cover the current limitations you should know about. It's in research preview right now but there are already some seriously cool use cases here.
TIMESTAMPS 0:00 What Is Computer Use 0:52 Starting OBS With Computer Use 1:24 How to Enable It 1:40 Demo: Sending a File via ClickUp 3:19 Troubleshooting Tips 3:52 Pairing With Dispatch 4:13 Demo: Calculator From Your Phone 5:07 Limitations to Know 5:47 Browser Use Workarounds 6:42 Best Use Cases 7:51 Final Thoughts
AI Generated·May contain errors
Anthropic just dropped another new feature which lets Cloud Code actually control your computer,
which means it can natively use your mouse, your keyboard, and take screenshots and do things for
you. Right now it is in research preview in both Cloud Cowork and Cloud Code and on Mac OS only,
which is why you guys might notice in this video I'm on my laptop. But the whole idea is that you
can basically do anything on your desktop, no matter if you are sitting there or if you're
out to lunch. Because you can also pair it with the new dispatch feature, so you can literally
reach your desktop from your pocket, shoot off a text, and it will control the computer locally
and do what you need to do. Or when it's time to check the build but you're on the train,
once again, shoot off a message and it will do what it needs to do on your actual local computer.
So today I'm going to show you guys a few demos, talk about when it's useful,
and then I'm going to talk about the limitations as well. So let's get into it.
So this is the official post from Cloud that just dropped about two hours ago and already
has over 4 million views, which is insane. And just as a little intro teaser, I actually had
Cloud Code start up this recording for me. So you can see here I said,
use computer use to open up OBS and start recording. And what we're going to see is
that it basically moves its screen over to the right. It has to navigate through my desktop
and has to open up OBS. And now it's going to take a screenshot of this interface to understand
where is the start recording button. You can see it just took a screenshot and now it's going to
say, okay, it's right here in the blue, and then it's going to click on it. And that is the video
that you're watching right now. The recording was started by Cloud Code. So once again,
you can use this in cowork and in Cloud Code. So today I'm going to be showing how you use it in
Cloud Code. Now, the only way right now is within the desktop app, which is not a huge deal. You
basically just come down to your settings. You go over to the general settings down here in desktop
app, and you can see at the bottom, we have computer use, which lets Cloud take screenshots
and control your keyboard and mouse in apps that you allow. You will have to give it some
accessibility like screen recording and other things like that. So let's just start off with
a quick demo here where I'm saying use computer use to find the how to use the ROI calculator PDF
in my downloads and send it in ClickUp as a DM. So what you'll see here is that first, I don't have
those two apps open. I don't have Finder open or ClickUp. So what I have to do real quick is I have
to allow Cloud to use these things. So it's going to allow it to use the downloads folder, and then
I'm going to have to allow it to use ClickUp as well. Now you want to do that every time, but in
that session, you may have to give it access. Now what it's doing here is it's pulled up ClickUp,
and it has to actually take screenshots and navigate through it. So the first thing that it
did is it found my DM. So there it is. There's Nate Hurk. This is where it's going to send that
file. And now it has to figure out how to actually send an attachment. So you can see what's going on
on the right hand side is it's continuously taking screenshots, and then it's going to press
different buttons and figure out what to do. Right here, you can see that it zoomed in on the bar,
and now it sees that it needs to click on this paperclip in order to attach a file. And just
to be sure, it even zoomed in even more. It's going to take another screenshot, and then it
should click on the right button for us. And now that it's in our downloads, it's going to have to
search for that correct PDF. So you guys didn't see that, but it just clicked on the search bar
on the top right. Now it's searching, and you can see that the files that popped up have changed,
and hopefully it's going to choose the right one. There we go. So it found the right one.
It's going to have to double click, open it up. Now you can see it's attached in our ClickUp
message, and hopefully it's going to be able to send that. It did come back and ask for confirmation
because it was doing a send action. So I'm just going to go ahead and say yes, and it should
open the computer back up, hit send, and then tell us that it has been sent. As you can see,
that has now been sent to myself. So a couple things. If you're having trouble getting this
to activate, or when you're telling it to use computer use, it's trying to use like a Cloud
and Chrome extension or something like Playwright, then you have to be more explicit in the prompts.
The other thing to check on is make sure that your Cloud desktop app is fully updated. So,
you know, delete it, maybe download it again. And then when you enable the setting,
quit out of desktop app and then open it back up again. And then you should see you're able to use
it. And now that I've already allowed those certain tools like Finder and ClickUp in this session,
next time it uses them, it won't have to ask me again for permission. Now what gets really cool
is when you pair this computer use functionality with the new dispatch feature, which is also
available right over here as you can see. This lets you control Cloud from anywhere. So I can
control it right here from my phone, which means if I'm out and about and I realize, oh shoot,
I left that file locally in my downloads folder and, you know, XYZ person needs it. I can just
text Cloud code to open up my browser, grab the file and send it to them. Okay. So here's an
example. I've got a message typed up in dispatch right here and I'm going to shoot it off and you
guys will see it pops through right there on my actual local machine. So I'm saying use computer
use to calculate 726 times 800 or not 800, 83,729. And it's going to open the calculator app and then
notes app. So once again, because this is a new session, I have to allow it. And now it took
control of the computer. As you can see, it opened up my calculator. It's going to have to clear out
this current search and it's going to have to type in the new numbers as you can see. And now it's
going to have to open up the notes app and store those results. So there's the notes app. You can
see that it just added in a calculation result, which has our calculator function right there.
And once again, as long as my computer is awake, which we can actually turn on natively
with cloud dispatch, then I can be anywhere from my phone and I can use cloud dispatch to start a
new session, use computer use and do whatever I need. Okay. So before we talk about some other
cool use cases here, I did want to talk about the limitations. So obviously right now, this is in
research preview. So sometimes it may feel a little slow or a little buggy. And right now it's only
available for Mac OS within the desktop app for cowork or cloud code. So it's going to be coming
to windows in just a few weeks here. Now, really the biggest one in my mind is the fact that it has
native limitations for browser use. So in an actual browser, it doesn't click around or type,
and that's for security reasons, but locally within any of your apps, it can use those really
well. So even if you have different connectors like your Gmail app or your Slack app or ClickUp
app, as you guys saw, it can use all of those perfectly fine. So let me just show you guys
what I mean by that. Use your computer use to open up Safari, search for images of cute dogs
and save them to my downloads folder. So I'll shoot this one off. And what we're going to see
is it's able to open up Safari, but then it can't actually type or click. And you can see here,
it actually even says Safari is restricted to read-only access on the system. However, I can
use Google Chrome extension to do this. So basically what that means is if you want to do
legit browser automation on the web, you would need to use Chrome extension or potentially like
Playwright CLI. But this is still super exciting because what's happening is Anthropic is shipping
a new feature like every day. And what this shows us is that they're moving towards having all of
these features natively within Cloud Code. So at some point, Cloud Code will be able to do this
in the web and all this kind of other stuff. It's just going to take a little bit more time,
but they're still shipping at an insane pace. But people are still having really great results with
this already. You can use your connected apps first. So Slack, Calendar, other integrations,
if there's no connector for the tool you need, it will ask for permission to open the app on your
screen directly. You can assign a task from your phone, you can turn your attention to something
else, and you can come back to finish work on your computer. So you could tell Cloud once to scan your
email every morning or pull a report every Friday, and it would handle it from there. Now, another
way that you could actually use this is you could pair it with the scheduled tasks. So you could
actually have a scheduled task kick off computer use. So it's pretty much going to be as if you
have a real human working on your laptop. And of course, right now in the research preview,
it is only available on Pro and Max, not on the Teams or Enterprise plans. But like I said,
there's a lot of unlocks here. If you think about the different use cases, like maybe legacy apps
that don't have an API, and you can just actually click around to do things, download reports,
look at different information. Of course, pairing it with scheduled tasks or something like Dispatch
is a huge unlock as well. And if you have certain things that you want to test, like maybe you're
building out some sort of app, and you're pulling that up in a sandbox environment, and you're just
having Cloud Code click around, find the bugs, test features, rather than you having to sit there
and do that manually. I hope you guys are having as much fun as I am playing around with Cloud Code
and exploring all of these new features. It's definitely not been a boring few weeks if you're
in the AI space. So if you enjoyed, if you learned something new, please give it a like. Helps me out
a ton. And as always, I appreciate you guys making it to the end of the video. I'll see you on the