I Gave My AI Assistant a Voice (ft. Fish Audio)

So that might sound like me, but it is not me. I never actually recorded that audio. A few weeks ago, I built an email assistant with no code. I had it read a folder of email, walk through it one by one, write me a reply for each one. And a bunch of you asked for a part two of that tutorial. So that's what this is. It's the same assistant, but now I'm giving it a voice. And in fact, I'm giving it my own voice. I don't know if that's vanity, but my AI assistant is going to talk like me. So by the end of this, my AI agent is going to read my morning briefing out loud in my voice, and it's going to repeat back all my draft email replies in my own voice, just to make sure it's how I sound without me doing much. So it took me about 15 minutes. You can do it today as well. So three things that you need, and I want to be just clear about the one that costs money. So first off, you're going to need a Claude code. Second off, you're going to need a paid Claude plan. I know a lot of people just like the free AI plans, but this won't work with that. You need, you know, the pro, the smallest packages is $20 a month. So you're going to need that. I would recommend you get that anyways. And then you're also going to need a fish audio account for the voice. And that part is free for now. So if you've not set up Claude code and have a Claude plan and you need help doing that, I'll leave a link to a video. I've walked to tutorial walking through how to do that. So we're going to skip that here, but we're going to assume you're coming to a terminal or a desktop where you have Claude code already installed. So we're in just this folder called email agent. There's nothing in here yet. This is an empty folder. So quick catch up for anybody who missed part one, but I have this folder full of fake emails. I was building an agent to check my emails and this is not my real inbox. Sorry. You don't get to see my real customer data and such, but we're going to open up Claude in this email agent folder. So you just type in Claude. We're using the Opus five model. It's the best right now. Well, fable five's the best, but it uses a lot of tokens. Opus five is fine for what we're doing. You can switch models too. If you do slash model, actually, honestly, Sonnet would probably be okay for what we're doing, but we'll just stick on Opus. And again, if you're not comfortable in a terminal as a software developer for 20 years. So I love the terminal. It's like a second home to me. But if you prefer, you can download the desktop app and do the same thing, but in the desktop app, it's a bit more user-friendly. So here in Claude code in this email folder, read the emails in this folder, show them to me one at a time. Give me a short summary and a suggested reply for each and let me approve, edit or skip before moving on. So that sentence is the entire program. There's no step two. It's just going to run through these emails. And in the last video, we made this email walk skill. So it goes even faster now, but here's the first email. It has the from the subject, the summary and the suggested reply already. And then you can just approve and in theory you could send it. So this is basically the loop that I use every morning on my real inbox. Just to clear it out. I wake up with about 150 emails. I run this and go through them as fast as I can and either approve, edit, or skip approve, edit, or skip. And this might sound lazy, but here's what has bugged me about this approach is I still have to read every one of those emails. I'm sitting at a desk, reading an agent's homework at six in the morning. And that's just my daily routine. I want to just talk to it. So I'm trying out a new tool, new AI tool called fish audio to give AI a voice. So fish audio does text to speech and it also does voice cloning. So cloning means it takes a copy of your actual voice from a recording of you, not a voice that sounds like you, but it tries to generate your actual voice and they have an MCP server, which means that you can make the voice on the website first. Then you can connect it to cloud code. So I can access fish audio in cloud code in my terminal here and have the, these emails read to me. So first we're going to go to my voices here and you can see that I have no voices. That's sad, but we're going to do build new voice. And there's three ways you can do this. You can instant voice clone where you just record 10 seconds of audio. Then you're good. You can describe the voice in plain text. So if you're a good writer and you know, you know, like what raspy means or you know, good adjectives to go with the voice, you can design it with text or they have professional voice clones that you can use. So I want to use my own voice and I also don't want to take one or two hours. So I'm going to do the instant voice clone here and I'm going to hit record. So you can do, so 10 seconds is the minimum. 30 seconds is enough. 210 seconds is the current plan limit. So obviously the longer you talk, the better AI will be able to clone your voice. All right. And they give you what to read. So let's do it. Oh, wow. I can't believe it actually worked. I've been trying to get this right for weeks, weeks, and now it finally happens just like that. Honestly, I'm a little proud of myself right now. How does that sound? I mean, actually, I'm just going to record a bit more like I'm talking to you because again, we want to try to get to that 30 seconds. So I'm going to try to talk naturally just like I do in my videos. This is pretty much how I talk in real life too. So we're just going to get enough. There we go. So I got 30 seconds. So we'll go ahead and start analyze. All right. So it graded my sample as good and it worked out that I'm speaking English. That's encouraging. My, uh, my homeschool paid off. I do speak English. So let's continue to details. All right. So it analyzed my voice and this is what it found out. This actually, this makes me happy. First off, it says friendly, young speaker. That's a good way to describe me. A friendly and energetic young male voice. Perfect for engaging in conversational content and put me young. So that's good. Medium, smooth, energetic, friendly, clear, crisp, expressive, and animated. You know, this might actually be fun. Just go try fish audio and just analyze your voice. That's kind of interesting thing to just see how it describes your voice. But yeah, I'll make it public. You people can use my voice if they want. Create. I can't believe it actually works. I've been trying to get this right for weeks and now it finally happens. Honestly, I'm a little proud of myself right now. I'm just trying to talk naturally like I do in my videos and in real life too. Honestly, that sounds pretty good. That sounds like me. All right. So we have a voice now, my voice, in fact. So now we're on the text to speech page for voice. Let's add a voice and friendly young speaker. So the model for this text to speech is fish audio S 2.1 pro, which is free for now. So you can, you can go try this out for free. So, and here's what we're going to have it say, good morning, James. You have three emails that needed decision and size sighing. The first one is from a sponsor asking about the invoice. So those square brackets are not actually read that just, they, they actually do that. They act them out. So when there's not a list to pick from, you can put anything in there you want. Like you put grown or complain or laugh or whatever you want and it'll do it. Okay. Let's do generate speech. Good morning, James. Three emails that need a decision. The first one is from a sponsor asking about the invoice. Good morning, James. You have three emails that need a decision. The first one is from a sponsor asking about the invoice. Okay. Give me two to see which is better as they're kind of working on the model. I like take two better. So we got our voice set up and I think it sounds pretty good, honestly. So we are going to hook this into Claude code, but you can, you can get an API key. You could build, put this into an app you're building or into your website, or you could just use it for fun to send to your friends or whatever, but we're going to actually put it into Claude code here. And particularly this email reader that we built in the last video. So actually let's close Claude here for a second. We're going to Claude MCP. And if you guys want me to do a full video on what an MCP server is, but it's basically just a way for AI to talk to a service. So in this particular instance, it's going to be fish audio, but it's going to let Claude code go talk to fish audio and do things. So Claude MCP add transport, HTTP fish audio. And the server is at API dot MCP. So MCP servers, you know, in terminal, you're typing it all in on the desktop app. It's a little bit easier. You can click, but again, I prefer typing a software developer for a long time. So that MCP server is added. So let's go back to Claude and then it says one MCP server needs authentication. So that's the one that we just entered. So we're going to do slash MCP and here's the different MCPs that we have. So I have a, some of these I've talked about in other videos, but I have a Gmail, Google calendar, Google drive, Claude in Chrome. Um, actually I have a whole bunch of just different stuff that I've used, but fish audio is the one we're working on today. So now you need to authenticate it. So I'm going to do authenticate and it actually opens my browser and just gives an approve button here, which is what authenticates it in Claude code. So now you'll see authentication successful connected to fish audio. So now I can say things in Claude code, like say this in my voice. Good morning. You have three emails that need a decision. So you can see it's calling fish audio. So it's using the MCP server. You have three emails that need a decision. So that's my voice. And again, I never actually said that sentence. So that's pretty cool. I can have myself saying anything that I want to say. This is something I probably play around with for quite a, quite a while. And actually, if you look down here, Claude code is suggesting, I have it now say the summary of the three emails in my voice. That's kind of a cool idea. Let's do it. Claude suggests prompts to you. It kind of tries to guess what you might want to do next. So let's do that. Here's what it came up with first North gate supply invoice in G 44 71, $1,240 due Friday. And that sounds exactly like my voice through how you actually use these agents day to day, 45 minutes remote. They handled the edit. She's holding September 4th. Honestly, I don't know that I would be able to tell myself that that is not actually me speaking. So that's pretty cool. Read the emails in this folder, write me a 32nd spoken briefing of what came in overnight and what needs a decision and say it out loud in my voice and start it playing again. Claude code calling fish audio, get the MCP server at work there. Five emails overnight, three needs you. North gates invoice is due Friday, 1240, about 190 over July for rush shipping and Dana can reissue it itemized. If you want build in public, once you on the podcast, September 4th or 11th and Marcus is asking. So this is the part I was actually wanting. I can make coffee. I can sit down. I can go take my dog on a walk and it will tell me what I need to look at for the day. And now it's doing two jobs at once. So we're connecting what I did in the last video, you know, looking at the emails, summarizing them, looking at what needs to be dealt with. And then this part reading it to me instead of me having to sit down to read it myself. I know I said that was going to be my last test, but there's one more thing I want to try from now on. When you show me a suggested reply, say it out loud in my voice before you ask me to approve it. So before I send out emails, I'm going to have it read it to me in my voice, just to see if that sounds like how I talk. That could be interesting. When you use language, just a little info about Claude code. But when I say from now on the difference, yes, please reissue with the rush shipping broken out as its own line. That makes it much easier to reconcile on my end. Either way, I'll have it paid by Friday. So no need to hold anything up. Thanks, James. Okay. That's, that's pretty cool. I like that. But I was saying is when I said from now on, it adds it to my Claude memories, which means from now on, it's going to remember to do that. So that's pretty cool. This is a good addition to my agent, my morning agent run. So in last step, I want to turn this into a skill so it can just be run every day very easily. So turn all of this into a reusable skill called morning briefing that I can run anytime and write it a short description. So you know when to use it. This is how you create skills, by the way, people, I get this question a lot. How do I create a skill? You just tell Claude to create a skill and what you want it to do, but it'll remember everything we did here today, like contacting fish audio, using the MCP server, and then it just turns it into one command that you can run for it to do it the same way every day. So it's using the skill creator. Look at all the cool code that it's writing. You know, you're a coder. Gave it a description. Like I asked five emails overnight, three needs you. Northgate's invoice is due Friday, 1240, about 190 over July for rush shipping and Dana can reissue it itemized if you want. Build in public once you on the podcast, September 4th or 11th, and Marcus is asking for a refund on the course three weeks in past the 14 day window hoping for a partial. The other two are a newsletter and thank you. Nothing needed there. Cool. So it works. Now anytime I want, I can just do this slash morning briefing and it'll do that. Check the emails, read the briefing to me. So how much does this all cost? So Clucko needs the paid plan. So $20 a month, fish audio S 2.1 pro is free for a limited time. So this pretty much doing something like this is pretty much free. Very, very cheap with what you can do. So we built an assistant that reads my emails, decides what matters, tells me in my own voice. We built this, I was going to say in an afternoon, but honestly we built this in like 30 minutes total. AI just speeds everything up so, so fast. So go try out fish audio, especially while fish audio S 2.1 pro is free. Go try it out. Try fish audios API. If you want to wire something of your own in link is all down in the description. Also pin a comment for go try it out. Thank you guys so much for watching. God bless you all and have a great rest of your day.