Saw this on HuggingFace and immediately had to make the video since this launch was super anticipated. Now while MiniMax 2.7 is just an iteration of M2.5 - the focus is *entirely* on basically making a model to replace Opus for OpenClaw. Boasting 230B (10B Active MoE) this model is going to be far too heavy for most to run on consumer, but there is more to that story and the reality of it may be closer than we think.
*Chapters* 0:00 Why Am I Filming At 8PM On A Saturday 0:31 About This Channel 1:06 About MiniMax & M2.7 2:35 Jumping in M2.7 Claims 8:16 What Are the Specs? 12:23 How To Run MiniMax M2.7 & What To Expect 15:14 The License Changed - This Might Matter To You 16:43 What Kind Of Hardware Is Needed? 18:59 That is MiniMax 2.7!
It is 8 p.m. on a Saturday.
So naturally, a giant Chinese model maker has dropped a huge open model that is crazy capable
and is actually on par with Sonnet 4.6 on basically all benchmarks and within a couple points of Opus
4.6. The model that I'm talking about is the Minimax M 2.7. It just dropped basically two
hours ago. There is a lot to talk about, though, so I'm just going to give the crash course.
For those of you who don't know me or haven't seen this channel before, my name is Timothy
Karambat. I'm the founder of Anything LLM. It's an all-in-one desktop app that brings you this
kind of cloud code and chat GPT-like experience but without any of the confusing setup or token
costs because we do everything locally. So when local models drop, I get excited because any
model that you can run outside of a massive cloud is always a good thing. Even today,
I believe that you can take a small local model, run it on your machine, and have a cloud-like
experience. That's what I do. That's what I believe in. That's what I built. Now let's get
to the video. Minimax is a Chinese company, much like Quen or any of the other popular open source
models that are out there that are super capable. They've been around for a while. This is not a
small company. This is a huge company. I think they're valued at something like two-something
billion dollars, and they're listed on the Hong Kong Stock Exchange and all that. So this is not
like some people rubbing nickels together to make a model. They've consistently delivered,
and the only downside is that these models are always crazy huge. You are not going to be running
this on your regular consumer laptop. But if you can run this, this might just be the thing you
need to basically get rid of token costs. And so yeah, a couple hours ago, OnHuggingFace,
one of the main devs, just dropped the link, just said, hey, here's the model. It was rumored to
come out this weekend. People didn't know where, and there's a bunch of people on the same thread
complaining about, where is it? Where is it? It's like, just relax, okay? The model's only
230 billion parameters. Probably takes a while to upload, I guess. And that is kind of a thing
that you're going to find. What we're going to talk about is a super capable model. I mean,
extremely capable. However, it is maybe out of reach for a lot of people, but there is always
a silver lining to stuff, and so we'll talk about that. So before getting in how you could even
possibly run this, where to look, all that other stuff, let's just talk about what they did,
because that's honestly where the cool stuff is. M2.7 is an improvement on the previous version,
M2.5. So basically, you can expect them to be around the same size and have the same
kind of requirements. You would just expect M2.7 to be a little bit more intelligent.
And the reason for that is this model has a different goal than its predecessor. If you
look at the highlighted line, they say M2.7 is our first model to deeply participate in its own
evolution. What they mean by that, it sounds very fancy. It is not that fancy. The entire idea is
having M2.7 build the tools and the things that it needs to do a job. You are already familiar
with this concept. If you use a tool like OpenCLAW or Cloud Code, you are familiar with
asking for something and it may be building a tool or writing a script to do something.
That is what this model is aiming to do and to be trained on specifically to succeed at.
So ideally, if you can run this, you can legitimately break away from ever having to pay
CLAWD or be rate limited or get randomly banned and all that other stuff. So if you can run this
model, you can unlock that reality. And M2.7 is free to download. It is open source. You and I,
if you have the hardware, can run it. You are going to notice a lot, I am going to say,
if you have the hardware. I am going to talk a little bit about hardware at the end of this
video. Now just to give an idea, just so I can get out of the way, this is GDP VAL AA. That is
not really important. The ones that people like to talk about are mostly like SWEBENCH and a lot
of that. And there is one also very interesting benchmark in here. This one right here. MMCLAW
BENCH. That is the MINIMAX CLAW BENCH. So basically what they did as part of this entire kind of
evaluation is because MINIMAX is basically obsessed with this specific model of seeing
what they can do to rip CLAWD out of OpenCLAW, they made their own evaluation set basically
around tasks that you would do in OpenCLAW and to see how they perform. Now this is where you start
to see the gaps between M2.7 and OPUS, but then also see M2.7 perform at at least SONNET levels
of the current anthropic generation. And if you look at all of these metrics, frankly, if you
don't know about some of these benchmarks, I'll link a video right now just so that you can kind
of get a crash course on benchmarks. But you can see just from looking at bar graphs that M2.7 is
definitely better than M2.5 and it is certainly capable as a SONNET replacement. OPUS, you may
find some rough edges. So in their report, they talk about, you know, kind of like where MINIMAX
can function as an employee. Now, not probably in the direct sense that you would entrust them
blindly with work, but in the sense that you could use them to augment your work. And so you can see
that they talk about professional software engineering, which I think is a given. That is
kind of Anthropic's main focus with the CLAWD models is kind of just coding tasks because that's
just, you know, a well-known problem set. And another area that they're talking about focusing
on, and a lot of people know this, is knowledge workers. So general kind of like office tasks,
people who spend their jobs in docs and PowerPoints and stuff like that. And there's
videos in this blog, which I'll be linking in the description for you to look at the demos if you'd
like to watch them. And you can see that the model here is still, it's performing, right?
Now, this is not bad. Specifically, the benchmark that's being talked about here, which is for this
general professional work situation, you can see that the Minimax 2.7 is doing well. Now,
it's doing better than Gemini 3.1 Pro, which in my opinion is a surprise, but it is not doing better
than Sonnet, Opus, and GPT. This is one of the benchmarks where M2.7 is not surpassing Sonnet 4.6,
but it is pretty close. I mean, within seven percentage points, but also the situation
varies dramatically. Benchmarks often aren't a great reflection of the thing you specifically do.
They're simply indicators of if it's worth your time. And it's very clear from this that M2.5
is not worth your time, but M2.7 is certainly worth a look at now. Lastly, one of the areas,
and this one was the weirdest to me, was entertainment. And not even in the way that
you would expect, like not even like generating like music, I guess, or like video or anything
like that. It's actually for like emotional intelligence, basically character building.
So they have like high emotional EQ and character consistency, which is great. So basically giving
a persona to your model. Now, I'm not crazy about this, to be honest. I know the use case exists.
I know there are people out there who want to have their own, I guess you would even say like Jarvis,
right? That is fine. But there's also like a weird edge here to like AI psychosis and like
online stuff. There's a whole video worth of information about this segment. And there's
a really interesting demo where basically they have this character that is running,
and this is powered by the Minimax M2.7 model. And it's like this model is basically interacting
with a Twitch chat. Every now and then it says something. It's just kind of odd. I don't know.
I'm not, I'm not crazy about this, but maybe you might be. And that's okay. That's totally fine.
It's just a thing they did. I don't know. This is just bizarre to me. Okay. So we know we've got
this Minimax M2.7 model. What are the details of this thing? Like what's the stuff that it can do?
Is it multimodal? How big is it? What's it going to take to even run it? Is it even human to run?
So first we're looking at basically 230 billion parameter model. Now, obviously that's crazy.
And if you laughed at that and said, wow, and then looked at, you know, your Lenovo think pad
and said, well, sorry, buddy, that's it. Well, don't, don't, don't be too sad. There is a silver
lining to this. This is actually a MOE model or mixture of experts. For those of you who have
never heard of that term before, an MOE is basically running a small model in a big model.
You still need crazy good hardware to load this 230 billion parameter model. But what it means
is that when you ask a question, you will get the speeds of a smaller internal model, which around
in this instance is about 10 billion parameter. And this is really good because you get the
intelligence of a much bigger model by running the smaller models inside of it.
MOEs are not perfect, but they are quite capable. And when we talk about the local domain and
getting stuff done locally, sure. Having stuff done in a dense model, which is a model that
isn't an MOE is always great, but there are limits, right? And so maybe in the future,
like we found with stuff that's coming out with other areas of optimization, like one bit models
and KV cache optimizations and all these things that make big models a lot smaller, who knows
what the future has. And it might not be crazy in the future for you to have a 230 billion parameter
model running on your 64 gig laptop. Who knows? I'm just, I'm not saying that that's around the
corner. I'm just saying that there's a lot more juice to squeeze when it comes to open source.
So don't be disheartened. This is as bad as the models will ever be. So when it comes to MOEs,
you have to jump into the config JSON of the actual model files themselves to get really
the kind of nuance, because for whatever reason, the technical side of things are really lacking
in all of the written documentation about this model, likely because this is kind of an iteration.
But for those of you who don't know about minimax, this may be the first time you've
ever heard about the model, but there's a property that says number of experts per token. That's what
that stands for. It says eight. Well, okay. So eight experts per token. All right. But out of
how many, 256. So at any given time, when you send a chat, you are having eight experts answer
your query. It could be very different experts. All of the experts kind of, it's not so simple
to explain experts really, but the idea is basically think of instead of having just one
really smart person to answer your question. You now have 256 really smart people in individual
fields that can come together to answer your question. You're wondering what the value of an
MOE is. It's basically that now again, M 2.7 is again, an iterative jump on the minimax M 2.5.
So a lot of the same functionality remains from the chat template, because this was not mentioned.
This is a reasoning model. You can reason with this model. So you should get great answers.
This is expected, but also there is no multimodal. Now we have about 200,000 tokens to play with,
which is great for a context size, but it doesn't look like there's any image support,
which frankly, I find myself using a lot. I like to send images and screenshots and stuff to my
model. And I just feel like without multimodal, there's still kind of a blind spot for a model
like this. If a lot of your stuff is text or can be put into text, then you should be fine.
And this is still a fantastic model. I'm still waiting though, for the kind of iterative jump
for a multimodal of this size of this nature scoring this well. Now, if you want to run this
model, that's where we kind of come up to a time-based thing. So right now this model just
dropped and it takes a lot of time to come up with really good quantizations of this model.
And like, for example, you may use tools like anything LLM or LM studio or Olama, and those
tools allow you to load the GGUF files and LLAMA CPP and stuff like that. Whatever tool you use,
that's those are usually the models people are downloading. You have to make those files. People
have to make them. It takes a lot of time. So right now you won't find quantizations,
but probably within a couple of days of this video dropping, there will be a lot.
We can go to our good friends at Unslaught who really provide a lot of these GGUF files that
people use in tools that you already probably use like LLAMA CPP, anything LLM, LM studio,
Olama, whatever it is that you run. This is the file that you're usually looking for.
To get an idea of what the file is probably going to look like, because this is very similar to the
generation, we can of course just look at the Minimax M2.5. So a lot of this, as you can see
right here on this side of the screen, if you wanted to run this thing at its full kind of
accuracy, basically very little accuracy loss, this is nearly half a terabyte of storage.
Now, if you are willing to go down to Q8, which I think if you're willing to run this at 16,
you can probably go to Q8, it'll be only 261 gigabytes, only 261. Me personally,
I'm probably going to have to find myself limited to at best the 2-bit quantization,
maybe this Q2K or Q2KL, and you could go down to 1-bit, but I think the accuracy is going to
suffer a lot. To get an idea of how much accuracy you're going to have by running a 1 or 2-bit
version of this super large 230 billion parameter model, we can actually refer to a similar graph
for the M2.5. Now, you can basically see that the red is the relative error, so obviously the
original has no error. If you go to Q4, you start to get one, and if you go to Q4KXL, so like the
larger version of Q4, it drops dramatically, but the real thing that's important is that this Q1
just is bad. Like, it is so bad that you almost might as well not run it. So if you can run a
model of this size, if you have hardware that can do this, at least strive for Q2. And if you are
laughing at me because you're sitting on like a server farm apparently and can run the Q8 or even
the full parameter FP16, excellent. Congratulations. That's all I can say really. In terms of just
general information about Minimax M2.7, there is one detail that could matter to you. It might not,
and if you're looking to run a model like this locally to help you with your job or do tasks or
as a replacement for Opus or Sonnet in your OpenCLAW instance or anything like that, then there
is a detail that you need to know about, and that is that the license for this model is actually
quite different from its original predecessor. So the one thing that is very different that I think
is gonna maybe be a sticking point for some people, especially when we talk about productivity
tasks and knowledge worker tasks, like does using this model to help you do your job fall under
commerciality? Because you cannot use this model commercially. I think the only angle they're
looking for is basically people reselling the kind of model as a endpoint to others. I'm not
exactly sure why they did this. The M2.5 model was broadly permissible, and it basically was just
like, hey, if you use the model, whatever you do with it, just, you know, redistribute this license
kind of stuff. So a lot of people really liked M2.5 and were using it in tools to help replace
their kind of like larger token cloud bills that they were using. I don't know how much this is
going to hurt M2.7 from being adopted. I don't know. For people that are dealing with models
like this, maybe it won't matter. Now, how do you even run this thing? So if you're looking at maybe
running this at a Q2 quantization or Q1, so super compressed, super, super compressed, you're still
probably going to need probably around like 100 gigs of space and maybe like 96 gigs of VRAM.
This thing is still huge. So for those of you who went out and bought the biggest Mac mini that you
could when OpenClaw was kind of like the talk of the town, you could actually run this model. And
that's frankly kind of like where I think this model has succeeded a lot. It's from people who
have bought those Mac mini kind of unified memory 128 gig devices. But there are some other devices
specifically I have too. And so if you'd like to see some tests about this model, or maybe just me
give an experiential kind of review about this model, I do have two things I can run it on.
I don't know how well it'll work. So the first is I have this DGX Spark. Now this thing is mostly for
fine tuning. And the reason for that is actually because the DGX Spark has limited memory bandwidth.
So basically, if you're looking to run super fast tokens, it's not really going to do that well.
But with an MOE, you kind of don't have that problem. So I may be able to load it on this
really big device, which I think I can get mostly loaded on and then still get good TPS. Because if
you put a small model on this thing, it's still plenty fast for me. I'm not a token snob. If I'm
like 40, 50 tokens a second, I'm very happy. And the next is I actually recently acquired
a this is basically an AMD Strix Halo device. This has 128 gigabytes of unified memory.
And I don't know how that one's going to work either. So I kind of have like a weird mix
of devices that I could run this on like the DGX is Linux. And the HP that I just sent is
that's Windows. So I don't know how this will all work. But if you're interested in at least me
showing some stuff, maybe you have some prompts or examples that you'd like me to run just to kind of
stress test it to see if it explodes. I'm happy to do that. Just leave a comment. But yeah,
so that's basically it. Minimax M 2.7 is hot off the press. It is a behemoth as it was already.
And it does look to be a serious kind of competitor for the kind of people who want to
have an open call like experience, but without the token anxiety of what it's going to cost you to
do it. Since now you can't use your cloud code plan with tools like OpenClaw or really anything
else anymore without getting banned. And the API costs are just not worth it. So a model like this
couldn't have come at a better time. Anyway, that's it. See ya.