Run Open-Weight Models for Free: DeepSeek, Qwen, and GLM on ModelScope Tutorial

So open source AI models have gotten really good and really gone mainstream just in the past few months. You've seen that with the Quen, the DeepSeek, the GLM, and most recently Kimi was the largest mainstream breakthrough there. And these kind of models now are trading blows with the state-of-the-art frontier GPT and Claude and Grok and Gemini models, despite the models themselves being free. Now, I do want to point out the infrastructure to host the models may not be free, so it is a little bit fallacious to say that these models themselves are free. But the problem is this. Actually testing all of these open source models yourself usually means signing up for two or three different API providers, entering your credit card each time, and then paying per token just to figure out which one you like and which one performs per the use case that's given. And so in this video, in the next few minutes, we're going to do it differently. I'm going to pit the top open source models head to head against one another using one platform and one free api that means no credit card required and then i will take the winner and wire it into a real working app for a coffee brand that we've built throughout several past videos so you can see that full path from hey this open source model looks interesting to it's actually doing useful work to me now the platform we're doing this with is model scope they're a kind of alternative to hugging face except model scope is a non-profit platform that means the resources available are free to use and that's important and you'll see why throughout the rest of this video if you build anything with ai or you've been wanting to start doing so without a very expensive api bill or token maxing as as they say this one is going to be worth your time so let me show you what it is right model scope is a global open source AI model community. You can think of it like an alternative to Hugging Face where it has models, data sets, and a place to build in one platform. The Chinese version is the largest open source AI community in China with over 26 million users. The international version for us in the West, that's at modelscope.ai, has your very popular open source models like Qen, DeepSeq, Minimax, and GLM, basically as soon as they're released. Now, they do have the model weights on there, and you can download model weights on ModelScope, but this is important, and this is how we can do what we're about to do. You get free API inference for popular models. So you get free API inference for popular models. You get free cloud notebooks with CPU and GPU access, free cloud storage, and then free app deployment. And now, if you are an individual developer, that is most of the stack that you would normally pay for so we're going to get into the demos now if you want to learn better from this video for a heads up you can sign up with my invite code john law that's j-o-n-l-a-w or the link in the description or comment model scope in the comment section of this video now you get an extra 100 magic cubes on top of the standard allowance if you do that and this is a free way to start with more headroom so then the plan for the workflow we're going to test again i'm going to test one use case and then have a model comparison through the free api and then do one integration ultimately and now let's jump into the demos okay so here we are inside of model scope this is a demo account that i've just set up and i've done so on purpose because i want to illustrate those free resources on a brand new account with no credit card or payment info attached but first i just want to show you the breadth of the models that are available on the platform so if you've seen something like hugging face you can start to understand this right you can see models data sets the studio or the kind of editor so to speak we can see a lot of different options here now if we jump into models you can see the true extent of the catalog so all the different ai models out there that vary across these verticals of cost of performance of specialty and of availability so you can see i mean you can just keep scrolling here there's a hundred pages right and there are double digit models per page that's a lot of models now models which have api inference enabled are the ones that you can access completely for free and to filter for those models we can go under experience and then turn on api dash inference and again i mean you can see here 19 617 results and if that doesn't give you an idea of the scope of the modern model landscape i think this right here is it in a nutshell and you can see a lot of familiar names here right your quen series out of alibaba your deep seek of course and now let's say we do want to run some inference on a model or play around with it what i can do is i can click on that model and you can see a summary of the model so what its advantages are for example in the case of this quen image model it is very small it is efficient and it's good for editing with a certain reference image a lot now this is fully open source meaning we can go ahead and download the entire model we could host it locally secure private data right we could get that whole nine yards of benefits from an open source model but if we want to access the free inference through model scope then we can scroll down from this summary to this code and then this plus an api key is how we're going to be able to run it we're going to run through that process in a second but first if you do just click generate now as well then you can do this in the web so you can go ahead and just use the model here now to go to the access token side of things we're going to click on our profile right here go to settings and then under here you can see your access token so we have our default token right here and so it's the code that we just saw plus this token that'll let us run the model locally for free so now we're going to run an experiment we're going to jump back to the model page with these 46,644 different AI models. We're going to go under experience, filter for inference, and then we're going to run a head-to-head test where we pit three models against each other and see which one we like the most. And we're going to do all of that and run all of these demos for our tests completely for free, which is the point. This is great for testing and great for trying out different models to see what's the best fit for your use case. So I'm going to go with DeepSeq V4 Pro and the Qen 3.8 and our GLM 5.2. Now for this test, what I have is a Python script. It's basically a big for loop that's going to run through these demos for these different models. And this was made with AI. So this is AI testing itself in a sense but we have our three open weight models three prompts to test them and then one api right so all of this is being routed through the model scope api and this is really the process i want to get across if you have an app a business a software and you want to test hey how do 15 20 30 50 100 whatever the number is different ai models work for this use case test them like this it's free a i'll write the script to do it for you and then you can very quickly judge the results yourself and figure out hey this is the one that i want to go ahead and use for my business now let's head into terminal we're going to go to our downloads and then let's go ahead and if i can not make a typo we can run this script we can see our nine calls three models three prompts. So first is our coding test for the agents. And you can see the three different prompts right here. We have coding, reasoning, and then writing. So I'm going to let this run for a few minutes as it completes the tests, and then we'll run through the results and pick our winner. Okay, we can see of our nine calls, seven of them are done. We're working through the last two and this has all been done in just over a minute minute and a half okay now it's opened all the results here so we can see coding reasoning and writing so what we can do is just look through these figure out what we like the most and that's going to be our winner for this competition we can also see the speed difference right 15 seconds 10 seconds and 19 seconds and also the token difference which is really interesting so i'm going to read through these for a minute and I'll tell you what my winner is. Okay, so I think for this one, the DeepSeek model is my winner for coding. Let's jump over into reasoning. For reasoning, I think I'm also going DeepSeek, right? It was much longer response than the other two, thought longer, and I think that translates into a better result for our coffee brand. And then for writing, let's see, I'll read this for a minute as well. For this, I think I'd actually go with the Quen 3.8 model. So overall, we had two deep seek one quen so we're going to go deep seek as the winner and that leads me to what i want to wrap this video with which let's say that you run this for your own business you try a bunch of models you picked your favorite now you want to go build something with it so you can you can go build something once more using the same process in your winning model completely for free so let's go ahead and do that okay i have our new script to complete a sample app build still through the model scope API with deep seek. Let's hop into terminal, go into our downloads, and then we're going to run the script. And here we go, I finished our app in about 10 seconds. So what this is, is an AI assistant to help us with customer support queries for Alto, or what we do is we paste a customer's email, and then it's going to go ahead and draft a reply with AI using deep seek. So it's an app built with AI that has the deep seek model running through it. And that's through that model scope api for inference right for user prompt now this was obviously super quick app but you can do a lot more with this and just heading back here i think that covers the full breadth the full workflow that i want to get across to you in this video of how to go test these models use these free resources and also if you want to download the data sets download the models locally host all of this stuff and either way really cool platform so big shout out to model scope and overall what we did here right we evaluated these three frontier open source models on my own real use case takes about 10 minutes for zero dollars and we're doing the same test across separate providers previously that meant three accounts three keys and a bill here it was one api completely free and a for loop okay now my take here on the limitations obviously no platform or product is perfect and you should know what you are walking into number one first the credit system does take a minute to understand there's the magic cube currency for compute heavy things and separate quotas for api calls and figuring out what draws from what is not immediately obvious you have to just sit down and kind of read for five or ten minutes that does sort it out but it will take that five or ten minutes of sitting down you should absolutely do that number one parts of the platform do feel like they grew up in that Chinese ecosystem first versus more western. Most of the international site is clean, but you will occasionally hit a dock page or community post that is not clearly translated. It doesn't block you, but it is noticeable. And then third, not every model on the platform does have API inference enabled. It's driven by community demand at the end of the day, so the popular models are covered. But if you have a super niche use case that may be something that you have to run in a notebook instead. And then fourth and finally, the free tier is built for individual developers and prototyping. So if you're shipping a production app with a lot of traffic, you're going to outgrow daily quotas and you'll need to think about proper hosting. And that's fair. That's what free tiers are for. But the bottom line is for what most of us actually need, which is testing models on real tasks and building working prototypes without a bill this is a pretty great deal and a workflow you're going to want to try out and so who should actually sign up i think three groups if you're a developer number one who wants to use open source models without paying per token or renting gpus this is a great move number two if you're learning ai development this is a fantastic sandbox that means free models free compute free deployment in one place and then finally if you're a builder or founder and you want to prototype AI features, you can validate ideas by plugging these models into your application at zero cost before committing to infrastructure or just token maxing with closed models, right? So all of that said, again, if you do want to get started, you can comment model scope under this video or go to this link in the description. If you sign up with this invite code, you get an extra 100 MagiCubes on top of the standard free allowance. So filter the catalog for API inference grab a token and you could be making your first api call if you didn't already within five minutes of watching this that in mind that's the video appreciate you being here and i will see you in the next one cheers