I forgot to make a video about this $150,000 server

We just kicked off spring cleaning at our studio, which wouldn't normally be a video, except we came across this. A $150,000 server with two CPUs, eight SSDs, and a whopping 6.3 terabytes of RAM. When we got this from Intel three years ago, we did a quick unboxing, promising a deeper dive into some of its more exotic features, and then promptly forgot about it. Well, that RAM I showed you guys isn't RAM at all, but Optane Persistent Memory Modules. You might know that Intel's Optane business was sadly shuttered in June of 2022, which means the contacts who originally sent us this have sadly moved on. We, on the other hand, haven't. So, all these years later, I am finally going to do what I said I was going to do and finish building the Ricer PC. I mean, sorry, I mean... Digging into what made Optane so special and cool, and what ultimately killed it. Just like I'm gonna kill this segue to our sponsor. Ugreen. They're getting into the NAS game, and they claim their new NAS Sync DXP-4800 Plus is the most powerful NAS under $500. It's got four bays for storage, a 12th gen 5-core Intel processor, 8 gigs of DDR5 memory that's expandable, and more. Check it out at the link down below. Before we go any further, some of you probably need a refresher. Isn't Intel Optane an SSD? What's it doing in a RAM slot? Well, Optane was more of a family than an individual product, ranging from inexpensive hard drive accelerators for pre-builds, where the manufacturer was too cheap to install a real SSD, to incredibly low latency, not to mention expensive storage, for the fastest servers in existence. What all of these uses had in common, though, was they looked like conventional memory and storage devices, but functioned somewhere in between. And that's exactly what we're looking at here. This appears to be a normal SSD, but the 3D cross-point chips inside it have just one hundredth the latency of regular NAND flash. As for this guy, well, it looks like a stick of DDR4 memory, but it's actually an Intel Optane Persistent Memory Series 100 module with a whopping 512 gigs of storage on it. And this server has 12 of them, for a total of 6 terabytes. That's enough to fit nearly 12 high-bitrate floatplane exclusives solely in system memory. Wait a second, Linus. Memory? I thought you just said it was storage. I did, but the simplest way to take advantage of it is in memory mode, where your Optane modules just show up to your operating system as a whole whackload of RAM. Now, the bad thing about this mode is that it forces Optane to have one of the big downsides of RAM. For security reasons, every time an Optane module in memory mode gets restarted, it generates a new cryptographic key, effectively wiping all of the data that it contained. The good news is that Optane can handle many more wipes than regular SSDs, so this is unlikely to harm it. And what's also cool is that no special coding is needed to immediately use memory mode, and the CPU's memory controller will even automatically move colder data off of the memory onto the Optane, and vice versa. Let me show you how this would work in a demanding professional application. Right now you're looking at SolidWorks Flow Simulator, but we aren't simulating anything at the moment, because first we need to generate a mesh, which is essentially a 3D space full of data sampling points, where some fluid, let's say air, is going to interact with some object, say this heatsink. It's not using that much RAM, though. What, 45 gigs? 48 gigs? That's a lot of RAM. Okay, it's a lot of RAM, but it's 2% of what we have. The issue is that it just takes hours for the CPU to process a large enough mesh like this for it to fill up all of that RAM, let alone the Optane that's in there, and this is a really silly way of doing this. So what you're saying is Flow Sim doesn't need that much RAM, Optane is pointless? No, no, no. Flow Sim does need heaps and heaps of RAM, just a model like this doesn't necessarily. If we were trying to run something that's super complex, like crashing a car into a barrier, having terabytes of memory could be the difference between a finishing in a day and a finishing in a week. Of course, though, this isn't memory. Its latency is several times higher, which would obviously impact performance. The question is, though, how much? To find out, we came up with a rather terrible idea. Why don't we just remove a bunch of the actual RAM so that the system has no choice but to use the Optane, and then we can do a before and after. It's not gonna do anything, is it? Nope. Oh, geez, there goes the alarm. Wait, maybe? There's no way. No. Sadly, Intel thought of that and requires each stick of Optane PMEM to be paired with a stick of RAM in order for the system to function at all, and they have a recommended ratio of 4 to 1 Optane to RAM for optimal performance. Ours, however, is more like a 16 to 1 ratio, and the server came straight from Intel, so what do we do with the rest of it? As it turns out, many customers simply used higher ratios, but there are some other ways to take advantage of the persistent part of our persistent memory. App direct mode and storage mode, which Alex is showing us now. Yes. App direct mode exposes the Optane directly to the application, which can use it like storage, RAM, or somewhere in between. Unfortunately, we aren't able to show this off, because doing so would require us to code our own program that explicitly takes advantage of it, and this is a task that almost no companies worth billions of dollars found to be worthwhile, so doing it for a YouTube video would have a dubious ROI, to say the least. What we can show you is storage mode. Have you ever wondered what would happen if you could plug your SSD into a high-speed memory slot? Well, basically this. Storage mode presents the Optane to your operating system much like an SSD, just with super low latency compared to traditional NAND flash. Setting this bit up, though, was what caused this project to be stalled for months, because we just couldn't figure out how to do it. Not because Alex is dumb, or because I'm dumb, but rather because the BIOS on this Intel motherboard is so poorly organized. Okay, okay, sorry, this is gonna be a bit of a tangent, but motherboard manufacturers, can you please figure your shiz out? I mean, we have this industry-wide problem where every single motherboard maker calls the exact same setting something ever so slightly different, and then puts it in a different place, meaning even if you are quite knowledgeable and not dumb, and you know exactly what you're looking for, there's a solid chance you won't be able to find it. We were easily able to figure out how to use AppDirect mode on Dell, Supermicro, Lenovo, and HP server boards, since they had decent documentation, but no, not the Intel one. We even called up Patrick from Serve the Home, who literally wrote the guide on how to do this, and he couldn't figure it out either. Okay, look at this, Linus. I'm going to go back to the main menu. We're looking for the memory controller spot. Where do you think that is? Well, I would go to advanced, and then I would probably go to memory configuration. So this is why we needed Emily to help us find it. Yes, it is under PCI configuration, option ROM controller, and that's where the memory controller is. Of course, I can't believe we didn't think of that. Anyway, by using storage mode, we're able to select how much of the Optane we want to be presented as an SSD, and we can create volumes with it, much like with a RAID array, with the maximum performance occurring when you interleave across multiple Optane DIMMs that are operating on the same CPU, which essentially creates a RAID 0 array on Optane. Now, as you might have guessed, this does come with the risk of data loss, so if you don't care as much about performance, you might want to mirror the data across both CPUs, or, I don't know, maybe just buy an SSD. But all of that aside, the coolest configuration is mixed mode. This allows you to set a target for how much of your Optane will be storage and how much will be RAM, creating a best of both worlds situation where the BIOS handles all of the complex bits and you get both cheap RAM and super low latency storage. All right, what's your memory percentage goal? What do you mean goal? It doesn't really do exact, it just kind of gets close to the percentage that you want. I think we all know the answer to that. Nice. So we reboot. Neat. Ooh, sorry, seemed mad. Okay, all right, no, you don't seem mad, you seem rightfully like we need to discuss this. Come on, come on, create namespace. Here we go. Hey, there we go. How about now? There we go. I love how surprised it is. It's making a YouTube thumbnail face. Speaking of which, we're working on a really cool video. Now that YouTube has A-B testing for thumbnails, Maria is creating a, like, cringe YouTuber face version of every thumbnail for a little bit and then just, like, a nice relaxed one, and then we're going to make a video about which ones people actually prefer, whatever they say they prefer. It could change our thumbnail practices. There we go. Okay, what we're looking at right now is our G drive, which is a conventional NVMe SSD, and our O drive, which is our Optane drive. And this measure is seconds per transfer, which is essentially the latency of each operation on the drive. Now what we're going to do is we're going to run CrystalDiskMark on both of these drives at the same time. And theoretically, yes. Yep. The regular NVMe is way more, which is bad. O is... Optane. Which makes sense because it barely even appears. That's how low the latency of this is, which doesn't matter for everything. And in fact, if we have a look at our CrystalDiskMark results, it's not that fast in terms of sequential transfer speeds. It's just that it's really, really responsive. Wow, we've done it. Okay, we can actually see what the crap we're looking at now. Here we go. This is what we expect to see. All three tests are running right now. There we go. Yes. And our SATA is much higher in terms of latency, not to mention that it can be very inconsistent. See that initial spike when we first hit it? That's a yikes. And while NVMe is a lot better, it doesn't have that super, super flat line when all of them are running the same test that we see with Optane, where the latency is negligible, which can be a huge benefit for certain applications and not terribly important for others. Which is cool. In some cases, really cool. So then why does Optane no longer exist? The answer is many, many reasons. The first and biggest problem is that Optane arrived two years late. It was supposed to come with Skylake in 2017 with speeds that matched the DDR4-2666 that would be paired with those processors. But Optane didn't arrive until Cascade Lake, which used 2933 megatransfer per second DIMMs. That meant that by choosing Optane, you would be giving up memory bandwidth to every application that doesn't need the extra capacity. That is not ideal. Also, in 2017, there was a global DRAM shortage as phones like the Samsung Galaxy S6, Google Pixel, and HTC One M9 started using DDR4, while Samsung and Micron were struggling to ramp up production. 16 gigs of RAM hit prices of over $200. That would have been a perfect opportunity for an almost-RAM product at a lower price. Except that by 2019, that same 16 gigs of RAM was down to more like a hundo. Also, remember how we mentioned that each stick of Optane PMEM requires a stick of RAM to pair with it in order for it to work? Well, yeah. So companies that were looking into Optane, they weren't just comparing the price of Optane versus RAM. They had to look at the cost of an Optane module and a stick of RAM compared to just getting two sticks of RAM that, oh, by the way, are gonna run faster. Tom's Hardware has an article from 2019 about the pricing, quoting a single stick of this 512-gig Optane at a cool $6,751, meaning that this server that we casually lost when it was new had $81,000 worth of Optane in it. That sounds like a lot, because it is. But in 2019, a single stick of 128 gigs of ECC memory was roughly $4,500, meaning Optane was only about one-third of the price. Wait, hold on a second. That was in 2019. That still sounds pretty good. I mean, the demo we just did looked pretty good. Optane for the win, right? Well, I do love your Optane-ism. Remember, guys, we spoiled the ending earlier, and everyone dies. But it doesn't have to be a total loss. There's maybe a valuable lesson here for the industry to maybe learn this time. Intel, if you're listening, and everyone, you need to stop locking your products because I didn't buy enough of your products. The Cascade Lake CPUs that did eventually ship with Optane had two variants, a standard version and a large memory footprint version. In the case of the Xeon Platinum 8260s that we've got in this server, the standard version supported only one terabyte of RAM, a number that's pretty easy to hit with regular old DRAM, meaning that if you wanted to really take advantage of Optane, you needed the Xeon Platinum 8260L. Oh, I mean, that's no problem, right? I mean, how much more could it co... $3,400? Oh, yeah, and since you're buying two of them, it's an extra $6,800 before you buy a single flipping Optane module. You can see why they used L as the suffix for those. Adding insult to injury, you know what also happened between 2017 and 2019 while Intel was struggling to get Optane out of the door? AMD launched Epic Rome and then Epic Milan, featuring more cores, more memory channels, support for four tebybytes of RAM without a silly large memory footprint tax. Oh, and by the way, they were also cheaper. This added up to not a whole lot of people buying Optane, and Micron, who owned the fab where all the Optane was being built, finally said, you know what? Enough is enough. It was and still is a very cool technology, but in 2021, Micron figured, you know what would be a lot cooler? How about not losing $400 million a year? So one year later, Intel killed Optane, which raises one interesting question. Why can you still buy it? Well, about that, even though there hasn't been a fab making the stuff for years now, in 2022, Intel revealed that they still had over half a billion dollars of inventory leftover, and they haven't exactly been pushing it that hard. I mean, what is this pricing? Intel, do you want this on your books forever? There are some awesome closeout deals out there. These 16 gig accelerators are amazing. There are go-to drives for a DIY network appliance or a NAS, because they're literally five bucks, and Optane features substantially more write endurance than NAND, which is great if you want to constantly retain and overwrite comprehensive system logs. And this one on Newegg is still among, if not the best drive on the market, even if it is starting to show its age in some ways. For example, it'll have outstanding latency, like we talked about before, but it runs at a quarter, the maximum speed of a modern gen five drive, which is a bit of a yikes. So in conclusion, Intel, do you want this thing back? We honestly really don't know what to do with it, but I do know how to segue to our sponsor. Delete.me. At LMG, we generally believe that, unless it gets you in trouble with the law, what you do on the internet is your business and yours alone. No judgment. Yes, that means you too, Ralph. You're safe this time. And Delete.me thinks the same, so that's why they make it their mission to keep your information from falling into the wrong hands. Their software and expert squad scour the internet looking for your data and can sweep it away in a heartbeat. Your phone number popping up in a cold callers list? Deleted. Your email cc'd in a copy paste template telling you to click this link to lock your bank account? Deleted. Delete.me averages over 2,000 data pieces gone for a customer in their first two years. Imagine doing all that manually. Yeah, no thanks. I won't. So head on over to the link below, use code LTT for 20% off a subscription and start locking your personal data down. If you guys enjoyed this video, maybe you watched the old one where we originally unboxed this beast. It was a lot newer and less dead then.