Spatial AudioMap

Okay, whenever you guys are ready. Okay, go ahead. Okay, so this is our device, which provides you a low-power spatial audio. So, this can be applied to an AR or VR device, or you can be like an add-on feature on a Google map. So, how this works is that the user will use the joystick to control the dots on the screen. So, that indicates where the user is on this College Town Crossing map that we created. Okay. So, when we press it down, spatial audio will be played through the headphone, and so the user will hear surrounding noise, like as if he or her is standing at a spot. Okay. So, you're moving this dot around through this animated intersection with the joystick there. Yeah. And you can see on the screen, there's a few sources of noise. Yeah, so here, this is the College Town map that we created. This is the Oishi Bowl. Okay. It will play a convenience store sound. Like a ding-dong. Yeah, like a ding-dong. Oh, yeah. So, we have three sounds. So, that's the Oishi Bowl sound at the corner, and then we also have the sound of a car driving by with this car, and then we also have a chirp sound for the little bird at the corner. Okay. And depending, so just to make sure I understand, depending on where you place the little dot, the audio from each of those sound sources will seem like it's coming from different places. Yeah. Okay. And that's kind of like we want to create an immersive experience in a map. Okay. Because you as a person, when you're moving around, you hear sound from different spaces, in different directions, in different amplitudes, and that's what we want to create. I see. Okay. So, the way how we created this spatial audio or sound localization feature is by tuning the delay between the left channel and the right channel of the deck, which is your left ear and your right ear. We tuned the delay and the amplitude ratio between left ear and right ear based on the angle you have between you and the object. That's all angle-based. And also, it seems like when in real world, like if we're far away from an object, the volume gets lower. We also have this intensity difference parameter being tuned for each sound source, so that's based on distance. So, if you get closer to it, it gets louder. Awesome. So, let me just make sure I've understood. So, for each of these audio sources, the closer we get to it, the louder it gets, just like in real life? Yes. And the sounds can overlay. Yeah. And the sounds overlay. And because, like, these sounds are originally different volume, they have, like, different audio range. So, like, for example, you can hear the car in every corner of the map. You can move on. But for the bell sound, and also the bird sound, you can only hear it when you're closer. Interesting. Okay. So, the rumble of the car is audible all over. Yes. Okay. And then, in order to get the directional audio, you've modeled the time delay associated with the sound getting from one side of your head to the other side of your head. Yes. And that time delay depends on the angle. Yes. That's a simplified version of the hatch-related transfer function. Sure. Yeah. There's also amplitude delay between, like, the left and right. Okay. Intensity. Cool. Can we try it? Yeah. Do you want to try it? Sure. I'll try it. Do you want to be the controller? Yeah. Someone can move me around. You can come around. Just move around. You can. Okay. Be close. So, you're walking out from Dryden. I see. Cool. You look so perfect. Okay. Awesome. So, right now, what I'm hearing, I hear the ding-dong of a wishy-bow on my right. I hear a car. The car seems to be right in front of me. Let me see if I can identify the bird location. The bird is, like, that way. Sort of forward and to my left. Yeah. Oh, I see. So, for me, is it the case that I'm looking this direction? Yeah. Okay. You're walking down from Dryden. Fascinating. Okay. Can you move me closer to, let's see, the bird, perhaps? Yeah. So, now the bird is louder. It's in about the same relative location. Sort of off to my left here. Yeah. The low rumble of the car is still present. It sounds like a sports car incidentally. Yeah. And I'm looking up. Yeah. And now the bird is decidedly off this shoulder. The car has moved a little bit to my right. Yeah. Because you're getting closer to it. Yeah. And let me see if I can hear the ding-dong. No. I'm too far away from it now. Fascinating. So, it's quite immersive. So, now the car is really close. Wow, that's really close. Move to this car. To the car. So, now the car is right off this right side of my body. Yeah. You can see it. Yeah. And it's quite loud. The bird is behind me. I can still hear it, but it's faint. Very cool. So, let me ask. The little icons, the car and all these things, did you guys come up with those yourself? Yeah. So, we used geometry of rectangle, circle, and triangle to combine them. And then overlay them. It's very artistic. Yeah. And I like how the intersection kind of looks like an integrated circuit, which is kind of nice. And how about the sounds? Are those audio files or are you synthesizing those sounds? We synthesize them. Okay. So, you... I see. So, is this direct digital synthesis? Yeah. Yeah, that sounds even better through these headphones too. It's a little rumble. That's really nice. We tried to do before, but it doesn't really work because the synthesis is too low. I see. Okay. Cool. Really, really compelling. Thank you, guys.