尼康好兄弟 尼康Z9二代會搭載新一代CMOS嗎？

G'day everybody, how are you going today? It is so good to see you and happy new year Wow, here we are in the new year And what that means is a whole lot more gear A whole lot more lenses and cameras A whole lot more images A whole lot more video And a whole lot more speculation About what's to come next Now I just shot for Federation Square last night Because it is New Year's Day for me right now The fireworks I'm hoping Federation Square will let me turn that into a video So I can show you what I did shooting for Federation Square Which is basically the town square for the city of Melbourne It's an absolutely massive New Year's Eve And for the 12 o'clock fireworks I ended up on the roof of a building at Federation Square Capturing the action I shot with four different cameras Z8, Z9, Z63 and Z7 Using four primes for the majority of it And just a great showcase of what this technology is capable of I've been shooting major events I think if memory serves me correctly 1999 Moomba Which is a kind of massive sort of street parade festival Maybe the first major event that I photographed And then I've been shooting enormous concerts And New Year's Eves And all sorts of major events here in Melbourne And I've just never experienced the technology that we have today The capacity and the ability to create just gorgeous cinematic images And now with the level of technology that we have today We can do this sort of thing on the fly Now shooting primes, that's part of it The whole point of a prime, just in case you didn't know Is a prime is absolutely extraordinary Or it should be at what it is Which is that one focal length 50mm, 85mm, 135mm Today's primes, what's so amazing about them And this is most definitely for the Z range S primes Is they're extraordinary wide open as well And this is something that as I've been photographing These sort of major night time events for, well, 25 years It just wasn't possible really I mean you could shoot them wide open But you just didn't get the same optical quality And of course the focus systems And the video systems Just the whole package has advanced so much For these very difficult And in this case night time And that's what makes it difficult Night time, lots of different light sources Lots of action, lots going on Everything's moving quickly You don't have much time And then it's all over I hope to bring you that But I'm sitting in the glow of a big night of shooting The traffic was so enormous getting out of the city I didn't get home till 2am If I skip a few beats here My apologies, it was a big night In this video we're going to speculate On what might be Nikon's next major sensor technology Speculation is sometimes considered just made up Which I suppose it can be I tend to like to speculate based on lots of data And in the case of this particular video We're talking about hard data That Nikon has released themselves Now this particular technology That I want to talk about today Is a sensor that runs at 1000 frames per second And has high dynamic range And you might go Well this is all theoretical And the reality is it's not Nikon has actually created prototype cameras Which we will see in this video You can actually see the camera in this paper That they have released publicly And we can also see some of the results And you might go Well it's just some little sensor And it's not It's actually a one inch sensor What's even more interesting about this Is the pixel pitch Well it's the sort of pixel pitch We see all the time in all sorts of devices So this is all very real world And they've been working on it for a while It's ridiculously fast It has high dynamic range And it's created on a 65 nanometer process Does anyone know what that means That's about the lithography required To etch these sorts of sensors Which is a conversation I was having recently In my Nikon museum visit video Which you should check out up there Nikon make etching or lithography Or stepper machines That are capable of working At that sort of resolution Which comes back to the conversation That well is it possible That Nikon's actual lithography machines Are etching their own sensors And from my perspective That's probably highly likely Nikon are very involved In the creation of their own sensors And we will see here in this video This is what they're doing right here Right now with this sensor This is not somebody else's technology This is their technology Let's get into looking at this document And look this is a big nerdy video If big nerdy videos are not for you Then maybe wait for the next one But if you're really interested in seeing A bit of a deep dive On some pretty serious technology That if it ever comes to pass Or even a version of this comes to pass As I said it runs at 1000 frames per second That means it can take images At 1000 frames per second Even if it was a half A quarter A fifth of the speed That would make it extraordinary This is the title A 1 inch 17 megapixel 1000 FPS block controlled Coded exposure Back illuminated Stacked CMOS image sensor For computational imaging And adaptive dynamic range control Now it could be argued That this sensor is not aimed At the photographic market But when we go on to read The first paragraph We do see that it's aimed at High dynamic range situations Now one area that I think Nikon Is very interested in And is going to be An enormous growth industry It's already enormous Is cameras in cars The automotive industry Tesla has something like Seven or nine cameras And if you think about the millions of cars That are sold every year And more and more technology Is going to be required Along with other usages Besides simply cars The need for sensors to see the world Is enormous And of course the world When you're outside driving Or in all sorts of exterior needs You do need high dynamic range sensors And a sensor running at a high frame rate Means that you are getting more data Into the computer To be able to make decisions About the world in front of it This sensor may not be For a global Nikon camera As you and I know it A Z92 perhaps It might be for automotive And other such usages Robots as well of course Tesla is making Optimus Their new robot And it also needs lots of sensors To see the world around it But of course any technology That's being worked on The byproducts of those technologies Can end up in other places And that's where we might see A version of this That's larger but slower And when we say slower Well a thousand frames per second Is ridiculously fast Even if we had one-fifth of the speed At 200 frames per second Or one-tenth of the speed At 100 frames per second Well I think we'd all be pretty excited If it still had awesome High dynamic range And thus it might not be this sensor It might be a derivative of this sensor Let's deep dive into what This 1000 frames per second High dynamic range One inch sensor is all about This study introduces a vision system That can acquire images at high speeds And high resolutions It can handle wide differences In brightness beyond human perception Image sensors are not only used In digital still cameras But also in a wide range of fields Such as robotic vision Object recognition And movement tracking At high speeds And high definitions Are essential In automated driving systems Especially in dark tunnels Or in a mid-summer sunshine However capturing a moving subject With a high contrast And achieving a high performance Concerning all aspects Is challenging Because of the trade-off Between high definition And high frame rate And pixel size And dynamic range We developed a high speed vision system Capable of 1000 frames per second Read out operation At a resolution of 4 by 4K Achieving a dynamic range Of 110 dB And fine pixels of 2.7 Now 110 dB I don't know how that translates To dynamic range Is that 10 stops 11 stops 14 stops Is that 20 stops If anybody out there knows I would love you all to let us know Because I think that is a pretty Critical part of this sensor And Nick Owen is stating It works in some very high Dynamic range situations So that is very cool These characteristics were achieved Simultaneously using Coded exposure CE Which divides the image plane Into smaller blocks And controls the exposure time Of each block individually Now this is critical This is how this sensor Is able to achieve High dynamic range With such small photo sites At such high speed An idea like this already exists Somewhere really common And really obvious That's in all of our homes The way that LED television panel Backlights have worked Is to have zones And those zones might be a grid Sort of like a chess board But obviously it's not square It's rectangular And there might be 40 or 60 Or 120 zones Backlit zones That can be turned on and off Or made brighter and dimmer And that's how we get Brighter and dimmer areas On our flat panel displays And as that technology has matured You've had smaller and smaller And more and more of those zones Thus creating a higher resolution Of intensity of light To the point where If you buy an OLED screen Well each individual pixel Is illuminated on its own But there are still Plenty of manufacturers Creating that other type of screen Where there is a massive grid Of illumination panels Behind the color pixels The color pixel array It's funny TVs are actually quite similar to sensors In a lot of ways That's how we get Differing levels of brightness This concept appears to be the same And Nikon has created These individual areas Which are very very small When you consider the pixel density And how many pixels are here It's a 17 million pixel sensor And these individual arrays That they can control the exposure with In these tiny tiny little blocks Are only 16 by 16 pixels This is really interesting technology In a way it's like having Tens of thousands of little individual 16 by 16 megapixel sensors That are all doing their own thing This is awesome And it's capable of doing it By the look of it At extraordinarily high frame rates A high speed readout Was achieved by arranging Analog and digital converters In parallel for each block A high resolution was realized By integrating both circuits And pixels at high density Using three dimensional Wafer stacking technology This system is expected to be applied To computational imaging Using CE In addition to applications That require high dynamic range In which dark and bright areas Co-exist in a scene That final part of that sentence Where they're talking about high dynamic range Well obviously we as photographers Are always looking for that type of sensor And the holy grail The absolute holy grail for everything And the ultimate all-rounder sensor The ultimate all-rounder camera And what would that next camera be From Nikon Well I think we would all think That is the Z9 II Well it's a sensor that's very very fast Which this is And it's a sensor that has Extraordinary dynamic range Which this does And they have a working prototype Now we go to point number one Which is the introduction Which straight away talks about photography Now it says that sensors are not only used For photography But that means that they are used For photography Image sensors are not only used For taking photos But also increasingly expected to serve As intelligent systems With surrounding configurations So in this opening introduction Nikon go on to talk about The various other ways They could have achieved a high readout And HDR sensor Talking about the pros and cons And why that they settled on this version And just to reiterate This is an immense amount of effort Going into the engineering And the potential ways Of solving this problem Look I'm not a scientist And I don't profess to know Exactly how to read This deeply technical document But we can see here That they are having One, two, three, four Maybe more attempts Or theorized attempts That how do we solve this problem And then they've come around to this one Out of various methods to do it This is absolutely Nikon doing this And if there's anybody out there That doesn't think that Nikon Are spending an enormous amount of time Energy Intellect Engineering Science Physics Electrical engineering Materials engineering There's just so many layers To getting a sensor That theoretically work There's also so many different ways That they can do this And we can see it here in this paper I will provide a link So you can get If you're interested And if you're interested in genuinely knowing How much effort Nikon put into their Sensor creating technology Then do yourself a favor Read this document Before you think anything differently There is this statement That Sony make Nikon sensors Let's just run with that ball Just for a second Maybe Sony bake the cake They literally get all of Nikon's ideas All of their technology And it's put into a cake tin And it's put in an oven That's what Sony do They put it in a cake tin They put it in an oven And they give it back And then what do Nikon do They take that cake And they decorate it with a color array With glass They use all of their raw converters Their raw technology Which is clearly different Their color science All of those things So they do all of this R&D All of this engineering All of this thinking That goes for years and years and years Then the ingredients are given to Sony They bake it And then that baked cake Is given back to Nikon And then they do all sorts of work On top of it after that That might be the case But in my mind That doesn't mean That Sony make Nikon sensors What it means is They fabricate them And at the end of the day If you know anything about how For example Apple make their products Apple don't own the manufacturing Other companies make their products And they are different companies For example for Apple To be able to grow in India They needed to manufacture some If not all of their phones in India They are being manufactured In another country Not by Apple Does this mean that Apple Aren't in control of what's going on Does this mean that Apple Didn't design it Does this mean that it's not Apple's technology No it's an Apple product And this situation is no different And part of why I wanted to show Everybody this document Is just to see the enormous length That have gone to To bring us the technology So we can do this Right here right now in a device Which is what Something like 2000 US dollars Astonishing And I just wanted to point out That Tim Cook two years ago Said that they worked with Sony Then there was a report saying That Apple would take Their sensor design in house And now we're hearing It might go to Samsung Things just never remain the same And I think it's safe to assume Whatever is happening with Nikon today May not be happening with Nikon tomorrow This paper that Nikon have published Is going to be literally The tip of the iceberg The amount of information And science and technology That will have gone into Getting this to the point that it's at Another big point about this Is the fact that Nikon Is publicly releasing it Would say to me that it's It's probably coming soon in some form Maybe and of course I believe recently Nikon Did announce a partnership I think it was with Mitsubishi In regards to automotive cameras There you go The world does not really move In mysterious ways If you join all the pieces Of the puzzle together What is talked about here Is the resolution of this chip Which is 4224x4224 pixels Then you can take a really technical look At how this chip is built Which is part of section 2 Which is the sensor architecture And you can see here The 16x16 pixel architecture Which can be individually exposed And they are calling them The exposure blocks And I think that's pretty cool Written on this diagram Is the global control logic I think I've seen it other times That this has been called a global sensor It's not specifically called A global sensor in this document But at this sort of speed I suspect it's reading globally Figure 1 shows a conceptual diagram Of the image sensor This one inch sensor has two layers The top chip comprises BSI pixels With the 65 nanometer process At the bottom chip And the bottom chip is used For signal processing The layers are bonded to each other The highest density areas Consist of two contacts per pixel The top chip has a pixel array Of 4224x4224 at 2.7 pitch Which is divided into A 264x264 exposure control blocks With the basic unit being A 16x16 pixel size As we move into Configuration of the experimental system Which is section 3 Figure 4 Which is an image of the actual camera Shows the system configuration The system comprises a camera head Camera control unit And host computer Which is a PC The camera head has a lens unit And image sensor board The camera control unit is composed of Two field programmable gate arrays And a power supply board And functions to control the image sensor Supply power and receive image data The PC controls the entire system process And then saves the images From what I'm reading here it says The exposure value can be updated In every 8 frames Which is still extraordinarily fast When you're talking about A thousand frames per second We are getting exposure information And being able to change the exposure That's more than a hundred times a second In section 4 Results and discussion They talk about how they were able to Achieve high dynamic range results At a thousand frames per second In 4.3 we talk about HDR tracking for moving objects Figure 7 shows the results of the Responsiveness of exposure control To moving objects The sensor was operated At a thousand frames per second And an object moved horizontally On the screen The image on the left Captured without an exposure control When the object dashes From the dark to the bright area Of frame 3, 4, 5 With respect to the initial state Of frame 1 The characters on the body are oversaturated The results of the dynamic exposure control In every 8 frames Are demonstrated on the right side Updating the exposure table In every 8 frames Suppresses the oversaturation Of characters on the body And realizes an HDR image 4.4 Sensor Specifications Table 1 presents a comparison Between the performance of our sensor With that of other sensors Presented in the existing studies Our developed sensor achieves High speed readout With a resolution of 17 megapixels In addition this paper demonstrates HDR with small pixels of 2.7 The total power of the sensor Was 7.4 watts At 1000 frames per second The chip micrograph Is shown in figure 8 Both chips were fabricated Using the 65 nanometer process 4.5 More applications Using coded exposure Figure 9 shows an application That overlays the information Number 5 Conclusions In this study We have newly developed A CMOS image sensor With a stacked structure That operates at 1000 frames per second While having a high resolution Of 17 megapixels With a small pixel pitch of 2.7 By using the Blockwise coded exposure function 110 dB dynamic range is achieved At 1000 frames per second In a single frame And 134 dB dynamic range in 16 frames This vision system can be applied To various computational imaging Using the coded exposure function In addition to high dynamic range imaging In scenes where dark and bright areas Of the subject are mixed in the frame Whether this ends up being a Z9 II Or not This is really interesting technology That may well be applicable To the photography industry Remember Nikon is showing us here What they're happy to publicly show us I did a little bit of maths In regards to this sensor Its pixel pitch And this sensor If it was expanded out To a 35 mm sensor Would be something I think If my maths is correct Something like 12000 pixels On the long edge That's significantly more than the current Just over 8000 we have On a 45 megapixel camera That is bringing us more up to A 100 megapixel sensor Interesting isn't it Because we've talked about Those sorts of things before We've talked about them wanting to be High dynamic range And that sort of technology We might want to see In a Z7 III So who knows A slower version of this We do not need Nobody wants 1000 frames per second In a Z7 III And I suspect the heat And the power consumption And just the data pipeline Would all be extraordinarily unreasonable But imagine if it was 20 frames per second One 50th Of everything that's going on here Speed wise But giving us all of the advantages Of that HDR And that very large sensor size And again the same thinking Could be applied to a Z9 II If we go from 12000 pixels in width Down to 8000 odd pixels in width We half or quarter the speed 250 frames Maybe 1 8th of the speed Around 125 frames Isn't that interesting 120 frames That seems to be a magic number In the industry right now So 1 8th of the speed Around 120 frames per second Pixel pitch becomes larger Uses far less power I don't know This technology to me Is very very interesting And I wonder what would be required To tweak it to fit Because there are so many layers to this Just to repeat Power consumption Heat Data pipeline And of course finally What is the cost We have no idea What the cost Of making these prototypes is I expect it's immense Because they're basically hand built This one prototype So of course We're not talking that If we think about the sensor That's in the Red Raptor It's obviously a lot more expensive Than the sensor in the Z9 Or the A1 A12 Or the A93 So it might be great To have this sort of technology But it might be too expensive The flip side of that is If you're going to build it for automotive And you want to sell it mass market Then it can't be super expensive And the thing about creating this sort of sensor Is that if you tweak any metric Any part of this Make it a little bit slower That changes everything Make the pixel pitch a little bit larger That changes everything And to me this just shows How deeply invested Nikon are In their sensor technology I had a discussion with someone recently Around the Z9 and the Z8 sensor It's still today And we're now in the fourth year Of its existence It's still the only sensor of its type Nobody else is doing this In the 35 mirrorless camera space So don't discount Nikon For being able to do Some pretty extraordinary things In this space in the future Along with the fact That they now have RED's alchemy RED's sensor technology To also add to their gamut of tools And expertise and engineering They have at hand The future is very bright And I'd love to know your thoughts On this look at this paper Published by Nikon It's publicly available on their website There have been patent applications For this going on For quite a number of years They have been working on it a long time Alright everybody It's been so good to see you And if this is your first time here I would love to see you again So please do subscribe Please share and please like Alright, bye for now