This math trick revolutionized physics

In this video, I want to show you how the study of hot objects broke classical physics, and the only way to fix this mess was by using a math trick that led to a revolution in physics called quantum mechanics. You may know the story already, but here I want to show you the mathematical shortcuts, the wild desperate guesses, and how the idea that energy comes in discrete amounts solves the problem. One of the great marvels of 20th century physics is quantum mechanics, a revolutionary understanding of nature at the scale of atoms and molecules that today enables the inner workings of every single electronic device, like the one in front of you right now. Most videos and textbooks explain that everything started when Max Planck had the weird but brilliant idea that energy comes in small packages called quanta, which he used to solve a mystery about light and heat. But it turns out that Planck first solved the problem using a guess, and later he worked backwards to find a physical interpretation to his conjecture using a math trick. This is the less told story of the dawn of quantum mechanics. If you have ever taken a course on quantum mechanics or read a popular physics book on the topic, you might have encountered terms like blackbody irradiation, the ultraviolet catastrophe, and how this led to the development of quantum mechanics. In case you haven't, here is a recap. During the late part of the 19th century, physics, chemistry, and engineering were changing the world. Steam power was literally the engine of the industrial revolution. More efficient machines and artificial lights required a better understanding of heat, gases, electricity, and light. A basic problem was the radiation emitted by hot objects. It was a well-known fact. Heat an object and it will glow. But scientists didn't know the details. They didn't have a mathematical description and a physical explanation for this common phenomenon. The lack of a clear understanding of such a basic observation was a scientific embarrassment. Imagine a metal box at a well-defined temperature with a little hole for the radiation to escape. Physicists decomposed the light coming from these hot objects and measured how much of each color was produced. In other words, they measured the intensity of light as a function of its frequency. The data looked something like this, with no light emitted close to zero, a bump in the middle, and a decrease at high frequencies. This curve is called the blackbody spectrum. The challenge was to find a mathematical expression that could explain this data from physical principles. One of the most renowned physicists in the study of heat, electricity, and light was Gustav Kirchhoff. In 1860, he formally defined the blackbody and from experimental observations established that the unknown curve describing the energy density of this spectrum should be a universal function of the frequency of light, nu, and the temperature, T, and challenged the physics community to find the mathematical form of this curve and its physical meaning. Experiments showed that when the temperature of the object was increased, the curve grew very fast and the peak shifted to higher frequency. In 1879, Joseph Stefan in Vienna noticed that the area under the curve was proportional to the temperature to the fourth power, where the constant sigma can be determined from data. Five years later, in 1884, his most famous student Ludwig Boltzmann used thermodynamics to derive Stefan relations, which became known as the Stefan-Boltzmann law. This is in fact the relation that I used in a previous video to explain the double flash observed in nuclear explosions. Make sure to check that video later if you haven't. After a few years and also using thermodynamics, Wilhelm Wien in Germany was able to derive a powerful relationship between the peak of the blackbody spectrum and the temperature. He found that their ratio was a constant. In other words, the frequency at the peak was proportional to the temperature. This relation became known as Wien's displacement law. These two laws allow determining the temperature of any object by simply measuring its spectrum and finding the area under the curve or the frequency at which the peak of the spectrum occurs, from an oven to our sun and distant stars. But the mystery of the unknown universal function persisted. After three years and using these two laws, Wien used Boltzmann's statistical mechanics to produce the first successful formula describing the blackbody spectrum, where A and B are constants to be determined by experiments. This formula works really well at high frequencies, but it fails to describe the data at low frequencies. But in 1896 and the following four years, this was the best description of the blackbody radiation, because measurements at low frequencies were challenging and quite unreliable. In 1900, in the UK, Lord Riley, with some later contribution by James Jeans, proposed a different formula that worked extremely well at low frequencies, but it failed completely in the mid and high range. This new formula incorrectly predicts an ever-increasing growth for high frequencies that was later called the ultraviolet catastrophe by Paul Ehrenfest, but this name was invented a decade later, so I find it irrelevant for this story. Lord Riley applied the statistical mechanics invented by Boltzmann and the electromagnetic theory of Maxwell. It was clear that this relation was just an approximation of the still unknown universal formula, but at least now we had the two limits. All that was needed was a good way to patch them in the middle. This is the point where most versions of the story introduce Max Planck as a person who was able to find a formula to interpolate between Wien's and Riley-Jeans limits by introducing energy quantization. But the truth is quite different. Although it's clear that he used the results from Wien, there is no evidence that Planck ever knew about Lord Riley's formula, so let's see what Planck actually did. Max Planck was a student of Gustav Kirchhoff, the physicist that defined the black body and challenged the physics community to find the universal function. Planck's expertise was thermodynamics, like most of the theories of the time, and like many, he was not a big fan of Boltzmann's statistical mechanics. Boltzmann's ideas worked for explaining many results in thermodynamics, and the whole idea of matter as an immense number of atoms and molecules following statistical rules had been fully embraced by chemists. But it took a while before physicists considered the idea of tiny, invisible particles seriously. The mystery of the universal function describing the black body radiation was a hot topic, and Planck took on the challenge proposed by his Dr. Fata from a pure thermodynamic point of view. For Planck, one of the key concepts in physics was entropy, usually described as the degree of disorder of a physical system. Following this approach, Planck described the problem as a collection of charged particles in the material that he called Hertzian electromagnetic oscillators, and used Maxwell equations to characterize their thermal equilibrium with the radiation. One of his initial findings in a series of papers published between 1887 and 1899 was that the energy density could be decomposed as a product of two factors, one describing the number of states for the oscillators, and the other given by their internal energy. So the universal function had to have this form, where c is the speed of light. In this way, the internal energy of a system described by Dean's law depends on two constants, the overall factor A and the term in the exponential, b. Using published data from experiments by Otto Lohmer and Ernst Brinstein, Planck found the constant A to have this value. This is what today we call Planck constant, and denoted by h. Comparing this with the modern value of h shows that Planck's value over a century ago was very precise. But this is a consequence of the incredibly precise experiments, so the credit here goes to the fantastic measurements of Lohmer and Brinstein. I believe that one point that is not emphasized enough in videos and textbooks is the vast and crucial work by experimentalists developing new and sophisticated ovens and cavities for radiation measurements, new techniques and sensors, as well as clever methods to detect very weak radiation and put the technology to the limit. Some of the heroes of this story are Otto Lohmer, Ernst Brinstein, Heinrich Rubens, and Ferdinand Kirchbaum. No theoretical development would have been possible without the precise measurements to validate and refute all the proposed ideas. Up to this point, Planck had not solved the problem of finding the universal function, but this decomposition allows a clear connection between the energy density, the internal energy, and the entropy. Planck claimed that the entropy of his Hertzian oscillators was given by this expression. He provided physical motivations for this formula, but without a mathematical proof. But it's not hard to find that this entropy was designed to produce Wien's law. From Planck's decomposition of the universal function for Wien's law, the corresponding internal energy is A nu times exponential factor. Inverting this expression and solving for 1 over temperature, we find that 1 over T goes like the logarithm of U. Now using the fundamental thermodynamic relation between temperature, entropy, and internal energy, we get an ordinary differential equation for S as a function of the internal energy U. Integrating this and using the appropriate boundary conditions produces exactly the entropy reported by Planck. In other words, what Planck found was the entropy to model his oscillators that leads to Wien's law. What he did next was completely new. He defined a quantity 1 over R given by the second derivative of the entropy with respect to the internal energy, which is equivalent to the derivative of the inverse temperature. This shows that R is proportional to the internal energy. After exploring the problem for years, Planck understood that this linear dependence on the internal energy was the reason that Wien's law was so successful at high frequencies. In May of 1899, Planck published his derivation of the successful Wien's law based on his entropy relation. But some new experiments at low frequencies show evidence of problems with Wien's formula. By early 1900, Blumer and Pringsheim had improved their measurements and found that Wien's law appears to fail at low frequencies. By this time, Heinrich Rubens had developed a new technique to intensify weak light signals. Using his new methods and pushing their measurements beyond the visible light to lower frequencies all the way to the infrared, Rubens and Kirchbaum confirmed that at these low frequencies, Wien's law completely fails. In addition to this remarkable result, they also found that the energy density becomes proportional to the temperature, consistent with the recent Rayleigh-Jeans formula in the infrared. Rubens and Kirchbaum not only confirmed the failure of Wien's law at low frequencies, but they also tested many existing proposals for the universal function against their precise measurements, and all failed to provide a simple fit to the new data. And if Wien's law fails, so does Planck's entropy relation. The 7th of October of 1900, a social event brought Rubens and his wife to Planck's house. By the way, you might have seen an apparatus for demonstrating acoustic standing waves in a tube with gas and flames called the Rubens tube. Yep, this is the same Rubens. He invented this demonstration in 1905. During the Sunday tea at Planck's house, Rubens told him about his new results. The failure of Wien's law meant that the entropy relation was incorrect. This should have been bad news for Planck, but the other finding was crucial. Rubens told him that their data indicated that for infrared light, the internal energy becomes proportional to the temperature. And as I will show you in a minute, this gave Planck a way to quickly fix his entropy relation. Before I continue, a big thanks to the viewers that have directly contributed to the channel on Ko-fi, and with super thanks on YouTube. You guys rock! If you find the content of this and other videos of value, please consider supporting the channel on Ko-fi. The link is in the description below. Another way you can always help this channel to grow is by liking, commenting, sharing and subscribing. Rubens shared that Kullbaum would present their findings in the next meeting of the Berlin Physical Society in less than two weeks. With this new crucial information, Planck could barely wait for his guests to leave the house to attempt to modify his entropy formula before the As soon as Rubens and his wife left, Planck knew that he had a long night ahead. Assuming that for low frequencies, the internal energy is proportional to the temperature leads to Rayleigh-Jeans formula, but Planck proposed it only as a limit to match the recent experiments. The Boltzmann constant there is just to convert the units of temperature into units of energy. Following his previous procedure, he started from his quantity R. He wrote T as a function of U to write the inverse of the temperature, and then calculated the derivative with respect to the internal energy, to find that now R depends quadratically on the internal energy, instead of linearly as in the case to the Rayleigh-Jeans law. Here Planck had the idea of using the sum of his old linear relation and this new quadratic term. His hope was that at high frequencies, the linear term would dominate in the entropy relation, leading to Wien's law, and at low frequencies, the quadratic term would be dominant, leading to a linear function on the temperature, making it consistent with the new measurements of Rubens and Courbant. Adding these two terms and working backwards, we get an ordinary differential equation for the temperature. Rearranging terms and integrating over U produces this logarithmic dependence. And finally, solving for the internal energy in terms of the temperature, we find this relation. Using this internal energy, Planck obtained in the middle of the night his proposal for the so badly desired universal function. His formula reproduced all the known results. At low frequencies, the exponential in the denominator can be expanded as a Taylor series. The ones cancel each other out, and we end up with an energy density proportional to the temperature, just like Reilly-Jeans formula and the new results described by Rubens that afternoon. At high frequencies, the exponential in the denominator is much larger than 1, and the approximation reduces to Wien's law. Also, the identification of Wien's law in this limit requires relating the constant V in the exponential with Boltzmann constant K and Planck's constant H found earlier. Plugging this back in the last relation, we find the final form of Planck's law. This formula is sometimes written in terms of wavelength instead of frequency. Planck's law also reproduced the two important laws described earlier. Integrating over all frequencies, applying some usual tricks of change of variables, and rearranging terms, what we get is a constant times the temperature to the fourth power. This is a Stefan-Boltzmann law, and now Planck gives us a way to write the Stefan-Boltzmann constant in terms of speed of light, Boltzmann constant, and Planck's constant. Similarly, if we search for the maximum of the distribution by setting the derivative with respect to the frequency to zero, after some manipulation we get a transcendental equation. In the recent video about the Newton diffusion equation, I show you how to visualize this type of equation. What you do is you plot the left-hand side and the right-hand side, and you look for the point where they cross. But here let me show you how to find the exact solution on a few lines of code. After importing a simple solver and defining the equation, we find that the solution is 2.82, and a simple graph confirms the result. And with this, we find that the frequency at which the maximum occurs is a constant times the temperature. This is exactly Wien's displacement law described earlier. Early the next morning, Planck gave his formula to Rubens. Excited with the new proposal, Rubens and Kirlbaum spent the day reanalyzing their data against the new formula to include it in their article. In the words of Planck, the very next morning I received a visit from my colleague Rubens. He came to tell me that after the conclusion of the meeting, he had that very night checked my formula against the results of his measurements and found a satisfactory accordance at every point. On October 19th, at the meeting of the Berlin Physical Society, Kirlbaum presented their new results and Planck unveiled his formula. In their paper, Rubens and Kirlbaum evaluated several proposals for the universal function. They concluded that Planck's formula was the simplest that correctly agrees with their data. Planck's formula describes our experiments for all temperatures. Planck's formula was celebrated, but the obvious follow-up question was, what is its physical meaning? It was great having a formula that worked, but that was just half of the challenge proposed by Kirchhoff 40 years earlier. Planck did not try to provide an explanation, he just wanted to find the correct formula. In fact, he called his results a consequence of lack of intuition. One could rightfully ask, why does the r-quantity stop at the quadratic term, why not a cubic term or any other higher power? In his own words, so long as it had merely the standing of a law disclosed by lack of intuition, it could not be expected to possess more than a formal significance. For this reason, on the very day when I formulated this law, I began to devote myself to the task of investing it with a true physical meaning. Planck tried many methods to derive his new formula of the entropy, but thermodynamics and electromagnetic theory were not enough. As usual, he stayed away from Boltzmann's statistical mechanics and probabilistic calculations, but in the end, this quest led me to study the interrelation of entropy and probability, in other words, to pursue the This is how Planck described this period. After a few weeks of the most strenuous work of my life, the darkness lifted and an expected vista began to appear. He went deeper into the methods of statistical mechanics and learned about the math trick that Boltzmann used to count the number of states in a gas of molecules. In his groundbreaking paper he supposed that the energy is not continuous but discrete, so the total energy is an integer multiple of some value epsilon. This epsilon is a fixed energy that later can be taken to zero to recover the continuous limit. Planck borrowed this trick from Boltzmann and wrote the energy of his oscillators in terms of a discrete amount epsilon and found that he could calculate the This is the expression that he found. The following was just the known steps. Write the inverse temperature as a derivative of the entropy with respect to the internal energy and then solve for U. I leave the calculation of the derivative as an exercise here, most terms will cancel each other and will lead to this expression. Now solving for U, we find an expression that looks identical to Planck's internal energy if we make the And here comes the most important point. This is the part where Planck should have taken the limit epsilon to zero, but he didn't. He had finally found a way to physically justify his formula without taking the limit of epsilon going to zero. The price he had to pay was adopting a discretization of the energy for his oscillators. He had to choose between giving up classical physics or keep looking for ways to modify the way the entropy is treated, but Planck believed that the laws of thermodynamics should not be altered. In his autobiography he wrote, What I did can be described as simply an act of desperation. By then I had been wrestling unsuccessfully for six years with the problem and I knew that the problem was of fundamental importance to physics. A theoretical interpretation therefore had to be found at any cost, no matter how high. It was clear to me that classical physics could offer no solution. This approach was open to me by maintaining the two laws of thermodynamics. The two laws, it seems to me, must be upheld under all circumstances. For the rest, I was ready to sacrifice every one of my previous convictions about physical laws. On December 14, 1900, almost two months after he unveiled his formula for the blackbody spectrum, Planck presented his final result at the next meeting of the Berlin Physical Society. He did not realize the consequences of his energy discretization. For him, this was just a mathematical formalism to obtain his phenomenological formula, but some of the great physicists of the time were worried. There is a famous picture of the Solvay Conference in 1927, showing one of the greatest collections of scientific intellect. Many well-known scientists can be recognized, Einstein, Lorentz, Madame Curie, Niels Bohr, and Erwin Schrödinger. Several names there have appeared in other videos, who will appear soon, like Werner Heisenberg and Paul Ehrenfest, the student of Boltzmann, who also coined the term ultraviolet catastrophe to describe the mess of the blackbody radiation that Planck solved. I hope to share more stories in future videos in which many more of the scientists in this picture play a fundamental role. But this famous picture is from the 5th Solvay Conference. The first one was organized by Lorentz in 1911, and got together a collection of moustached dudes with the topic The Theory of Radiation and Quanta. Madame Curie was there, as well as many other important names of the time, like Ernest Rutherford, Wilhelm Wien, James Jeans, our hero Heinrich Rubens, Arnold Sommerfeld, the great teacher of Heisenberg, Pauli, Hans Bethe, and Rudolf Peierls, Henri Poincaré, and Albert Einstein. Max Planck can be seen at the back, next to his now famous radiation formula. Planck didn't know it at the time he presented his work in 1800, but the real prize for maintaining the laws of thermodynamics was the birth of a new physical theory, quantum mechanics. After this point, the discretization of energy was just a math trick, but he set the stage for others to take the next conceptual leap.