Modern AI is built on ML. Although builders can go far without understanding its details, they inevitably hit a technical wall. In this guide, I cover the ML essentials that engineers need to know.
References [1] https://youtu.be/alfdI7S6wCY [2] The Royal Society. Machine Learning: The Power and Promise of Computers That Learn by Example. The Royal Society, 2017. https://royalsociety.org/~/media/policy/projects/machine-learning/publications/machine-learning-report.pdf [3] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. “ImageNet Classification with Deep Convolutional Neural Networks.” Advances in Neural Information Processing Systems, vol. 25, 2012, pp. 1097–1105. [4] Silver, D., Huang, A., Maddison, C. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016). https://doi.org/10.1038/nature16961 [5] Williams, R.J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8, 229–256 (1992). https://doi.org/10.1007/BF00992696
Introduction - 0:00 Intelligence & Models - 0:40 3 Ways Computers Can Learn - 1:50 Way 1: Machine Learning - 2:47 Inference (Phase 2) - 3:36 Training (Phase 1) - 4:27 More ML Techniques - 9:07 Way 2: Deep Learning - 10:43 Neural Networks - 12:06 Training Neural Nets - 15:29 Way 3: Reinforcement Learning (RL) - 21:56 The Promise of RL - 23:25 How RL Works - 25:16 Data (most important part!) - 30:30 Key Takeaways - 33:32
AI Generated·May contain errors
Hey everyone, I'm Shaw.
In this video, I'll be covering machine learning fundamentals that every AI engineer needs
to know.
While you'll find endless textbooks and articles on this subject, my goal with this video is
to give builders a short and accessible guide to the most critical concepts in machine learning.
Here I'm going to focus on five key points.
I'll first talk about intelligence and world models, then I'll talk about three different
ways computers can learn world models through machine learning, deep learning, and reinforcement
learning.
Finally, I'll talk about the most important ingredient in machine learning, which is data.
Let's talk about intelligence.
Intelligence requires understanding how the world works, but of course the world is a
big and complicated place.
In order to understand it, you need to develop models of the world so that you can compress
the complicated and vast reality that we live in into something you can fit into your head.
And put simply, a model is something that allows you to make predictions.
For example, if you were to look out the window and were to see dark clouds like this, your
mental model of the world and understanding that dark clouds typically mean it's going
to rain later will allow you to make a prediction that it's going to rain later.
We all have mental models of the world that allow us to make predictions and to achieve
desired outcomes.
And there are two main ways that we develop these models.
The first is learning from others like our parents or mentors, and the second is learning
directly from our experiences.
And as it turns out, computers can learn models of the world in very much the same way.
Here, I'll talk about three different ways computers can develop these models of the world.
So here, when I say the word learning, what I mean by that is getting computers to do
things without explicit instructions.
So this is in contrast to traditional software development, where programmers use their own
intelligence to develop step-by-step instructions for performing tasks or making decisions,
and then translating those instructions into computer code.
Here, I'm going to focus on three different ways we can get computers to do things without
explicit instructions.
The first is machine learning, which is actually an umbrella term for everything that I'm going
to talk about in this video.
The second is deep learning, which is a special type of machine learning.
And then finally, I'll talk about reinforcement learning, which is also a special type of
machine learning, but often you'll see it combined with the deep learning in modern
applications.
Starting with way number one, machine learning allows computers to learn tasks directly from
data.
And so machine learning consists of two key phases.
The first phase is called training, and this involves gathering a training data set, so
basically a set of examples, passing this training data to a so-called learning algorithm,
and the result of this learning algorithm is a machine learning model.
Once we have this machine learning model, we can use it to make predictions.
So this brings us to phase two, which is called inference, where we give our machine learning
model new data to make predictions, which we can use in making decisions or solving
problems.
Let's dive into a bit more detail on how each of these two phases works.
So I'll actually start with inference, which is phase two, because it's a little easier
to understand.
Inference involves using a model to make predictions.
For example, suppose we have this simple linear model for predicting tomorrow's high temperature.
The way this works is we take today's temperature high, which is represented by x, we plug it
into this equation to make a prediction about tomorrow's high temperature, and this mapping
between today's temperature and tomorrow's temperature is determined by these two so-called
parameters, m and b.
The quality of our predictions are going to be determined on how well our parameters are
aligned with reality.
But you might be wondering, how do we pick these parameters so that these are going to
be accurate predictions?
That brings us to the training phase, which is where we fit a model's predictions to reality.
The key intuition here is that in training, we want to minimize the discrepancy between
our model's predictions and real-world data.
So the first step in doing this is quantifying the discrepancy between our model's predictions
and reality.
One way we can do this is through this equation shown here.
There's a lot of math on this slide, but I'm going to go through it one step at a time.
The first thing to look at is this first term here, which corresponds to the actual values
of the quantity we're trying to predict.
For example, if we're trying to predict tomorrow's temperature high, y is just going to be a
collection of temperature recordings that we collect from reality.
The second thing to look at is this term on the right, which are our predicted values.
So this is exactly the same equation we saw on the previous slide, but now we've written
it out in terms of multiple examples.
Instead of just one input x, we have n inputs, x1, x2, all the way to xn.
So here are our input values, and then we have our two parameter values, m and b.
m gets multiplied by each value in this matrix, and then we just write out this matrix of
one so we can multiply b by each of these elements to make the math work out here.
But you can just think of this as n different linear equations.
So m times x1 plus b, m times x2 plus b, and so on and so forth.
By taking the difference between these actual values and our model's predictions of the
values, we are quantifying the discrepancy between our model's predictions and reality.
We call this quantity our loss function.
Now that we've quantified this discrepancy, the goal of training is to find the parameter
values, m and b, that corresponds to the smallest possible loss function.
While we could just randomly guess a bunch of parameter values to try to find the one
with the smallest loss function, it turns out that we can solve this problem in a closed
form using math.
The way that looks is we're just going to rewrite some terms.
So we'll define this matrix x, which will be all our input values and all these ones
stacked side by side.
Then we'll put our parameter values in their own little column matrix.
Finally, we'll just define our column of actual values as this vector y.
With these new terms, we can rewrite our loss function like this.
So we have the norm squared of y minus x theta.
Writing this out a different way, it looks like this.
So it's y minus x theta transpose times y minus x theta.
And then we can expand this term out and it looks like this.
So it looks all long and scary, but you don't really have to worry about that.
The key thing to know here is that we can find the values of theta that minimize this
loss function by computing its gradient, setting it equal to zero, and then solving for our
parameter values.
The way that looks is as follows.
The gradient behaves a lot like a derivative, which you probably learned in high school
calculus.
If we're taking the gradient of L with respect to theta, we can see that this term goes away.
This term is linear in theta, so this just becomes this.
And then this term becomes this term here.
This is our gradient of L. Then we can set it equal to zero, which gives us this equation.
And then rearranging it, it'll give us this expression here.
And then finally, we can just solve for theta and those values will be our optimal parameter
values.
So if we assume that this matrix here is invertible, then this becomes our optimal parameter values
for theta.
Okay, so what just happened?
We just went through all this math and we computed this gradient, did all this complicated
stuff.
But here's the punchline.
We can get the optimal parameter values for our linear model simply from training examples.
So what is X?
X is a set of input data that we collect.
Y is a set of target data that we collect.
And we can combine them to find our optimal parameter values.
So put another way, the key point of machine learning is that computers can fit models
to reality using data and math.
While a linear model is one of the simplest models you can work with in machine learning,
this idea of fitting model parameters to real-world data is a common thread you'll see throughout
all machine learning models.
And just to list a few other popular machine learning techniques you might come across,
there's also logistic regression, which is a lot like linear regression, but instead
of predicting a continuous target, it predicts a binary target.
So that's why it lets you do classification instead of regression.
There's also decision tree, which is a fundamentally different way of training models, but it's
also very powerful and relatively easy to interpret.
And then you have extensions of decision trees by combining many trees together into a so-called
forest or boosting decision trees on top of each other, like an XGBoost, as well as more
traditional approaches like support vector machines, which can be used for both regression
and classification.
So all the approaches that we see here is what we might call traditional machine learning.
While you can go very far with them, they do have their limitations.
One such limitation is that often to get these approaches to work well, it requires a significant
amount of time and domain expertise to choose the right input variables for your model.
For example, we used today's temperature to predict tomorrow's temperature.
And so that's probably a pretty good predictor of tomorrow's daily high.
However, if we use the number of cappuccinos I drank as our predictor, it might have some
sort of correlation with tomorrow's temperature high, but it probably wouldn't do a very good
job at this prediction task.
So this process of picking the right input variables to your machine learning model is
called feature engineering, and it is something that can make or break your model's performance.
However, the importance of feature engineering began to wane around 2020-2012 with the rise
of deep learning.
Deep learning is a specific type of machine learning which involves training neural networks
which can learn optimal features for a specific task all on their own.
A cartoon diagram of this, which Chajabuti helped me put together, is let's say you're
trying to create a deep learning model to do image classification.
The way this would work is that you would just take raw image pixels as the inputs to
your network.
The early layers of the neural network would represent pretty basic features like edges
and textures of images.
The layers toward the middle would start to represent things like eyes or ears or whiskers,
higher level features that might be in an image.
And then the later layers would represent higher level concepts like entire objects
or animals or something like that.
These final layers make it a lot easier to do this prediction task because if you have
a quantity that represents the cat-ness of an image, you can basically use a linear model
to make this prediction.
This is the power of deep learning.
By simply curating raw data with corresponding labels, you can train a neural network to
do basically anything.
The key technique in deep learning are neural networks, which are a series of operations
that can approximate practically any function.
The fundamental building block of a neural network is a so-called neuron, which involves
the following steps.
It takes a set of inputs, multiplies them by some corresponding weights, and sums them
together.
It applies some bias term, so just adds a scalar value to the summation.
And then it passes this whole thing through a non-linear function called an activation
function.
And then this will spit out the neuron's output.
These three stages of summing inputs, adding bias, and passing through an activation is
fundamentally what a neuron is.
Writing this out mathematically, xi are the inputs to the neuron, wi and b are the parameters
of the neuron, g is this non-linear function, and z is the output of the neuron.
Looking at this, we see that this looks a lot like the linear model that we saw in the
previous slide.
The key difference, however, is that we now have this non-linear activation function.
This is actually a critical aspect of a neural network because it is these non-linearities
that allow the network to be able to approximate practically any function.
The way we can build up networks from neurons is we can take these inputs and we combine
them into a neuron, so basically compressing everything we saw in the previous slide to
this single circle.
But we can combine multiple neurons together to form so-called layers, and then we can
combine multiple layers together to form entire neural networks.
This is how we take that fundamental building block of a neuron, build them into layers,
and then build layers into networks.
But of course, neural networks in practice are much more sophisticated than the example
I showed on the previous slide.
While that vanilla neuron we saw on the previous slide is widely used, there's also another
one called a long short-term memory neuron, or LSTM, which is also pretty common when
working with sequenced data.
Also there are a wide range of activation functions that are commonly used.
There are many more than just these, but there's the ReLU, which is shown here.
There's a sigmoid, which is kind of like an S-shaped thing.
There's a tanh, which is also an S-shaped thing, but it has a different output range.
And then there's a softmax, which basically translates any inputs to a probability distribution.
There are also several different types of layers that you'll commonly see in the wild.
So before we just saw these fully connected layers, where every neuron from a previous
layer is connected to every neuron in a subsequent layer.
But there are also recurrent layers, convolutional layers, attention layers, which are critical
in transformers, which is probably the most popular architecture today.
There are also pooling layers, normalization layers, and dropout layers.
Finally, there are a zoo of different network architectures.
Before we just saw a simple feed-forward neural network, which is the most basic architecture.
There's also a recurrent neural network, which handles sequential data.
There's a convolutional neural network, which is used a lot in image processing.
And then of course, there's the transformer, which is revolutionized natural language processing
and the architecture behind large language models.
So that's the anatomy of neural networks.
But how do we actually train these objects on our data?
At the most basic level, just like how we fit our linear model to real world data, we
can also fit our neural network to real world data.
However, the trouble with neural networks is that the loss function is going to be significantly
more complicated than the one we saw for the linear model.
Just to show you a cartoon diagram of this, the loss function of a neural network will
have this jagged and messy terrain of different peaks and valleys, which we have to figure
out a way to navigate so that we can find the smallest value.
Again, the fundamental goal of training is to minimize the discrepancy between our model's
predictions and reality, which is equivalent to picking parameter values for our model,
which minimizes the loss function.
So in terms of this picture here, we want to pick the parameter values that correspond
with this point in the loss landscape.
But unfortunately, since this loss landscape is so bumpy and perilous, we can't just simply
write out a single equation that spits out the optimal values like we saw with the linear
model.
So in practice, what we need is some kind of algorithm to search this space of possible
parameters in such a way that we get closer and closer to this minimal point that we're
trying to reach.
So it turns out we can again turn to our handy gradient to help us do this.
The gradient is helpful because at any spot in this loss landscape, if we compute the
gradient, it will point in the direction of greatest ascent.
Basically, it'll point in the direction where the slope of the loss landscape is steepest.
So if we take the minus of the gradient, basically pointing in the opposite direction, this is
going to point us in the direction of greatest descent.
What that means is we can use the gradient of our loss function with respect to our parameters
as a guide to point us toward the minimal value of this loss landscape.
So what that looks like is if we're at this spot here, we'll compute the gradient and
then move in the direction of this black arrow and take a step.
And then we'll be in a new spot and then we'll do the same thing.
We'll compute the gradient and then take a step in the opposite direction.
Again, compute the gradient, take a step in the opposite direction, and then just keep
doing that until we get closer and closer to this optimal value.
This algorithm of computing the gradient, using it to decide what direction to move
in, taking a step, and then repeating the process is called gradient descent.
Put another way, it's an algorithm for updating model parameters to minimize the loss.
The equation for this is shown here, where theta i are the old parameters of our model.
This will be a vector.
So you can imagine we have hundreds, thousands, millions, billions of parameter values in
our neural network.
Theta i1 are the new parameters.
So we're going to update the old parameters in some way to generate the new ones.
And the way we do that is using this term on the right here.
There are two key components of how we update our model parameters.
The first is the gradient, which we just talked about in the previous slide.
So the gradient is telling us which direction to go in.
And then we have this other term here, gamma, also called the learning rate, which controls
how big our steps are.
A very large learning rate means we'll take really big leaps when following the gradient,
while a really small learning rate corresponds to very small steps in this descent algorithm.
The learning rate is something that you need to tweak and refine, because if your learning
rate is too big, you might overshoot the optimal value, while if the learning rate is too small,
you may never get there.
But in practice, it's rare people use gradient descent because you can have convergence issues.
The data you use in computing your loss function is the full dataset and this causes some problems.
At the other extreme, there's also stochastic gradient descent, where basically the updates
are based on picking a random data point in your training dataset and doing the parameter
updates based on that.
What's much more common is to use so-called mini-batches.
Instead of using the full dataset or just a single data point, you'll take a random
sample of, say, 8 or 16, 32, whatever examples and compute your loss based on those.
But by far the most common optimizer or update strategy for neural networks today is ADaM,
which is short for Adaptive Momentum Estimation.
This takes into account not just the gradient itself, but this momentum term to make the
training process more stable and efficient.
A couple of things came up when we were talking about optimizers.
One is this idea of a learning rate, which is a number that we have to choose as model
developers, but also this idea of the mini-batch, in which we choose the number of samples to
include in our loss calculation.
The learning rate and the mini-batch size can have a significant impact on the success
of your model training.
This brings up the idea of hyperparameters, which are values that guide the training process.
These are not like the model parameters, which we learn from data, but rather these are values
that we set manually or that we tune through some other optimization strategy to develop
the best final model.
Some of the most common hyperparameters you'll see are epochs, so basically the number of
epochs is the number of times that your optimizer goes through the entirety of your training
dataset.
The learning rate we already talked about, it controls the step size in your optimization
process.
The batch size we also talked about, which is the number of samples you use to compute
the loss.
Then finally, there's this dropout term, so we can introduce dropout layers into our neural
network and then have some number, maybe between 0 and 0.2, which is representing the fraction
of neurons to randomly set to 0 during the training process.
Dropout is a very powerful approach to help avoid overfitting and making your models more
robust.
So far in talking about machine learning and deep learning, we've focused on models that
learn by example.
In other words, we curate a training dataset of inputs and outputs where we train a model
to use the inputs to predict the output.
This is a lot like how you and I might learn from a teacher or a mentor who shows us how
to perform a task through explicit examples.
However, this isn't the only way that we can learn.
In fact, a lot of times we don't learn from other people through examples, but rather
we learn from our own personal experience of basically trying things, making mistakes,
and getting negative feedback from our environment, or trying things and getting positive feedback
from our environment.
It turns out computers can learn in very much the same way, and this process is called reinforcement
learning.
This is where computers can learn through trial and error.
So far, we've talked about supervised learning where you have a human who curates these examples.
So here's our training dataset with a set of inputs and outputs, and then we use these
inputs and outputs to train a machine learning model.
However, with reinforcement learning, the strategy is a bit different.
We have a model that can interact directly with reality or some artificial environment
that we construct, and then the model gets rewarded for good actions, which indirectly
influences the model parameters.
And it just continues in this feedback loop until we have a model that does what we want
it to do.
While reinforcement learning is notoriously hard to get working well, it does hold a ton
of promise and potential.
And the central reason for this is that with reinforcement learning, models are not bound
by human labeling or expertise.
In supervised learning, the performance of our model is going to be bound by our ability
to label high quality data or our existing knowledge or expertise.
However, with reinforcement learning, since the model is interacting with the environment
on its own, it can find its own ways of solving problems and potentially surpass even expert
human performance.
A real world example of this is AlphaGo, which is a deep learning model that DeepMind developed
to play the game of Go.
So there's actually a whole movie about this.
But in this Nature paper, they showed this plot here comparing a reinforcement learning-based
deep neural network, a supervised learning-based deep neural network, and the ELO rating of
this Go grandmaster.
And we can see that the grandmaster has just below a 4,000 ELO rating and that the supervised
learning model gets good really quickly because it's learning from experts and grandmasters.
But you see that it never can quite surpass the grandmaster level.
On the other hand, the reinforcement learning algorithm, even though it really sucks at
the beginning, it learns to play the game of Go not from experts, but by playing itself.
And iteratively and slowly getting rewarded for winning games.
This allows the model to be completely decoupled from how humans play the game of Go and develop
its own novel strategies for playing the game and eventually surpassing human grandmaster
performance.
Going one layer deeper into how reinforcement learning works, the basic idea here is to
update model parameters to maximize some reward function.
In the game of Go, the reward they used was winning the game, but this could be anything.
More recently in training DeepSeq R1, DeepSeq used the correctness to math encoding solutions
as the reward to their model.
And so a key difference between reinforcement learning and the supervised learning approaches
we've seen so far is in the quantity we're trying to optimize.
So in linear regression and neural networks, we saw that we had a loss function that we
were trying to minimize, but in reinforcement learning, we instead have a reward function
that we're trying to maximize.
One way we can write out this reward function is like this, and I know this is all scary
complicated math, but let me walk through it one piece at a time.
So J is our objective.
This is the reward that we're trying to maximize.
Log of pi theta is the model's output, and R is the reward at time t.
The way to understand this is that in reinforcement learning, the model isn't producing a prediction.
Rather, it's generating this probability distribution of different actions it might take given the
current state of its environment.
So pi is just representing a probability distribution, and pi theta is just saying that this probability
distribution is parameterized by the parameters theta.
A is representing the action at time t, and then S is representing the state of the environment
at time t.
And the reason we use log here is that it makes the math a bit nicer to work with.
And then this reward at time t is basically the feedback the model is getting for its
specific action.
So these probabilities and rewards are going to be summed over an entire episode or trajectory.
And so if we're playing the game of Go, an entire episode would be, let's say, an entire
match of the game.
Or if this was like a self-driving car, an episode would be from when the car starts
moving to when it reaches its destination or crashes.
And then a bit more details here.
So again, we already said that pi theta is the probability of taking action A given state
S based on model parameters.
Tau is this full rollout of states and actions.
So we have an initial state to which the model has an action to and gets a corresponding
reward.
Then there's another state which the model has an action to and gets another reward,
and so on and so forth.
And then t is just the number of time steps in this full trajectory.
So I know there's a lot of math and terms happening here, but here's the key intuition.
What we want is to update these model parameters such that our model gives a high probability
to actions which result in a high reward.
So in that case, if this probability is big, when this reward is big, then this whole term
will be big.
And then once we average over the whole trajectory, this objective will be as big as possible.
So maximizing the objective is equivalent to having high probabilities for high reward
actions.
But from there, we can update our model parameters in very much the same way we did in deep learning.
Namely, we have this parameter update rule where we have theta k, which are the old parameters,
theta ki, which are the new parameters, and then we have this update rule.
So again, we take the gradient of our objective to serve as our guide in navigating this landscape.
And then we have the learning rate gamma.
However, since we're not trying to minimize the reward and we're trying to maximize it,
all we need to do is change the minus sign from gradient descent to a plus sign to give
us this gradient ascent update rule.
So everything I just described is based on this reinforce algorithm, which is the reinforcement
learning algorithm that underlies all modern reinforcement learning techniques.
However, that came out in 1992, and since then there have been a lot of improvements
and new approaches introduced.
So here's a summary table of some popular ones.
Reinforces what we've just been talking about on the previous slides.
There's this trust region policy optimization approach, which adds this KL divergence term
and compares the current model action to the previous version of the model's action.
And this just improves training stability.
There's PPO, which was used in reinforcement learning with human feedback, which is what
OpenAI used to make ChatGPT.
And they basically do a similar thing as trust region policy optimization, but they don't
have this KL divergence term.
Instead, they have this clipping term to ensure that training is stable.
Finally, there's this group relative policy optimization approach developed by DeepSeq.
And this is what they used in DeepSeq R1.
And it basically combines PPO with this KL divergence penalty to make training more stable.
Additionally, they batch different examples into a group when computing this objective
and updating the model parameters.
For most of this video, I've talked about algorithms for optimizing parameter values
and loss functions and neural networks and sophisticated fancy math.
And this might give you the impression that algorithms are the most important thing when
it comes to machine learning.
However, that is very much not the case because at the end of the day, we are simply fitting
models to our data.
So if you have bad data, it doesn't matter how sophisticated your algorithm is or how
much you tune your hyperparameters.
If you're fitting a model to bad data, you're going to have a bad model.
That's why I wanted to finish this talk to review what makes good data.
And so there are really two properties that make data good.
The first is quantity.
All things equal, more data are better than less data.
When you don't have enough data, your model is prone to overfitting.
So this is basically a situation where your model memorizes your training dataset and
doesn't generalize well to new examples.
In other words, it leads to models that just aren't very helpful in practice.
However, there is a situation where more data is not necessarily a good thing.
And this is when your data is garbage.
Because if you take bad data and train a model, even if you have a lot of it, you're going
to have a bad model.
And this is summarized by the old adage of garbage in, garbage out.
That brings us to the second property of good data, which is quality.
The essence of quality is having data that faithfully represents reality.
So there are two key aspects of this.
The first is accuracy, which is the correctness of your data.
For example, if your dataset says someone is 35 years old, when in reality the person
is actually 53 years old, that is bad data.
Another example is if your data says someone makes $12,000 a year, when in fact they make
$12,000 a month, that's another example of bad, inaccurate data.
The other aspect is diversity, which is basically the representativeness of your data.
For example, if you have some kind of SaaS product and you're trying to predict customer
churn, and you take data from a bunch of pro users of your SaaS product and you use it
to develop a customer churn model, this is completely fine if you're trying to predict
churn of users of your pro plan.
However, if you have an enterprise customer, this model probably won't be helpful in predicting
the churn of an enterprise.
In other words, it's not only important that your data are correct and accurately represent
reality, but that they also contain examples of all the different scenarios you want to
use your final model in.
Just to summarize, good data comes down to quantity and quality.
We want more data over less data, we want high-quality data over low-quality data, and
we want less high-quality data over more low-quality data.
I covered a ton of information in this video, so I just wanted to recap with a few key takeaways.
First is that solving problems requires an accurate model of the world, and machine learning
gives us a way to align models to reality using data and math.
A specific type of machine learning is deep learning, which involves using neural networks
to learn useful features and mappings from raw data.
A lot of times, deep learning is combined with reinforcement learning, which allows
computers to learn by interacting with the world through trial and error.
Finally, although there's a lot of math and fancy algorithms in machine learning, training
good models comes down to having good data.
What this means is you need high-volume, high-quality training datasets.
Although there were many things I couldn't cover in this short overview here, I hope
it gave you more clarity around some buzzwords in machine learning and the confidence to
continue your learning on your own.
Toward that end, if you have any questions or suggestions for future topics you want
me to cover, please let me know in the comments below.
As always, thank you so much for your time, and thanks for watching.