AI & ML in Finance - Lecture - 24 - Regression Trees

Smart Investing
3 followers
2 Video Views·Mar 21, 2025
24th lecture in the online class "Artificial Intelligence & Machine Learning in Finance" held at Leipzig University, Faculty of Economics.
Hello everyone and welcome back to our class in artificial intelligence and machine learning
and finance. In the last videos we've seen classification trees and we now want to use
basically the same model for a different purpose for regression analysis. So we'll
continue our application but we now need a different data set. We again use a data set
from Kaggle but this time it's one that is on insurance premia. You can download the
data here under this link and we want to predict insurance premia which are expenses for customers
and potential policyholders and we have data on close to 1,300 health insurance policies
and we have four numeric features, H, BMI, number of children and expenses and three
nominal features, sex, smoker, region. So obviously we want to forecast the expenses,
the insurance premia and we are going to use H, BMI, number of children, sex, gender, smoker,
yes, no, region, yes, not yes, no, but the different reasons, regions to predict
the insurance premia. We need to import the data so we use the readr library, we download the CSV
file, import this. There seems to be a duplicate entry so actually we need to remove the duplicate
entry. If we now use sum is.na insurance we get zero so there are no missing values which is
important in this context and we have a look at the insurance premia to get a summary. As you can
see the mean is $13,000 probably, the maximum is $63,000 and the minimum is $1,122 so that's
the data. We use the TIBL library, we convert the data into a TIBL and we print the first
eight observations and you can see we have female, male, male, male, male, female, female, female,
the body mass index, number of children, smoker, yes, no and the reason, regions and the different
expenses. Now we create the training and test data which by now is pretty standard. Again you
can see the 80% of the observations go into the training data, 20% in the test data, x train,
y train, x test, y test as a standard and as it has been done before a number of times. Again,
we use the carrot package, we set train control, the option to cross validation,
tenfold, we set seed in line three and as single trees are fitted very fast we do not need parallel
computing here and we actually consider various hyperparameters so the length of our tuning
step is 100. The metric needs to be different now, we cannot use accuracy which is for classification
but this is a regression tree, y train is now a metric variable so the function sees that even
using rpart, regression partitioning, it sees that this is not a classification but a regression task
and the metric then is the relative mean squared error, there cannot be accuracy as before.
And then we print the complexity parameter 0.0061 and this is what happens, you can see now
with increasing complexity parameter the RMSE increases, it stays constant sometime and then
it jumps up. Now to visualize the optimal tree for the optimal complexity parameter,
you can see here that on the first level it's quite interesting to see smoker, yes or no.
Actually what you can see here, these are the expenses, the insurance premia and as you can see
if you're no smoker you're immediately in this area where actually the premia are relatively low.
Then if you're young it's the lowest premium, if you're old and then it's a question if you're
not as old or if you're rather old then this is a distinction made here, if you're a smoker and
if you have a high body mass index and if you're old then five percent of the observations have
very high insurance premium. Now what is the predictive accuracy here?
Again we predict based on the test data sample and you can see the R squared is 87 percent,
the root mean squared error is 4238 and the mean absolute error is 2900. Obviously the R squared
is easiest to interpret and we get that in this model close to 87 percent of the variants can be
explained by this regression tree. We could also use boosted regression trees, however XGBoost
only works with numerical features and as we've seen actually smoker, yes or no, a categorical
feature binary variable should be included because it seems to be the feature that has the highest
predictive power. If we leave this out this would lead to an inferior model. We can do this but you
can see the R squared immediately drops to nine percent. Instead if you rely on bagging via the
method tree bag and if you use all the features this does not in this scenario lead to an improved
performance compared to the single tree model. The R squared drops to 81 percent but it's a good
thing to try this out. Okay so this is a regression tree. Classification regression can be done by
trees. We've seen a few examples and some nice applications and in the next video in the next
section we will start with using and discussing a huge part in artificial intelligence which are
neural networks.
