AI & ML in Finance - Lecture 36 - AI/ML & Ethics

Smart Investing
3 followers
3 Video Views·Apr 4, 2025
36th and final lecture in the online class "Artificial Intelligence & Machine Learning in Finance" held at Leipzig University, Faculty of Economics.
Hello everyone and welcome back to our class in Artificial Intelligence and Machine Learning in Finance.
And in this last video of our lecture, we are going to cover a huge topic and a topic that will become even bigger in the next couple of years and decades.
That is the ethical side of the use of AI and MLs and some ethical considerations.
We'll start with some considerations that are taken by a study by Thomas et al. in this year actually.
And it shows that we have ethical risks at numerous levels when using AI and machine learning.
For example, if we start with the data, the training data must be free of biases and it should include all relevant types of stimuli.
For example, racial, sexist biases should be excluded.
And this is one thing you need to consider when training machine learning models.
AI should only be deployed in environments it has been trained in.
That is, we shouldn't have any cross use for a different purpose.
This might be problematic, especially in the financial sector.
If you think about models that have been trained on, let's say, lending data, on loan data that are used in a different context.
And especially in Germany, privacy laws must be taken into account.
Data privacy needs to be respected when collecting and processing the data.
Turning to the algorithm side, we should think about unethical coding that might be introduced by the programmers and the developers themselves.
And the coding should be free of biases.
That is, it doesn't necessarily mean that an ethical algorithm becomes unethical due to the selection of training data.
But it could be that developers have already set up the model and set up AI such that, for example,
when we consider the fact that most developers are white males, it could be that there are some biases already included in the coding.
And AI must be controlled for emerging biases during the lifecycle.
And last but not least, the business use.
The purpose of using AI should be ethical in the first place.
It shouldn't be, for example, to discriminate against certain parts of the population.
And the unintended impact of AI can also be unethical.
So these are some problems associated with AI and ML.
And we'll come back to some of these later on in more detail.
And we'll start with racism and sexism in AI.
Now, the machines and the models themselves, they obviously have no bias.
But the learning data sets and the training data sets and algorithms are most likely as biased as our current society.
So most of the AI developers are usually white males.
They lack the perspectives of minorities.
And consequently, if everything goes wrong, existing biases will be introduced into the machines.
And they can be amplified if we are not actively working against them.
And there are several organizations, at least in the United States, that are working against biased algorithms.
For example, the ACLU, the American Civil Liberties Union.
And they, for example, fought in a wide variety of issues.
And they exposed Amazon's recognition of them as racially biased.
Also, the Algorithmic Justice League, which was founded in 2016 with a mission to raise public awareness to biases in AI,
is one of these organizations that have highlighted the dangers of using AI algorithms in situations where previously we had human interactions,
where we had human managers, for example, that had to make decisions on, let's say, loan applications.
And this is obviously not immune to biases and racism and sexism just because it's a machine.
But if the algorithm itself or the training data, if they include these biases,
we might still end up with an AI method that produces biased results.
The study by Thomas et al. also gave some practical recommendations.
For example, one should implement a general statement on the firm's and the institution's intention for AI ethics,
ideally an AI ethics framework or charter that should be an extension to the firm's already existing mission or purpose statement.
Usually you have this in place, especially in large companies.
One should implement an internal application-specific design plan,
and you should regularly audit the processes so that if you see ethical risks and some concerns that these programs might produce unethical decisions,
this should be flagged and in the end one can do something about this.
And, of course, one should keep records of decisions concerning ethical tradeoffs for transparency.
Sometimes, of course, it might be that you need to do a tradeoff.
This doesn't mean that you have to act unethically, but in some cases you need, of course, to, for example,
weigh data privacy concerns against business purposes and also not just for your company, but also for the benefit of your customers.
And in this case, these tradeoffs should be recorded just for transparency.
How are artificial intelligence and ML regulated, especially from an ethical perspective?
Well, actually, this is quite in its infancy.
Artificial intelligence and machine learning have not yet even been clearly defined.
We will later on see the definition by German Barfin, but this is something in the making.
For example, the European Commission has stated in its digital finance strategy that it intends to clarify together with the European supervisory authorities by 2024 at the latest.
So in almost three years, whether and how existing financial market regulation should be applied to the use of big data and artificial intelligence.
Now, Barfin has proposed several principles for the general use of algorithms in decision-making processes of financial firms.
This is the link to the document.
You can find it under Barfin and then Big Data und Künstliche Intelligenz.
And these are principles that represent preliminary considerations.
It's not yet regulation, but these are some first ideas by Barfin for minimum supervisory requirements regarding the use of artificial intelligence in supervised financial institutions.
And this is the definition of AI according to Barfin.
They say artificial intelligence is a combination of large amounts of data, big data, computing resources and machine learning.
This is rather a different definition than in our very first lecture, but this is the one that Barfin chose.
So in machine learning, they say computers are given the ability to learn from data and experience on the basis of special algorithms.
And compared to rule-based methods, learning takes place without programmers specifying which results are to be derived from certain data constellations and how.
So big data plus machine learning and high computational power, that's artificial intelligence in the definition of German Barfin.
And again, they mentioned this not in regulations, but actually this is basically a newsletter, a publication in which they summarize their first ideas on what AI is, what it entails and what they might be doing in the future on its regulation.
They also say algorithms are rules of action that are usually integrated into a computer program and solve an optimization problem or a class of problems.
And in addition to the distinction according to the type of algorithms, how is the problem technically solved, applications of machine learning can also be differentiated according to result types.
The basic distinction between classification, regression, clustering, we've seen that, and data types.
So that's what Barfin says.
They also have some overriding principles, and they mentioned this in their publication.
Again, this is also in a way typical for German supervision and regulation.
It usually states the responsibility of top management, that is management, just like risk for risk management.
Management is responsible for company-wide strategies and guidelines or policies for the use of algorithm-based decision-making processes.
We can find the same kind of recommendation regulation, actually, when it comes to risk management, that management is responsible for overall enterprise risk management.
Now, potentials of such processes as well as the limits and risks should be taken into account, clearly stated,
and a company-wide strategy for the use of algorithm-based decision processes should also be reflected in the IT strategy.
So you can see that at this point, they have these overriding principles that are very similar to the qualitative regulation we've seen in Basel 2 and Basel 3.
Adequate risk and outsourcing management.
Now, financial institutions should establish risk management systems adapted to the use of algorithm-based decision-making processes.
If applications are sourced from a service provider, management must also have an effective outsourcing management, clearly.
If you're using AI and ML and you're outsourcing this, then obviously you should also keep track of your service provider.
Responsibility, reporting, and controls structures must be clearly defined.
And when establishing adequate risk management, one needs to consider the risks of an algorithmic decision-making processes.
Risk mitigating measures and processes should start exactly where risk originates according to the polluter pays principle.
Also, one should prevent a bias, avoidance of a bias that is the systematic distortion of results in algorithm-based decision-making processes.
And they must be avoided in order to be able to make business decisions based on results that are not systematically distorted
and exclude the possibility of a systematic bias-based discrimination against certain customer groups.
And also, and this is important to stress, in the end, and probably this is one of the reasons why financial supervisors and regulators will be concerned about AI and ML,
because in the end, if you have ethical concerns within AI and ML in its use in a financial institution, this might cause a reputational risk.
This might cause a damage to your company's reputation.
And at that point, this will become a concern for the regulator.
And in accordance with the polluter pays principle, the risk of bias must be identified where it can arise.
It must be analyzed and either eliminated or at least mitigated.
Now, some things need to be regulated, but actually some other things are also prohibited by law.
So for some financial services, it is actually stipulated by law that certain characteristics may not be used for differentiation.
That is for calculating risk, calculating prices, premium.
And the danger of discrimination exists if these characteristics are replaced by an approximation.
That is, for example, if instead of using, let's say, gender or ethnicity, you suddenly use hometown, age and income groups,
then in the end, it might lead to the fact that algorithms will simply substitute one feature by three other features which are correlated.
Now, this, again, will be associated with increased reputational risks and also legal risks.
So it's in the best interest of a financial institution to actually prevent this from happening.
And companies should establish statistical verification processes that exclude discrimination and such substitution of features within AI and ML processes.
Now, this was true actually for all financial institutions.
We can also find some more information by IOPA, the European Union's Insurance and Pension Fund Supervisory Authority.
And they have a group on digital ethics and they also have formulated AI governance principles.
So they are subdivided into human oversight, robustness and performance, data governance, record keeping, transparency and explainability, fairness and non-discrimination.
And last but not least, principle of proportionality.
Let's talk about human oversight first.
Now, again, this is for European insurance companies, but obviously most of the things are also applicable to financial institutions in general.
Insurance firms should establish adequate levels of human oversight, taking into account the impact of specific AI use cases and other governance and control measures in place.
They should select the level of human oversight and the selection should be proportionate to the nature, scale and complexity of the risk inherent to the specific AI use case in that insurance company.
Now, different roles and different responsibilities for the staff involved in AI processes should be clearly defined in policy documents.
That's human oversight.
Robustness and performance.
Now, the firm should assess and monitor the performance of the AI systems on an ongoing basis and take due consideration of their limitations and potential shortcomings.
Now, performance metrics should be adapted to the objective, pursuit and the nature of the data used.
You should check whether you are actually achieving the goals you've set yourself.
Sound data management obviously is key to ensure the performance of AI systems and they should produce stable outcomes over time.
Otherwise, it doesn't make too much sense from a business perspective, but obviously also for the regulator.
And insurance firms should develop resilient IT systems and infrastructure that cannot be tampered with, for example.
Data governance and record keeping.
Now, insurance firms should adapt the data governance and record keeping measures to the impact of specific AI use cases.
And data used in AI models should be accurate, complete and appropriate.
Again, sound data governance should be applied throughout the AI model lifecycle.
The data used in AI models should be handled and stored in a secure manner.
Obviously, especially in insurance, this is highly sensitive and confidential customer data.
And appropriate records of the data and the modeling methods should be kept to ensure reproduction and traceability.
Transparency and explainability.
The firm should adapt the types of explanations to specific AI use cases and to the recipient stakeholders.
Now, firms should adapt their explanations to the different types of stakeholders.
And they should strive to use explainable AI models in particular in high impact AI use cases.
Now, data used need to be transparently communicated.
And as a result, again, we need data security, data governance and sensible data management.
Fairness and non-discrimination.
Sound and transparent governance processes are key to ensure fairness and non-discrimination,
especially when it comes to the calculation of insurance premium.
Otherwise, this could lead to reputational risks.
Insurance firms should conduct their business in a fair manner when using AI
and make reasonable efforts to take into account the outcomes of AI systems.
Consumers not willing to share very personal and sensitive data are not strictly necessary for risk assessments,
should still have access to affordable insurance coverage.
And insurance firms should respect the principle of human autonomy
by developing AI systems that support consumers in their decision-making process
and avoid unfair nudging practices introduced by the use of AI methods.
And last but not least, the principle of proportionality.
Insurance undertaking should establish the necessary governance measures
that are proportionate to the nature, scale and complexity of their operations.
And AI use cases and use case impact assessment and the governance measures
should be proportionate to the potential impact of a specific AI use case on consumers and or insurance firms.
Insurance firms should then assess the combination of measures put in place
in order to ensure an ethical and trustworthy use of AI within, for example, premium calculation.
These are the principles set forth by IOPA's GDE group on digital ethics and ethics and AI in insurance.
You can see many of these things can also be applied to other financial institutions.
And this will be a huge topic in the years to come with more applications,
with more models and even a higher utility stemming from the use of AI and ML in finance.
We will see more ethical problems, more use cases and obviously more regulation
and principles set forth by regulators and supervisors like IOPA.
And thus, we are at the end of the lecture.
If you are interested in more of the literature and the tools and the references we've used,
you will find all of these on this last slide.
I hope you've enjoyed the lecture.
You've learned a little bit about the usage of AI, ML in finance and good luck with the exam.
Thank you.
