Is data management the secret to generative AI?
1 Video View·Dec 12, 2024
Every conversation starts with generative AI, but ends with data. Why? Because there is no AI without data. Generative AI is changing the way we think about data. To understand how enterprises use generative AI to contribute to a competitive advantage, we must think about the relationship between gen AI and data. In this episode of AI Academy, explore why high-quality data is essential for the successful use of generative AI.
Patterns and relationships in vast amounts of data
unlock entirely new possibilities.
Sometimes we learn about our past,
or we discover something that helps us predict the future.
For some time, we've been collecting data
without even knowing what might come of it,
and then the volume just becomes overwhelming.
And that's when the relationship between data and AI
gets really interesting.
Hello, and welcome to AI Academy.
My name is Love Agarwal,
and I'm the worldwide sales leader for IBM Watson X.
And my partner today is Edward Kalvisberg,
vice president of product management
for the IBM Watson X platform.
It's great to reconnect with you here today
in this amazing IBM research facility.
I know. This place is incredible.
This is an active working lab,
and we're in front of a prototype system designed for AI.
You and I both talk to a lot of clients,
and it seems like every conversation these days
is about generative AI or genAI.
Well, every conversation starts with AI,
but it usually ends with data,
because the truth is there is no AI without data.
And data is the only sustainable source
of competitive advantage in business.
Generative AI is changing how we think about data
in a couple of ways.
First, genAI models can make much more effective use
of unstructured data,
which is the majority of all new data.
GenAI can dive into large volumes of language data,
primarily documents or software code,
which is also a language,
and spot patterns or make connections
without much preparation or supervision.
It can see things that we likely wouldn't see.
Yes. Second, we can apply genAI
to the data management problem itself,
and it can help us organize, refine, and enrich data
so that it's higher quality and easier to consume.
Right. So, for example,
if a client has different legacy applications
that have been encoding data in different ways,
like different formats or dates
or people's names and initials,
or different columns and column headings
and that sort of thing,
genAI can help make sense of that.
And the data is often scattered everywhere.
GenAI can look at those siloed systems
and begin to understand what the data is
and how it could be related
so you can take advantage of all of that data holistically.
And much more efficiently.
Big savings in time and energy.
How effectively and efficiently you manage your data
has real cost and business performance implications.
It becomes intellectual property
and a point of competitive advantage.
So let's talk more about genAI and data
as a source of competitive advantage.
High quality data is essential
to helping enterprises use genAI to improve their business.
Almost every one of our customers
is at least experimenting with genAI today,
which means that everyone has access
to essentially the same technology.
So it's the customization or tuning
of the large language models with enterprise data
and the infusion of genAI
into new and existing enterprise applications
that drives productivity gains,
improve business performance, and competitive advantage.
So every company has data,
but there's really a spectrum
of how different organizations take advantage of that data.
Some might be stuck with an architectural problem,
like data being inaccessible or locked in silos
across on-prem and cloud environments.
The data silos problem is pervasive.
And you can't solve it by creating a new data silo in the cloud.
So we have different approaches to solving it,
such as building a virtual data layer
and querying data across multiple sources,
or consolidating data onto one platform,
like a data lake house,
which is open and cost-effective.
And for other companies, what's holding them back
from unlocking the full value of their data
is something more subtle.
It's almost a psychological barrier.
They have to shift how they think about data
and turn the business into data
so that they can then turn their data into a business.
Data monetization is nirvana.
When you can literally sell a version of your data,
effectively a byproduct of your core business
as a product itself.
To create those products,
you need to ensure data quality, security, and governance
so that it doesn't become a business risk
or regulatory exposure.
So let's talk about those.
First, you need just good traditional data quality practices.
Do the things you already know you should be doing.
You need to catalog or organize your existing data
into a business glossary.
Now, AI can assist with that work,
but the more you have ready,
the faster you can get value from AI.
And the financial incentives to do so are just going up and up.
Second is having thoughtful data access policies.
Like with any user, you really need to define
what information you want AI to be able to access.
Or said differently, which information you want to remove
or redact in some way,
like social security numbers
or other personal identifiable information.
And the third aspect regarding governance
is monitoring and enforcement.
So you don't just set policies and call it a day, right?
Ideally, you set policies centrally and enforce them locally
while actively monitoring model inputs and outputs
to ensure that the policies are effective,
including how the output of the model is changing or drifting
as it's exposed to real-world interactions.
I don't think we can emphasize that enough.
Data and AI governance is absolutely critical.
Risk and uncertainty about data and model outputs
are two of the biggest barriers to the adoption of AI today.
Everybody wants AI,
but companies can't risk having their own data
or their clients' data exposed.
I think it's clear that quality, trusted data
is essential to successfully implementing Gen AI in business.
But a lot of our clients are still struggling
with moving beyond a handful of single prototypes
to the next phase of customizing the models with their own data
and deploying them into production across the enterprise.
Okay, so let's talk about how a company can customize Gen AI
with their data and integrate it
with their enterprise applications and workflows.
There are two main ways to customize Gen AI
with your own data to make it work for you.
The first is by tuning the model with your data.
And the second is through retrieval augmented generation
or as it's commonly referred to by its acronym RAG.
Tuning a model involves instructing it
or partially retraining it with good examples
from your enterprise data
of how it should respond to certain prompts.
The model quickly learns from these examples
and adapts to incorporate the language and structure
of your business so that it can become
an integral part of your enterprise systems.
On the other hand, RAG doesn't customize the model,
but rather leverages a knowledge base,
again, of your quality enterprise data
to improve the accuracy and even limit the responses
of the model to known facts,
thereby mitigating the new risk of hallucinations.
What about the data that's used to train the models themselves?
Well, that's a great question,
considering we're here at IBM Research
where we train our foundation models.
As an example, our training system
is built on a data lakehouse architecture,
and it breaks down the walls
between data, AI, and governance.
First, we source, catalog, filter, and transform the data
that is going to be used in the models.
Next, we use it to train, test,
and tune our large language models.
And finally, we govern that end-to-end lifecycle
so we have the complete lineage of our data sets,
data pipelines, and AI pipelines,
and we're able to stand behind our models.
And that's the sort of comprehensive approach
that enterprises are going to want to build
as a core capability.
So another way to think about a data lakehouse
is like a commercial kitchen.
And I've used this analogy before,
but if you think about a business like a restaurant,
the ingredients for everything you make is the data.
Now, it'd be silly to send one truck
to get carrots from a farm in Connecticut,
another truck to get peas from a grower in California,
and a third truck to get beets from Minnesota,
and then wait around while they bring it all back to the kitchen.
What are you making, man?
You know, I don't know, I'm not a chef,
but it's proprietary.
The point is that this is how many businesses
approach their data architecture today,
when what you really need is a well-stocked pantry
where everything is already on hand
and neatly organized in one place
and labeled to be quickly accessible.
So ultimately, I believe the open data lakehouse architecture
is the best way to achieve that well-stocked pantry.
It combines the flexibility, scalability,
and cost advantages of data lakes
with the performance and functionality of data warehouses, right?
So it's the best of both worlds.
And as an industry,
we're really moving past the old monolithic architectures
to these more flexible, open, and interoperable architectures
that let you choose the right tool
for the right job at the right price.
So, for example, we can use one query engine
for data ingest and transformation,
another for interactive queries,
and yet another for embedding documents as vectors for RAC.
That sounds ideal.
And this flexibility also enables optimal price performance,
which is an essential enterprise consideration
when deploying this technology at scale.
I'm actually a huge fan of all the experimentation and innovation
we are seeing with GenAI in the market,
in open source and across the ecosystem.
But in addition, the choice of tooling and cost considerations
there's another important lesson to be learned
from our experience over the last decade or so
scaling the adoption of AI.
A lot of good machine learning models
never got deployed into production
because they were built by data scientists
working on the sidelines of core enterprise IT
and could not be properly assessed for risks,
including regulatory compliance.
Data and AI governance is very hard to do after the fact
because so much of it involves managing
and tracking the end-to-end lifecycle.
So in order for all of that experimentation
to someday make its way efficiently into production,
companies need to implement data and AI governance frameworks
or platforms from the very beginning
and not as an afterthought.
Right, and that all makes sense, Edward,
but I see you sneaked in your beloved open source.
So what role do you think open source plays
in the evolution of this technology and the market?
I think it plays a huge role.
Simply put, open source technology
and community-based innovation
delivers superior transparency, security,
and project oversight,
which I think is critical at this early stage of maturity.
The transparency and security points are straightforward.
With open source technology,
you have more users and developers
finding and fixing bugs and vulnerabilities.
But the point on project oversight is more subtle,
and it's about having a broad and diverse community
of both vendors and users of the technology
driving its long-term roadmap,
which ensures that it's aligned with the interests
of the broader community
and not just those of a particular vendor.
The productivity gains of Gen AI are so massive
that how and how rapidly enterprises implement it
using their own data
can make the difference between the winners and losers
in almost every industry.
Being well-prepared is great,
but the best way to start building a new capability
is to just do it,
with the proper guardrails, of course.
Absolutely.
Well, thank you, Edward.
And for everyone else, thank you for watching.
Keep an eye on this space for more episodes of AI Academy
with expert perspectives and real talk
about some of the most important issues in AI for Business.

