Pyramid Analytics at Big Data London
3 Video Views·Oct 5, 2024
Architecture & Engineering
It's London, it's data, and it's big. This is Big Data London, and Scott Taylor of Data
Whisperer is sitting down with conference chair Mike Ferguson, as well as the VP of
sales of Pyramid Analytics, Peter Walker. Let's hear what they have to say.
So Mike, I know you've talked about how technology can be a spickless fashion.
So what are some of the best dressed data leaders wearing this season?
They definitely got their Gen AI suits on, no doubt about that.
There's lots of people wondering around, calling themselves AI agents at the moment.
There seems to be modern data architectures, a pretty hot space that they're looking at.
And I think as well, the whole democratization of all of this towards domain-oriented citizen
data engineers, citizen data scientists, and the adoption of generative AI in there to help
widen the use of all of this kind of tooling.
There's probably an Emperor's New Clothes analogy in there somewhere.
Don't ask me what I'm wearing next year, though.
How do you see the idea of converged analytics really taking shape in the space today?
Well, I think historically, we've had a problem. In the past, what we've done is had
siloed analytical systems. So we've had maybe streaming analytics over here,
then we've had data science with a data lake over there, a data warehouse.
Then we've had graph databases and graph analytics. I think that's coming down.
We're tearing down the silos and really beginning to get towards the ability to
create shared data products across multiple analytical workloads.
So I think there's a number of things happening there, things like open table formats.
So platforms like Pyramid can sit on top of that, take advantage of data products in that
environment, and then bring together these different analytical workloads.
Whereas in the past, we couldn't mix and match them, I think converged analytics is
going to allow us to mix and match. So the ability to use a co-pilot with
machine learning and also graph analytics, or do things we've not been able to do before,
graph analytics and machine learning. Because increasingly now, these things are
becoming part of SQL standard and stuff like that, of the queries that we're generating.
And so that really opens up a whole opportunity of brand new insights.
And of course, the ability to embed all of that into applications and processes around
the enterprise as well. So yeah, I mean, it's an exciting time. And I think we're
really at the beginning of that whole space. Finally be able to cut across all those
vertical silos with some horizontal value. That's what's really going to deliver maximum value.
And anyway, also the output of data science is the input to the business analysts. They want
to leverage all of that stuff that data scientists create. And not much point creating all this stuff
if we can't maximize the value out of it. And I think converged analytics is going to make
that happen. So we can kind of get what Peter and I've talked about a lot, which is the whole idea
of mass contribution to strategic outcomes. Peter, if you can elaborate a little bit on that,
how Pyramid Analytics brings that value across the enterprise.
What we're seeing is, yeah, in terms of a convergence of analytics is those different
flavors of challenge that exist within enterprises today need to be addressed not by multiple tools,
but by the potential of what a broad platform offer can deliver, combining a multitude of
access to operational sources, modeling appropriately so that it's analytics ready,
and then delivering the opportunity for citizen data scientists, business analysts,
and consumers across enterprises, whether they be inside the firewall or outside the firewall,
out to the public domain, customers, partners, suppliers, the general public, to be able to
receive insights that's data-driven, that means that they can optimize their actions and their
activities on a day-to-day basis without really needing to worry about the actual data side.
That's being managed automatically. We're ensuring that all of the stakeholders,
wherever they may be located, with whatever experience they'd like to receive insight,
are able to do their day-to-day job and leverage data to maximum effect.
The idea of being able to bring the right insight to the right folks at the right time,
regardless of where they are in the organization.
Yeah, exactly. Everyone needs a different experience because they've got different
personas, different roles, and as a result, they've got different access to the data that
means that they can do their job efficiently and effectively. That can be determined at
a strategic level by the enterprise, but at the same time, enabling self-service for those
consumers to be able to interpret the data in the way that suits them in order that they can
get their job done. Mike, I love always in your keynotes, you always bring it back to the idea
and the need for data governance and data management. How does that play in this and
how do we make sure that folks who are using Pyramid Analytics are able to pull from that
source of truth? It's vital, really, because whether you're building machine learning models
with Pyramid on the data science side or whether you're a business analyst or just a consumer,
we can't do this on inconsistent data. The most important thing here is it really does
have to be rock solid if you're going to trust it and have the confidence to use it to make
really critical decisions, whether those decisions are even automated at the operational level,
whether they're tactical or whether they're strategic. If you're going to do that by
standing on quicksand of inconsistent data, we're in trouble. To me, it's absolutely critical that
we get that foundation. I often say, why do you think they call it a foundation? Peter,
adoption is key. If you can share some anecdotes possibly about how Pyramid Analytics has helped
drive adoption at a typical enterprise. The adoption challenge has always been there. I've
been in this space for 25 plus years and it's always been that significant challenge. Most
enterprises have typically got anywhere between 15% to 30% of their employee base covered from
an analytics adoption perspective. In terms of latest generation from Pyramid, we've delivered
the capability through our GenBI offer to enable different experiences at different levels within
an organization to really drive significantly greater adoption, whether that be at the
business analyst, analytics developer angle by we can leverage generative BI to enable
developers and business analysts to create dashboards in a phenomenally swift manner versus
previous experiences. Then furthermore, the end user consumer being able to leverage generative
BI to have conversations with the data. They don't have to understand the backend data and how it's
structured. They just need to know in conversation what they'd like to get out of the data in order
they can do their job. Having an iterative conversation, leveraging generative BI is
something that we offer. We're agnostic in terms of the LLMs that we're able to leverage, which I
think is really important, especially as we see the emergence of LLMs more and more and we'll
continue to do so. Being agnostic in that respect is key. Then furthermore,
ensuring that the self-service that we're delivering out to the different stakeholders
is governed. All handled very capably within the Pyramid platform.
GenAI, GenBI, what's next? GenCI, maybe? No, no. Well, stop at BI. Peter, can you share some
names, if you like, but some key clients and customers you've worked with and how
Pyramid Analytics really brought value to that? What we're seeing in particular is a
real need for broad enterprise use case analytics scenarios being addressed rather than single use
case scenarios. That means that large enterprises are having to tackle demands from every angle of
their business, whether that be the traditional form of operational analytics to more specific
supply chain analytics, marketing analytics, sales analytics, financial analytics, etc.
But in particular, wanting to extend that out to a broader audience in the manufacturing space,
in particular, a great example, a prospect, as it was at the outset, was challenged by
having adopted multiple tools over multiple years. They realized they weren't getting the scale,
they weren't getting the value from the data that they held, and they needed to modernize
their platform proposition as it related to analytics, so approached Pyramid.
Initially, we're just focused on delivering analytics for their enterprise internally
within their organization across multiple countries to enable them to drive greater
efficiencies across sales, marketing, finance, supply chain, etc. But then when we started
talking to them about the potential to extend that even further, their dealers, their distributors
out in the different markets, how could they leverage Insight in a way that meant that they
were still connected to them as a manufacturer but able to deliver more value out to their customers
in each of those geos? Doing so in an embedded format is something that they were able to
leverage, and as a result, drive significant gain. We're seeing a number of examples within
retail, in particular, store-based analytics, so enabling people within stores to make
quick decisions that are driven by insights from their data that mean they're optimizing
the opportunity to sell more products, ultimately. But by delivering that in a way whereby they don't
need to have experience of using a tool, experience of data, it's just there. It's delivered to them
in a form, whether it be embedded within an application, in a portal, or as a service generally,
they're able to access that insight and ultimately drive the value right at the
point in time that they need it, but when the customers need it as well. And in return, in
that example, that also impacts supply chain. If they're able to see the impact of what they're
doing day-to-day and make that decision that ultimately impacts the customer, at the same time
they can impact the organization themselves in terms of where their supply chain needs to be
optimized in order to serve that customer base. Across multiple geos again, across multiple
retail experiences and a broad product range, that's particularly complex, and it needed a
platform like Pyramid in order to deliver it. That multi-tier situation can get very complicated
very quickly when you've got different parties, which are essentially separate enterprises,
trying to share the same insight. Very much so, and so when we break that down,
we look at sort of three core components. So there's the data component, there's the content
component, and then there's the experience component. So from a data perspective, being
able to connect directly high-performing connections to any operational source, being
able to model that data appropriately and deliver content that's tuned to the different personas,
the different experiences that individuals are expecting dependent on their roles within an
organization or outside the firewall with their customers, partners, and suppliers.
And then that experience again becomes tuned. How do I deliver that experience to those different
personas in order that they can consume that appropriately in order that they can make a
decision or take the next best action? And so they're the three core elements that we've been
focused on within the platform that we enable organizations to leverage.
Those kind of business benefits really seem applicable to almost any kind of enterprise.
We're seeing opportunities come at us and we've delivered on those opportunities across
almost every single vertical, in every single geo, and at different ranges in terms of
customer adoption. From relatively small use cases, it could be 50 user
requirements, all the way up to hundreds of thousands, and in some cases, millions of users,
especially public domain-orientated analytics, embedding Pyramid Discovers within a web page
for a large financial institution in order that their customers can interact with that data
in real time and ultimately make the best decisions for them. So that's got a phenomenal reach,
phenomenal level of adoption, and the critical factor is it's based on the latest and greatest
data that's governed, but giving a self-service experience that means there isn't a push back
onto the enterprise from a workload perspective. It's just there, and it's an easy to adopt,
experience, true self-service, and as a result, what tends to happen is we might start off with
a few areas of an individual enterprise, and then the word will get out, and others will come to
the core team and say, hang on a minute, I've seen what you're doing over in supply chain.
Well, we'd like that in marketing, and we'd like that in sales, and actually, we'd like to give our
customers and our business partners a better experience. How do I do that? Well, Pyramid's
the answer, and we're delivering on that every single day. And clearly, you're at that scalability
capability. Limitless scalability, and ultimately, there's a combination of reasons why,
but we've got the most modern data and analytics platform from a data prep, data science, and
business analytics perspective. That means we're leveraging the latest and greatest code base,
and as a result, we are not restricted in terms of enabling an organization to integrate to their
existing assets, but at the same time, when those different assets need to change, we just repoint,
and so significant opportunity there. At the same time as addressing complexity,
we're making it easy for every stakeholder within that journey from source to screen, if you like,
to have an optimized experience, leveraging AI across each of those different components of that
journey, and focused on the strategic value that's required from the enterprise core initiatives,
but also consuming the requirements on an ongoing basis that are ever-changing
from all of the users, internal or external. How do enterprises balance the human element
with the idea of automated decision-making? Think of an enterprise as a pyramid in the sense that
at the lower part of the pyramid, we have operations, then we've got tactical decisioning,
and we've got strategic decisions. Yeah, but if it's a pyramid, it's probably the perfect shape.
Decision automation is more likely to be widely successful where there are literally
hundreds of thousands of decisions that need to be made pretty rapidly in relatively mundane areas
or pretty well-defined areas. So I think operational decision-making is very,
very suitable for decision automation, where you can delegate responsibility
to machine learning models and things like that to do that, or even to recognize a situation
and alert or recommend to a human to make the final decision. But certainly, I think that's
probably where we're going to see the most of it. Yes, we may see some of that at a tactical level
and probably a little bit or more like a recommendation at the higher levels within
the enterprise where the decision's left to the human. Today, fraudulent transactions
happening every day in banking. They're happening millions per second, if you like,
and you can't get a human in the loop. It has to be automated. These, to me, are repetitive
decisions. As long as, as Peter said, we're talking about solid, governed data in a governed
analytical environment, yes, we can get that right, and we can overcome those burdens and
make sure we keep costs down, we lower risks, we improve efficiency, or seize opportunities
as a result of some of those automated decisions or even automated alerts and recommendations.
Anything else you want to add on the future around decision support and analytics?
The one thing I would also say is we often deliver insights, and then
maybe the human makes the decision, or even it's automated. But what we don't seem to do enough of
is record the decision that's made and understand the effectiveness of it. I think that's where
we're going to go next. We're going to record these decisions, look at the impact of them,
and are they actually improving the outcomes or not? This is where I think reinforcement
learning, which is reward-oriented, and if you're putting out targets as the reward to get to as a
business, and then we get into self-learning in the platform, then we're going to get progressively
better at making decisions so that we optimize the business and exploit all this to the max.
How does that start to look when you record decisions? It seems really interesting,
that idea of being able to look back and see whether something was done right or not.
You have to know, well, how do you know if it's effective or not? I think that's really where we
need to understand business strategy in the software. To me, that's where we're looking at
business plans, and we're looking at accountability, target metrics, and whether or not
the decisions are resulting in improvement of those metrics to reach those targets.
Because if they're taking us in the other direction, then very, very clearly,
it's not very effective. Good to spot that early because at least then you can police that.
Similarly, also, if it turns out that the data is indicating, actually, by the way,
your objectives and your strategy are wrong, that's also a good idea to know early.
But I do think it's linked to business strategy, and I think in the future,
what we're going to see is all of that coming together.
Wrapping up here, I'd love to hear from both of you. What's next? What do you think is next
on the horizon, and how does Pyramid Analytics fit into that?
I still think we've got a long way to go with Gen AI. I think that's going to have a big
impact on the number of concurrent users around the enterprise able to take advantage
of this, as Peter talked about, with different roles around the enterprise.
So that's really going to widen the number of users able to take advantage of this stuff,
and I think scaling that out and understanding it within the enterprise
is really, really important. But I think the future is we have to
not only look at data science, look at analytics, generative AI, embedded analytics, et cetera,
but we have to coordinate it. We can't have these as just silos of stuff around the enterprise.
It's really got to be coordinated and linked to business outcomes so that we get what I think I
said earlier, mass contribution to common goals. Because if everybody's pulling on the same rope
within an organization, it's a darn sight more effective than if we're all moving in different
directions. You know what I mean? So in that sense, I think we're going to see a lot more
embedding going forward, a lot more integration with applications and processes, because people
don't always get their insights through tools. They need it built into their everyday workflow
in their day job. And I think, as Peter already alluded, it has to be seamless.
The moment it's seamless, you don't even notice it. It's just there.
That, to me, is really where we're getting into this opportunity for mass contribution,
and what I actually call intelligent business. In terms of future state, I think certainly
the human intelligence combined with artificial intelligence is something that we need to nurture,
and we need to continuously learn as humans. As much as we're relying on machines,
we have to continuously learn as to the art of what's possible. And so that thought leadership
piece is key. And in particular, I see that being anchored around predictive-orientated
and prescriptive-orientated analytics as being the next wave from a point of view of
ease of adoption. We need to make it easier for adoption in that respect, and make decisioning
support and decision intelligence come to life. And so we're very focused, in terms of our
platform proposition, on those key points of predictive and prescriptive, as well as descriptive
and diagnostic, of course. But that's, from a future state perspective, enabling citizen data
scientists to leverage data in a way that they've never done before by combining human intelligence
and artificial intelligence is going to be absolutely key. And yeah, the future's really
bright. Super. Well, gentlemen, thank you so much for your thoughts today. It was great talking to
you. Seems like exciting times ahead for Pyramid Analytics and the whole data space. This is Scott
Taylor of DataWhisperer coming to you from Big Data London. We'll see you next time.

