Lec 51 05 04 2023

IIT Kanpur July 201
20 followers
1 Video View·Oct 23, 2023
Welcome to the last week of the course, Computed Aided Decision Support Systems, this will
be last one or two lectures, where we will be discussing data visualization.
We have recently discussed the HTML and PHT, how do we connect that, when we connect that
we try to get the user interface, where the user interface we need to present the data,
we need to present the function and the form of the data types that you have.
Now the data visualization would be covered in this lecture, where we will be covering
the definition of it, then some advantages and disadvantages, yes data visualization
obviously it is used for multiple merits but there could be certain demerits, some points
could be missed that also we will discuss.
Then significance of data visualization in industry 4.0 or in today's world that is
big data, how this is being used to present the structured or user form of it, then framework
of data visualization, then visualization types and some examples.
Let me start with the definition, defining data visualization, by definition data visualization
is only representation data using common graphics, these graphics would be charts, plots, tables,
infographics or anything like even animations, now by definition data visualization helps
us to have a representation of the data in various forms, in various graphics, so that
we are able to communicate, communicate the message, this message could be the relationships
between data, the data driven insights, so that one is able to understand or one can
relate with the trends, outliers, patterns, this helps us to take the data driven decisions.
Developing an interface, developing your program, developing your website, developing
your PHP and HTML programs is okay but finally how to present that, what kind of colors would
you choose, what is your audience, trying to understand the audience, I will talk to
talk about major tips when we try to have a good data representation, the form of the
data, the function of the data, the form should not kill the function, the function should
not be such represented that the form is not even interactive, it is not even aesthetically
looking well, so it is important to have good understanding of what your data is, what are
you trying to represent, who is the audience, all these things when you are trying to generate
an idea, even when we are talking about data visualization, design thinking sessions also
helps us to understand it, when we try to have a concept of design thinking, we try
to first empathize with what the user would like to have, what the user would like to
see, what the user would like to visualize, then we try to define what our problem statement
is, then we try to bring up some ideas over it, so this is how data visualization framework
also we will talk about in the coming slides.
So, data visualization is a very important step in data science process which allows
the teams or the individuals to convey the data more effectively to colleagues, to decision
makers, so that the teams can organize the reporting systems, specifically define template
views to monitor performance, we cannot miss that the data visualization is not only limited
to the performance dashboards for example, text mining as an analyst may use the word
cloud to capture key concepts and trends or hidden relationship with unstructured data.
Alternatively, these might also use the graph structure to have in relationship between
the entities of the knowledge graph itself, so there are certain ways data visualization
is used, yes there are certain upsides and downsides, if I talk about the advantages
or the merits, number 1 is the clarity, data visualization helps us to have great clarity
on what we are trying to present to our audience that means, one graph represent 1000 words,
telling just in a theory or writing just in a report that okay, the GDP has risen by
5% in the last 5 years or the average income has risen by around 2% in the last 1 month
or presenting them in a graph, putting in a pie chart, India, various states how the
GDP has risen.
So, one graph represent 1000 words telling the whole story and putting them in a one
graph that is why we need data visualization, so our eyes just flow like water, it tries
to follow the equilibrium, it tries to follow the lowest path, so data visualization is
such design, so that the colours are also chosen, the best part or the most prominent
part that we would like to show is highlighted in a way that might be put in a red colour,
that might be put in a yellow colour, so that the person sees or that might be put in a
bold letters or bigger sizes, so that one is able to see what is the overall zest of
the visual representation that we are trying to bring in.
So, we can quickly identify red from blue, squares from circles, so that is why our culture
is visual which includes everything from the arts, the reports, advertisements, journals,
information, the data visualization is a form of visual art that grabs our interest
and keeps our eyes on the message that the person who has designed is trying to present,
so when we try to look at a chart, we very quickly try to see the trends and outliers
in it.
The precision is also one of the advantages, in data visualization what specifically purposes,
so we can have a target value that could be bold or a contrasting colour or maybe highlighted
using certain other ways, so data visualization helps us to have an efficiency or an ease
of sharing the information, so we interactively explore the opportunities, we visualize the
patterns and the relationships, when we visualize the patterns and the relationships, we also
maximize the ideas with minimum inks, here I think I would say 1000 words is equal to
one chart, with the upsides, certain downsides are there that I can say it demerits.
Sometimes in the big data, we have a lot of information and what we are trying to target
to or the primary goal is missed, that is why I showed you the framework or the process
for the big data presentation, how do we present the data, now sometimes the basic message
is missed, we try to sometimes go into the form of the data, the function is missed,
this is always a battle between form and function, for instance you have an engineer or a data
scientist who is trying to write a code, who understand what is the function of the data
or we have the data administrator, who has a goal to be put in, the data scientist sometimes
has a different view point on what color of colors could be chosen, so data administrator
has to take the call that the function we cannot miss, this is more important, function
for instance if I need to say, so IIT Kanpur has risen in its QS ranking in the last 10
years or so, I would only say a rank chart or I can say may be a histogram showing the
ranks between different years of IIT Kanpur and other institutions in India, this is my
function and form both coming up.
Sometimes I just might like to put okay, the form is more important, the color is more
important and the basic function is sometime missed, the basic function for instance the
rank of IIT Kanpur has risen in the last 10 years and if I try to say, if I try to compare
with the other IITs or other institutions in India or if I try to compare with the world
top rankings, IIT Kanpur would still see its ranking down there may be at 48 or 100 or
so, wherever the rank is in the last years.
So the form of the data, the function of the data, function is to present that our data
or IIT Kanpur has risen or enhanced its ranking in the last few years, accordingly it has
to be put.
I will talk about this more when I will talk about the cheating with skills.
So data visualization sometimes gives the information that is either incorrect or biased.
For example when viewing a visualization with many different data points, it is easy to
have an interpretation or an inaccurate assumption or sometimes the visualization is just designed
even wrong, so that it is biased or it is confusing the person.
So correlation does not indicate cause, so the causation is always missed in correlation.
Correlation could only tell you in comparison to the boys in the institution, the girls
have scored more marks but what is the reason of that, for that we need to do further study,
for that we need to have certain regression analysis or so, based upon the certain parameters,
what were the parameters, the girls might be spending more hours in study, the boys
might be spending more hours in just roaming around or the girls might have higher ranks
in the admission itself, those who are there, there might be certain parameters which could
be used or which could be set and regression analysis or certain other kinds of analysis
could be taken to have exactly what is the cause of this.
Sometimes as I said the core statements can be lost in translation, with translations
when you talk about the data visualization framework, we try to map the data with the
basic data and try to have a visual graph of it, the basic or the core statements are
sometimes missed, so this could be certain demerits of the data visualization even.
Now, let me talk about significance of data visualization in the world nowadays, so big
data it is important that data should make a good sense, it should be more understandable,
so the importance of data visualization is very simple, it helps people see, interact
with and better understand the data, so whether simple or complex, the data visualization
can bring everyone on same page regardless of their level of expertise.
Audience though versatile, everyone has to be brought on same page because these days
definitely audience that you get we do not know a school, great coder even or even a
person who is there as a CEO might not able to understand what is the basic code in which
this website is designed or so, so different audience or different parts of the audience
who your audience is and what kind of the interaction you are trying to do, are you
really going to present the data, are you going to interact with them or are you going
to tell a story, that also I will discuss what kinds of types of the data interaction
or data visualizations that you have.
So, this is important and also it is hard to think of a professional industry that does
not benefit from the data nowadays, so the concept of the citizen data scientist is on
rise where skill sets are changing to accommodate the data driven world, so it is increasingly
valuable for professionals to be able to use data to make decisions and use visuals to
tell stories of when the data informs who, what, when, where and how the specific field
or the specific topic that you are trying to discuss is presenting, so data visualization
if I say in the big data, it sits between the data analysis and you need to have a storytelling,
so here do we have data visualization, now this is the age of big data where we are trying
to talk about the trillions of the rows of the data that is being generated each day,
so in this case data visualization is not only to dress up a graph, to look up the information
or the infographic is there, the effective data visualization is a delegate balance between
the form and function, I will have to put that here once again, form versus function.
A very simple graph could be too boring that is of the form is too form, simple is boring
in the case of form, so that no one notices it, but it can still make a, still a powerful
information could be extracted from it, simple is boring for instance if just in a black
and white form itself only a line graph shows from the number of students who have reason
to take NPTEL courses, it could be just given okay from 500, it has reason to 50,000 in
last 10 years, this could be very simple but this 500 to 50,000 is 100 times rise, it is
a powerful message that it is a 100 times rise of the number of students who are taking
the courses in specific field may be mechanical engineering or computer science and engineering,
so this could be very simple but it could give a powerful message.
On the other hand form versus function if I say the form could be very stunning visualization
with lot of colours, I would say a complex, complex visualization but in this case as
I said in the previous slide as well the core statement might be missing, for instance if
I just say okay in mechanical engineering these many students have taken the courses,
in computer science these many, in electrical these many, these many people have cleared
the courses, a lot of information is given is just one in A4 size poster.
So it has a lot of information but finally what it is trying to tell is that the number
of people is reason by 100 times in who are taking the courses in NPTEL, so sometimes
the complex visualization could miss the core statement or it could not present the
message very powerfully, so the data and the visuals needs to work together, there is always
a balance between them and this art of combining these two form and function is actually the
data visualization itself.
So in big data because we have lot of data, we have lot of ways the data is coming, what
specifically is the function, what specifically is the message we are trying to give from
there itself the data has to be driven, to understand this let us try to see what is
the framework of data visualization, when I talk about the word framework it means how
do we design our data visualization, it could be simply three layers, we have data layer,
we have mapping layer and the graphic layer, where actually we are going to choose the
kind of the plot or the kind of the chart that we have and what kind of data inputs
that we have.
So here generally we have different data sources, may be data source 1, data source 2 and data
source 3 which I am putting it as my stored data, so here the data is collected from the
data sources, we call it data mining, we call it data collection, we call it data map reduce
it is collected here and then we try to map the layer, here the map reduce function comes
into play more prominently, we have a mapper, different data points come here, from here
itself we now go to the graphics, let me say I am putting a histogram here and from the
mapper different kind of data points are taken, from the mapper itself different data layers
could be taken, I might say okay this layer is going here, so it is going here, this one
is also coming here, from here the information goes here, how does this work?
Let us try to talk about this thing, the data layer is having a purpose of locating and
obtaining the data, locates the data or obtains the data, now the importing data in a proper
format is also important here, I talked about the different kinds of the data if you remember
NOIR, nominal, ordinal, interval and ratio scale data, this is just a categorical data
NNO, INRR, quantitative data, depends upon what kind of data we have, the relating data
or proper for proper correspondence is also important, here in the data layer itself the
data analysis and aggregation is taken here, because big data multiple points, millions
of or trillions of the data sets could be there, whether the data is in a discrete form
or continuous form, it is ordered, unordered or do we need to transfer the categorical
data into quantified form or is the quantified data is it is only to be presented in the
low or high or yes or no, so these translations could happen, this is known as data aggregation
where we try to get the final data, because we are not talking about the data analytics
right now, we will be talking about the data visualization, in which the aggregate data
is only to be presented finally for the proper visual views that we are trying to get.
Now this aggregate might be, we might like to have a pie chart, we might like to have
a line diagram, we might like to have a scatter plot or may be more complex plot such as the
fan plot, the eye plot could be taken depends upon the understanding of the person who is
trying to read the data, if the person is simple just a data administrator or may be
the business investor only, he would better understand the basic graphics, the bar graph,
the line graph, pie charts, this could be easily understood by them.
If the person is a student of statistics, the person should be, could be given the data
different kinds of the graphs such as box and whisker plots could be given, the bullet
graph could be given, then fan plot could be given, so that the person understand what
is the data, what is the overall information the data is trying to extract.
Now here data aggregation is important and the kind of the data that we have is important,
this data layers tries to give us the information that what are we going to feed into the mapping
layer.
Next comes the second layer which is a mapping layer which means we associate the appropriate
geometry with corresponding data channels, so it could be the data as I said in a quantitative
form or it could be in a categorical form, in a quantitative form it could be may be
the position of the data, length of the data, position, length, then may be area, volume,
density, I am just taking some examples of the I and R.
In the case of nominal ordinal, it could be just may be the hue, texture, the names, etc.
These are connected, the data analysis algorithms are put, so that we try to get the corresponding
data channels which could be connected to our next layer that is the graphic layer.
Now in the graphic layer what we have is the final conversion of geometry into, when you
say geometry, geometry is the kind of the data points that we have taken aggregate from
the data layer and in the mapping layer we have try to map it what the kind of the results
that we are trying to present here.
So here the conversion of geometry into display which means we try to select the charts or
graph or plot or table and we try to select the colours and we try to see whether it has
to be just presentation or interaction, this is important.
So when I talk about presentation or interaction or the interactive story telling, this comes
in our styles of the data visualization, data visualization styles they are majorly 3 styles
we try to have the interactive visualization, next is we try to have the presentation visualization
and it could be even the interactive story telling.
When I say the word interactive means there is an interaction between the audience and
the presenter who is going to present the data or the website that you are trying to
present, so it has to have the user input as well, here we have user input in between
is taken that means it could be intended for single investigator or it could be for
the collaborators.
For example, if I am trying to define proposal that I have put before a ministry, I will
take my presentation, I will take to take different kinds of the graphs or a different
kind of the options available, I would say this is option 1, if they have a query I will
come to the option 3, if we have a query I can say okay I have another option that is
the option 5 that is also available, so interaction also happens and the data is presented that
we are ready with the graphs, we are ready with different kinds of the website presentations,
we are able to connect with that and we are trying to have an interface or the input from
the user as well, the user input is always there.
So here for example, if I have also developed a prototype for already taken project, prototype
is to be presented before the final review committee, the prototype has to have an interaction,
the person would like to use the prototype, the prototype how do we present that okay,
this is my android interaction, this is my phone application, if you click this button
this graph would open, if you try to click, if you try to input these values maybe for
example, if it is a fluid mechanics experiment, if I say okay, if this is a fluid flow between
5 milliliter per seconds to 22 milliliters per seconds, this is the opening of the valve
that you need to keep.
So these kind of interactions that could also help us to have an interactive visualization
where we need to design it accordingly, so the pre-renders based upon inputs are there,
so Tableau public is also one of the examples that use the interactive data visualization,
this is an example.
Next comes are presentation visualization, it is only used for communication, this means
I am not trying to interact with them, I am only giving the information, the example could
be a poster that I am trying to present in a hard copy or the poster that is displayed
okay, this is the output, number of students have risen from 500 to 50,000 in last few
years, this is just a poster, there is no interaction or input from them, but this is
just a communication that means no user input is there, so it does not support any user
input.
So it is highly polished prior, which means we cannot interact with them, whatever the
information is taken by them or it is understood by them has to be final, so we have to be
very careful in designing it and it is polished prior, it is well tested or well thought of
prior itself that how would it present, example could be anything, example I would say rank
of the institute, institutes if I compare, just simple presentation.
Another kind of presentation or the data visualization is storytelling, when we are trying to narrate
something that we have an interactive web pages, so in this case the presentations are
always there, but these are done through interactive web pages, here the story is if you do this,
this happens, so if else loop is well used here in this kind of this data presentation
or data visualization, where we try to narrate something, we narrate using data, we tell
a story, the purpose here is we invite the viewers to explore further, for instance if
I am trying to present my proposal before ministry, it is a technical proposal or maybe
fluid mechanic itself, I would say fluid mechanics has certain experiments, these experiments
are very important for the students of the first year of civil engineering, of mechanical
engineering, of production engineering to understand, now in this way the word civil
engineering, mechanical engineering, production engineering could be a link, where if they
click, they try to understand what is civil engineering or mechanical engineering.
In further itself, if I say there are certain experiments such as Bernoulli's theorem or
it could be some turbine, maybe Pelton turbine, so this Bernoulli's, Pelton, if they click
that, a picture pops up, which tries to tell them okay, what is this theory, okay what
are the kind, what is the Pelton turbine, what are the other kinds of turbines which
are there, so this interacts or invites the viewers to click more and more to have an
understanding of that, the most of the web pages that you see, for instance if you just
see Wikipedia itself, the words which are there, this always try to have a link over
it which are complex words or which are words not very common, also the names of the person
in a specific Wikipedia page, if you are trying to read, if the name of the person who is
a prominent figure, the name of the person would also have a link, if you try to link
that, if you try to click that, it would open the page of the person, you can see the information
about that person, so this is known as interactive story telling.
These are the ways or the styles we try to present our visualization, there could be
certain tools depending upon what kind of the business information that you have, it
could be used for business intelligence, it is only to be used by the document presentation,
it depends upon what kinds of the tools that you have, what kinds of the final output or
final presentation that you have or final targets that you have, the data visualization
can help you to approach all of these.
Now, there are certain tips when we are trying to talk about data visualization specifically,
what we are trying to see or what we are trying to present and whom we are going to present,
these are very important, so that means we need to understand 4 major points, number
one what is your story, I will just jot them down here, then I will try to discuss them
further, so what is the kind of data, so which chart fits best, the presentation or the visualization
that you are trying to display to your audience and who is your audience.
So, when I say what is your story, it means what your data is trying to tell even, so
all the data that you are trying to present, the data visualization that you are trying
to present here is going to tell a story, is this story easy to understand, for example
it could be are your sales up or down, is the admission in the different institutes
or an engineering institutes in Uttar Pradesh is on rise or decrease, are the expenses within
control, so what is the reaction towards the new products which are launched the last year,
so what is the story, whether it has to be interactive, presentation or a storytelling,
this only is taken once you are very sure that what your story is, what is your final
output that you are trying to target.
So, what do you want the other person to understand, once your target is fixed that what you are
trying to present, what finally you are going to get out of it, then comes what kind of
data do you have or what do you want to explain through this data, so is it a qualitative
that is we have become better than before or we have added new features in our product
or is it even quantitative that means you have specific objective, objectivity is to
be targeted, it could be your sales and earnings, it could be the population statistics, it
could be the number of products that were sold in the last year, so what is the data
that you are trying to present here.
Now, if you are trying to present data depending upon the audience that you have, what kinds
of the charts that fits your data is very important, so nearly all the organizations
has quantitative information which they collect or they put it in their database system itself,
data visualization now allows the persons or the organizations to present this data
in a simple visual way, so that it makes sense at a glance itself.
Quantitative data is ideal for visualization because when you put numbers, we can compare
them, we can have a good plot out of them, it summarizes the essential characteristics
that allows you to present disregard the exceptions, sometimes the data is only qualitative that
means you can only present yes or no but this yes or no, how many numbers of yes or no,
how many times, how many people have made yes or no, those percentages, this could be
quantified further as well, which chart fits your data that is very important.
Now, depending upon the kind of the audience, these 2 are connected, one thing is if my
data is qualitative, it is only yes or no, I can only have very limited amount of, limited
kinds of the charts, I could only have maybe number of people who have said yes or no that
could be put in a pie chart itself for example or if it is a quantitative data with different
detailed information, we have the height and the weight of the students, right or the
number of students who are there in a specific class, this is one data, height, weight then
also age of the students in a class or the month in which they were born.
Then height, weight, the month in which they are born and their eating habits for example,
the person who is obese would have lesser height to weight ratio, so these kinds of
the quantitative information is important, so which chart fits best depends upon the
kind of the data and also the kind of the audience as I just mentioned before, these
are the 4 tips or 4 points that we need to keep it in mind.
If an audience is a data science intellect, those who are maybe able to understand the
complex charts itself, you can present accordingly but if the audience here are the maybe the
directors of the certain companies or you are certain investors or those who have only
money or you have people in ministry, those who only have to give the funds, then data
has to be in a simpler form, anyway data has to be in a simpler form, so it depends upon
the kind of audience that you have, the professionals, the investors, the directors would have different
view, the data scientists, the students, the learners, the organization middle or low level
managers would have different view point, so these 4 points are always to be kept in
mind.
With this, I would like to just have a break with my data visualization lecture and I would
like to show certain examples and what are the different categories of the charts, plots,
graphs that we have that I will discuss in the next lecture, thank you.
