Paper/Presentation Description: This paper examines the challenges and opportunities that "big data" poses to scholars advancing research frontiers in the social sciences. It examines the strengths and weaknesses of machine-based and human-centric approaches to information extraction and argues for use of a hybrid approach, one that employs tools developed by data scientists to leverage the relative strengths of both machines and humans. The notion of a progressive, supervised-learning approach is developed and illustrated using the Social, Political and Economic Event Database (SPEED) project's Societal Stability Protocol (SSP). The SSP generates rich event data on civil strife and illustrates the advantages of employing a supervised-learning approach in contrast to conventional approaches for generating civil strife data. We show that conventional event-count approaches miss a great deal of within-category variance (e.g., number of demonstrators, types of weapons used, number of people killed or injured). We also show that conventional efforts to categorize longer periods of civil war or societal instability have been systematically mis-specified. To demonstrate the capacity of rich event data to open new research frontiers, SSP data on event intensities and origins are used to trace the changing role of political, class-based and socio-cultural factors in generating civil strife over the post WWII era.
Speaker Bio: After completing his doctoral work in political science at Northwestern University, Professor Althaus joined the University of Illinois faculty in 1996 with a joint appointment in the Political Science and Communication departments. He is currently a professor in both departments, and also associate director of UIUC's Cline Center for Democracy, where he has been a faculty affiliate since 2004. Professor Althaus's research and teaching interests center on the communication processes by which ordinary citizens become (in theory, at least) empowered to exercise popular sovereignty in democratic societies, as well as on the communication processes by which the opinions of these citizens are conveyed to government officials, who (in theory, at least) must transform the will of the people into political action. His work therefore focuses on three areas of inquiry: (1) the processes and constraints that shape the journalistic construction of news about public affairs, (2) the processes and constraints that influence how news audiences receive and utilize public affairs information, and (3) the channels of communication that allow individual members of a polity to speak in a collective voice as a public. He has particular interests in the quantitative analysis of political discourse, opinion surveys as channels for mass communication and political representation, the impact of strategic communication activities on news coverage and public opinion, the psychology of information processing, and communication concepts in democratic theory.
Professor Althaus serves on the editorial boards of Critical Review, Human Communication Research, Journal of Communication, Political Communication, and Public Opinion Quarterly. His research has appeared in the American Political Science Review, the American Journal of Political Science, Communication Research, Journalism and Mass Communication Quarterly, Journal of Broadcasting & Electronic Media, Journal of Conflict Resolution, Journal of Politics, Public Opinion Quarterly, and Political Communication. His book on the political uses of opinion surveys in democratic societies, Collective Preferences in Democratic Politics: Opinion Surveys and the Will of the People (Cambridge University Press, 2003) , was awarded a 2004 Goldsmith Book Prize by the Joan Shorenstein Center on the Press, Politics and Public Policy at Harvard University, and a 2004 David Easton Book Prize by the Foundations of Political Theory section of the American Political Science Association. He was named a Merriam Professorial Scholar by the UIUC Department of Political Science and the Cline Center for Democracy (2012-4, 2010-2), a 2004-5 Beckman Associate by the UIUC Center for Advanced Studies, and a 2003-4 Helen Corley Petit Scholar by the UIUC College of Liberal Arts and Sciences.
The median number of people involved is just one, and it's not a very impressive event,
but we can detect it in the news.
Whereas when we're talking about mass expression events like strikes or big demonstrations
and protests, the median number of participants is 2,500.
Well, for a government who is looking, whether at the local level or national level, at this
unrest event, it matters a lot whether you've got 2,500 people or one for predicting what
the government response is going to be.
We want to capture those differences in intensity so we can better understand how the political
systems and economic systems respond to these civil unrest occurrences.
So, bringing humans and computers together allows us to measure that very reliably.
A second benefit that we get by bringing these two systems together is that it provides us
increased precision for identifying when civil unrest events happen.
Here's an example.
It's a very complicated set of graphs.
I'm going to just draw your attention to this one here.
This is showing the country of El Salvador from 1979 through 2009.
The country of El Salvador experienced a civil war in this period.
The lines at the top represent three of the highest quality, purely human-generated data
sets that are out there that identify by country year when a particular country is
in a civil war situation or not.
If the line is there, it's been designated, in this case El Salvador, as being in civil
war.
When the line goes away, it's not.
The red bars come out of the Climate Center Speed Project that actually count, use humans
to extract how many people have been killed, as reported in these news articles.
We can come up with a much lower resolution picture of how these civil unrest events are
transpiring over time than the standard human method.
Now we know that in 1980 through 1983, the civil war was very intense in El Salvador.
But then it really died off with a few isolated incidents, even though the fully human-based
measures would say, oh, it's still a civil war going on in El Salvador.
We've got the ability to see civil unrest events at a much lower resolution than purely
human systems have been able to do.
The third benefit is that we get increased spatial and temporal resolution for observing
civil unrest, and hopefully what I'm about to do is going to work.
I'm going to show you a demonstration that Joe pulled together for us.
This is using our New York Times data, where we coded a random sample of stories that our
classifier deemed relevant for containing civil unrest data.
We coded, what, 50,000 stories, I think it was, something like that.
This shows just a portion of the world, but the distribution of these reports over time.
We're beginning right now in 1945, and I'm just showing Europe and the Middle East as
an example.
As we move this time through, you can see, for example, the early 1950s, there are a
few protests in Europe.
Those blue Ps are protest events, mass expression events.
The red flames are political violence events, and the yellow, I don't see any yellows,
there's one right there, is an example of a state attack.
Any of these units, you can click on them, there's actually a series of them in there,
you can, if I can do this easily, we get event information out of them.
We can go back to the original news article, if we need a human to do that, to go recover
all that information.
We can see how this plays out in time.
Not much protest in Europe, not a whole lot happening in the Middle East, but some protest
events.
As we move on, you will see that the world obviously changes very dramatically.
Here we have North Africa, by the time we get to the late 1950s, parts of North Africa
engaged in a great deal of civil unrest.
The 1960s, we start seeing more protests in Europe.
We also see protests, interestingly enough, in Israel and across other areas of South
Asia.
As we move forward into the 60s, we get into the time of very heightened protests in Europe,
but also lots of conflict and violent conflict in other places.
As we move on through the late 70s, we start getting to, finally, the Iranian revolution.
It shows up right here.
As we continue on, we will see Iraq and Afghanistan on either side, erupting in conflict.
This is the level of acuity that we get on civil unrest data through this hybridization
of machine and human systems.
This has not been possible before.
Nothing like this has been seen before this project.
We're very excited about what it portends for the future.
That's our third benefit.
Our fourth benefit is that bringing these two kinds of systems together into a hybrid
composite allows us to better understand why civil unrest is happening and how the reasons
for civil unrest are evolving over time.
You can imagine that this lady that I pointed out before, who had many different reasons
why she's upset, could be expressing sentiments against her government at whatever level.
She might desire retribution against some other group.
She might be concerned about economic instability.
There might be some sociocultural tensions that she is expressing.
There might be a desire to hold on to power.
There might be other kinds of positive political desires that you want government to act on.
The machines are going to have a hard time sorting these different things apart.
Humans have a hard enough time, but when they're well trained, they can do that.
What we're looking at now is a graph from our New York Times data that shows in each
of these subplots from 1946 to 2006.
The dotted lines are incidents of political violence, which can go all the way down to
the level of one person attacking another person at that level to much larger scale
examples of political violence.
The solid lines are mass expression events, speeches, demonstrations, protests.
The actual numerical values of these lines are not as important as how they change over time.
What this shows is that economic concerns were a more important part of motivation for
political violence and mass expression events in the 40s, 50s, and 60s.
Today, less of these civil unrest events are motivated by economic concerns.
More of them over time have been growing, particularly expressions of political violence,
terrorism, and so on, have been motivated by positive political desires.
I want something from government that I am not getting.
But the two biggest categories driving, or at least the news accounts relate, are the
reason why political violence or mass expression events have been happening, have been changing
dramatically over time.
For anti-government sentiments being expressed, that has been a main reason that is expressed
by people who are perpetrating political violence or who are described as the reasoning in the
news for people who are perpetrating political violence.
And social-cultural animosities becoming increasingly the reason why civil unrest is happening
around the world.
This is a look at around the world.
Before this project, this sort of acuity into the underlying reasons for these civil unrest
events had never been studied systematically.
But it's the combination of bringing computers and humans together that allow us to do this.
So, I want to end with a quote from Donald Rumsfeld, who was the Secretary of Defense
in the United States for many years in the Bush administration.
Though his native language is English, most of what he says sounds Greek to most of us.
And so, let me read you the quote and see if you can make sense out of it.
Donald Rumsfeld says,
As we know, there are known knowns.
There are things we know we know.
We also know there are known unknowns.
That is to say, we know there are some things we do not know.
But there are also unknown unknowns.
The ones we don't know, we don't know.
So, if I were to map out for you the social scientific understanding of civil unrest processes,
the known knowns are very small.
We don't know very much.
The unknown unknowns are very large.
And we believe that until Dan Roth and his group can overcome some of the challenges
that they've been doing and bring the fully automated processes up to the level that humans can operate at,
the future, in the near term, looks like a hybrid system that brings humans and computers together.
And, you know, there's not many opportunities for those of us who live south of Green Street
to come on this side of campus.
It's kind of nice.
But we're definitely interested in following up with anybody who might want to explore
some of these challenges with us from the north of Green Street side.
So, thank you.
The Correlates Award data set provides us with some administrative data that we find
very useful for coordinating our case studies and so on.
But Pete, who's stepping out, do we have...
Yeah, you've got to go.
Okay.
That's all right.
The Correlates Award data, the problem is that it gives us event data at the country year.
Right?
So, country, yes or no, it's in civil conflict in a given year.
That is not a low enough level of resolution to really get at the underlying dynamics
and processes and understand just what's going on in these civil unrest events.
So, hopefully, we could, by understanding them better, perhaps prevent them for governments
to be more responsive to their citizens or so on.
Correlates Award data don't get us very far.
So, that's human coded data.
The other machine coded data, purely like the KEDS system and others, don't give us
enough resolution on the underlying causes, the who, what, where, when, why, and how,
that we want to know in order to map these things.
Because few of these civil unrest events are completely independent of the others.
Right?
You get changed sequence.
Somebody sets himself on fire in Tunisia.
We don't even have a picture of that.
And then the Arab Spring comes out of it.
And we want to be able to trace those dynamic processes as well.
Not in a way that Don Rubles is able to trace all these complexities in the atmospheric
aspects of how the world works.
But we'd like to get at least a little deeper in understanding some of those complexities
in the social, cultural dimensions of how the world works.
And we think this is a way forward.
Yes, it comes up in our analysis quite a great deal.
And the biggest challenge with assimilating information from multiple sources is the same
as Dan was talking about before.
To the extent you're relying on machine systems, switching from the New York Times to Genois
or Agence France-Presse or something else, there are different conventions of language,
there are different terms, there are different entities.
And so you can't just take the same model and plunk it down somewhere else.
And that's where the humans come in for us.
Because the humans are able to navigate those subtle differences in language a bit better
than the machines are.
And so that's actually one of the ways that we're bringing about the progressive part
of this supervised learning system.
Once we get the humans training the machines on, say, the New York Times, then we put the
humans on to Phibos or SWB to come up with a new training set.
And we'll see how well the New York Times classifier is going to work on those new data.
So that's how we try to inch our way forward.
But the problem is a tremendous problem.
Entities are called by different names.
There are different transliterations of groups' names and so on.
So we are working to try to work that out.
And that's where we fall back on our human coders to be able to do the corrections for
us until such time as the machine systems get better at being able to do that.
I mean, there are efforts to try to do that.
The American government, particularly in the military, are very interested in that.
I think the problem is that large-scale social unrest events are very rare.
And so it's hard to predict things that are very rare, even if you have a lot of data.
I think we're going to have more success at predicting more regularized things.
So one of the things that we're interested in doing is looking at climate change data
and looking at incidents of drought or other kinds of climate change and see how that correlates
with the civil unrest events that we see.
If we can see patterns over time, we might be able to model that.
The weather is a little bit more regular in human society, even though it's far more
complex than we concurrently expect.