More than a million customers around the world buy something at Walmart every hour. As they do, the retail giant collects data on their buying habits and adds it to a database containing the equivalent of 50 million four-door filing cabinets of text. Analyzing this torrent of data to understand and predict consumer behavior can be a profitable endeavor. Indeed, “big data” already helped Walmart’s US competitor Target to detect pregnancies by identifying changes in buying habits, so that it could adjust its marketing accordingly and increase sales dramatically.

For the private sector, big data is the next frontier in terms of creating value. But being able to understand when and why human behavior changes also offers huge potential for the public good, not least in the area of conflict prevention.

Right now, though, big data presents us with a new kind of digital divide—one that hinges on access to analytics rather than consumption of content. As large corporations leverage their own datasets and a number of governments release treasure troves of “public” data, poorer governments, civil society organizations, and multilateral bodies struggle to analyze big data for peacebuilding, development, and humanitarian ends. And conflict-prevention actors in particular may be getting left behind.

Key Conclusions

• The private sector has embraced big data analytics to drive innovation. Building on initiatives in the development and humanitarian fields, conflict-prevention actors should foster partnerships with corporations willing to share anonymized data and analytical expertise that could contribute to preventing violence and conflict.
• As governments increasingly embrace open-data policies, those seeking to better understand the structural causes of conflict should capitalize on already-public datasets.
• Public and private sector actors need to develop a framework for identifying the levels of trust, transparency, and control that citizens, corporations, and governments are willing to accept when it comes to sharing data in contexts of violence and conflict.

Analysis

Big data can offer insights into deep social, political, and economic trends as well as individual and group behavior. As previously outlined in this publication, it could therefore play a valuable role in both long-term efforts to address the root causes of conflict and short-term efforts to prevent outbreaks of violence. If plummeting wheat imports in Arab countries helped spark the Arab Spring, for example, or if phone call volumes increase before violence breaks out, then those seeking to prevent conflict need to draw on a much wider variety of data than before to make more informed decisions.

Two reports published last month make it clear that international actors, governments, and civil society organizations recognize big data’s potential in both conflict prevention and crisis response. But who has the data, and how can it be tapped?

From Walmart and Target to Google, Facebook, and Amazon, large corporations are amassing vast quantities of information on humans and their behavior. And while the private sector still struggles to separate the relevant data from the rest of the deluge and analyze effectively, it largely controls access and use of its vast datasets, and attracts the lion’s share of technical and analytical talent.

Some companies have indicated a willingness to share information extracted from their data for the public good, particularly during natural disasters and humanitarian crises. When it comes to analyzing the data itself, Google Trends provides an example of partial access and basic analytics by allowing users to explore the frequency of particular Google search terms. The company’s use of aggregated search data to estimate flu outbreaks demonstrated this tool’s potential in other areas.

Perhaps more promising, however, is the idea of data philanthropy, in which corporations would make the data itself, once anonymized, available for others to use. In a recent experiment in data philanthropy, the telecommunications company Orange offered academic and scientific researchers around the world access to a dataset of 2.5 billion anonymized text messages and phone calls from Côte d’Ivoire. One of the winning groups in this “Data for Development Challenge” showed how mobile phone data can be used to estimate social divisions within a population such as linguistic, ethnic, or political differences. This could be a significant finding for conflict-prevention actors. But when it comes to the public good, so far data philanthropy and sharing by the private sector has focused on healthcare, humanitarian, and development ends. Nonetheless, peaceful—not insecure—business environments usually help the bottom line.

In most countries, public sector bodies also gather enormous amounts of data from censuses, tax returns, and public health surveys, for example. Much of this data is technically “public,” but accessing it is not always easy, and mining it for relevant insights can require technical expertise and training that organizations and governments with limited resources can’t always afford.

A number of governments have begun systematically opening national data to the public by putting it online. The Kenya Open Data Initiative includes datasets on health, poverty, and energy. This democratization of data is a promising step forward. And if private sector startups like Enigma succeed, infrastructures for acquiring, indexing, and searching “public” data that is actually “scattered across a dizzying array of data silos” could make this data much more accessible and easy to analyze. In the politically sensitive field of conflict prevention, public data could represent a gold mine, since policymakers and societies at large have already decided that it should be shared.

Meanwhile, civil society organizations such as Ushahidi are attempting to gather and analyze their own data in crisis contexts—crowdsourcing information, often outsourcing the data processing to volunteers, and displaying relevant events visually on a map. This model may be promising in terms of connecting crisis warning and response on the ground, but relatively low participation levels can lead to inconclusive or skewed findings over a large area. It is also less suited to long-term conflict prevention than crisis response.

When it comes to preventing conflict, international actors, governments, and civil society organizations need to ensure that they have access to relevant information and the capacity to understand and use it. Having a better understanding of what’s going on in a conflict setting may not always mean that action is taken to prevent or manage the violence—more information does not necessarily generate political will. But better-informed decisions are undoubtedly a step in the right direction.

While data philanthropy may sometimes be more difficult to achieve for politically sensitive conflict-prevention efforts, this need not be true when it comes to mining data and correlating it with past events to better understand the structural causes of conflict. In addition, beyond sharing data itself, private companies and individuals can also contribute to bridging the digital divide by donating technical expertise or data-crunching tools. This could pay significant dividends for companies in terms of corporate social responsibility. And, to truly foster partnerships, international organizations, governments, and NGOs could team up with companies from the early stages of a project’s conception when appropriate.

The need to democratize access to public data and analytical tools is particularly acute in conflict contexts, where affected populations and grassroots organizations may be best placed to contextualize the analytics in a meaningful way. With this in mind, NGOs also need to start investing in their own technical talent and human capital to leverage this public data.

As noted in IPI’s recent report on new technologies and conflict prevention, much more work is needed to identify the levels of trust, transparency, and control that citizens, corporations, and governments are willing to accept when it comes to sharing data in contexts of violence and conflict. Seeking out partnerships with the private sector and increasing the accessibility of data that is already public are two good places to start bridging the big data divide.

Marie O’Reilly is an Associate Editor at the International Peace Institute. She tweets at Marie_O_R.