The GDELT Project

Visualization credit GDELT Project.

Supported by Google Ideas, the GDELT Project monitors the world's broadcast, print, and web news from nearly every corner of every country in over 100 languages and identifies the people, locations, organizations, counts, themes, sources, and events driving our global society every second of every day, creating a free open platform for computing on the entire world.

The GDELT Project is a realtime network diagram and database of global human society for open research

Visualization credit GDELT Project.

Watching The Entire World

GDELT monitors the world's news media from nearly every corner of every countryin print, broadcast, and web formats, in over 100 languages,every moment of every day.

GDELT monitors print, broadcast, and web news media in over 100 languages from across every country in the world to keep continually updated on breaking developments anywhere on the planet. Its historical archives stretch back to January 1, 1979 and update daily (soon to be every 15 minutes). Through its ability to leverage the world's collective news media, GDELT moves beyond the focus of the Western media towards a far more global perspective on what's happening and how the world is feeling about it.

From the Global Twitter Heartbeat to the SyFy Opposite Worlds Show (and many more to be announced shortly) we are exploring how social media is used around the world and how people and societies express themselves and talk about the world online. As these projects increase our collective understanding of the social sphere and especially how it is used in the non-Western world, we will be increasingly integrating social media into GDELT's monitoring streams.

In the words of George Santayana "those who cannot remember the past are condemned to repeat it" - history is highly cyclic and contemporary events are often deeply rooted in historical contexts, making the understanding of the past of critical importance to interpreting the present. Already GDELT is the first truly multi-decade global event database and through an array of collaborations and partnerships we are expanding GDELT's coverage all the way back to the year 1800, which, when complete, will offer more than two centuries of codified global history.

Even the largest teams of human translators cannot read and translate every word published by the world's news media each day. In partnership with Google, we are leveraging Google Translate for Research to augment human translation of the world's news media to eventually translate the entire world's news media in realtime 24/7/365. Already, our pilot system is finding that 75% of the material in the non-English news media appears nowhere else in the world.

Monitoring nearly the entire world's news media is only the beginning - even the largest team of humans could not begin to read and analyze the billions upon billions of words of new information published each day. GDELT uses some of the world's most sophisticated computer algorithms, custom-designed for global news media, running on "one of the most powerful server networks in the known Universe", to create a realtime computable record of global society that can be visualized, analyzed, modeled, examined, and even forecasted. Two datasets are created, one codifying physical activities around the world in over 300 categories, and one recording the people, places, organizations, and themes underlying those events and their interconnections.

The GDELT Event Database records over 300 categories of physical activities around the world, from riots and protests to peace appeals and diplomatic exchanges, georeferenced to the city or mountaintop, across the entire planet dating back to January 1, 1979 and updated daily (soon every 15 minutes).

Essentially it takes a sentence like "The United States criticized Russia yesterday for deploying its troops in Crimea, in which a recent clash with its soldiers left 10 civilians injured" and transforms this blurb of unstructured text into three structured database entries, recording US CRITICIZES RUSSIA, RUSSIA TROOP-DEPLOY UKRAINE (CRIMEA), and RUSSIA MATERIAL-CONFLICT CIVILIANS (CRIMEA).

Nearly 60 attributes are captured for each event, including the approximate location of the action and those involved. This translates the textual descriptions of world events captured in the news media into codified entries in a grand "global spreadsheet" that can be mapped to explore and understand evolving situations like Ukraine or Nigeria or broader topics like global protest trends.

Much of the true insight captured in the world's news media lies not in what it says, but the context of how it says it. The GDELT Global Knowledge Graph compiles a list of every person, organization, company, location, and over 230 themes and emotions from every news report, using some of the most sophisticated named entity and geocoding algorithms in existance, designed specifically for the noisy and ungrammatical world that is the world's news media.

The resulting network diagram constructs a graph over the entire world, encoding not only what's happening, but what its context is, who's involved, and how the world is feeling about it, updated every single day.

Visualize the Global Conversation in a single glance, make World Leader Wordclouds, or explore the connections among Iran's leadership or the evolving narrative around Edward Snowden. You can even drill into reaction to key policy decisions like Syria and Ukraine and leadership assessment. Subscribe to the World Leaders Index to receive a daily summary report of how the world's heads of state are being portrayed in the world's news media.

The GDELT Analysis Service is a free cloud-based service that offers a variety of tools and services to allow you to visualize, explore, and export both the GDELT Event Database and the GDELT Global Knowledge Graph. This is a great way to get started exploring GDELT and what it can do for you, even if you don't have a technical background.

The entire quarter-billion-record GDELT Event Database is available in Google BigQuery, updated daily. You can query, export, and even conduct sophisticated analyses and modeling of the entire dataset using standard SQL, with even the most complex queries returning in near-realtime.

Advanced users and those with unique use cases can download all of the underlying records in CSV format. The GDELT Event Database alone is more than 100GB, and few software packages can deal with even small subsets of the database, so most users will likely wish to use the GDELT Analysis Service or Google BigQuery.

The blog is also where we feature a steady stream of examples showcasing projects that use GDELT in new and innovative ways as well as "getting started" examples that show you how to perform basic analyses using GDELT, from mapping to modeling. Want to see an example of how to apply one of the tools in the GDELT Analysis Service? The blog offers examples of how to use each tool and the kinds of analyses they support.

Do you have a cool new application or visualization you've built using GDELT? Writing a paper or analysis using GDELT? Hosting a hackathon using it? Have an awesome new analysis or visualization software package you used GDELT to show off? Write a great story about GDELT or using GDELT? Drop us an email and we'd love to feature it on our blog!