Verification Handbook Mixes Tools, Tips and Culture for Fact-Checking

Last week Twitter and CNN announced a major partnership with the data analysis startup Dataminr to shift the way journalists use Twitter as an early alert system for breaking news. Dataminr worked with CNN to fine-tune the algorithms they use, to help close the gap “between the eyewitness wanting to be heard and the journalist who wants to listen,” according Twitter’s head of news, Vivian Schiller, in a blog post. That gap is not just one of distance or time, but also one of trust.

Dataminr says its algorithms can not only identify emerging patterns and trends, but also help journalists focus in on the most relevant and reliable information. As an example of this, The Verge’s Ben Popper points out that “Dataminr told its financial clients that the AP tweet about an explosion at the White House was false five minutes before the AP itself corrected the facts.”

This is clearly a promising tool for newsrooms, but in breaking news, it is not just the tools, it is how you use them. It is not enough to have a great verification tool if the culture inside and outside the newsroom doesn’t value accuracy above all. To that end, last week also saw the release of an important new guide to verifying digital content. The Verification Handbook is free online, and was produced by the European Journalism Centre with contributions from an all-star cast of journalists.


In their introduction, Craig Silverman and Rina Tsubaki write that verification is “about achieving a harmony of two core elements: Preparing, training and coordinating people in advance and during an emergency; and providing them with access and resources to enable them to take full advantage of the ever-evolving tools that can help with verification.” The Verification Handbook’s greatest strength is its dual focus on tools and tips alongside process and culture.

The handbook has 10 chapters broken up around basic newsroom tasks such as verifying user-generated content, verifying images, verifying videos, crowdsourcing verification and computer-driven verification. Each of the chapters includes at least one case study. I thought Mathew Ingram’s profile of Andy Carvin’s work, Malachy Browne’s case studies on verifying video at the Boston Marathon bombing, and Anthony DeRosa’s use of Twitter as a police scanner when a plane went down in the Bronx were particularly useful.

These case studies are valuable to understanding how these tools and techniques actually work in a real newsroom. Writing for Nieman Reports about the role of social media in the aftermath of the Boston Marathon bombing, Hong Qu argued, “Tools and processes for assessing source credibility need to catch up with ever evolving social media technology and culture.” Together, the case studies included in the Verification Handbook highlight this tension. They give the sense that verification is often a matter of negotiating competing cultures and contexts: the fast-paced culture of social media, the high pressure culture of newsrooms, the culture of the community where the breaking news is happening, and the cultural context of the individual journalists themselves. These negotiations are happening through the lens of technology, tools and social platforms that are themselves constantly changing and evolving.


While the editors position the Verification Handbook as a resource for anyone — “journalists, emergency responders, citizen reporters and everyone else” — it is clearly designed and tailored for the news and media professional. There are a few good examples from human rights work, but the majority of the book is aimed at newsrooms.

And yet, one of the most interesting parts in the book for me was how reporters and their communities can and should work together to support more accurate and reliable news and information. In their introduction, Silverman and Tsubaki write:

“The public relies on official sources such as news organizations, emergency services and government agencies to provide credible, timely information. But, at the same time, these organizations and institutions increasingly look to the public, the crowd, to help source new information and bring important perspective and context.”

When it works, it creates what the editors call a “virtuous cycle,” but it can quickly turn into a vicious circle when errors spiral and spread through news and social media, each amplifying the other.

In my experience, people who are deeply engaged on social media want to be trustworthy participants in media. They are hungry for more training on how to be skeptical media consumers, to assess the content they are seeing online and how to handle errors and mistakes when they arise. To that end, “On The Media” created a“Breaking News Consumer’s Handbook.” It’s a good start, but the Verifcation Handbook shows that much more could be done.

As more and more people take up the tools of journalism, there is a huge opportunity for journalists to openly and honestly engage their communities around these issues. Journalists can help model best practices and train their communities in verification tools and techniques. The benefits to communities are access to skilled media professionals who can help them create, curate and make sense of the flood of information they are faced with daily. The benefits to newsrooms are communities who are better trained and prepared to help be sources and collaborators when news breaks, and who feel a greater affinity and trust for their local newsroom.

In his chapter, Mathew Ingram suggests journalists build relationships with their communities that go beyond traditional notions of sources.

“It’s not just a tool or a process that allows you to do your job or complete a task faster and more efficiently,” he writes. “It’s a collaborative effort, and you should be prepared to give as much as you receive.” If journalists leverage the opportunity of supporting their communities around news and media literacy in this way, then the lessons of this book could reach far beyond newsrooms.


The book opens and closes with verification fundamentals and concrete checklists for people to use in their own newsroom. The last chapter is a comprehensive list of verification tools. For many, these elements of the text will be most immediately useful, but the authors are careful not to suggest that anything in the book be seen as a one-size-fits-all solution.

In his chapter on fundamentals, Steve Buttry reminds the reader, “Verification is the essence of journalism, but it also illustrates the difficulty of journalism and the need for high standards: The path to verification can vary with each fact.” As such, verification is more of an art than a science, and so the Verification Handbook is more about asking questions than providing answers. Asking questions is, after all, at the heart of journalism, and today we have more people than ever to ask those questions with.

A year ago I started curating links to the best writing and case studies on verification of social media content because there was no central hub of information on these issues. A few months later I began to highlight verification tools. The new Verification Handbook pulls the best of these debates and discussions together in one place. It is important for that fact alone, but also because it further establishes the work of verification as a central part of how journalism is done today.

This post originally appeared at PBS MediaShift

1 Comment

Leave a Comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s