Dark Data: Why What You Dont Know Matters

(Paperback)

Available Formats

Publishing Details

Classifications

Physical Properties

Description

A practical guide to making good decisions in a world of missing data In the era of big data, it is easy to imagine that we have all the information we need to make good decisions. But in fact the data we have are never complete, and may be only the tip of the iceberg. Just as much of the universe is composed of dark matter, invisible to us but

Reviews

"[A] penetrating study of missing (dark) data and its impacts on decisionsskewing stats, enabling fraud, embedding inequity and triggering preventable catastrophes. Advocating data science judo, Hand offers expert training, from recognizing when facts are being cherry-picked to designing randomized trials. A book illuminating shadowed corners in science, medicine and policy."---Barbara Kiser, Nature
"A tour de force. . . . Hand is a good and able guide to take us through the many aspects of dark data that are potentially skewing our understanding of real world observations and potential scientific breakthroughs. He writes in an accessible and understandable way too."---Simon Cocking, Irish Tech News
"Well-written and accessible."---Tim Harford, Undercover Economist
"You need to read [Dark Data], and be convinced by Davids reasoning and his examples of cases in which unseen or unreported data play a critical and sometimes even a fatal role. You are likely to walk away with the feeling that the term dark data is indeed a very effective one to arouse both curiosity and suspicion, mixed with happiness that finally a great term was coined by a statisticianand sadness that the statistician is not you."---Xiao-Li Meng, IMS Bulletin
"An exploration of a major problem in data analysis with an attempt of classification, analysing causes, mechanisms, and to some extent also suggest mitigations."---Adhemar Bultheel, European Mathematical Society
"An excellent guide to the many reasons for caution in interpreting data."---Diane Coyle, Enlightened Economist

Author Bio

David J. Hand is emeritus professor of mathematics and senior research investigator at Imperial College London, a former president of the Royal Statistical Society, and a fellow of the British Academy. His many previous books include The Improbability Principle, Measurement: A Very Short Introduction, Statistics: A Very Short Introduction, and Principles of Data Mining.