You are here
Home > Big Data: Tools are not a substitute for thought
- Albert Shar [sharethis]

Would you be surprised if you flipped a coin seven times and got seven heads? You should be since the probability of that occurring with a fair coin is less than 1% (.0078125). Should you still be surprised if a told you that that streak of seven heads was part of a sequence of 1,000 coin tosses? You shouldn’t be since it’s a virtually certainty that you’d get that result. And therein lies one of the problems with trying to make decisions around big data (not that 1,000 data points has much to do with big data).

I recently wrote about my concern about people misinterpreting what superficially appear to be actionable results that come from mining big data. That was just part of the problem since big data more and more relies on complex analysis and extraction methods that can obscure thelogic behind the results. There is a school of thought that argues that big data results eliminate the need for theory. This is evident in the search for new drugs via high-throughput screening where candidates are discovered by testing for reactions in literally millions of micro-experiments. While this may be valuable in some circumstances, I believe that there on some inherent dangers that must be addressed.

  1. As alluded to in the first paragraph, it is important to understand the probabilities in big data experiments. Rare events will occur with high probability. It is important to not look at just superficial results.
  2. Big data techniques are exceptionally good for developing hypotheses. They are not, in and of themselves, proofs.
  3. In some highly sensitive situations, it is not sufficient to demonstrate that something is true. It is critical to provide an underlying logic to that demonstration. This is most often the rule in health interventions. Sophisticated big data tools most often fail along this dimension.

The best course of action is to think carefully about the purpose of your work and act in a way that’s consistent with that purpose. If you’re looking for interesting hypotheses, think about the methods for testing them. If you’re looking to test and implement an intervention, consider the costs of being wrong. The degree of certainty needed to act should consider the consequences of all possible outcomes. If you’re using complex tools, you may need to develop simpler models to justify the logic of even valid findings. These are not insurmountable obstacles; they are reasonable activities that go hand in hand with big data and their associated tools.

In short, powerful tools that one can unleash on interesting data repositories are not substitutes for thinking. A few years ago I was working in a facility with huge technology resources. I carefully set up an experiment to test an hypothesis, using essentially every tool we had to work overnight on the data. When I came in the next morning and scanned the results I realized that had I thought about the problem for five minutes, the answer was obvious and easily articulated. Just because we have cannon doesn’t mean that they are the right tool.

Linda Musthaler About the Author
Albert Shar, PhD, is a principal at QERT (, a technology consultancy and serves as vice president the Robert Wood Johnson Foundation. Shar previously worked as director for IT Research and Architecture at the RWJ Pharmaceutical.
Research Institute, a J&J company. He has also served as CIO at the University Of Pennsylvania School Of Medicine. He holds a patent in medical imaging and is the author of more than 50 articles in medical informatics, computer science, and pure and applied mathematics.

188 thoughts on “Big Data: Tools are not a substitute for thought

  1. Pingback: 2019
  2. Pingback:
  3. Pingback: #macron #Lassalle
  4. Pingback: a2019-2020
  5. Pingback: facebook
  6. Pingback: facebook1
  7. Pingback:
  8. Pingback:
  9. Pingback: hs;br
  10. Pingback: tureckie_serialy
  11. Pingback:
  12. Pingback: +1+
  13. Pingback: watch
  14. Pingback: HD-720
  15. Pingback: 2020
  16. Pingback: Video
  17. Pingback:
  18. Pingback: Watch TV Shows
  19. Pingback: Kinokrad
  20. Pingback: filmy-kinokrad
  21. Pingback: kinokrad-2019
  22. Pingback: serial
  23. Pingback:
  24. Pingback:
  25. Pingback: dorama hdrezka
  26. Pingback: movies hdrezka
  27. Pingback: HDrezka
  28. Pingback: kinosmotretonline
  29. Pingback: LostFilm HD 720
  30. Pingback: bofilm
  31. Pingback: 1 seriya
  32. Pingback:
  33. Pingback:
  34. Pingback:
  35. Pingback:
  36. Pingback:
  37. Pingback:
  38. Pingback:
  39. Pingback:
  40. Pingback: See-Season-1
  41. Pingback: Evil-Season-1
  42. Pingback: Evil-Season-2
  43. Pingback: Evil-Season-3
  44. Pingback: Evil-Season-4
  45. Pingback: Dollface-Season-1
  46. Pingback: serial 2020
  47. Pingback: Dailymotion
  48. Pingback: Watch+movies+2020
  49. Pingback:
  50. Pingback:
  51. Pingback: #1plus1
  52. Pingback: 1plus1
  53. Pingback: Film
  54. Pingback: Film 2020
  55. Pingback: Film 2021
  56. Pingback: human design
  57. Pingback: dizajn cheloveka
  58. Pingback: human-design-space
  59. Pingback: koma 2020
  60. Pingback: The-Gentlemen
  61. Pingback: led-2
  62. Pingback: pod-vodoi
  63. Pingback: vk 2020
  64. Pingback: DSmlka
  65. Pingback: viagra
  66. Pingback: viagra online
  67. Pingback: +
  68. Pingback: ¯jak Son³k
  69. Pingback: astrolog
  70. Pingback: generic cialis
  71. Pingback: cialis 20mg
  72. Pingback: viagra generic
  73. Pingback: viagra online usa
  74. Pingback:
  75. Pingback: cialis
  76. Pingback: viagra pills
  77. Pingback: buy cialis
  78. Pingback: generic viagra
  79. Pingback: buy cialis cheap
  80. Pingback: levitra
  81. Pingback: cialis pills
  82. Pingback: strelcov 2020
  83. Pingback: film t-34
  84. Pingback: online pharmacy
  85. Pingback: canadian pharmacy
  86. Pingback: sildenafil citrate
  87. Pingback: sildenafil
  88. Pingback: cialis generic
  89. Pingback: Beograd film 2020
  90. Pingback: viagra online
  91. Pingback: cialis
  92. Pingback: buy viagra
  93. Pingback: viagra
  94. Pingback: psiholog
  95. Pingback: cialis online
  96. Pingback: psyhelp_on_line
  97. Pingback: coronavirus
  98. Pingback: prices of cialis
  99. Pingback: PSYCHOSOCIAL
  100. Pingback: buy cialis pills
  101. Pingback: generic viagra
  102. Pingback: cialis cost
  103. Pingback: levitra online
  104. Pingback: Us viagra
  105. Pingback: film doktor_liza
  106. Pingback: cbd
  107. Pingback: djoker film
  108. Pingback: generic viagra
  109. Pingback: buy viagra online
  110. Pingback: cialis 5mg
  111. Pingback: generic ed pills
  112. Pingback: natural viagra
  113. Pingback: cialis 5mg
  114. Pingback: tylenol otc
  115. Pingback: viagra 50mg
  116. Pingback: viagra 100mg
  117. Pingback: cialis free trial
  118. Pingback: viagra generic
  119. Pingback: buy cialis
  120. Pingback: tadalafil 20 mg
  121. Pingback: buy viagra
  122. Pingback: amoxicilina 500 mg
  123. Pingback: viagra for sale
  124. Pingback: buy cialis online
  125. Pingback:

Comments are closed.