Patients and doctors often need to make decisions based on the results of medical tests. When these results are presented in the form of conditional probabilities, even doctors find it difficult to interpret them correctly. There is over 20 y of research supporting the finding that people are better able to calculate the correct positive predictive value of a test when given information in natural frequencies, as opposed to conditional probabilities. Natural frequencies are one of a few psychological tools that have made it into evidence-based medicine. Recently, Pighin and others (Med Decis Making 2016;36:686–91) argued that natural frequencies could hinder informed decision making, a critique based on a single task and a crude scoring criterion we refer to as the 50%-Split. Our commentary addresses these criticisms based on three analyses. First, we show how the 50%-Split scoring used by Pighin and others misclassifies known errors, such as solely attending to the hit rate (true-positive rate) of the test, as strategies that support understanding. Second, we reanalyze data from 21 additional problems completed by various participant groups to show that their scoring criterion does not support their results in 19 out of 21 cases. Third, we apply the mean deviation scoring method and show that, when given information in natural frequency formats, participants provide estimates that are closer to the correct Bayesian solution than for conditional probability formats. In each analysis, natural frequencies lead to more correct judgements and therefore promote informed decision making relative to conditional probabilities. We welcome further discussions of performance metrics that can provide insight into how the public and therefore patients understand the implications of medical test results.
Background. Although people are likely to underestimate the frequencies of risks to health from common diseases and overestimate those from rare diseases, we still do not know much about reasons for this systematic bias, which is also referred to as “primary bias” in the literature. In this study, we take advantage of a series of large epidemics of mosquito-borne diseases to examine the accuracy of judgments of risk frequencies. In this aim, we assessed the perceived v. observed prevalence of infection by Zika, chikungunya or dengue fever during these outbreaks, as well as their variations among different subpopulations and epidemiological settings. Methods. We used data drawn from 4 telephone surveys, conducted between 2006 and 2016, among representative samples of the adult population in tropical regions (Reunion, Martinique, and French Guiana). The participants were asked to estimate the prevalence of these infections by using a natural frequency scale. Results. The surveys showed that 1) most people greatly overestimated the prevalence of infection by arbovirus, 2) these risk overestimations fell considerably as the actual prevalence of these diseases increased, 3) the better-educated and male participants consistently yielded less inaccurate risk estimates across epidemics, and 4) these biases in the perception of prevalence of these infectious diseases are relatively well predicted by the probability weighting function developed in the field of behavioral decision making. Conclusions. These findings suggest that the primary bias, which has been found in laboratory experiments to characterize a variety of probabilistic judgments, equally affects perception of prevalence of acute infectious diseases in epidemic settings. They also indicate that numeracy may play a considerable role in people’s ability to transform epidemiological observations from their social environment to more accurate risk estimates.