Subscribe
Prev | Table of Contents | Next
Science
Vol. 342 no. 6163 p. 1169
DOI: 10.1126/science.1246680
  • Technical Comments

Comment on “Poverty Impedes Cognitive Function”

  1. Annemarie Zand Scholten2
  1. 1Jelte Wicherts, Department of Methodology and Statistics, Tilburg University, P.O. Box 90153, 5000 LE, Tilburg, Netherlands.
  2. 2Department of Child Development and Educational Sciences, University of Amsterdam, Amsterdam, Netherlands.
  1. ↵*Corresponding author. E-mail: j.m.wicherts{at}uvt.nl

Abstract

Mani et al. (Research Articles, 30 August, p. 976) presented laboratory experiments that aimed to show that poverty-related worries impede cognitive functioning. A reanalysis without dichotomization of income fails to corroborate their findings and highlights spurious interactions between income and experimental manipulation due to ceiling effects caused by short and easy tests. This suggests that effects of financial worries are not limited to the poor.

Mani et al. (1) recently presented four laboratory experiments and a field study that aimed to show that poverty impedes cognitive functioning. We criticize their results on statistical and psychometric grounds.

Mani et al. ran three randomized experiments in which U.S. adults were assigned to read one of two sets of financial scenarios that differed in their activation of financial concerns. Although participants’ income varied from $7560 to $160,000, Mani et al. used a median split to analyze income data. This procedure has been criticized strongly for being associated with lower power, loss of information on individual differences, and its inability to pinpoint nonlinear relations (2). Of the two measures of cognitive functioning in Mani et al.’s studies, only the Raven’s scores are fairly symmetrically distributed. We therefore submitted these data to linear regressions involving family income (mean-centered to facilitate interpretation) and an interaction between income and the type of scenario. Results are given in Table 1. In none of the three core experiments (1, 3, and 4) was the interaction significant when analyzed without unnecessary dichotomization of income. We also analyzed data from study 2, which aimed to show that the effect of poverty-related worries could be distinguished from a form of test anxiety and would not occur in similar, but nonfinancial, scenarios. We note that the second experiment is appreciably smaller (N = 39 people) than the other three experiments (N > 95 people) and so is associated with lower statistical power. Of importance are the regression weights; those from study 2 are not appreciably different than those in the core studies.

View this table:
Table 1 Linear regressions of Raven’s accuracy on mean-centered income and scenario and the interaction between income and scenario.

Income is mean-centered to improve interpretability and avoid multicollinearity. Conditional and unconditional bootstrapping corroborated these results. B indicates unstandardized regression weight, with standard error (SE).

The second measure of cognitive functioning employed by Mani et al., cognitive control, showed non normal distributions that render them unsuitable for linear analyses (see Fig. 1). The measure was developed specifically to assess cognitive control among children and showed clear ceiling effects, as it did in earlier work involving adults (3). Because higher-income adults outperform lower-income adults, the easiness of the control test is particularly problematic in the higher-income range; more than half of the participants in the above-median income group acquired a perfect or near-perfect score (11 or 12 correct out of 12 items). In fact, the negative skew was so extreme that satisfactory normalization of the scores using a Box-Cox transformation was impossible. However, if the transformed, platykurtotic scores are subjected to a linear regression, the interaction is no longer significant in two of the three core experiments. Had the test been able to discriminate between higher levels of cognitive control, the difference between financial scenarios might have been established for the rich participants also. Hence, the core interaction that was meant to indicate that the poverty-related scenario only affected the poor may be an artifact of the cognitive control test’s being too easy (4, 5). Latent variable modeling could be used to deal with such issues (6–8).

Fig. 1
View larger version:
Fig. 1 Histograms of scores on the Cognitive Control test scores for easy and hard scenarios and the above- and below-median income groups in studies 1 to 4 from Mani et al.

We note that a highly relevant potential confound in the field study presented by Mani et al. is the possibility of retesting effects. The lack of any retesting effect in Mani et al.’s field study involving Indian farmers is clearly at odds with one of the more robust findings in the literature on cognitive testing (9). Retesting effects on the Raven’s tests are particularly profound among test-takers with little education (10).

Mani et al. go beyond the data by concluding that “The poor…are less capable not because of inherent traits, but because the very context of poverty…impedes cognitive capacity.” We note that the correlation between income and IQ also appears in longitudinal studies in which IQ was measured years before incomes (11). Further research is needed to fully grasp whether poverty indeed affects cognitive performance, as proposed by Mani et al., or whether the effect found in their experiments is a test artifact.

The stronger cognitive impediment experienced by the poor could merely be the result of an inappropriate statistical test and an overly easy cognitive control measure. The latter could obscure an equally “threatening” effect in the rich, simply because they were unable to obtain higher scores when not threatened. With such methodological issues remaining to be addressed, the authors’ proposal of far-reaching policy changes, such as timing HIV educational campaigns to harvest cycles, seems premature.

  • Received for publication 1 October 2013.
  • Accepted for publication 29 October 2013.

References

  1. ↵
    1. A. Mani,
    2. S. Mullainathan,
    3. E. Shafir,
    4. J. Zhao
    , Poverty impedes cognitive function. Science 341, 976–980 (2013). doi:10.1126/science.1238041 pmid:23990553
    Abstract/FREE Full Text
  2. ↵
    1. R. C. MacCallum,
    2. S. Zhang,
    3. K. J. Preacher,
    4. D. D. Rucker
    , On the practice of dichotomization of quantitative variables. Psychol. Methods 7, 19–40 (2002). doi:10.1037/1082-989X.7.1.19 pmid:11928888
    CrossRefMedlineWeb of ScienceGoogle Scholar
  3. ↵
    1. M. C. Davidson,
    2. D. Amso,
    3. L. C. Anderson,
    4. A. Diamond
    , Development of cognitive control and executive functions from 4 to 13 years: Evidence from manipulations of memory, inhibition, and task switching. Neuropsychologia 44, 2037–2078 (2006). doi:10.1016/j.neuropsychologia.2006.02.006 pmid:16580701
    CrossRefMedlineWeb of ScienceGoogle Scholar
  4. ↵
    1. S. Kang,
    2. N. Waller
    , Moderated multiple regression, spurious interaction effects, and IRT. Appl. Psychol. Meas. 29, 87–105 (2005). doi:10.1177/0146621604272737
    Abstract
  5. ↵
    1. S. Embretson
    , Item response theory models and spurious interaction effects in factorial ANOVA designs. Appl. Psychol. Meas. 20, 201–212 (1996). doi:10.1177/014662169602000302
    Abstract/FREE Full Text
  6. ↵
    1. J. M. Wicherts,
    2. C. V. Dolan,
    3. D. J. Hessen
    , Stereotype threat and group differences in test performance: A question of measurement invariance. J. Pers. Soc. Psychol. 89, 696–716 (2005). doi:10.1037/0022-3514.89.5.696 pmid:16351363
    CrossRefMedlineWeb of ScienceGoogle Scholar
    1. J. M. Wicherts,
    2. W. Johnson
    , Group differences in the heritability of items and test scores. Proc. Biol. Sci. 276, 2675–2683 (2009). doi:10.1098/rspb.2009.0238 pmid:19403538
    Abstract/FREE Full Text
  8. ↵
    1. J. M. Wicherts,
    2. A. Zand Scholten
    , Test anxiety and the validity of cognitive tests: A confirmatory factor analysis perspective and some empirical findings. Intelligence 38, 169–178 (2010). doi:10.1016/j.intell.2009.09.008
    CrossRefWeb of ScienceGoogle Scholar
  9. ↵
    1. J. P. Hausknecht,
    2. J. A. Halpert,
    3. N. T. Di Paolo,
    4. M. O. Moriarty Gerrard
    , Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. J. Appl. Psychol. 92, 373–385 (2007). doi:10.1037/0021-9010.92.2.373 pmid:17371085
    CrossRefMedlineWeb of ScienceGoogle Scholar
  10. ↵
    1. J. M. Wicherts,
    2. C. V. Dolan,
    3. J. S. Carlson,
    4. H. L. J. van der Maas
    , Raven’s test performance of sub-Sarahan Africans: Average performance, psychometric properties, and the Flynn Effect. Learn. Individ. Differ. 20, 135–151 (2010). doi:10.1016/j.lindif.2009.12.001
    CrossRefGoogle Scholar
  11. ↵
    1. T. Strenze
    , Intelligence and socioeconomic success: A meta-analytic review of longitudinal research. Intelligence 35, 401–426 (2007). doi:10.1016/j.intell.2006.09.004
    CrossRefWeb of ScienceGoogle Scholar

ADVERTISEMENT

Headline Jobs