From under the microscope: A FAQ about Sen. Coburn’s report on “frivolous” research at NSF

Last week Sen. Tom Coburn (R-OK) released a report attacking the National Science Foundation for waste, fraud, and mismanagement. Included in his report is a long list of research projects which he deemed frivolous and a waste of tax-payer money. I was quite surprised to learn that a recent paper I wrote with Rob Goldstone about social influence and individual decision making was one of the projects called out in the report. I have since been contacted by a number of people requesting information about NSF’s involvement in the research. In response, I worked up a short frequently asked questions (FAQ) which I thought I’d share here.

 

1. Coburn’s report alleges that “Armed with a $1 million grant from NSF, researchers at Indian (sic) University-Bloomington and New York University analyzed baby names to determine trends in parents’ naming decisions.” Did Indiana and NYU actually receive $1 million from NSF to study baby naming?

No.

First, NYU did not receive any funds from NSF related to this project as I was not a PI or Co-PI on the grant in question.

Second, the NSF grant in question was awarded to a co-author of the paper (Robert Goldstone from Indiana University). The footnote included in Coburn’s report references NSF award #0910218. The title of this grant is “Transfer of perceptually grounded principles” and originated from NSF’s Division of Research on Learning in Formal and Informal Settings. Thus, the grant was not to study names as claimed in the report. This fact alone largely undermines the criticism.

More troubling is that, actually, this particular grant was never even acknowledged in connection with the baby names research. Instead, we acknowledged NSF Award #0527920, a grant awarded to Rob Goldstone concerning “the use of interactive computer simulations to teach scientific concepts governing complex adaptive systems.” (Total award amount for this grant was $196,086 over 4 years, a far cry from $1 million). Again, as is clear from the abstract and title of this award, this grant was not to specifically fund baby naming research.

In contrast to the claim in the report, nobody received $1 million from NSF to specifically fund the research reported in Gureckis & Goldstone (2009).

Rob credited NSF in our paper because the paper deals with complex systems: namely, cultural transmission systems. Since the aforementioned grant was designed to improve the way we teach students about such complex scientific phenomena, he included a mention of his support in the paper. This paper counts as a “synergistic” activity related to the primary focus of Goldstone’s awarded project. Goldstone has also published many highly cited peer-reviewed papers on student learning which credit his NSF support (a list appears in the above link). Note that NSF requires all products of funded research to acknowledge said funding even if the ideas are only partially related to the original award. The paper also acknowledged funding from NIH/NIMH (a Mathematical Modeling Training Grant awarded to Indiana University which paid for my post-doc when I was at Indiana University), and the Department of Education. Other aspects of the writing of the paper were supported by private funds given to me by NYU when I first took my job here.

Thus, the claim in the report is objectively false, sensationalized, but also suggests a troubling lack of understanding about how scientific research is funded (confusing the multiple products of that research for the grant itself). The bottom line is that nobody received $1 million from NSF to study naming behavior (although, as described below, there is no reason why research on this topic shouldn’t be funded). The actual cost to the tax payer through NSF to produce this particular research report had to be less than $300 (and was quite possibly $0) but NSF was credited out of an abundance of caution (and thanks) for their support.

2. Did Coburn or his staff even look at the scientific paper in question?

Rather than examine the actual peer-reviewed research paper (Gureckis, T.M. and Goldstone, R.L. (2009) How You Named Your Child: Understanding The Relationship Between Individual Decision Making and Collective Outcomes. TopiCS in Cognitive Science, 1 (4), 651-674. available for download here), the Coburn report exclusively references a USA Today article written by an individual not involved in the original research. This news story is not an authoritative source on the contributions of our scientific paper. It would be like referring people to a left-wing blog to describe Coburn’s stance on political issues instead of letting them look at his own voting record or website.

The Coburn report exclusively references a USA Today article written by an individual not involved in the original research.

Had those developing this report actually looked at the research paper they were criticizing, they would know that we were not specifically interested in baby names except in so far as they offer a unique opportunity for studying such the impact of social influence on decision making. We all know that iPhones are popular but the underlying reasons for this cultural success is distorted by the role that advertising budgets and existing computer technologies play in determining which ideas win out and which die off in the consumer marketplace. In contrast, the popularity of names is more organically determined by processes of social influence (there is no company out there trying to convince you to name you child something in particular). Baby names thus represent an important and relatively “pure” empirical test of theories of cultural transmission and social influence in large groups.

The Coburn report makes it seem as though this research spent money to determine the frequency and popularity of names. Fortunately, this data was provided for free by the Social Security Administration which has recorded and published the most popular baby names in the United States since the 1880s (freely available here: http://www.ssa.gov/oact/babynames/). Many of the popular websites that analyze naming trends rely on the same data source. Any NSF funds used toward this effort paid exclusively for the statistical/mathematical analysis of this data. In fact, in the context of a discussion about government waste, this is a great example of government efficiency since data collected for one purpose (issuing social security cards), which would have been very expensive to collect otherwise, turns out to be very useful to NSF and NIH supported peer-reviewed science.

Note that many researchers agree that this data is unique for studying the interactions of individual decision making and social behavior. Similar analyses on the same data set were nearly simultaneously reported with our paper in esteemed peer-reviewed journals like Proceedings of the National Academy of Sciences (Berger & Le Mans, 2009), the Proceedings of the Royal Academy (Hahn & Bently, 2003), among other peer-reviewed journals (Bentley, Lipo, Herzog, & Hahn, 2007; Fryer & Levitt, 2004) and naming trends and patterns have been extensively studied and discussed by economists (Steven Levitt and Steven Dubner in the best selling book Freakonomics) and sociologists (Stan Lieberson in A Matter of Taste: How Names, Fashion, and Culture Change). The work we published was peer-reviewed in a journal by scientific experts and went through multiple revisions with extensive debate.

Our paper reports novel findings which suggest a refinement of leading theories of cultural transmission of ideas. The report gets the basic finding from our research wrong when it claims our conclusion was the tautology “popular names are popular with parents.” If only it was so simple. One prediction of the idea that “popular names are popular” would be that the most popular names would never change from year to year (the same popular names would keep being popular). In fact, the historical record provided by the Social Security Administration shows that there has been dramatic changes in the popularity of names over the last few years. Our paper proposes and evaluates possible reasons for these changes in time. Our theory is rigorous and mathematically specified, and may thus be used by other researchers studying the cultural transmission of other ideas (such as political ideologies, health-related habits and decisions, or purchasing decisions). The ideas in the paper borrow from recent mathematical theories of human decision making and learning as well as cultural transmission and cultural evolution.

As authors, we made a concerted effort to communicate the broader impacts of this work to the public at large. The paper is available for free from my website (http://gureckislab.org), and both NYU and Indiana University jointly issued a very nice press release with details and discussion about the merits of the paper which went far beyond the third-party source the Coburn report extensively quotes (the USA Today article).


3. Why publish a paper about naming patterns in the first place? Is this a
useful scientific topic?

It is easy to be distracted by the seemingly trivial nature of “baby naming.” However, as noted above and in the paper, we did not choose this topic for frivolous reasons. Baby naming just happens to be a cultural practice for which there are extensive historical records about the aggregate decisions of millions of individuals. Thus, it provides an important domain in which to test theories of how other people influence our opinions, decisions, and judgements. These theories are far from trivial and involve detailed mathematical arguments about how the distribution of cultural tokens (such as names) should change in response to societal forces. Research should not be singled out simply for pursing theoretically motivated research that just happens to reference a popular culture phenomena. In fact, this feature of this work helps many Americans recognize the potential of NSF funded research to transforming our understanding of the world around us.

Baby naming represents a unique cultural domain in which to test theories of how other people influence our opinions, decisions, and judgements.

While Coburn’s report suggests this research is obvious or trivial, many areas of both public and private research funding are currently very interested in this type of research. For example, understanding the factors that influence how ideas spread through a human groups may help our military better influence the “hearts and minds” of people we are trying to help. In addition, it is noted that social influence has an effect on the health decisions that people make (e.g., Christakis & Fowler, 2007). The application of the ideas in this research may be later used to enact positive societal outcomes. We find evidence that names go through boom and bust cycles not unlike the recent economic bubbles that lead to the current budget situation. Understanding the factors that contribute to these bubbles could be important in preventing these events in the future.


4. Should NSF support for social and behavioral sciences be eliminated?

Coburn’s report recommends that funding for social and behavioral sciences should be terminated within NSF (page 53). Coburn cites the past successes of astronomy, biology, chemistry, and physics as examples of the important research that NSF supports. As someone originally trained as an electrical engineer, I couldn’t agree more that basic research in these areas deserves continued investment. However, investing in basic research based primarily on past success is bad science policy.

Investing in basic research based only on past success is bad science policy.

Many of the future challenges that face our society have to do with human behavior. For example, how can we get people to make better decisions for their health? How does the brain contribute to behavior? How can we best intervene to improve student learning and retention? How can we develop better treatments for language disorders or developmental disabilities? How do trends propagate through society and how might these contribute to “bubble”-like market phenomena? Research funded under NSF’s Social, Behavioral, and Economics Sciences initiative supports ground-breaking research on many of these issues.

Contrary to the impression given in the report, NSF’s SBE includes divisions for “hard” sciences (such as the National Center for Science and Engineering Statistics), decision making, economics, and computational social science (under the division of Social and Economic Sciences), cognitive neuroscience, linguistics, and developmental and learning sciences (under the Behavioral and Cognitive Sciences division), as well as numerous programs focused on improving science education in our country (such as the Research Experiences for Undergraduates program and the Science of Learning Centers). These are not “frivolous” topics but important areas of research that contribute greatly to our economy, our health, our status as a worldwide leader in scientific research, and our national security.

Critically, NSF funds basic research of theoretical importance which has the potential to make truly transformative progress. I would argue that many of the advances in science that we will be talking about in future generations will come from further development of a detailed, quantitative, and mathematically-based science of human behavior, exactly of the type exemplified by the baby naming paper.

Citations:

Bently, R.A., Lipo, C.P., Herzog, H.A., Hahn, M.W. (2007) Regular rates of popular culture change reflect random copying. Evolution and Human Behavior, 28(3), 151-158.

Berger, J., Le Mens, G. (2009), How Adoption Speed Affects the Abandonment of Cultural Tastes, Proceedings of the National Academy of Sciences, 106, 8146-8150.

Fryer, R.G. and Levitt, S.D. (2004) The causes and consequences of distinctively black names. Quarterly Journal of Economics, 119(3), 767-805.

Gureckis, T.M. and Goldstone, R.L. (2009) How You Named Your Child: Understanding The Relationship Between Individual Decision Making and Collective Outcomes. Topics in Cognitive Science, 1 (4), 651-674.

Hahn, M.W. and R.A. Bentley (2003) Drift as a mechanism for cultural change: An example from baby names. Proceedings of the Royal Society London B, Biology Letters, 270:S120-S123.

Christakis, N.A. and Fowler, J.H. (2007) “The Spread of Obesity in a Large Social Network Over 32 Years,” New England Journal of Medicine. 357(4): 370-379.







  1. Just had the opportunity to read the blog on Sen Coburn’s erred ‘assessment’ of frivolous research projects. A very well stated and evidence-based defense of the significance of your research findings on selection of baby names. I bet the OK and DC lobbyists would disagree with his (obvious staff written) report if they ever read it. Instead, I bet they realize the importance of learning how to effectively win the ‘hearts and minds’ in the congressional offices they visit daily. Maybe this area of research is not as ‘frivolous’ as some would want us to believe.
    Thanks for the defense of solid research!

    KG, DMD
  2. How thrilling to learn that the NSF supports scientific research intended for the benefit of human beings and voting citizens. Perhaps Mr. Coburn would like to talk to the District Attorney in Philadelphia, with the highest homicide rate in the nation, about the correlation between parenting and societal problems urgently in need of solutions? Or perhaps he is unaware of the paltry half billion dollars recently committed toward ameliorating the effects of abusive and/or poverty-stricken parents?

    http://www.ustream.tv/recorded/14949645

    Q – Could you reveal your research w/r/t the popularity of a particular name, say, Thomas? Has it risen in popularity in recent decades, or fallen? When parents choose that name, are they fating their child to become a “doubting Thomas” requiring personal experience rather than accepting scientific proofs as evidence?

    jana brubaker
  3. Thanks for the comment. You might check out http://www.babynamewizard.com/ to look at the popularity of particular names (like Thomas). It’s pretty fun to explore.

    Todd Gureckis
  4. Read your paper? I don’t think he read his.

    I refer you to the bottom of page 35:

    “The researchers did find that dick shrimp did not perform as well and did not recover as well from exercise as healthy shrimp.”

    http://coburn.senate.gov/public/index.cfm?a=Files.Serve&File_id=f6cd2052-b088-44c3-b146-5baa5c01552a

    MB
  5. [...] Making and Collective Outcomes.” Gureckis was surprised at Coburn’s criticism, and responded on his website: “The Coburn report makes it seem as though this research spent money to determine the [...]

    I feel ya, Gureckis « Measure of Doubt
  6. as you point out, possibly the most infuriating part of this is that he is generating his recommendations based on third party sources. i think i might start numerous fake news websites about how nonparametric Bayesian modeling has saved babies, baby seals, lowers oil prices, and spreads democracy.

    good luck fighting the good fight. he really did pick on social sciences in particular.
    =joe

    josephausterweil
  7. [...] The Coburn report blamed one of my papers (co-authored with a bunch of other people in our lab) as being a waste of taxpayer money and framed it in such a misleading manner that I initially did not think it was worth responding to. Apparently, many other researchers were also “attacked” with factually incorrect interpretations of their work (see Gureckis’ post on his research on baby names). [...]

    Why should we provide funding to social science research « Arctic Penguin
  8. I’m going to guess that he doesn’t care whether he characterized your work correctly, or anyone else’s. Neither will his primary audience. None of them will read your reply, or care about what you said even if they did read it. He wants to cut the funding, and what he said will probably convince some people to support him; job accomplished. The only type of reply that would hold his interest is a political one.

    Aaron
  9. That’s probably true, sadly. However, searching for “coburn report nsf” in google, the third link is a BoingBoing.net post linking to this page. So, at least the information is out there for people who care to learn. I think you are right though that the only solution is to vote for pro-science legislators. This isn’t something most scientists care to get involved with, but it important for continued public funding of basic research. Also, my understanding is that Coburn’s report was largely ignored when it was released by the congressional panel in charge of science funding (he is not a member of this committee so his influence on this issue is limited).

    Todd Gureckis