Folks who do not have a lot of experiences with systems that don’t work well for them find it hard to imagine that a well intentioned system can have ill effects. Not work as advertised for everyone. That is my default because that is my experience.
– Bashir, Advancing How Science is Done
A couple of months ago, a tenured white male professor* from an elite research university wrote a blog post about the importance of replicating priming effects, in which he exhorted priming researchers to “Nut up or shut up.”
Just today, a tenured white male professor* from an elite research university said that a tenured scientist who challenged the interpretation and dissemination of a failed replication is a Rosa Parks, “a powerless woman who decided to risk everything.”
Well then.
The current discussion over replicability and (more broadly) improving scientific integrity and rigor is an absolutely important one. It is, at its core, a discussion about how scientists should do science. It therefore should include everybody who does science or has a stake in science.
Yet over the last year or so I have heard a number of remarks (largely in private) from scientists who are women, racial minorities, and members of other historically disempowered groups that they feel like the protagonists in this debate consist disproportionately of white men with tenure at elite institutions. Since the debate is over prescriptions for how science is to be done, it feels a little bit like the structurally powerful people shouting at each other and telling everybody else what to do.
By itself, that is enough to make people with a history of being disempowered wonder if they will be welcome to participate. And when the debate is salted with casually sexist language, and historically illiterate borrowing of other people’s oppression to further an argument — well, that’s going to hammer the point.
This is not a call for tenured white men to step back from the conversation. Rather, it is a call to bring more people in. Those of us who are structurally powerful in various ways have a responsibility to make sure that people from all backgrounds, all career stages, and all kinds of institutions are actively included and feel safe and welcome to participate. Justice demands it. That’s enough for me, but if you need a bonus, consider that including people with personal experience seeing well-intentioned systems fail might actually produce a better outcome.
—–
* The tenured and professor parts I looked up. White and male I inferred from social presentation.
The Hardest Science 
Watching the replication “crisis” unfold has been very odd, in particular seeing people apparently attempt to one-up others in just how awesome they think pre-registration and replication is. It’s hard not to get the sense that some of them think they’re trying to make sure other people work up to their standards–” well there’s no concerns about *my* effects replicating!”.
There was a great question from a postdoc at APS about this switch to pre-reg and replication focus is ahead of any culture shift on how tenure review treats publication records.
I’m the tenured white professor who wrote the “nut up or shut up” article. This is not the first time that someone has indicated their displeasure with the title. I’m sorry that people may feel it is inappropriate. But I’m not apologizing for the title itself though, because that would be insincere. The title was a reference to Woody Harrelson, and it allowed me to incorporate zombies in my narrative. I believe that when it comes to the opportunity of adding zombies to your story, almost any title is excusable.
Now to the real issue, namely, the issue of minorities in academia. With respect to the replication debate at hand, all I personally care about is good arguments, regardless of whether they are put forward by tenured white professors, untenured female Asian PhD students, or even a bunch of zombies (OK, this is an unlikely scenario). Of course, science as a whole has a huge problem in as far as minorities are concerned. In the Netherlands, for instance, women are still very much underrepresented in academia. This is an important problem and we need to fix it. By the way, my own department contains many women actively involved in the replication debate, most noticeably Marjan Bakker and Dora Matzke. Their contribution was acknowledged many times during the APS session on replicability. By the way, Bobby Spellman has also played a pivotal role, and so does Christine Harris. In addition, the APS session “Improving Analyses and Interpretation of Published Results in Psychology” had three women (all PhD students) and one man. The Open Science Framework is also supported by many women. If you want to see minorities being underrepresented, I suggest you attend the annual meeting of the Society for Mathematical Psychology (one of the very best meetings in psychology, by the way).
Finally, I firmly believe that good science is *not* a question of taste, culture, or an issue that minorities may feel differently about. Cleanly separating hypothesis-generating from hypothesis-testing research is a matter of methodology 101. What happens when you don’t adhere to this distinction? Well, just look at the results from the Social Psychology special issue, and the intermediate results from the Reproducibility Project. It is evident that in the future, we need results that are more reliable than they are now. This will also be of concrete benefit for junior researchers who will not have to waste their time trying to replicate that which does not exist.
Cheers,
E.J. Wagenmakers
The topic of the post was diversity –of thought, experience, culture, race, ethnicity, and gender (for starters). Your “translation” of that to “minorities in academia” is an excellent example of the problem.
E.J., thanks for taking the time to respond. In making the point that women are underrepresented in this discussion, I did not mean to say that none are involved. So thank you for mentioning some of the women doing terrific work on this issue.
A great many women have had experiences being treated poorly because of their gender, both in the workplace and out of it. How they will respond to this specific instance will of course vary, and I don’t want this to spiral off into the specifics of that phrase (especially if that turns into two tenured male professors arguing in a comments thread; oh, the irony). But at least some find it off-putting to take part in a professional discussion where casual references to male genitalia are the norm. The fact that it was funny in a different context does not necessarily help. In fact, sometimes the opposite, as many women have had unpleasant experiences of being told to “lighten up” when they raise these issues.
You say you are interested in good ideas regardless of who has them. So let’s create an atmosphere where everybody with good ideas wants to participate.
I wrote a bit about this last year, after I had visited the symposium on Solid Science http://asefixesscience.wordpress.com/2013/06/14/but-where-oh-where-are-the-women/
The Nut Up didn’t bother me – actually didn’t quite reflect on that (I usually say Butch up, or shit or get off the pot or something similarly crude), but, then again, I’m one of the OSC bloggers, and, as a fellow alumni of IU Bloomington enjoyed the little Purdue Jab. Rosa Parks, though…
But, yes, I’m a bit bothered by the “chimpanzee politics” style of battling, which frequently is more attempts at style than substance, and I’m really bothered that reforming science seems to get lost in this, and I’m really bothered that so few of my majority group (women) are engaging.
Thanks, EJ!
E.J. – I think the point that Sanjay was making wasn’t that females or minorities would necessarily feel differently about good science but that there is a conversation taking place that they might not feel comfortable joining. And that’s everyone’s loss because you never know where the good ideas will come from.
I like Sanjay’s call for making the debate more welcoming to others, but I would also like suggest that there are ways that we can actively encourage others to join the debate and how we can shape the debate. There is a entire back-channel conversation that goes on behind the scenes, and I have been guilty of not encouraging people to say something publicly when valuable points are raised. It is very appealing to hold conversations in safe venues that are free from public attack, but perhaps we are doing ourselves and the field a disservice by doing this.
Thanks for the clarification. I would like to see more people get involved, and I can see why my post (and perhaps the tone of it as well) may not have contributed to that. And I also acknowledge that for some of the proposed solutions to the replicability crisis, one may justifiably argue that it is easy for me to say (being tenured).
EJ
I have been thinking about this post over the past 36 hours. I agree 100% that we want to, as Cindy put it, make this a debate that everyone feels comfortable joining, and for this reason language that makes women, minorities, or anyone else feel excluded or unwelcome should be avoided.
Along those lines–and very much consistent with what Cindy suggested about making this a discussion that everyone feels comfortable joining–I think it’s a huge, glaring, issue that one of the most senior people in our field is attacking more junior people (the authors on the Schnall replication paper are two graduate students and an associate professor) in strikingly personal, ugly ways. Language like “shameless little bullies,” “assholes who never did anything but complain,” “second stringers”–this creates an environment where structurally less powerful people are understandably afraid to participate. To me, this sort of direct, blatant (I’ll say it) bullying is much, much worse for the conversation than an inapt historical comparison, offensive as that was.
Even more mystifying, I have seen little direct pushback on this behavior, at least publicly (I can think of one exception, perhaps not coincidentally from a woman). Maybe there is back-channel stuff going on I’m not privy to. But don’t we need to publicly say, as a field, that this is out of bounds? You don’t treat junior people (well, anyone really, but junior people especially) like this. That one of the most well known and powerful figures in our field seems to be doing so with impunity is troubling and, to me, shameful.
He’s been long at it, and there has been some push-back. But, you know, sometimes you just have to give ’em enough rope…
I’m afraid this is par for the course. I’ve been following discussions about CBT in psychosis (for no good reason other than some of the players are interested in improving psychology also, and I can never keep to the topic), and it is just as ugly, and occasionally truly embarrasing. Similar… clashes took place last year over priming. I’m suspecting things like this have gone on for long, but wasn’t done in cyber-space.
Brian Nosek tweeted that passions were running high because we all care about psychology, and care that it is doing well. I think that is a very gracious attitude.
This is the most bizarre academic debate I have ever seen. Kahneman talks about an impending train wreck and says “your field is now the poster child for doubts about the integrity of psychological research.” But the wikipedia page for priming effect, he’s the only critic listed. He lists a research methodology that no one follows, especially him. I cannot, for the life of me figure out why he chose to attack priming effects when they are among the most replicated effects in all of psychology. Why is he so passionate about this? Is it because priming researchers don’t pretend that he invented the unconscious/conscious distinction, intuitive/rational distinction. Does he really believe the bizarrely termed “system 1 vs. system 2” is his idea?
Hi Jennifer – the term “priming” applies to a very broad set of findings in psychology, some of which are considered very robust and others that have been contested recently. The Wikipedia page on “Priming (psychology)” does not give a very good or comprehensive background of the present debate, nor does it represent the context of Kahneman’s remarks very well. I don’t know of a single comprehensive overview, but if you have full-text access one good entry point into the discussion might be a recent special issue of the journal Perspectives on Psychological Science:
http://www.psychologicalscience.org/index.php/publications/observer/obsonline/perspectives-on-behavioral-priming-and-replication.html
The original APS 2013 series of symposiums that dealt with the isssues of replicability and data quaility were recommended by me, an Associate Professor and woman. Janet Ruscher who was the program chair at the time helped in expanding the symposium from 1 to 3 different symposiums crossing all areas of psychology and the issues that we have not only with replicating and generalizing our results but also designing studies with enough power to actually test the hypothesis we want to test. This is a problem across psychology and something I thought needed to be addressed in one of our premier conferences. I introduced the series at the meeting in 2013. We needed a correction to the proliferation of marginal studies in our literature and we are making strides in that direction. This is an issue for anyone who uses the label “psychologist” and a burden we all share to do our science as well as we can not matter our sex, gender, race, county of origin, or any other label we can create.