Improving Psychological Science at SIPS

Last week was the second meeting of the Society for the Improvement of Psychological Science, a.k.a. SIPS[1]. SIPS is a service organization with the mission of advancing and supporting all of psychological science. About 200 people met in Charlottesville, VA to participate in hackathons and lightning talks and unconference sessions, go to workshops, and meet other people interested in working to improve psychology.

What Is This Thing Called SIPS?

If you missed SIPS and are wondering what happened – or even if you were there but want to know more about the things you missed – here are a few resources I have found helpful:

The conference program gives you an overview and the conference OSF page has links to most of what went on, though it’s admittedly a lot to dig through. For an easier starting point, Richie Lennie posted an email he wrote to his department with highlights and links, written specifically with non-attendees in mind.

Drilling down one level from the conference OSF page, all of the workshop presenters put their materials online. I didn’t make it to any workshops so I appreciate having access to those resources. One good example is Simine Vazire and Bobbie Spellman’s workshop on writing transparent and reproducible articles. Their slideshow shows excerpts from published papers on things like how to transparently report exploratory analyses, how to report messy results, how to interpret a null result, and more. For me, writing is a lot easier when I have examples and models to work from, and I expect that I will be referring to those in the future.

The list of hackathon OSF pages is worth browsing. Hackathons are collaborative sessions for people interested in working on a defined project. Organizers varied in how much they used OSF – some used them mainly for internal organization, while others hosted finished or near-finished products on them. A standout example of the latter category is from the graduate research methods course hackathon. Their OSF wiki has a list of 31 topics, almost all of which are live links to pages with learning goals, reading lists, demonstrations, and assignments. If you teach grad research methods, or anything else with methodsy content, go raid the site for all sorts of useful materials.

The program also had space for smaller or less formal events. Unconferences were spontaneously organized sessions, some of which grew into bigger projects. Lightning talks were short presentations, often about work in progress.

As you browse through the resources, it is also worth keeping in the back of your mind that many projects get started at SIPS but not finished there, so look for more projects to come to fruition in the weeks and months ahead.

A challenge for future SIPS meetings is going to be figuring out how to reach beyond the people physically attending the meeting and get the broadest possible engagement, as well as to support dissemination of projects and initiatives that people create at SIPS. We have already gotten some valuable feedback about how other hackathons and unconferences manage that. This year’s meeting happened because of a Herculean effort by a very small group of volunteers[2] operating on a thin budget (at one point it was up in the air whether there’d be even wifi in the meeting space, if you can believe it) who had to plan an event that doubled in size from last year. As we grow we will always look for more and better ways to engage – the I in SIPS would not count for anything if the society did not apply it to itself.

My Personal Highlights

It is hard to summarize but I will mention a few highlights from things that I saw or participated in firsthand.

Neil Lewis Jr. and I co-organized a hackathon on diversity and inclusion in open science. We had so many people show up that we eventually split into five smaller groups working on different projects. My group worked on helping SIPS-the-organization start to collect member data so it can track how it is doing with respect to its diversity and inclusion goals. I posted a summary on the OSF page and would love to get feedback. (Neil is working on a guest post, so look for more here about that hackathon in the near future.)

Another session I participated in was the “diversity re-hack” on day two. The idea was that diversity and inclusion are relevant to everything, not just what comes up at a hackathon with “diversity and inclusion” in the title. So people who had worked on all the other hackathons on day one could come and workshop their in-progress projects to make them serve those goals even better. It was another well-attended session and we had representatives from nearly every hackathon group come to participate.

Katie Corker was the first recipient of the society’s first award, the SIPS Leadership Award. Katie has been instrumental in the creation of the society and in organizing the conference, and beyond SIPS she has also been a leader in open science in the academic community. Katie is a dynamo and deserves every bit of recognition she gets.

It was also exciting to see projects that originated at the 2016 SIPS meeting continuing to grow. During the meeting, APA announced that it will designate PsyArXiv as its preferred preprint server. And the creators of StudySwap, which also came out of SIPS 2016, just announced an upcoming Nexus (a fancy term for what we called “special issue” in the print days) with the journal Collabra: Psychology on crowdsourced research.

Speaking of which, Collabra: Psychology is now the official society journal of SIPS. It is fitting that SIPS partnered with an open-access journal, given the society’s mission. SIPS will oversee editorial responsibilities and the scientific mission of the journal, while the University of California Press will operate as the publisher.

But probably the most gratifying thing for me about SIPS was meeting early-career researchers who are excited about making psychological science more open and transparent, more rigorous and self-correcting, and more accessible and inclusive of everyone who wants to do science or could benefit from science. The challenges can sometimes feel huge, and I found it inspiring and energizing to spend time with people just starting out in the field who are dedicated to facing them.

*****

1. Or maybe it was the first meeting, since we ended last year’s meeting with a vote on whether to become a society, even though we were already calling ourselves that? I don’t know, bootstrapping is weird.

2. Not including me. I am on the SIPS Executive Committee so I got to see up close the absurd amount of work that went into making the conference. Credit for the actual heavy lifting goes to Katie Corker and Jack Arnal, the conference planning committee who made everything happen with the meeting space, hotel, meals, and all the other logistics; and the program committee of Brian Nosek, Michèle Nuijten, John Sakaluk, and Alexa Tullett, who were responsible for putting together the scientific (and, uh, I guess meta-scientific?) content of the conference.

3 thoughts on “Improving Psychological Science at SIPS

  1. I just want to note that i can not see any projects about how researchers can signal, promote, and (hopefully) get rewarded for engaging in open practices. After all the debate in the recent 5 years or so about incentives and how to change them, i find this strange.

    If i am not mistaken, it was a topic last year but i couldn’t find anything concrete (e.g. a paper with advise and tools) that resulted from it. I was hoping this would be(-come) a more prominent issue to talk about this year but again i am left disappointed.

    If you want to change incentives, i reason an important part of it would be providing the opportunity for researchers that have followed open practices to be able to signal that they used open practices in some way or form in the first place. I think history has taught us that we should not depend on employers, funders, etc. to actually read the papers of the candidates to in turn be able to spot open practices used.

    1. The “Resources for changing culture” hackathon (https://osf.io/jtqbp/) was working on the flip side of that – how to encourage and support search committees, funding agencies, etc. to change their practices to better recognize and incentivize open practices. Ultimately that is the way that things need to go.

      But I think what you may be saying is that in the meantime there may be room to support or guide people in signaling their open practices more effectively. There was a breakout group last year that talked about that, but I’m not sure if it ever came to a finished product. That sounds like it would be a good topic for the next SIPS meeting in Grand Rapids (or maybe someone will work on it in the meantime).

      1. Thank you for the information and reply!

        1) “But I think what you may be saying is that in the meantime there may be room to support or guide people in signaling their open practices more effectively.”

        Not just “in the meantime”. It seems reasonable to me to assume this incentive-thing is a two-way street: candidates signal open practices so funders/committees can spot them (and hopefully reward them). I feel the 1st part is completely left out of any discussion and/or possible action thus far.

        2) “That sounds like it would be a good topic for the next SIPS meeting in Grand Rapids (or maybe someone will work on it in the meantime).”

        Yes, please!

        I have tried to get attention for this at this year’s SIPS and have tried to work on an idea on how to achieve signaling, promoting, and rewarding open practices for researchers (use badges for CV’s), but i got the feeling nobody was really interested, or thought it was a useful idea to say the least :). I posted something about this all on the OSF discussion board. Somewhere in that thread i posted a paper in the attachment about the idea of badges for CV’s. Perhaps it can be useful in some way or form:

        https://groups.google.com/forum/#!topic/openscienceframework/7oeul9pusKg

        I hope you guys will give this a shot at SIPS 2018. I really think it deserves some more discussion and possibly some concrete action!!

        Thank you for all your efforts.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s