Replication, Robustness, and Reproducibility in Psychophysiology


Interested in learning more about issues affecting reproducibility and replication in psychophysiological studies? Check out the articles in this special issue of Psychophysiology edited by Andreas Keil and me featuring articles by many notable researchers in the field.

Andreas and I will be discussing these issues and more with other researchers at a panel the opening night of the Society for Psychophysiological Research (SPR) annual meeting in Quebec City October 3-7

Decoding the contents of working memory from scalp EEG/ERP signals

Bae, G. Y., & Luck, S. J. (2018). Dissociable Decoding of Working Memory and Spatial Attention from EEG Oscillations and Sustained Potentials. The Journal of Neuroscience, 38, 409-422.

You've probably seen MVPA and other decoding methods in fMRI, but did you know that it's possible to decode information from the scalp distribution of EEG/ERP signals?

In this recent paper, we show that it is possible to decode the exact orientation of a stimulus as it is being held in working memory from sustained (CDA-like) ERPs.  A key finding is that we could decode both the orientation and the location of the attended stimulus with these sustained ERPs, whereas alpha-band EEG signals contained information only about the location.  

Our decoding accuracy was only about 50% above the chance level, but it's still pretty amazing that such precise information can be decoded from brain activity that we're recording from electrodes on the scalp!

Stay tuned for more cool EEG/ERP decoding results — we will be submitting a couple more studies in the near future.

How to p-hack (and avoid p-hacking) in ERP Research

Luck, S. J., & Gaspelin, N. (2017). How to Get Statistically Significant Effects in Any ERP Experiment (and Why You Shouldn’t)Psychophysiology, 54, 146-157.

Figure 3b.jpg

In this article, we show how ridiculously easy it is to find significant effects in ERP experiments by using the observed data to guide the selection of time windows and electrode sites. We also show that including multiple factors in your ANOVAs can dramatically increase the rate of false positives (Type I errors). We provide some suggestions for methods to avoid inflating the Type I error rate.

This paper was part of a special issue of Psychophysiology on Reproducibility edited by Emily Kappenman and Andreas Keil.

Announcing the Virtual ERP Boot Camp Blog

N2pc CSD+Waveform.jpg

We have started a new blog called Virtual ERP Boot Camp on to provide tips, advice, and other information about best practices for ERP research. We will also be highlighting new research that is relevant to the field.

To submit a question to our advice column, send an email to We can't answer every question, but we will post answers to questions that we believe will be of general interest.

You can also follow us on our new Twitter account: @erpbootcamp.

Steve and Emily