Flattened logistic regression vs. empirical logit

I first learned about quasi-logistic regression and the "emprical logit" from Dale Barr's (2008) paper, which just happened to be right next to the growth curve analysis paper that Jim Magnuson, J. Dixon, and I wrote. I came to understand and like this approach in 2010 when Dale and I co-taught a workshop on analyzing eye-tracking data at Northwestern. I give that background by way of establishing that I'm positively disposed to the empirical logit method. So I was interested to read a new paper by Seamus Donnelly and Jay Verkuilen (2017) in which they point out some weaknesses of this approach and offer an alternative solution.

New media and priorities

I was disappointed to read a (a draft of) a forthcoming APS Observer article by Susan Fiske in which she complains about how new media have allowed "unmoderated attacks" on individuals and their research programs. Other bloggers have written at some length about this (Andrew Gelman, Chris Chambers, Uri Simonsohn), I particularly recommend the longer and very thoughtful post by Tal Yarkoni. A few points have emerged as the most salient to me:

First, scientific criticism should be evaluated on its accuracy and constructiveness. Our goal should be accurate critiques that provide constructive ideas about how to do better. Efforts to improve the peer review process often focus on those factors, along with timeliness. As it happens, blogs are actually great for this: posts can be written quickly and immediately followed by comments that allow for back-and-forth so that any inaccuracies can be corrected and constructive ideas can emerge. Providing critiques in a polite way is a nice goal, but it is secondary. (Tal Yarkoni's post discusses this issue very well).

Second, APS is the publisher of Psychological Science, a journal that was once prominent and prestigious, but has gradually become a pop psychology punchline. Perhaps I should not have been surprised that they're publishing an unmoderated attack on new media.

Third, things have changed very rapidly (this is the main point of Andrew Gelman's post). When I was in graduate school (2000-2005), I don't remember hearing concerns about replication and standard operating procedures included lots of stuff that I would now consider "garden of forking paths"/"p-hacking". 2011 was a major turning point: Daryl Bem reported his evidence of ESP (side note: he was working on that since at least the mid-to-late 90's when I was undergrad at Cornell and heard him speak about it). At the time, the flaws in that paper were not at all clear. That was also the year a paper called “False-positive psychology” was published (in Psychological Science), which showed that “researcher degrees of freedom” (or "p-hacking") make actual false positive rates much higher than the nominal p < 0.05 values. The year after that, in 2012, Greg Francis's paper ("Too good to be true") came out showing that multi-experiment papers reporting consistent replications of small effect sizes are themselves very unlikely and may be reflecting selection bias, p-hacking, or other problems. 2012 also the year I was contacted by the Open Science Collaboration to contribute to their large-scale replication effort, which eventually led to a major report on the reproducibility of psychological research.

My point is that these issues, which are a huge deal now, were not very widely known even 5-6 years ago and almost nobody was talking about them 10 years ago. To put it another way, just about all tenured Psychology professors were trained before the term "p-hacking" even existed. So, maybe we should admit that all this rapid change can be a bit alarming and disorienting. But we're scientists, we're in the business of drawing conclusions from data, and the data clearly show that our old way of doing business has some flaws, so we should try to fix those flaws. Lots of good ideas are being implemented and tested -- transparency (sharing data and analysis code), post-publication peer review, new impact metrics for hiring/tenure/promotion that reward transparency and reproducibility. And many of those ideas came from those unmoderated new media discussions.

Post-doctoral research position available

We are hiring a post-doctoral research fellow to start in 2017. Research in the lab focuses on spoken language processing and semantic memory in typical and atypical speakers. Current research projects investigate: (1) The processing and representation of semantic knowledge, particularly knowledge of object features and categories, and the events or situations in which they participate. (2) The organization of the spoken language system by mapping the relationships between stroke lesion location and behavioral deficits.

Research methods include:

• behavioral and eye-tracking experiments
• lesion-symptom mapping
• computational modeling
• non-invasive brain stimulation (tDCS)

Qualifications:

• Doctorate degree in Psychology, Cognitive & Brain Science, CSD/SHLS, or related discipline. Must be completed before starting post-doctoral fellowship.
• Experience with one or more of the research methods and/or content domains.
• Programming experience in R, Matlab, python, or similar language will be preferred.

The post-doctoral fellow will be expected to contribute to ongoing projects and to develop an independent line of research. Mentorship, training, and professional development opportunities will be provided to facilitate the fellow’s future career in academic, research, or industry settings.

About the Language & Cognitive Dynamics Lab

LCDL has recently relocated to the Department of Psychology at the University of Alabama at Birmingham. UAB is a comprehensive, urban research university, ranked among the top 25 in funding from the NIH. Postdoctoral training at UAB is enhanced by the Office ofPostdoctoral Education. The medical school is routinely ranked among the top in the US, and interdisciplinary programs are a particular strength, including the Psychology Department’s undergraduate and graduate neuroscience programs. Birmingham is a growing, diverse, and progressive city located in the foothills of the Appalachians. It was recently rated #1 Next Hot Food City by Zagat, it is home to several world-class museums and performing arts venues, and the region offers excellent sites for hiking, camping, boating, swimming, and fishing.

To Apply, submit the following

• A letter of interest that describes your training, research experience and interests, and career goals
• CV
• 2-3 letters of recommendation

Applications will be considered until the position is filled. For full consideration please apply by November 1, 2016. Only complete applications will be considered. Questions and applications can be addressed to LCDL Director Dan Mirman.

MAPPD 2.0

About 5 or 6 years ago my colleagues at Moss Rehabilitation Research Institute and I made public a large set of behavioral data from language and cognitive tasks performed by people with aphasia. Our goal was to facilitate larger-scale research on spoken language processing and how it is impaired following left hemisphere stroke. We are pleased to announce that we have completed a thorough redesign of Moss Aphasia Psycholinguistics Project Database site. The MAPPD 2.0 interface is much simpler and easier to use, geared toward letting users download the data they want and analyze it themselves. 

The core of this database is single-trial picture naming and word repetition data for over 300 participants (including 20 neurologically intact control participants) with detailed target word and response information. The database also contains basic demographic and clinical information for each participant with aphasia, as well as performance on a host of supplementary tests of speech perception, semantic cognition, short-term/working memory, and sentence comprehension. A more detailed description of the included tests, coding schemes, and usage suggestions is available in our original description of the database (Mirman et al., 2010) and in the site's documentation.

Acceptance and rejection rates

There was a recent blog post at Frontiers pointing out that journals' publicly-available rejection rates are not associated with their impact factors. Their post discusses several factors that contribute to this, but I've been thinking about how rejection rates are calculated, particularly publicly stated rejection rates. For example, the 2013 rejection rate for both JEP:LMC and JEP:HPP is 78% and JEP:General is slightly higher at 83%. These are top-tier experimental psychology journals and those rejection rates seem intuitively appropriate for selective outlets, but I think they might be inflated because many papers are rejected with an invitation to revise and resubmit.

15th Neural Computation and Psychology Workshop

NCPW15 – August 8-9, 2016 – Philadelphia, PA, USA

15^th Neural Computation and Psychology Workshop

Contemporary Neural Network Models:
Machine Learning, Artificial Intelligence, and Cognition