Behavioral research at Journal of Neuroscience

In a recent issue of Journal of Neuroscience, the editors described an ongoing policy of increasing the percentage of papers that are “desk-rejected”, that is, rejected without peer review. The data showed that from Jan. 2014 to March 2016, the percentage of such decisions increased from 5% of submitted papers to around 25%. What concerned me was that the policy appeared to focus on a specific group of neuroscientists: those who primarily relied on behavioral experiments to infer function of the brain. We conducted a survey to quantify this impression.

130 people responded to the survey, 62% of whom had published a paper in JNeurosci. These authors represented about 342 articles in JNeurosci (this is a lower bound because a handful of authors had published 10 or more articles but the survey counted their contributions as 10). Of the papers that the authors had published in JNeurosci, about 40% were behavioral, without explicit neural correlates. Therefore, the survey participants represented a group of scientists that relied substantially, but not exclusively, on behavioral research.

The participants reported that they had experienced 148 desk-rejections. Among the desk-rejected manuscripts, about 75% were rejected because the editors felt that the work “lacked insights into neural mechanisms”. Here are some examples of the letters sent to the authors:

• “For purely behavioral studies, our criteria have evolved to require that a behavioral study provides novel insights into the underlying neural representations and mechanisms.”

• "The study does make some experimental predictions, but they are all behavioral."

• "while the study had a number of strengths, the emphasis was on behavioral processes rather than neural mechanisms which are the focus for The Journal of Neuroscience.”

• “It is, unfortunately, the case that it is not possible for us to consider manuscripts where the emphasis is on behavior.”

•  “I am afraid that it has become much rarer for behavior only manuscripts to be sent out for review at The Journal of Neuroscience.”

This policy has left a strong impression on the participants, as noted in their comments.

• "It is well-known that it is useless to submit behavioral studies to this journal even if neural mechanisms are discussed in the Introduction and Discussion."

• "To understand how the brain works, we have to understand the structure of the information processing and predict the behaviour before jumping to see the neural correlates of it. Therefore, I believe behavioural research is a fundamental part of neuroscience. It is sad to see one of the journals I respect is underestimating the value of behavioural neuroscience."

• "Although my personal experience included neural data, I definitely share the impression.  Quite a few journals including j neurosci are rejecting papers at the editorial stage and I have a sense behavioral studies are being excluded more; also agree that this is not wise for the journal."

• "I gave up submitting papers to JN or reviewing for them, a position which I also advertised to colleagues who came to me asking for advice whether they should consider JN as an outlet for their own work. My feeling has become that submitting papers that are mostly behavioural/computational to JN is a waste of time and effort."

• "The loss of behavioral research is dangerous to neuroscience. The idea that we will understand the brain without a nuanced understanding of behavior, i.e. the only neural output that ultimately matters, is misguided."

• "It will be a huge mistake to give any kind of low priority to behavioral/psychphysics/cognitive only studies! These papers are the basics of neuroscience. After all, this is what we aim to understand : human (or animal) behavior! Moreover, even as an electrophysiologist, i would say without hesitation that the contribution of purely behavioral papers to our understanding of computations in the brain is huge, and equal to that of invasive studies. Sometimes it is even more convincing to draw a conclusion about the neural correlates by observing the behavior, than by observing the actual neural correlate. this is due to two main factors: 1. Limitations and technical confounds of the physiological measures; 2. The superiority of a clean targeted well-thought-of purely behavioral design, compared to when compromised when one need to accommodate technological confounds of physiological methods. To sum, behavioral and psychophysics is, almost by definition, the foundation of neurosciences."

Fortunately, the editors are beginning to respond to these concerns. Senior Editor Dan Sanes left a constructive comment noting that this issue will be discussed at the journal's board meeting in November. Senior Editor Yavin Shaham spoke with me and noted that a policy statement that appeared to focus on behavioral research was problematic. Editor-in-Chief Marina Picciotto mentioned that she has asked for data regarding desk-rejections in the past 6 months and will compare them to similar data from 3 years ago to ascertain whether behavioral work has been unfairly impacted.

Meanwhile, David Herzfeld has begun a systematic, quantitative analysis of all papers published in the history of JNeurosci. The data has allowed us to quantify the citation impact of behavioral research, with the results published here.

In a thoughtful perspective, John Krakauer, Asif Ghazanfar, Alex Gomez-Marin, Malcolm MacIver, and David Poeppel considered the question of whether behavioral experiments are fundamental to advancing neuroscience. They noted that whereas the focus of neuroscience appears to have shifted to neural circuits, behavioral experiments provide the guiding vision of what that circuit might be doing. They wrote: "when scientists ask 'how does the brain generate behavior,' they are in fact asking a question best approached through behavioral work, specifically task analysis, aided by theory, that allows behavior to be decomposed into separable modules and processing operations... The neural basis of behavior cannot be properly characterized without first allowing for independent, detailed study of the behavior itself."

The positive immigrant

My friend Youseph had just returned from a trip to Iran. His father, a former foreign minister of Iran, under house arrest for many years, had passed away, and he had returned to Tehran for the 40th day ceremony. 

We met for breakfast. I found a table at the rather busy Einstein Bagels, and when he walked in, the first thing he did was to hand me a jar, with a small package of saffron in it. The jar was the one that I had given him some months back, with my homemade tomato soup. 

I sat mesmerized as he described the trip, what it's like to hear about your father from people that remembered him. Reminded me of when I had made a similar trip, some 8 years ago. 

Youseph is a positive person (aren't all immigrants?). So many changes to Tehran: signs that read come to the mosque, where you'll find a quiet place to think, where you can find shelter from an abusive relationship, where you can find peace. The mayor's initiative to fill the city with flowers. A new and interesting bridge, architecturally gorgeous, designed by a young woman. 

He recalled that as he mentioned his observations, people in the room, those who lived in Tehran, would occasionally dismiss it. "I think you're the first person that has actually read this propaganda that they put on walls." And "who cares about the flowers when after going to college for 4 years, you have to drive a taxi."

He fell into thought when I told him that he should write down his observations. I'm pretty sure I get this desire to record things from my dad. He loved to identify places and people that were in the picture on the back of that printed image. 

I look at his handwriting, and I see a record of his movements, as well as his thoughts, still here today.

Is Journal of Neuroscience discriminating against behavioral research?

Journal of Neuroscience is the flagship scientific publication for the Society for Neuroscience, a journal that was established in 1981 “to publish research in the field of neuroscience and to serve the membership of the Society for Neuroscience”. Unfortunately, available data suggest that the journal’s impact has been declining, at least as evidenced by the Journal Impact Factor (JIF). Perhaps as a reaction to these trends, in the past two years policy statements from the journal’s editorial board, and anecdotal evidence from the review process, suggest that the journal has instituted a policy of discrimination against a specific kind of neuroscience research: non-invasive, psychophysical experiments are rejected without peer review.

I am alarmed by this change. I believe that the new policy is flawed for a number of reasons.

• The available data suggest a poor relationship between rejection rates and journal impact factor, making it unlikely that increasing rejection rates will result in improved impact factor.
• The focus on rejecting a specific body of research is discriminatory and runs counter to the principle of representing research of all members of the Society for Neuroscience.
• The current policy, as applied to the journal’s own data would eliminate some of the highest cited papers in the journal’s history. That is, the new policy may worsen the plight of the journal.

The declining impact factor and the journal’s new policies

The Society for Neuroscience is an organization that represents scientists who study the field of neuroscience at all levels, including molecular, cellular, systems, behavioral, and computational. Unfortunately, over the past 20 years the flagship journal of this society has seen a general decline in its impact factor.

Data source: Thomson Reuters InCites Journal Citation Reports.

The reasons for this decline are unclear, though I speculate that one factor is the rapid rise of new, competing journals. Perhaps in response to these trends, in January 2016 the journal published an article describing changes to the review process. The new policy aimed to increase the rate of rejection of papers at the editorial stage, before they are submitted to peer review. The data that were included in a follow up article showed that from January 2014 to March 2016, the total percent of papers rejected increased modestly from around 70% to nearly 80%, while the percent of papers rejected before peer-review increased by a factor of five: from 5% to about 25%.

The new policy described the kind of research that the editors rejected without review. They wrote: “… purely biophysical or behavioral studies should provide novel insights into, and make specific predictions about, neural mechanisms or neural representations.” A typical rejection letter from the senior editor gave the following reason for rejection without review of a study that used psychophysical, non-invasive recording techniques to measure behavior: “we felt that The Journal of Neuroscience is not the right venue because the findings do not inform us about or implicate a specific neural mechanism.”

It appears to me that the journal’s new policy is targeting a specific group of neuroscientists: those who primarily rely on behavioral data to infer function of the brain. Given that this is the official journal of the Society, the discrimination would divide the membership into classes that can and cannot publish in the journal.

What to do

To help address these issues, we need data that quantifies how the policy is affecting the ability of the community to publish in the journal. To help with that, I ask that you to take a few moments to complete this survey.

The objective of the survey is to collect data regarding the publishing experience of scientists who may have been affected by this policy. We hope to be able to answer a simple question: what has been the impact of papers published in the journal that have relied primarily on psychophysical tools to measure behavior? We hope to present the data to the journal’s editors, clarifying the impact of their new policy.

Survey results are accumulated here.

A follow-up article appears here.

A paper with analysis of the data appears here.

Thank you for your help.