Behavioral research at Journal of Neuroscience

In a recent issue of Journal of Neuroscience, the editors described an ongoing policy of increasing the percentage of papers that are “desk-rejected”, that is, rejected without peer review. The data showed that from Jan. 2014 to March 2016, the percentage of such decisions increased from 5% of submitted papers to around 25%. What concerned me was that the policy appeared to focus on a specific group of neuroscientists: those who primarily relied on behavioral experiments to infer function of the brain. We conducted a survey to quantify this impression.

130 people responded to the survey, 62% of whom had published a paper in JNeurosci. These authors represented about 342 articles in JNeurosci (this is a lower bound because a handful of authors had published 10 or more articles but the survey counted their contributions as 10). Of the papers that the authors had published in JNeurosci, about 40% were behavioral, without explicit neural correlates. Therefore, the survey participants represented a group of scientists that relied substantially, but not exclusively, on behavioral research.

The participants reported that they had experienced 148 desk-rejections. Among the desk-rejected manuscripts, about 75% were rejected because the editors felt that the work “lacked insights into neural mechanisms”. Here are some examples of the letters sent to the authors:

• “For purely behavioral studies, our criteria have evolved to require that a behavioral study provides novel insights into the underlying neural representations and mechanisms.”

• "The study does make some experimental predictions, but they are all behavioral."

• "while the study had a number of strengths, the emphasis was on behavioral processes rather than neural mechanisms which are the focus for The Journal of Neuroscience.”

• “It is, unfortunately, the case that it is not possible for us to consider manuscripts where the emphasis is on behavior.”

•  “I am afraid that it has become much rarer for behavior only manuscripts to be sent out for review at The Journal of Neuroscience.”

This policy has left a strong impression on the participants, as noted in their comments.

• "It is well-known that it is useless to submit behavioral studies to this journal even if neural mechanisms are discussed in the Introduction and Discussion."

• "To understand how the brain works, we have to understand the structure of the information processing and predict the behaviour before jumping to see the neural correlates of it. Therefore, I believe behavioural research is a fundamental part of neuroscience. It is sad to see one of the journals I respect is underestimating the value of behavioural neuroscience."

• "Although my personal experience included neural data, I definitely share the impression.  Quite a few journals including j neurosci are rejecting papers at the editorial stage and I have a sense behavioral studies are being excluded more; also agree that this is not wise for the journal."

• "I gave up submitting papers to JN or reviewing for them, a position which I also advertised to colleagues who came to me asking for advice whether they should consider JN as an outlet for their own work. My feeling has become that submitting papers that are mostly behavioural/computational to JN is a waste of time and effort."

• "The loss of behavioral research is dangerous to neuroscience. The idea that we will understand the brain without a nuanced understanding of behavior, i.e. the only neural output that ultimately matters, is misguided."

• "It will be a huge mistake to give any kind of low priority to behavioral/psychphysics/cognitive only studies! These papers are the basics of neuroscience. After all, this is what we aim to understand : human (or animal) behavior! Moreover, even as an electrophysiologist, i would say without hesitation that the contribution of purely behavioral papers to our understanding of computations in the brain is huge, and equal to that of invasive studies. Sometimes it is even more convincing to draw a conclusion about the neural correlates by observing the behavior, than by observing the actual neural correlate. this is due to two main factors: 1. Limitations and technical confounds of the physiological measures; 2. The superiority of a clean targeted well-thought-of purely behavioral design, compared to when compromised when one need to accommodate technological confounds of physiological methods. To sum, behavioral and psychophysics is, almost by definition, the foundation of neurosciences."

Fortunately, the editors are beginning to respond to these concerns. Senior Editor Dan Sanes left a constructive comment noting that this issue will be discussed at the journal's board meeting in November. Senior Editor Yavin Shaham spoke with me and noted that a policy statement that appeared to focus on behavioral research was problematic. Editor-in-Chief Marina Picciotto mentioned that she has asked for data regarding desk-rejections in the past 6 months and will compare them to similar data from 3 years ago to ascertain whether behavioral work has been unfairly impacted.

Meanwhile, David Herzfeld has begun a systematic, quantitative analysis of all papers published in the history of JNeurosci. The data has allowed us to quantify the citation impact of behavioral research, with the results published here.

In a thoughtful perspective, John Krakauer, Asif Ghazanfar, Alex Gomez-Marin, Malcolm MacIver, and David Poeppel considered the question of whether behavioral experiments are fundamental to advancing neuroscience. They noted that whereas the focus of neuroscience appears to have shifted to neural circuits, behavioral experiments provide the guiding vision of what that circuit might be doing. They wrote: "when scientists ask 'how does the brain generate behavior,' they are in fact asking a question best approached through behavioral work, specifically task analysis, aided by theory, that allows behavior to be decomposed into separable modules and processing operations... The neural basis of behavior cannot be properly characterized without first allowing for independent, detailed study of the behavior itself."

The positive immigrant

My friend Youseph had just returned from a trip to Iran. His father, a former foreign minister of Iran, under house arrest for many years, had passed away, and he had returned to Tehran for the 40th day ceremony. 

We met for breakfast. I found a table at the rather busy Einstein Bagels, and when he walked in, the first thing he did was to hand me a jar, with a small package of saffron in it. The jar was the one that I had given him some months back, with my homemade tomato soup. 

I sat mesmerized as he described the trip, what it's like to hear about your father from people that remembered him. Reminded me of when I had made a similar trip, some 8 years ago. 

Youseph is a positive person (aren't all immigrants?). So many changes to Tehran: signs that read come to the mosque, where you'll find a quiet place to think, where you can find shelter from an abusive relationship, where you can find peace. The mayor's initiative to fill the city with flowers. A new and interesting bridge, architecturally gorgeous, designed by a young woman. 

He recalled that as he mentioned his observations, people in the room, those who lived in Tehran, would occasionally dismiss it. "I think you're the first person that has actually read this propaganda that they put on walls." And "who cares about the flowers when after going to college for 4 years, you have to drive a taxi."

He fell into thought when I told him that he should write down his observations. I'm pretty sure I get this desire to record things from my dad. He loved to identify places and people that were in the picture on the back of that printed image. 

I look at his handwriting, and I see a record of his movements, as well as his thoughts, still here today.

Is Journal of Neuroscience discriminating against behavioral research?

Journal of Neuroscience is the flagship scientific publication for the Society for Neuroscience, a journal that was established in 1981 “to publish research in the field of neuroscience and to serve the membership of the Society for Neuroscience”. Unfortunately, available data suggest that the journal’s impact has been declining, at least as evidenced by the Journal Impact Factor (JIF). Perhaps as a reaction to these trends, in the past two years policy statements from the journal’s editorial board, and anecdotal evidence from the review process, suggest that the journal has instituted a policy of discrimination against a specific kind of neuroscience research: non-invasive, psychophysical experiments are rejected without peer review.

I am alarmed by this change. I believe that the new policy is flawed for a number of reasons.

• The available data suggest a poor relationship between rejection rates and journal impact factor, making it unlikely that increasing rejection rates will result in improved impact factor.
• The focus on rejecting a specific body of research is discriminatory and runs counter to the principle of representing research of all members of the Society for Neuroscience.
• The current policy, as applied to the journal’s own data would eliminate some of the highest cited papers in the journal’s history. That is, the new policy may worsen the plight of the journal.

The declining impact factor and the journal’s new policies

The Society for Neuroscience is an organization that represents scientists who study the field of neuroscience at all levels, including molecular, cellular, systems, behavioral, and computational. Unfortunately, over the past 20 years the flagship journal of this society has seen a general decline in its impact factor.

Data source: Thomson Reuters InCites Journal Citation Reports.

The reasons for this decline are unclear, though I speculate that one factor is the rapid rise of new, competing journals. Perhaps in response to these trends, in January 2016 the journal published an article describing changes to the review process. The new policy aimed to increase the rate of rejection of papers at the editorial stage, before they are submitted to peer review. The data that were included in a follow up article showed that from January 2014 to March 2016, the total percent of papers rejected increased modestly from around 70% to nearly 80%, while the percent of papers rejected before peer-review increased by a factor of five: from 5% to about 25%.

The new policy described the kind of research that the editors rejected without review. They wrote: “… purely biophysical or behavioral studies should provide novel insights into, and make specific predictions about, neural mechanisms or neural representations.” A typical rejection letter from the senior editor gave the following reason for rejection without review of a study that used psychophysical, non-invasive recording techniques to measure behavior: “we felt that The Journal of Neuroscience is not the right venue because the findings do not inform us about or implicate a specific neural mechanism.”

It appears to me that the journal’s new policy is targeting a specific group of neuroscientists: those who primarily rely on behavioral data to infer function of the brain. Given that this is the official journal of the Society, the discrimination would divide the membership into classes that can and cannot publish in the journal.

What to do

To help address these issues, we need data that quantifies how the policy is affecting the ability of the community to publish in the journal. To help with that, I ask that you to take a few moments to complete this survey.

The objective of the survey is to collect data regarding the publishing experience of scientists who may have been affected by this policy. We hope to be able to answer a simple question: what has been the impact of papers published in the journal that have relied primarily on psychophysical tools to measure behavior? We hope to present the data to the journal’s editors, clarifying the impact of their new policy.

Survey results are accumulated here.

A follow-up article appears here.

A paper with analysis of the data appears here.

Thank you for your help. 

Elizabeth's hopes and dreams

Early mornings, as I walk the path from the university parking lot to my lab, I stop and pick up occasional potato chip bags or candy wrappers that were left behind by children and other pedestrians that shared the sidewalk, maybe on their way home yesterday from the nearby school, or maybe on their way back from the corner store.

Sometimes there is a crumpled piece of paper, someone’s graded test, perhaps a homework, little windows into a child’s life.

It’s worth looking, because the way they see the world is a perspective akin to that of poets: by recasting the obvious, they help you rediscover the beauty that’s right in front of you.

And so a few days ago I picked up this crumpled piece of paper, where a girl named Elizabeth had listed her hopes and dreams, crossed out a few things, and then finally put check marks next to the ones that, I imagined, she identified as the key ingredients for the recipe of her life.  

From Jonathan to Joni

My colleague Joni began her presentation by describing a patient named “Elliot”, a businessman who had a tumor in his frontal lobe. The surgeons had successfully removed the tumor, but now Elliot “wasn’t Elliot anymore.” His life had started to fall apart. He lost his job, where he had been a successful executive. He divorced his wife, and then married a prostitute (which also ended badly).

Elliot had gone to see Antonio Damasio, the eminent neuroscientist. Damasio noticed that Elliot did fine in all the usual tests, exhibiting intact memory and excellent intelligence. But when asked to look at pictures of emotionally charged events, like an auto accident, or people in distress, Elliot did not register the usual signs of being affected by what he was seeing. The damage to Elliot’s brain had caused him to be unable to access his emotional memory to assign value to things that he sensed or actions that he might do. 

Without access to this emotional context, options could only be evaluated based on “cold-blooded logic”, Damasio noted. And when all you have is logic, options appeared of the same value in the landscape of decisions, and decision-making became quite difficult.

So Elliot might take hours trying to decide between various options for dinner. Food, after all, is something that can be viewed logically as having some caloric value, and some health benefits, but for a normal person the choice of what to eat is often based on memories of how it felt when you ate that food. Without access to memory of pleasure associated with eating the food (which is derived largely from the emotional context in which the food was eaten), the caloric content of food becomes a sterile dimension to base decisions.

Joni used this example to motivate her own research, which focused on the activity of neurons in the orbitofrontal cortex of animals. She showed that the neurons increased their activity when the option promised greater reward, and lowered their activity when the option required the animal to exert greater effort (through lever presses). She showed that the neurons of the orbitofrontal cortex were encoding value of the option, taking into account a measure of the reward that it promised, minus the effort that it required.

And so perhaps when there was damage to these neurons, and their connections to the brainstem regions that were important for registering emotional context of events, people lost the ability to make choices based on the subjective value of that option, that is, how they felt about that choice.

But damage to the frontal lobe, particularly the regions that so dramatically altered the ability to make choices, also changed the way people reacted toward the patient. Joni brought this up later as she sat in my office, describing that a major problem that patients faced following these forms of surgery was in their ability to continue their relationships with people whom they had known all their lives. These friends and family members had a hard time adjusting to a person who looked exactly the same as before, but now suddenly had changed internally. That is, Elliot wasn’t Elliot anymore.

As Joni explained this problem, a tear formed in her eyes.

I suddenly understood that in some ways, a similar problem was faced by people who changed their gender. For them, on the inside they were exactly the same person as before, but now on the outside, they looked something else entirely. That outwardly change made it hard for lifelong friends, children, family, to maintain the relationships that they had built together. Just when they needed it most, the critical bonds were being strained.

You see, about a year ago, Joni was Jonathan, married and father to children. Now, she was an elegantly dressed woman with eye shadow and curves.

“I’m still the same person I always was. I’m just being more authentic now.”