Monday, November 16, 2015

Exercise and dopamine

The November sun was shining through the glass roof, making moving shadows that lit up the shallow end of the swimming pool.  I was doing a few laps, thoroughly enjoying my afternoon at the local rec center.  I wondered, why is it that exercise makes me feel good?

Over the last two decades, much of the research on the neural basis of reward has focused on dopamine, a neurotransmitter that is released by a relatively small number of neurons in the midbrain, and sent to pretty much all the rest of the brain and other organs.  When an animal sees something that it likes, for example food, these neurons in the midbrain release dopamine.  The magnitude of the release is related to the subjective value of the rewarding stimulus.  So when you see ice cream, your brain probably releases more dopamine than when you see broccoli. 

However, when you have to work to acquire the rewarding item, then the magnitude of dopamine release becomes smaller as the required effort becomes larger.  That is, if you have to run a mile to get the ice cream, then the broccoli may seem like a better choice.  So it is puzzling that exercise, which basically involves generating a lot of effort, should result in an increase in the production of dopamine.  But there is indeed some evidence for this.

In 1994, Satoshi Hattori, Makoto Naoi, and Hitoo Nishino in Nagoya, Japan trained 6 rats to run on a treadmill.  After training, they measured dopamine levels via micro-dialysis in a region of the basal ganglia (striatum) at 20 minute intervals (this became the baseline measure of dopamine).  They then had the animals run for 20 minutes at a slow, medium, or fast speed, re-measured dopamine levels during the run, and then each 20 minutes after completion of the run (for another 3 hours).  They found that during the medium and fast runs, dopamine levels increased.  Interestingly, dopamine levels remained elevated for about 1.5 hours after completion of the run (in the figure below, the x-axis has bins of 20 minute duration).  As a result, the study demonstrated that dopamine levels increased with physical exercise, and remained elevated beyond completion of the exercise.

But this result was in rats.  Does the same thing happen in humans?  In 2000, Gene-Jack Wang, Nora Volkow and colleagues at State University of New York at Stony Brook asked 12 people who regularly exercised to participate in a PET study, where they used a scanning technique to indirectly measure dopamine levels in a region of the basal ganglia (striatum).  They scanned the brain and measured dopamine levels at baseline, and then they had the volunteers run on a treadmill for 30 minutes.  Following the run, they again scanned the brain.  Surprisingly, they found no significant changes in dopamine levels (as shown in the figure below).  In these people who exercised regularly, the run on the treadmill seemed to have had no particular effects on the dopamine levels.

Given the results in rats, this result in humans was puzzling, and to my knowledge still remains unresolved.   

Fortunately, over the last decade there have been  advances in our ability to directly record from dopamine neurons in the primate brain.  With these recordings it is possible to see how dopamine responds to both reward and effort.

In 2015, Chiara Varazzani, Sebastian Bouret, and their colleagues in Paris, France, trained thirsty monkeys to squeeze a handle in order to receive juice.  There were 3 juice amounts (small, medium, and large amounts of juice) and 3 amounts of effort (small, medium, and large amounts of force), producing a total of 9 conditions. 

There was a symbol associated with each of the 9 conditions.  For example, when the cue was a long, narrow rectangle, it meant that the monkey would have to squeeze the handle by a small amount to get a small amount of juice.  When the cue was a circle, it meant that the monkey would have to squeeze by a large amount to get the same small amount of juice.  In this way, the symbols defined both the amount of reward and the required effort.  Once the symbol was removed, a “go” cue appeared, instructing the animal to actually produce the force.  If the animal produced the right amount of force, it received the reward.

The authors recorded from 90 dopamine neurons in the substantia nigra, another region in the basal ganglia, and found that when the monkey saw the symbol indicating the reward and effort levels, the dopamine cells responded more with increasing reward.  However, as the required effort increased, the dopamine response became smaller.  Therefore, when the animal received information regarding effort and reward contingencies of the upcoming trial, it produced more dopamine if the trial was to include a large amount of juice, but less dopamine if the trial was to require a large amount of force.  In a sense, dopamine acted like a sum of reward minus effort, signaling the value (or utility) of the upcoming event.  At the time of cue, dopamine levels signaled how much the animal “liked” the following trial: the brain got more dopamine if the cue promised a lot of juice, and required only a small amount of effort.

Interestingly, during the time that the animal actually squeezed the handle and produced the required force, dopamine cells once again responded, but now with increased rates when the required force was larger (in the figure below, the x-axis is time, with each interval 200ms).  That is, the same cells that earlier had reduced their discharge when told that the trial would involve a large amount of effort, now increased their discharge during the actual production of the large effort. 

These results highlight the dual nature of dopamine.  When the brain is deciding between two options that each promise some amount of reward, and require some amount of effort, dopamine response becomes larger with greater promised reward, and becomes smaller with greater required effort.  Maybe this is why we tend to pick the more rewarding, less effortful option.  However, when the brain is sending motor commands to actually perform the option that we selected, dopamine responds more with the increasing effort, and now seems impervious to the promised reward.  Maybe this is why when we exercise, that is, when we spend effort, we tend to feel as if we are rewarded: because during exercise, dopamine makes effort seem like reward.

Satoshi Hattori, Makoto Naoi, and Hitoo Nishino (1994) Striatal dopamine turnover during treadmill running in the rat: relation to the speed of running.  Brain Research Bulletin 35:41-49.

Chiara Varazzani, Aurore San-Galli, Sophie Gilardeau, and Sebastien Bouret (2015) Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys.  Journal of Neuroscience 35:7866-7877.

Gene-Jack Wang, Nora D. Volkow, Joanna S. Fowler, et al. (2000) PET studies of the effects of aerobic exercise on human striatal dopamine release.  Journal of Nuclear Medicine 41:1352-1356.

Sunday, June 14, 2015

Happiness and the aging brain

In 2008, the Gallop organization randomly called around 355,000 people in the United States, asking them about their state of happiness.  They wanted to quantify how this state changed as people transitioned from youth, to middle age, to old age.  To assess global well-being, they asked the following:

“Please imagine a ladder with steps numbered from 0 at the bottom to 10 at the top.  The top of the ladder represents the best possible life for you, and the bottom of the ladder represents the worst possible life for you.  On which step of the ladder would you say you personally feel you stand at this time?”

They found that youngest people, those in their late teens and early 20s, felt quite upbeat about their life, placing themselves high up on the ladder.  Unfortunately, this sense of well-being dropped as age increased, reaching its lowest point around the age of 50.  But as age increased beyond 50, the sense of well-being increased dramatically, continuing to grow even into the 8th and 9th decade of life. 
Well-being ladder as a function of age in America, as assessed in 2008
The trend was consistent in both men and women.  And so amazingly, people in America felt that they reached their lowest point on the ladder of life around the time they reached midlife.  What is it about aging beyond the teen years that made people feel worse about their sense of well-being, and why did this process reverse in the fifth decade of life?

Stress and worry decline in mid life

Statistical analysis of the data revealed that some obvious things that one might think affects sense of well-being did not alter the age-dependent process.  For example, gender, being unemployed, having a child living at home, and not having a partner had no significant effects on the trends.  That is, regardless of these factors, people simply became less happy as they aged toward midlife, and then something seemed to change, allowing them to regain happiness as they got older.

The authors of the study, writing in the Proceedings of the National Academy of Science, speculated that perhaps this change had something to do with increased wisdom and emotional intelligence of the aged.  They wrote: “older people have an increased ability to self-regulate their emotions and view their situations positively.”

They provided some data to back their speculation.  That same Gallop poll had also asked the respondents to evaluate how they felt yesterday.  They asked about specific affects like worry, stress, and anger: “Did you experience stress during a lot of the day yesterday?  Did you experience worry during a lot of the day yesterday?”

They found that regardless of age, women reported that their yesterday had greater stress and worry than men.  However, there was a dramatic age-dependent effect.  When young people evaluated their yesterday, they reported much more stress and worry than older people.  But for people in their late 40s and early 50s, their yesterday was suddenly much less stressful than for people 10 years younger.  For people in their 60s, their yesterday was even more peaceful.

The emotional response to what could've been

This questions of why people (at least Americans) feel a reduced sense of wellbeing with age, and why this process reverses at midlife, remain unanswered.  One intriguing line of research is that with age, the brain alters how it evaluates lost opportunities.  

Stefanie Brassen and colleagues asked a group of young people (around 25 years old), a group of healthy older people (around 66 years old), and a group of late-life depressed elderly (also around 66 years old), to watch a monitor which showed 8 boxes.  Seven of those boxes contained gold, but one had a devil in it.  Boxes could be opened in sequence, and as long as the box contained gold, they could keep accumulating the reward.  But if the box contained a devil, they would lose everything.  So the participants could decide to stop and collect their gains, or continue.  Importantly, if they decided to stop, the position of the devil was revealed.  This indicated how far they could have safely continued, thereby showing them the missed opportunity.

Young people responded to the missed opportunity by being aggressive on the next trial—the greater the missed opportunity on the current trial, the greater the risk that they took on their next attempt.  Surprisingly, following a missed opportunity the depressed elderly did the same as the young people, taking a bigger risk.  However, healthy elderly did not respond this way.  Following a missed opportunity, they did not increase their risk taking.

To measure the emotional response to the missed opportunities, the authors measured skin conductance and found that this measure was modulated in the depressed elderly but not in the healthy elderly.  While these data were being collected, the authors also measured brain activity using fMRI and found that whereas all groups responded similarly to winning and losing, the main difference was in the response to a missed opportunity.  Both the young and the depressed elderly had a strong response when they observed the missed opportunity.  The healthy elderly, however, only responded to real losses, and not missed opportunities.

The data suggested that healthy aging was associated with a reduced responsiveness to lost opportunities.  When the healthy elderly made their decision, they were happy if they gained something, and did not care so much if that gain was less than optimal.  That is, they did not respond emotionally to the fact that they could have made even a better decision.

On the other hand, both the young folks and the depressed elderly responded emotionally once they found out that they could have made a better decision, despite the fact that the decision that they had made had produced a gain.  That gain, in retrospect, was not good enough.  And in some ways not being as good as it could have been felt like a loss to the young folks and the depressed elderly.

So it is possible that in youth, a decision that results in a positive outcome, but represents a lost opportunity (because it could have been better), produces an emotional, stressful response. But when that same decision is made after midlife, the brain is less sensitive to the fact that the results could have been better, and more concerned with the fact that the decision produced a gain.  Curiously, this is true for elderly who are healthy, but not elderly who are depressed.  

In his poem, "The Bridge", Henry Wadsworth Longfellow writes:

For my heart was hot and restless,
      And my life was full of care,
And the burden laid upon me
      Seemed greater than I could bear.

But now it has fallen from me,
      It is buried in the sea;
And only the sorrow of others
      Throws its shadow over me.

Yet whenever I cross the river
      On its bridge with wooden piers,
Like the odor of brine from the ocean
      Comes the thought of other years.

And I think how many thousands
      Of care-encumbered men,
Each bearing his burden of sorrow,
      Have crossed the bridge since then.

I see the long procession
      Still passing to and fro,
The young heart hot and restless,
      And the old subdued and slow!

And forever and forever,
      As long as the river flows,
As long as the heart has passions,
      As long as life has woes;

The moon and its broken reflection
      And its shadows shall appear,
As the symbol of love in heaven,
      And its wavering image here.

Andrew Steptoe, Angus Deaton, Arthur A. Stone (2015) Psychological wellbeing, health and ageing.  Lancet 385: 640–648.

Stone, A. A., Schwartz, J. E., Broderick, J. E., & Deaton, A. (2010). A snapshot of the age distribution of psychological well-being in the United States.Proceedings of the National Academy of Sciences, 107:9985-9990.

Stefanie Brassen, Matthias Gamer, Jan Peters, Sebastian Gluth, and Christian B├╝chel (2012) Don’t look back in anger! Responsiveness to missed changes in successful and unsuccessful aging.  Science 336:612-614.

Sunday, June 7, 2015

The problem of planets: from Aristotle to Newton

The ancient Greeks, along with a number of other civilizations, noticed five “wandering stars” that over many nights appeared to travel against the background of fixed stars in the sky.  These “stars” went along the same path as the Sun and the moon, but in the opposite direction.  They named the wandering stars Hermes, Aphrodite, Ares, Zeus, and Cronos.  The Romans translated these names into Mercury, Venus, Mars, Jupiter, and Saturn.  The names of the wandering stars, along with the Sun and the moon, became the names of the 7 days of the week.  Saturday, Sunday, and Monday are associated with Saturn, the Sun, and the moon.  Tuesday is thought to be associated with the Germanic god Tyr (Mars), Wednesday with Wotan (Mercury), Thursday with Thor (Jupiter), and Friday with Frigga (Venus). 

The wandering stars, of course, were no stars at all, but planets.  It took about two thousand years to understand why the planets appeared to wander.  The story begins around the time of Aristotle, and ends with Newton.  Along the way, humans learned how to use mathematics to represent observations in nature, and this led to the birth of science.  In a recent book titled “To Explain the World”, Stephen Weinberg, a physicist and Noble Laureate, tells this story.  Here, I simplify his eloquent and thorough text, and highlight the key ideas.

Aristotle and Ptolemy

Anaxagoras, an Ionian Greek born around 500 BC, reasoned that the earth is spherical because when the Sun placed the earth’s shadow on the moon, one could see the round outline of the earth.  Aristotle repeated this idea in his book “On the Heavens”, writing: “In eclipses the outline is always curved, and, since it is the interposition of the Earth that makes the eclipse, the form of the line will be caused by the form of the Earth’s surface, which is therefore spherical.”  But he also argued that the earth must be stationary and not moving, because if it were moving a rock thrown upward would not fall straight down, but to one side.  He wrote: “heavy bodies forcibly thrown quite straight upward return to the point from which they started, even if they are thrown to an unlimited distance.” 

Given that the earth is not moving, how does one explain the fixed and the wandering stars (the planets)?  Aristotle, citing an earlier work by Eudoxus of Cnidus, suggested that the fixed stars are carried around the earth on a sphere that revolves once a day from east to west, while the sun and moon and planets are carried around the earth on separate (and transparent) spheres.   Now there were lots of problems with this scheme.  For example, because the planets were thought to shine with their own light, and the spheres were always the same distance from the earth, the brightness of the planets should not change, which disagreed with observations.

This issue remained unresolved until 650 years later, with Claudius Ptolemy, who in AD 150, working in Alexandria, Egypt, wrote Almagest.  Ptolemy gave up on the notion that earth was the center of rotation for the planets, and instead suggested that each planet had a center of rotation that itself went around the earth.  For the nearby planets of Venus and Mercury, he proposed that the centers of rotation were always along a line between the earth and the sun, and went around the earth in exactly one year.  For Mars, Jupiter, and Saturn, the centers of rotation were beyond the sun.  

Ptolemy's planetary model
Ptolemy wrote: “I know that I am mortal and the creature of a day; but when I search out the massed wheeling circles of the stars, my feet no longer touch the earth, but, side by side with Zeus himself, I take my fill of ambrosia, the food of the gods.”

Copernicus and Tycho Brahe

For centuries the idea that the earth was stationary remained, so that even in the middle ages, scholars like Jean Buridan would reject the idea that the earth could be rotating, not realizing that if earth rotated, then its rotation would give everything, including an arrow that was shot straight up, an impetus.  Like all good mentors, Buridan had a student who thought independently.  His name was Nicole Oresme.  Oresme studied with his mentor Buridan in Paris in 1340s.  In his book “On the Heavens and the Earth”, Oresme rejected Aristotle’s arguments for a stationary earth, stating that when an archer shoots an arrow vertically, the earth’s rotation carries the arrow with it (along with the archer).  Therefore this observation is not a demonstration of an immovable earth, but also consistent with a rotating earth.  Aristotle’s argument on a stationary earth took its first major blow.

The idea that the earth might be rotating took center stage with Nicolaus Copernicus, who in 1510 wrote a short, anonymous book titled “Little Commentary”.  The book was not published until after the author’s death, but in it he put forth a new theory.  He began by asserting that there is no center for the orbits of the celestial bodies: the moon goes around the earth, but all other heavenly bodies go around a point near the sun.  He further asserted that the night sky has fixed stars that are much farther away than the sun, and appear to move around the earth only because the earth is rotating on its axis and about the sun.

Tycho Brahe was impressed with the simplicity of Copernicus’ theory, but pointed out a huge problem:  if the earth is moving, what is moving it?  After all, earth was made of rocks and dirt, materials that would make something the size of earth weigh an enormous amount.  In contrast, ever since Aristotle it was thought that the heavenly bodies were nothing like earth, made of some kind of substance that gave them a natural tendency to undergo rapid circular motion.  The problem was, if earth was moving around the sun, what was pushing it, and what was keeping it there in its orbit?

In an ironic twist, to explain motion of the earth it was the Copernican astronomers who called on divine intervention.  In a letter to Brahe, Copernican Christoph Rothmann wrote: “These things that vulgar sorts see as absurd at first glance are not easily charged with absurdity, for in fact divine Sapience and Majesty are far greater than they understand.”

Being unimpressed with divine intervention, in 1588 Tycho Brahe pointed out that if one took Ptolemy’s theory and put the moving center of all the planets (except earth) on the sun, and have the sun go around the stationary earth, then much of the observed data would fit just as well as Copernicus’ theory.  This “Tychonic” system kept the advantage of a stationary earth, and was mathematically identical to the model of Copernicus.

Tycho Brahe's planetary model
In January of 1610, Galileo used his newly built telescope to look at Jupiter, and saw that “three little stars were positioned near him, small but very bright.”  The next night he noticed that the little stars seemed to have moved, and eventually he concluded that the little stars were actually satellites of Jupiter, its moons.  This observation was critical, as it was the first discovery of heavenly objects that circled something other than earth.  They were a miniature example of what Copernicus had proposed.  But Tycho Brahe’s model remained a viable alternative, because the fundamental question for a sun-centric theory remained that if the earth is moving, what could be so powerful as to move it?  

Newton and calculus

In 1665, Issac Newton asked a simple question: how does one compute speed of some object if the distance traveled as a function of time is not constant (or uniform).  Suppose x(t) represents position as a function time t.  Newton argued that in order to calculate speed, we need to think of an infinitesimally small period of time, which he called o.  Speed becomes:
For example, suppose that
For o an infinitesimal period of time, we can ignore terms that include squared and cubic powers of o.  This means that:

Newton called this the "fluxion" of x(t).  We now call it the derivative of x(t).  

Newton was considering this question because he wanted to ask about the acceleration that a body would experience as it travels in constant speed about a circle.  At any time t, the velocity of this body is a vector tangent to the circle, with amplitude v.  

Suppose that the circle is radius r.  After an infinitesimal time o, the body will have traveled by a distance vo, and angle q about the circle.  At this new location the speed would still be v, but the velocity vector will have rotated by an angle q.  We now have two isosceles triangles that are scaled versions of each other.  Therefore, the ratio of the short side to the long side of the two triangles is equal: 
We can re-write the above equation as follows:
Eq. (1)
The term on the left of the above equation is a derivative.  It represents the length of the acceleration vector that the body experiences (pointing to the center of the circle) as it rotates with constant speed around the circle.  

Newton realized that this acceleration toward the center is due to a force that is pulling the body toward the center of the circle (otherwise, it would fly off in a straight line, tangent to the circle).  That force, he assumed, is proportional to square of the velocity v, divided by radius r.  

Next, Newton considered Kepler’s observation (his third law) that the square of the period of a planet in its orbits is proportional to the cube of the radius of its orbit.  The period of a body moving with speed v around a circle of radius r is the circumference  2pr divided by speed v.   And so Kepler’s third law says that 
We can re-write the above equation as follows: 
Eq. (2)
If we now compare Eq. (1) with Eq. (2), we see that the acceleration that was keeping the body moving in circular motion, is also proportional to the reciprocal of squared r.  This means that the force that is pulling the body toward the center is proportional to the inverse of the squared distance of the body from the center.  This is the inverse square law of gravity.

But the incredible discovery was still one step away.  Newton now asked whether the acceleration of the moon in its orbit around the earth is the same acceleration that a body undergoes when it is falling here on earth.  To calculate moon’s acceleration, he estimated the distance of the moon to the center of the earth to be around 60 times the radius of earth, or around 314 million meters.  Next, he computed the speed of the moon by dividing the circumference of one orbit around the earth by its period of travel (27.3 days, or 2.36 million seconds):
He then used Eq. (1) to compute the acceleration of the moon toward the earth:
This is the moon’s acceleration toward earth.  It is quite small, but Newton understood that the acceleration is small because the moon is very far away from earth.  An object on the surface of earth accelerates faster because it is at a distance of one radius of earth away from the earth’s center.  The moon is 60 times farther.  Therefore, using Eq. (2), he argued that the moon’s acceleration should be 1/60^2 that of an object on the surface of the earth.  

Multiplying moon’s acceleration by 60^2 we find the result that a body on the surface of earth should accelerate at around 8 m/s^2.   (The actual value is 9.8 m/s^2.  The greatest source of error in Newton’s calculations was that the distance of moon from earth, which he underestimated by around 15%).  He then writes:  

“I began to think of gravity extending to the orb of the moon and (having found out how to estimate the force with which [a] globe revolving within a sphere presses the surface of the sphere) from Kepler’s rule of the periodical times of the plants being in sesquilterate proportion of their distances from the center of the orbs, I deduced that the forces which keep the planets in their orbs must [be] reciprocally as the squares of their distances from the centers about which they revolved and thereby compared the moon in her orb with the force of gravity at the surface of the earth and found them answer pretty nearly.”

So what Newton had done was to show that the motion of the moon around the earth described an acceleration toward earth that was due to a force quite identical to the force that acts on an apple on the surface of the earth.  The only reason that the moon accelerates much slower toward earth is because the moon is much farther, and therefore the force that it feels from earth is much weaker.  

The acceleration of the apple, the moon, and the planets around the sun, are all governed by the same rules: force grows weaker as the squared distance of one body from another.

Dennis Danielson and Christopher M. Graney (2014) The case against Copernicus.  Scientific American, January 2014, pp. 74-77.

Stephen Weinberg (2015) To Explain the World: The discovery of modern science. HarperCollins.


Tuesday, December 30, 2014

Remembering David Yue

My friend and colleague, David Yue, Professor of Biomedical Engineering at Johns Hopkins, suddenly passed away in his laboratory on Tuesday December 23, 2014.  This is to his memory.

On Saturday my phone rang.  I went to pick it up, and was surprised that is said "David Yue".  Puzzled, I thought wouldn't it be amazing if it were my dead friend calling?  I answered it, tentatively saying "hi", and heard the voice of David’s wife, Nancy.

She was calling to ask if I could speak at his memorial.  I said of course and asked how she was doing.  She said that it gets better with every passing day.  Just the night before she and the three boys had opened the presents that David had placed under the Christmas tree.  One of the presents got the four of them laughing. "It was so David", she said.  "He gave me 27 years of happiness".

David gave a quarter century of happiness to us, his colleagues and students at the Biomedical Engineering Department at Hopkins.  With his hands, he constructed some of the pillars of our undergraduate education, a course called Systems Bioengineering, and a course called Ion Channels.  What was it like to be a student in his class? 

Joseph Greenstein writes:  “I go back nearly twenty years to when I was a student taking his class on ion channels. At the time he was reading a biography of Sir Isaac Newton, and David shared his favorite bits with us in class. David took to using the phrase made famous by Newton, “standing on the shoulders of giants,” when referring to Hodgkin, Huxley, and many other pioneers of quantitative electrophysiology. Beyond his extraordinary skill and passion as a scientist, he had a rare talent for transforming explanations of biological mechanism into engaging, eloquent stories. I recall him likening a CaMKII molecule hovering over a calcium channel to the massive mother ship in the movie Independence Day hovering over a city. As I teach students how to model channel gating and pursue my own lines of research, David is the giant whose shoulders I stand on.”

Gerda Brietwieser writes: “His explanations of the electrocardiogram were the clearest and most beautiful I have ever heard. He was a stellar scientist and a wonderful human being.”

He mentored over 30 PhD students and postdoctoral fellows.  What was it like to be a student in his lab?  Let’s hear David describe it, in this 2002 note that he wrote for his student Carla DeMaria:

“Carla, you have had a ride to remember the past few years in lab, forging friends, colleagues, science, and truth.  Thank you for sharing a path of self-discovery borne of waging exciting battles together, and laboring arm-on-arm with courage.  Treasure these years, as I will.  They have nurtured you with a strength that will sustain you in life and work.“

Manu Ben-Johny, a graduate student in his lab writes: “He was an incredible mentor and an exceptionally kind and generous person. He was always there to help us - whether it was b/c our electrophysiology rigs weren't air-floated properly or if it was a personal struggle we needed advice on. He spoke with eloquence and enthusiasm about science that was truly inspirational. His absence leaves behind a large void in my heart but I know his memories will continue to guide my life.”

What was it like to be one his colleague?  When one of his sons started taking physics at college, David began re-learning thermodynamics on his own.  One morning he walked into my office, wearing his black shirt, the coffee cup from the 3rd floor cafe in his hands, the rainbow colored band that held his badge around his neck, and started telling me about a fundamental equation.  I joined him at my black board, handed him a piece of chalk, and asked him to start at the beginning, because I wanted to understand it too.

Together, David and I built the PHD program from 50 students, to nearly 200.  Just a couple of days ago, I was sitting in my office and thinking of ways to organize this year’s admission process.  I had a thought, and like always, I wanted to bounce it off David.  Does this idea make sense David?  If we do it this way, would it make a difference?  I was about to leave my chair and go up to the 7th floor, to find him in his office, but then I sat down.

One of the worst things about growing old is that you lose people that you love.  And so it is for me.

John Keats wrote: "A thing of beauty is a joy forever: its loveliness increases; it will never pass into nothingness."  And so it is with David.

Friday, June 27, 2014

Effort of movements in Parkinson’s disease

The very terms that we use to describe the motor symptoms of Parkinson’s disease (PD) imply a subjective scaling of time and space: bradykinesia (slowness of movement), tachyphemia (cluttering of speech), and micrographica (smallness of handwriting).  Although these symptoms are stable features of the disease, a remarkable property of PD is that under some conditions the symptoms can spontaneously improve.

In 1965, R.S. Schwab and I. Zieper, two neurologists at the Massachusetts General Hospital, described the case of a 62-year old male PD patient who exhibited severe tremor and severe rigidity and was totally dependent on his wife.  His wife would start her day by dressing him, laying out his breakfast, making his lunch, go to work, then come back in the afternoon to make his dinner and finally get him undressed and ready for bed.  One evening his wife had severe abdominal pain and had to be taken to the hospital for emergency surgery.  The next day she woke worried about her husband, and was surprised when the nurse told her that he had come to visit her.  He had dressed himself, made his own breakfast, and then took a taxi to the hospital.  At the hospital his neurologist noticed him and upon examination found that he was able to walk 50% faster than in past examinations.  “All his motor tests were improved in spite of the presence of the same amount of rigidity and tremor that had been present before.”

A second case was another elderly male with advanced stage PD with severe rigidity who was confined to a wheelchair, unable to walk alone, living on the first floor of his home in Providence, RI.  A hurricane approached the city and his wife left to get some supplies from the drugstore.  “As a result of the storm the harbor overflowed 10 feet into the street.  The patient, sitting in his wheelchair, suddenly saw the door blown in and a wall of water entered the house.  Exactly how he did it is not clear, but he managed to get out of his wheelchair and climbed the steps to safety on the second floor where he was found several hours later by his wife, the waters having subsided. She found him seated in a chair as helpless as he was before.”

While these examples are anecdotal, there are other more controlled instances in which the PD patients show marked improvements in their movements.  One example of this is in the movements that are made during sleep.  Although healthy people do not move during REM sleep, people with PD sometimes experience REM sleep behavior disorder (RBD).  Valerie Cochen De Cock and her colleagues studied movements made during sleep by PD patients and reported that the movements were “surprisingly fast, ample, coordinated and symmetrical, without obvious signs of parkinsonism”.  They found one patient singing a song with a “strong and sonorous voice, a wide smile on his face” (he used to sing before his PD), another “declaiming political speeches with a loud voice” (he used to give speeches at the town council), another “shouting and getting hold of a heavy oak table and throwing it across the room”, and another “fighting with an invisible foil, with great agility” (apparently to save his lady-love from an attacking knight).

The mechanisms with which the brain of a Parkinsonian patient produces these feats remain a complete mystery.  But these observations do hint that latent in the PD brain is the ability to make fairly normal movements.  Yet, the movements are apparently unavailable for expression except under extraordinary circumstances.  Why?

Neuroeconomics of movements

Pietro Mazzoni, Anna Hristova, and +John Krakauer studied this question by asking PD patients and healthy controls to reach with their dominant (and more affected) arm to a target.  Visual feedback for the hand was removed at reach onset, and at the end of each reach the volunteers were given feedback with regard to the speed and accuracy of their movement.  Crucially, the trial had to be repeated if the speed was outside the requested range.  The authors found that for a given reach velocity, the endpoint accuracy of the movements made by the PD patients was similar to controls.  This again illustrated the latent abilities of the patients.  However, the patients required many more attempts in order to produce a reach that was as fast as the requested speed.  That is, the patients were capable of producing movements of normal speed and accuracy, but it took them more trials to become motivated to make the fast movements.  The authors proposed that under normal conditions, the patients seem to lack the “motor motivation” that healthy people possessed in generating their movements.

I have suggested that one way to view this result is to consider the possibility that in the brain, each movement is a balance between two factors: the reward that one expects to acquire at the end of the movement, and the effort (or motor cost) that will be spent in generating that movement (Shadmehr et al., Journal of Neuroscience, 2010).  The reward that we expect to acquire represents the subjective value of the movement.  For example, if you see a dear friend, the subjective value for the steps that you are about to take toward your friend are higher than if you are walking to greet someone that you may not be so fond of.  As a result, you will walk faster toward the dear friend.  (I have often thought that to examine how my brain currently values people in my life, I should measure the speed at which I walk toward them.)

Indeed, humans and other animals tend to move faster toward things that they value more.  This was first illustrated by Okihide Hikosaka and his colleagues in saccadic eye movements of monkeys.  In these experiments, thirsty monkeys were trained to move their eyes to a location in exchange for a reward (juice).  In some blocks of trials, the juice volume was a little larger, and in some blocks the volume was a little smaller.  The peak velocity of the saccadic eye movements in blocks in which there was more juice at stake was larger.  That is, the monkey’s eye movements were faster when the subjective value of the movement was higher.

In the real world we do not make saccadic eye movements in exchange for juice.  Rather, we move our eyes to place the part of the visual scene that we are interested in examining on our fovea.  Do we make faster saccades to things that we value more?  In humans, this idea was first illustrated by my former student +Minnan Xu-Wilson.  She asked people to make a saccadic eye movement to spots of light, but after the saccade was completed she ‘rewarded’ them by showing them a picture of a face, an object, or simply a noisy picture.  She found that saccades that were made in anticipation of viewing a face were faster.

These experiments illustrate that one of the factors that influences the speed by which we move, that is the vigor of our movements, is the subjective value of the reward that we expect to attain at the end of the movement.  The higher this expected value, the faster the movement.

The second factor is the subjective cost of the effort that is required to make the movement.  If the subjective value of the reward associated with two potential movements is the same, people pick the movement that requires less effort.  

Now suppose that we have to move a given distance.  How does the brain decide on the speed of the movement?  The faster the speed with which we move to cover that distance, the greater force we have to produce.  If effort is related to force (perhaps because of metabolic cost of generating force), then the subjective cost of effort will be higher for the faster movements as compared to the slower movements that cover the same distance.  So if we move slower, we will produce smaller forces with our muscles and have a lower subjective cost of effort. 

However, the slow movement will bring us to our goal later.  Time discounts reward.  That is, it is better to arrive at a valuable state sooner rather than later.  So the subjective value of the movement drops if we arrive later at the destination, making it better to move fast so we get to our goal sooner. 

In summary, the subjective cost of effort makes it better to move slow so we produce smaller forces, but passage of time makes reward less valuable.  These two factors compete and the movement that the brain produces appears to be one that is the best possible given these two competing factors.  That is, the speed at which we move is one that produces the smallest possible effort (encouraging us to move slow), while at the same time maximizing the subject value of the reward we hope to attain (encouraging us to move fast).

Dopamine disorders alter the neuroeconomics of movements

In Parkinson’s disease, some of the neurons in the substantia nigra, a nucleus in the basal ganglia, gradually degenerate and die.  These neurons provide dopamine to much of the brain, and in particular the striatum, another region of the basal ganglia.  Dopamine appears to play a critical role in regulating the two factors that control movements: subjective value of reward and subject cost of effort.

In the course of the last two decades, John Salamone and his colleagues have been investigating the effects that loss of dopamine has on behavior of rats.  When rats are offered a choice between pressing a lever a few times to obtain good food, vs. eating a less preferred food for which they do not have to press levers, they choose to spend the effort and press the lever to get the preferred food, but only if the lever pressing requires modest effort.  But when a drug is injected into their basal ganglia that acts as an antagonist to dopamine, the rats become less willing to press the lever and forego the better food, settling for the less effortful choice.  On the other hand, if a drug is injected that enhances action of dopamine, the animal becomes more willing to press the lever, even if it has to press it many times in order to earn the better food.  

Therefore, it appears that when dopamine’s actions are disrupted, the balance between subjective value of reward and cost of effort shifts.  Loss of dopamine shifts the balance by increasing cost of effort and decreasing value of reward, whereas increase of dopamine shifts the balance by decreasing cost of effort and increasing the subjective value of reward.  

In this framework, loss of dopamine in PD shifts the neuroeconomics of movements towards ones that have smaller effort costs, which include movements that are slow. This speculation would not explain why certain movements of the patients are better during REM sleep, but does provide a framework for understanding the paradoxically fast and able movements that they exhibit under extraordinary circumstances: perhaps under these conditions, a greater proportion of available dopamine is engaged, increasing the expected reward for the movement, countering the effort costs.

Sunday, April 20, 2014

Breaking habits and erasing memories

The summer day on Okinawa Island of Japan was so warm that those us who were there to teach had given up on going to the beach in the afternoon and instead had decided to try some early morning tennis.  On this particular early morning, a distinguished scientist and friend had joined our group, and there he was holding the ball and warming up for his serve.  As he tossed the ball up, he bent his body sideways and then back and twisted upwards and finally had his racquet make contact with the ball, delivering a pretty good serve. 

I stood there marveling at how he had learned to serve this way.  He said: “Well, I learned on my own, and despite lots of coaches who have tried to break it down and rebuild it, I haven’t been able to change it much.”

Memories, like that of how to hit a tennis serve, can become so persistent that the brain seems unable to change them.  This, at the surface, may not appear that important, as the cost is looking a little silly and not being able to do something as efficiently as possible.  But what if you are traveling with family on a peaceful day and stop at a gas station, and suddenly the smell of petroleum brings back memories of combat, paralyzing you with fear?  What if you are watching a movie where the hero is climbing the face of a rock and when she reaches the peak, she stands and looks down, and you find your knees shaking?  Does the brain have a mechanism in place to rebuild or even erase unwanted habits and fear-inducing memories?

Until about 15 years ago, it was generally assumed that when the brain learns something new, the newly acquired memory is initially in a labile state and can be readily changed, but after a short period of time (hours), it becomes ‘consolidated’, meaning that it becomes resistant to change.  For example, when rats were given a single pairing of a tone with a food-shock, this made it so that the next time they heard that tone, they got scared and stopped moving.  If a drug was given to them that disrupted the molecular pathways that are involved in consolidation (protein synthesis inhibitors), the next day when they heard the tone they were not scared of it.  However, the drug had to be given soon after the animal’s first experience of the tone-shock pairing.  If it was given even a few hours after the first experience, it did not have much of an effect; the animal still feared the tone.  And so it seemed that once an emotional or fear-inducing memory was acquired, there was little that could be done to change it.

The basis for this idea was a century of work that had described how memories form.  Neurons communicate with each other via their synapses, tiny junctions where one neuron sends and receives messages from another neuron.  Eric Kandel, a Columbia University neuroscientist, had shown that short-term memories, things that last for a few minutes, are due to transient changes to the synapse to make it more efficient, but these changes were sustained only for a short period of time.  To make memories last, the changes at the synapse had be sustained indefinitely, and this required manufacture of new proteins.  If the initial experience was strong enough, with passage of time these new proteins were made by the neuron and the memory was maintained, apparently becoming permanent.

But in the year 2000, Karim Nader, an Egyptian born neuroscientist who was raised in Canada, made a discovery that completely overturned this idea.  He was working in Joseph LeDoux’s laboratory in New York University where he took rats and gave them a single pairing of a tone with a foot-shock, and indeed, the next day he found that when they heard the tone, the rats froze in their tracks (rats express fear by ‘freezing’).  However, right after they heard this tone, he injected into their amygdala (a region of the brain involved in storing fearful memories) a drug that inhibits protein synthesis.  Amazingly, he found that a day later, when they heard the tone their fear was reduced by half (measured by the time spent ‘freezing’).  Interestingly, if the drug was given without the reactivation of the memory (that is, on day 2 don’t play the tone), it had no effect.   And if the animal heard the tone but 6 hours later was given the drug, it still feared the tone the next day.  So the key idea was that the fear-inducing memory could be weakened if the drug was given right after the memory was reactivated, but it could not be weakened if the drug was given alone, or if the memory was reactivated but without the drug.

Unfortunately, protein synthesis inhibitors cannot safely be given to humans, and so until recently, it was unclear whether this new understanding could be applied to fear-inducing memories in people.  In 2009, Merel Kindt and colleagues in Amsterdam asked a few undergraduate students to look at a picture of a spider and then a few seconds later played a loud sound, followed by a mild shock to the hand.  When the students heard the loud sound, they had a startle reflex, producing an eye blink.  They also showed them a picture of another spider, followed by another loud sound, but no shock to the hand.  So the students learned to fear the picture of the 1st spider, but not the second.  The amount of fear was measured by how they reacted to the loud sound.  Indeed, the students feared the 1st spider more than the second.  The students returned on Day 2, and Kindt showed them the picture of the 1st spider, but did not shock them.  Right after this, they gave them a drug called propranolol, which is often used to prevent stage fright, and works to inhibit actions of norepinephrine.  When the students returned on the next day, they did not show fear of the spider.  Importantly, if they gave the drug but did not show them the picture of the spider, the fear-inducing memory remained. 

So it seems possible that in humans, certain fear-inducing memories can be weakened by a combination of reactivation of that memory and consumption of certain drugs like propranolol.  Later work from the Kindt group showed that the key step is that during recall of the memory, there must be a prediction error.  That is, during recall, the brain appears to predict that a bad thing is going to happen (a shock), and if it does not happen, and the drug is present, then the memory is weakened.  Both the prediction error and the presence of the drug seem to be required, as one without the other is much less effective. 

These approaches are now being studied for treatment of PTSD.  In a recent study, propranolol was given to people who were involved in a serious car accident.  Those people were less likely to develop PTSD symptoms in the following 3 months compared to people who were given placebo. 

Notice, however, that all the successes have been on weakening newly formed memories.  What about the old fear-inducing memories?  The news there is less clear.  Older memories may be less likely to be affected when they are reactivated.   Which brings me to one of my favorite quotes from Margaret Thatcher, who was quoting her father when she said:

Watch your thoughts for they become words.
Watch your words for they become actions.
Watch your actions for they become habits.
Watch your habits for they become your character.
And watch your character for it becomes your destiny

Kindt M, Soeter M, Vervliet B (2009) Beyond extinction: erasing human fear responses and preventing the return of fear. Nature Neurosci 12:256-258.
Nader K, Schafe GE, Le Doux JE (2000) Fear memories require proten synthesis in the amygdala for reconsolidation after retrieval. Nature 406:722-726.
Sevenster D, Beckers T, Kindt M (2013) Prediction error governs pharmacologically induced amnesia for learned fear. Science 339:830-833.