## Thursday, June 23, 2016

### Agency promotes risk-seeking

People and other animals are generally risk-averse.  We tend to prefer sure things, despite their low payoff value, to risky things that may have a high payoff value.  However, a recent study demonstrates that people can change their behavior dramatically, becoming risk-seeking, if they participate in games in which there is an illusion of control.  That is, if you believe that your actions are responsible for the outcome of the gamble, then you are more likely to be risk-seeking.

In a paper that was recently published in the Proceedings of the National Academy of Sciences, Sam McDougle, Jordan Taylor and colleagues at Princeton and Berkeley designed a game where there were two targets.  The participant chose one of the targets, and depending on the hit probability of that target, and the payoff value of the target (both manipulated by the experimenters), the participant received points, which would later be exchanged for money.  The hit probability and payoff were randomly varied but always in proportion to each other.  This way, the expected value of each target (hit probability times payoff value) was constant in time, as well as equal among the targets.  This made it so that there was no “optimum” behavior, that is, there was no target that they could pick which would maximize their winnings.

If a participant was risk-averse, then she should usually pick the target that had the higher hit probability, ignoring the fact that the other target, though less likely to hit, had a higher payoff value.  On the other hand, if the participant was risk-seeking, then she should usually pick the target that had the low hit probability, gambling to get the higher payoff value.

The authors divided the participants into a few groups.  In the Standard group, the volunteers pressed one of two keys, selecting a right or left target.  These participants tended to usually pick the high hit probability target.  They exhibited risk-averse behavior.

Does risk-aversion change if people believe that they can control the outcome of the gamble through their behavior?  To test for this, the authors changed the task: now instead of pushing a button to indicate the chosen target, the participants reached toward the target while holding a joystick.  As they moved the joystick, a cursor indicated whether they landed on target or not.  This was manipulated through the hit probability.  In reality, it actually did not matter how they moved the joystick.  If the hit probability was low, the cursor would not land on target (but close to it), giving the illusion that they missed the target because of a poor movement.  Remarkably, these participants behaved in a risk-seeking manor.  They usually picked the target that was less likely to hit, gambling that they could capture the high payoff by improving their movement.

The key difference between the two groups was that in the key-press group, the action had very low variability: you press a button, giving the impression that there is little relationship between what you do and the hit probability of the target.  However, in the reach group, the action had high variability: you see that the cursor that appears to be connected to your actions did not hit the target.  Perhaps by reaching slightly differently, you will hit it the next time.  In reality, in both cases the hit probability was predetermined, having nothing to do with your actions.

The work suggests that greater risk-seeking arises in circumstances where there is a likelihood that the actions themselves were responsible for the outcome, and not some external agent.  If, on the other hand, one believes that the outcomes were predetermined, due to an external agent, then they are likely to be more risk-averse, avoiding actions that have low probability of success (but potentially high reward).

Let me take these results and make some generalizations.  Suppose you wanted to build a casino.  The results of this study would say that you should buy slot machines that required some large motion of the arm to pull down a lever, rather than a button-press.  That would make your customers believe that the way they pull the lever has something to do with their winnings, and this belief would make them more risk-seeking.  (The down side is that pulling a lever may get tiring.).

Cultures and religions differ in the relative importance of agency and concept of destiny.  Perhaps a belief in pre-determination of events and an “ultimate cause” would reduce the sense of agency, which in turn would promote risk-averse behavior.

References

Samuel McDougle, Matthew Boggess, Mattew Crossley, Darius Parvin, Richard Ivry, and Jordan Taylor (2016) Credit assignment in movement-dependent reinforcement learning.  Proceedings of the National Academy of Science (USA) 113:6797-6802.

## Tuesday, December 1, 2015

### A visit to Tehran

The taxi service that my host had arranged called the night before to tell me that because of morning traffic, and the distance of the conference to our apartment, he would need to come at 7:30am.  I said OK and hoped that I would be able to sleep and wake up in time, as the night before the whole family was up at 2 am having “breakfast”, and then sound sleep till around noon.  The traffic turned out to be surprisingly light as we drove southwest of Tehran, toward a region that had a number of fruit orchards.

The conference, 22nd Iranian Biomedical Engineering meeting, was held at a rather beautiful location, a series of buildings placed in a nicely landscaped campus.  Our building had intricate Islamic style carvings on the ceilings and walls.  The main conference hall was decorated with mosaic of mirrors and tiles, and unexpectedly, two styles of chairs: large, heavily upholstered rows of chairs up front, and smaller, more modestly upholstered chairs in the rows back.  The professors sat up front, the students in the back.

The conference started with speakers who were not scientists, but government officials.  They were there mostly to talk to the cameras (their speech was recorded for the evening news apparently): talk of budgets, lack of budgets, and need for budgets.  One lovely thing was the moderator, who read a short poem before introducing each speaker.  This mixture of “hard” sciences with the “soft” arts, equations and poetry, is one of the pleasures of attending a meeting in Iran.

One of the officials described a new institute, set up about a year ago following a large donation from a wealthy Iranian in Canada.   The institute was named after the donor: Movafaghian Research Center in Neurorehab Technologies.  A young scientist working there, Saeed Behzadipour, later described that the first floor housed a rehab clinic where patients tried out student-designed devices like an exoskeleton robot, and a balancing system.  The second floor housed the students and scientists, and the third floor housed start-up companies that commercialized the devices.  How nice, I thought; finally the wealth and success of a few expatriates coming back to do some good.

My talk was in the afternoon.  Looking at the conference hall, it seemed that most of students were female (later I learned that almost 70% of the undergraduate in Iran were female).  I gave a talk that described my student Yousef Salimpour’s work on non-invasive cortical stimulation in Parkinson’s disease.  My talk was in English, which is a little embarrassing.  I remember my dad coming to one of my talks many years back.  Afterwards he told me that he thought no one understood a word I said.  But after I finished there was a line of students who asked questions.  The last student in line was a young lady who had brought me a gift, a beautiful little book.

I ended my talk with a poem from Abo Ali Sina, the 12th century Persian physician.  I had read the poem on the plane ride, having discovered it in my new Iranian passport.  Each page had a drawing of a monument in remembrance of a Persian poet or artist, and a few words from them.  What a nice passport, I thought, focusing on Persian artists and scientists.

The conference had a modest exhibition hall.  There I saw a company that displayed an exoskeleton robot that wrapped around the legs and torso and walked stroke patients, and another company with a robot that moved a patient out of their bed and had them stand up.  Many of the companies displayed hospital instruments, dialysis machines and the like -- all quite proud of the fact that they were able to build the machines despite crippling economic sanctions.  (Perhaps because I was raised in the American west, I have always admired people who aspire to be self-sufficient.)

During the regular sessions I heard a talk by a student , Sahar Jahani, who had done an experiment using functional near infrared spectroscopy, measuring activity in the prefrontal cortex during an attention task.  It was a nicely designed and executed study, showing that activity in the dorsolateral prefrontal cortex increased with task difficulty.  But what was really impressive was the fact that she was using an instrument designed and built by an earlier group of engineering students.

The conference book listed around 200 abstracts, all in English.  It seemed that despite years of sanctions, science had survived.

IPM

Some of the very best neuroscientists in Iran work at a place called Institute for Research in Fundamental Sciences (IPM), in the School of Cognitive Science.  What stands out is that while almost everywhere else in the world research on non-human primate brain is declining, with laboratories closing due to the high monetary and political costs of neurophysiological research, here is a place where research is expanding.  I saw three laboratories working on the cerebral cortex and the neural basis of vision.

I gave a one day short-course, summarizing the work of the last couple of decades on how the brain controls movements of the eyes; contributions of the basal ganglia, superior colliculus, and the cerebellum.  We started at 9:30am, with a room that was so full that, to my delight, they kept on bringing in chairs.  The sight of all those eager students, and their wonderful questions that followed every slide, was a source of energy that fed me for the whole day (the lecture ended at 4:30pm).  I then met with individual students and visited labs.  The labs were recording from neurons in the brain with amplifiers and other electronics that were all homemade, another example of survival in the face of sanction imposed scarcity.  When the day had come to an end, I felt that I had experienced one of the best visits that I had had to any scientific institution.

 With the students at the short course on Decisions and Actions, held at Institute for Research in Fundamental Sciences (IPM), Center for Cognitive Science.

Tehran

The most unusual feature of the city of Tehran is its sheer number of private banks (that is, not-government owned): driving through the city, it is hard to pass a block of a major street without passing a couple of bank branches.  There are more numerous than corner delis.  On the display windows are rates for certificates of deposit: 20% annual interest.  This implies that inflation must be much higher than that.

I sat and listened to the song of the caged love birds, hanging outside the corner grocer, and watched a  Persian alley cat cross the street.  The best thing about my short visit, however, was the quality of food.  The bread baker a few doors down from the apartment baked a thin flat bread, and sold it hot as it came out of the clay oven for around 15 cents.  The vegetables, particularly the ordinary humble tomatoes, were as red in the inside as the gorgeous color on the outside.  The dairy products, the sheep’s milk cheeses, were spectacular.  The cream pastries that I bought from the confectionery a block away were as good as those that I had had in Paris.  But what made the experience special was that this quality came at a fraction of the price that I had paid back home in America.

## Monday, November 16, 2015

### Exercise and dopamine

The November sun was shining through the glass roof, making moving shadows that lit up the shallow end of the swimming pool.  I was doing a few laps, thoroughly enjoying my afternoon at the local rec center.  I wondered, why is it that exercise makes me feel good?

Over the last two decades, much of the research on the neural basis of reward has focused on dopamine, a neurotransmitter that is released by a relatively small number of neurons in the midbrain, and sent to pretty much all the rest of the brain and other organs.  When an animal sees something that it likes, for example food, these neurons in the midbrain release dopamine.  The magnitude of the release is related to the subjective value of the rewarding stimulus.  So when you see ice cream, your brain probably releases more dopamine than when you see broccoli.

However, when you have to work to acquire the rewarding item, then the magnitude of dopamine release becomes smaller as the required effort becomes larger.  That is, if you have to run a mile to get the ice cream, then the broccoli may seem like a better choice.  So it is puzzling that exercise, which basically involves generating a lot of effort, should result in an increase in the production of dopamine.  But there is indeed some evidence for this.

In 1994, Satoshi Hattori, Makoto Naoi, and Hitoo Nishino in Nagoya, Japan trained 6 rats to run on a treadmill.  After training, they measured dopamine levels via micro-dialysis in a region of the basal ganglia (striatum) at 20 minute intervals (this became the baseline measure of dopamine).  They then had the animals run for 20 minutes at a slow, medium, or fast speed, re-measured dopamine levels during the run, and then each 20 minutes after completion of the run (for another 3 hours).  They found that during the medium and fast runs, dopamine levels increased.  Interestingly, dopamine levels remained elevated for about 1.5 hours after completion of the run (in the figure below, the x-axis has bins of 20 minute duration).  As a result, the study demonstrated that dopamine levels increased with physical exercise, and remained elevated beyond completion of the exercise.

But this result was in rats.  Does the same thing happen in humans?  In 2000, Gene-Jack Wang, Nora Volkow and colleagues at State University of New York at Stony Brook asked 12 people who regularly exercised to participate in a PET study, where they used a scanning technique to indirectly measure dopamine levels in a region of the basal ganglia (striatum).  They scanned the brain and measured dopamine levels at baseline, and then they had the volunteers run on a treadmill for 30 minutes.  Following the run, they again scanned the brain.  Surprisingly, they found no significant changes in dopamine levels (as shown in the figure below).  In these people who exercised regularly, the run on the treadmill seemed to have had no particular effects on the dopamine levels.

Given the results in rats, this result in humans was puzzling, and to my knowledge still remains unresolved.

Fortunately, over the last decade there have been  advances in our ability to directly record from dopamine neurons in the primate brain.  With these recordings it is possible to see how dopamine responds to both reward and effort.

In 2015, Chiara Varazzani, Sebastian Bouret, and their colleagues in Paris, France, trained thirsty monkeys to squeeze a handle in order to receive juice.  There were 3 juice amounts (small, medium, and large amounts of juice) and 3 amounts of effort (small, medium, and large amounts of force), producing a total of 9 conditions.

There was a symbol associated with each of the 9 conditions.  For example, when the cue was a long, narrow rectangle, it meant that the monkey would have to squeeze the handle by a small amount to get a small amount of juice.  When the cue was a circle, it meant that the monkey would have to squeeze by a large amount to get the same small amount of juice.  In this way, the symbols defined both the amount of reward and the required effort.  Once the symbol was removed, a “go” cue appeared, instructing the animal to actually produce the force.  If the animal produced the right amount of force, it received the reward.

The authors recorded from 90 dopamine neurons in the substantia nigra, another region in the basal ganglia, and found that when the monkey saw the symbol indicating the reward and effort levels, the dopamine cells responded more with increasing reward.  However, as the required effort increased, the dopamine response became smaller.  Therefore, when the animal received information regarding effort and reward contingencies of the upcoming trial, it produced more dopamine if the trial was to include a large amount of juice, but less dopamine if the trial was to require a large amount of force.  In a sense, dopamine acted like a sum of reward minus effort, signaling the value (or utility) of the upcoming event.  At the time of cue, dopamine levels signaled how much the animal “liked” the following trial: the brain got more dopamine if the cue promised a lot of juice, and required only a small amount of effort.

Interestingly, during the time that the animal actually squeezed the handle and produced the required force, dopamine cells once again responded, but now with increased rates when the required force was larger (in the figure below, the x-axis is time, with each interval 200ms).  That is, the same cells that earlier had reduced their discharge when told that the trial would involve a large amount of effort, now increased their discharge during the actual production of the large effort.

These results highlight the dual nature of dopamine.  When the brain is deciding between two options that each promise some amount of reward, and require some amount of effort, dopamine response becomes larger with greater promised reward, and becomes smaller with greater required effort.  Maybe this is why we tend to pick the more rewarding, less effortful option.  However, when the brain is sending motor commands to actually perform the option that we selected, dopamine responds more with the increasing effort, and now seems impervious to the promised reward.  Maybe this is why when we exercise, that is, when we spend effort, we tend to feel as if we are rewarded: because during exercise, dopamine makes effort seem like reward.

References

Satoshi Hattori, Makoto Naoi, and Hitoo Nishino (1994) Striatal dopamine turnover during treadmill running in the rat: relation to the speed of running.  Brain Research Bulletin 35:41-49.

Chiara Varazzani, Aurore San-Galli, Sophie Gilardeau, and Sebastien Bouret (2015) Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys.  Journal of Neuroscience 35:7866-7877.

Gene-Jack Wang, Nora D. Volkow, Joanna S. Fowler, et al. (2000) PET studies of the effects of aerobic exercise on human striatal dopamine release.  Journal of Nuclear Medicine 41:1352-1356.

## Sunday, June 14, 2015

### Happiness and the aging brain

In 2008, the Gallop organization randomly called around 355,000 people in the United States, asking them about their state of happiness.  They wanted to quantify how this state changed as people transitioned from youth, to middle age, to old age.  To assess global well-being, they asked the following:

“Please imagine a ladder with steps numbered from 0 at the bottom to 10 at the top.  The top of the ladder represents the best possible life for you, and the bottom of the ladder represents the worst possible life for you.  On which step of the ladder would you say you personally feel you stand at this time?”

They found that youngest people, those in their late teens and early 20s, felt quite upbeat about their life, placing themselves high up on the ladder.  Unfortunately, this sense of well-being dropped as age increased, reaching its lowest point around the age of 50.  But as age increased beyond 50, the sense of well-being increased dramatically, continuing to grow even into the 8th and 9th decade of life.
 Well-being ladder as a function of age in America, as assessed in 2008
The trend was consistent in both men and women.  And so amazingly, people in America felt that they reached their lowest point on the ladder of life around the time they reached midlife.  What is it about aging beyond the teen years that made people feel worse about their sense of well-being, and why did this process reverse in the fifth decade of life?

Stress and worry decline in mid life

Statistical analysis of the data revealed that some obvious things that one might think affects sense of well-being did not alter the age-dependent process.  For example, gender, being unemployed, having a child living at home, and not having a partner had no significant effects on the trends.  That is, regardless of these factors, people simply became less happy as they aged toward midlife, and then something seemed to change, allowing them to regain happiness as they got older.

The authors of the study, writing in the Proceedings of the National Academy of Science, speculated that perhaps this change had something to do with increased wisdom and emotional intelligence of the aged.  They wrote: “older people have an increased ability to self-regulate their emotions and view their situations positively.”

They provided some data to back their speculation.  That same Gallop poll had also asked the respondents to evaluate how they felt yesterday.  They asked about specific affects like worry, stress, and anger: “Did you experience stress during a lot of the day yesterday?  Did you experience worry during a lot of the day yesterday?”

They found that regardless of age, women reported that their yesterday had greater stress and worry than men.  However, there was a dramatic age-dependent effect.  When young people evaluated their yesterday, they reported much more stress and worry than older people.  But for people in their late 40s and early 50s, their yesterday was suddenly much less stressful than for people 10 years younger.  For people in their 60s, their yesterday was even more peaceful.

The emotional response to what could've been

This questions of why people (at least Americans) feel a reduced sense of wellbeing with age, and why this process reverses at midlife, remain unanswered.  One intriguing line of research is that with age, the brain alters how it evaluates lost opportunities.

Stefanie Brassen and colleagues asked a group of young people (around 25 years old), a group of healthy older people (around 66 years old), and a group of late-life depressed elderly (also around 66 years old), to watch a monitor which showed 8 boxes.  Seven of those boxes contained gold, but one had a devil in it.  Boxes could be opened in sequence, and as long as the box contained gold, they could keep accumulating the reward.  But if the box contained a devil, they would lose everything.  So the participants could decide to stop and collect their gains, or continue.  Importantly, if they decided to stop, the position of the devil was revealed.  This indicated how far they could have safely continued, thereby showing them the missed opportunity.

Young people responded to the missed opportunity by being aggressive on the next trial—the greater the missed opportunity on the current trial, the greater the risk that they took on their next attempt.  Surprisingly, following a missed opportunity the depressed elderly did the same as the young people, taking a bigger risk.  However, healthy elderly did not respond this way.  Following a missed opportunity, they did not increase their risk taking.

To measure the emotional response to the missed opportunities, the authors measured skin conductance and found that this measure was modulated in the depressed elderly but not in the healthy elderly.  While these data were being collected, the authors also measured brain activity using fMRI and found that whereas all groups responded similarly to winning and losing, the main difference was in the response to a missed opportunity.  Both the young and the depressed elderly had a strong response when they observed the missed opportunity.  The healthy elderly, however, only responded to real losses, and not missed opportunities.

The data suggested that healthy aging was associated with a reduced responsiveness to lost opportunities.  When the healthy elderly made their decision, they were happy if they gained something, and did not care so much if that gain was less than optimal.  That is, they did not respond emotionally to the fact that they could have made even a better decision.

On the other hand, both the young folks and the depressed elderly responded emotionally once they found out that they could have made a better decision, despite the fact that the decision that they had made had produced a gain.  That gain, in retrospect, was not good enough.  And in some ways not being as good as it could have been felt like a loss to the young folks and the depressed elderly.

So it is possible that in youth, a decision that results in a positive outcome, but represents a lost opportunity (because it could have been better), produces an emotional, stressful response. But when that same decision is made after midlife, the brain is less sensitive to the fact that the results could have been better, and more concerned with the fact that the decision produced a gain.  Curiously, this is true for elderly who are healthy, but not elderly who are depressed.

In his poem, "The Bridge", Henry Wadsworth Longfellow writes:

For my heart was hot and restless,
And my life was full of care,
And the burden laid upon me
Seemed greater than I could bear.

But now it has fallen from me,
It is buried in the sea;
And only the sorrow of others

Yet whenever I cross the river
On its bridge with wooden piers,
Like the odor of brine from the ocean
Comes the thought of other years.

And I think how many thousands
Of care-encumbered men,
Each bearing his burden of sorrow,
Have crossed the bridge since then.

I see the long procession
Still passing to and fro,
The young heart hot and restless,
And the old subdued and slow!

And forever and forever,
As long as the river flows,
As long as the heart has passions,
As long as life has woes;

The moon and its broken reflection
As the symbol of love in heaven,
And its wavering image here.

Sources
Andrew Steptoe, Angus Deaton, Arthur A. Stone (2015) Psychological wellbeing, health and ageing.  Lancet 385: 640–648.

Stone, A. A., Schwartz, J. E., Broderick, J. E., & Deaton, A. (2010). A snapshot of the age distribution of psychological well-being in the United States.Proceedings of the National Academy of Sciences, 107:9985-9990.

Stefanie Brassen, Matthias Gamer, Jan Peters, Sebastian Gluth, and Christian BÃ¼chel (2012) Don’t look back in anger! Responsiveness to missed changes in successful and unsuccessful aging.  Science 336:612-614.

## Sunday, June 7, 2015

### The problem of planets: from Aristotle to Newton

The ancient Greeks, along with a number of other civilizations, noticed five “wandering stars” that over many nights appeared to travel against the background of fixed stars in the sky.  These “stars” went along the same path as the Sun and the moon, but in the opposite direction.  They named the wandering stars Hermes, Aphrodite, Ares, Zeus, and Cronos.  The Romans translated these names into Mercury, Venus, Mars, Jupiter, and Saturn.  The names of the wandering stars, along with the Sun and the moon, became the names of the 7 days of the week.  Saturday, Sunday, and Monday are associated with Saturn, the Sun, and the moon.  Tuesday is thought to be associated with the Germanic god Tyr (Mars), Wednesday with Wotan (Mercury), Thursday with Thor (Jupiter), and Friday with Frigga (Venus).

The wandering stars, of course, were no stars at all, but planets.  It took about two thousand years to understand why the planets appeared to wander.  The story begins around the time of Aristotle, and ends with Newton.  Along the way, humans learned how to use mathematics to represent observations in nature, and this led to the birth of science.  In a recent book titled “To Explain the World”, Stephen Weinberg, a physicist and Noble Laureate, tells this story.  Here, I simplify his eloquent and thorough text, and highlight the key ideas.

Aristotle and Ptolemy

Anaxagoras, an Ionian Greek born around 500 BC, reasoned that the earth is spherical because when the Sun placed the earth’s shadow on the moon, one could see the round outline of the earth.  Aristotle repeated this idea in his book “On the Heavens”, writing: “In eclipses the outline is always curved, and, since it is the interposition of the Earth that makes the eclipse, the form of the line will be caused by the form of the Earth’s surface, which is therefore spherical.”  But he also argued that the earth must be stationary and not moving, because if it were moving a rock thrown upward would not fall straight down, but to one side.  He wrote: “heavy bodies forcibly thrown quite straight upward return to the point from which they started, even if they are thrown to an unlimited distance.”

Given that the earth is not moving, how does one explain the fixed and the wandering stars (the planets)?  Aristotle, citing an earlier work by Eudoxus of Cnidus, suggested that the fixed stars are carried around the earth on a sphere that revolves once a day from east to west, while the sun and moon and planets are carried around the earth on separate (and transparent) spheres.   Now there were lots of problems with this scheme.  For example, because the planets were thought to shine with their own light, and the spheres were always the same distance from the earth, the brightness of the planets should not change, which disagreed with observations.

This issue remained unresolved until 650 years later, with Claudius Ptolemy, who in AD 150, working in Alexandria, Egypt, wrote Almagest.  Ptolemy gave up on the notion that earth was the center of rotation for the planets, and instead suggested that each planet had a center of rotation that itself went around the earth.  For the nearby planets of Venus and Mercury, he proposed that the centers of rotation were always along a line between the earth and the sun, and went around the earth in exactly one year.  For Mars, Jupiter, and Saturn, the centers of rotation were beyond the sun.

 Ptolemy's planetary model
Ptolemy wrote: “I know that I am mortal and the creature of a day; but when I search out the massed wheeling circles of the stars, my feet no longer touch the earth, but, side by side with Zeus himself, I take my fill of ambrosia, the food of the gods.”

Copernicus and Tycho Brahe

For centuries the idea that the earth was stationary remained, so that even in the middle ages, scholars like Jean Buridan would reject the idea that the earth could be rotating, not realizing that if earth rotated, then its rotation would give everything, including an arrow that was shot straight up, an impetus.  Like all good mentors, Buridan had a student who thought independently.  His name was Nicole Oresme.  Oresme studied with his mentor Buridan in Paris in 1340s.  In his book “On the Heavens and the Earth”, Oresme rejected Aristotle’s arguments for a stationary earth, stating that when an archer shoots an arrow vertically, the earth’s rotation carries the arrow with it (along with the archer).  Therefore this observation is not a demonstration of an immovable earth, but also consistent with a rotating earth.  Aristotle’s argument on a stationary earth took its first major blow.

The idea that the earth might be rotating took center stage with Nicolaus Copernicus, who in 1510 wrote a short, anonymous book titled “Little Commentary”.  The book was not published until after the author’s death, but in it he put forth a new theory.  He began by asserting that there is no center for the orbits of the celestial bodies: the moon goes around the earth, but all other heavenly bodies go around a point near the sun.  He further asserted that the night sky has fixed stars that are much farther away than the sun, and appear to move around the earth only because the earth is rotating on its axis and about the sun.

Tycho Brahe was impressed with the simplicity of Copernicus’ theory, but pointed out a huge problem:  if the earth is moving, what is moving it?  After all, earth was made of rocks and dirt, materials that would make something the size of earth weigh an enormous amount.  In contrast, ever since Aristotle it was thought that the heavenly bodies were nothing like earth, made of some kind of substance that gave them a natural tendency to undergo rapid circular motion.  The problem was, if earth was moving around the sun, what was pushing it, and what was keeping it there in its orbit?

In an ironic twist, to explain motion of the earth it was the Copernican astronomers who called on divine intervention.  In a letter to Brahe, Copernican Christoph Rothmann wrote: “These things that vulgar sorts see as absurd at first glance are not easily charged with absurdity, for in fact divine Sapience and Majesty are far greater than they understand.”

Being unimpressed with divine intervention, in 1588 Tycho Brahe pointed out that if one took Ptolemy’s theory and put the moving center of all the planets (except earth) on the sun, and have the sun go around the stationary earth, then much of the observed data would fit just as well as Copernicus’ theory.  This “Tychonic” system kept the advantage of a stationary earth, and was mathematically identical to the model of Copernicus.

 Tycho Brahe's planetary model
In January of 1610, Galileo used his newly built telescope to look at Jupiter, and saw that “three little stars were positioned near him, small but very bright.”  The next night he noticed that the little stars seemed to have moved, and eventually he concluded that the little stars were actually satellites of Jupiter, its moons.  This observation was critical, as it was the first discovery of heavenly objects that circled something other than earth.  They were a miniature example of what Copernicus had proposed.  But Tycho Brahe’s model remained a viable alternative, because the fundamental question for a sun-centric theory remained that if the earth is moving, what could be so powerful as to move it?

Newton and calculus

In 1665, Issac Newton asked a simple question: how does one compute speed of some object if the distance traveled as a function of time is not constant (or uniform).  Suppose x(t) represents position as a function time t.  Newton argued that in order to calculate speed, we need to think of an infinitesimally small period of time, which he called o.  Speed becomes:
For example, suppose that
For o an infinitesimal period of time, we can ignore terms that include squared and cubic powers of o.  This means that:

Newton called this the "fluxion" of x(t).  We now call it the derivative of x(t).

Newton was considering this question because he wanted to ask about the acceleration that a body would experience as it travels in constant speed about a circle.  At any time t, the velocity of this body is a vector tangent to the circle, with amplitude v.

Suppose that the circle is radius r.  After an infinitesimal time o, the body will have traveled by a distance vo, and angle q about the circle.  At this new location the speed would still be v, but the velocity vector will have rotated by an angle q.  We now have two isosceles triangles that are scaled versions of each other.  Therefore, the ratio of the short side to the long side of the two triangles is equal:
We can re-write the above equation as follows:
 Eq. (1)
The term on the left of the above equation is a derivative.  It represents the length of the acceleration vector that the body experiences (pointing to the center of the circle) as it rotates with constant speed around the circle.

Newton realized that this acceleration toward the center is due to a force that is pulling the body toward the center of the circle (otherwise, it would fly off in a straight line, tangent to the circle).  That force, he assumed, is proportional to square of the velocity v, divided by radius r.

Next, Newton considered Kepler’s observation (his third law) that the square of the period of a planet in its orbits is proportional to the cube of the radius of its orbit.  The period of a body moving with speed v around a circle of radius r is the circumference  2pr divided by speed v.   And so Kepler’s third law says that
We can re-write the above equation as follows:
 Eq. (2)
If we now compare Eq. (1) with Eq. (2), we see that the acceleration that was keeping the body moving in circular motion, is also proportional to the reciprocal of squared r.  This means that the force that is pulling the body toward the center is proportional to the inverse of the squared distance of the body from the center.  This is the inverse square law of gravity.

But the incredible discovery was still one step away.  Newton now asked whether the acceleration of the moon in its orbit around the earth is the same acceleration that a body undergoes when it is falling here on earth.  To calculate moon’s acceleration, he estimated the distance of the moon to the center of the earth to be around 60 times the radius of earth, or around 314 million meters.  Next, he computed the speed of the moon by dividing the circumference of one orbit around the earth by its period of travel (27.3 days, or 2.36 million seconds):
He then used Eq. (1) to compute the acceleration of the moon toward the earth:
This is the moon’s acceleration toward earth.  It is quite small, but Newton understood that the acceleration is small because the moon is very far away from earth.  An object on the surface of earth accelerates faster because it is at a distance of one radius of earth away from the earth’s center.  The moon is 60 times farther.  Therefore, using Eq. (2), he argued that the moon’s acceleration should be 1/60^2 that of an object on the surface of the earth.

Multiplying moon’s acceleration by 60^2 we find the result that a body on the surface of earth should accelerate at around 8 m/s^2.   (The actual value is 9.8 m/s^2.  The greatest source of error in Newton’s calculations was that the distance of moon from earth, which he underestimated by around 15%).  He then writes:

“I began to think of gravity extending to the orb of the moon and (having found out how to estimate the force with which [a] globe revolving within a sphere presses the surface of the sphere) from Kepler’s rule of the periodical times of the plants being in sesquilterate proportion of their distances from the center of the orbs, I deduced that the forces which keep the planets in their orbs must [be] reciprocally as the squares of their distances from the centers about which they revolved and thereby compared the moon in her orb with the force of gravity at the surface of the earth and found them answer pretty nearly.”

So what Newton had done was to show that the motion of the moon around the earth described an acceleration toward earth that was due to a force quite identical to the force that acts on an apple on the surface of the earth.  The only reason that the moon accelerates much slower toward earth is because the moon is much farther, and therefore the force that it feels from earth is much weaker.

The acceleration of the apple, the moon, and the planets around the sun, are all governed by the same rules: force grows weaker as the squared distance of one body from another.

Sources:
Dennis Danielson and Christopher M. Graney (2014) The case against Copernicus.  Scientific American, January 2014, pp. 74-77.

Stephen Weinberg (2015) To Explain the World: The discovery of modern science. HarperCollins.

## Tuesday, December 30, 2014

### Remembering David Yue

My friend and colleague, David Yue, Professor of Biomedical Engineering at Johns Hopkins, suddenly passed away in his laboratory on Tuesday December 23, 2014.  This is to his memory.

On Saturday my phone rang.  I went to pick it up, and was surprised that is said "David Yue".  Puzzled, I thought wouldn't it be amazing if it were my dead friend calling?  I answered it, tentatively saying "hi", and heard the voice of David’s wife, Nancy.

She was calling to ask if I could speak at his memorial.  I said of course and asked how she was doing.  She said that it gets better with every passing day.  Just the night before she and the three boys had opened the presents that David had placed under the Christmas tree.  One of the presents got the four of them laughing. "It was so David", she said.  "He gave me 27 years of happiness".

David gave a quarter century of happiness to us, his colleagues and students at the Biomedical Engineering Department at Hopkins.  With his hands, he constructed some of the pillars of our undergraduate education, a course called Systems Bioengineering, and a course called Ion Channels.  What was it like to be a student in his class?

Joseph Greenstein writes:  “I go back nearly twenty years to when I was a student taking his class on ion channels. At the time he was reading a biography of Sir Isaac Newton, and David shared his favorite bits with us in class. David took to using the phrase made famous by Newton, “standing on the shoulders of giants,” when referring to Hodgkin, Huxley, and many other pioneers of quantitative electrophysiology. Beyond his extraordinary skill and passion as a scientist, he had a rare talent for transforming explanations of biological mechanism into engaging, eloquent stories. I recall him likening a CaMKII molecule hovering over a calcium channel to the massive mother ship in the movie Independence Day hovering over a city. As I teach students how to model channel gating and pursue my own lines of research, David is the giant whose shoulders I stand on.”

Gerda Brietwieser writes: “His explanations of the electrocardiogram were the clearest and most beautiful I have ever heard. He was a stellar scientist and a wonderful human being.”

He mentored over 30 PhD students and postdoctoral fellows.  What was it like to be a student in his lab?  Let’s hear David describe it, in this 2002 note that he wrote for his student Carla DeMaria:

“Carla, you have had a ride to remember the past few years in lab, forging friends, colleagues, science, and truth.  Thank you for sharing a path of self-discovery borne of waging exciting battles together, and laboring arm-on-arm with courage.  Treasure these years, as I will.  They have nurtured you with a strength that will sustain you in life and work.“

Manu Ben-Johny, a graduate student in his lab writes: “He was an incredible mentor and an exceptionally kind and generous person. He was always there to help us - whether it was b/c our electrophysiology rigs weren't air-floated properly or if it was a personal struggle we needed advice on. He spoke with eloquence and enthusiasm about science that was truly inspirational. His absence leaves behind a large void in my heart but I know his memories will continue to guide my life.”

What was it like to be one his colleague?  When one of his sons started taking physics at college, David began re-learning thermodynamics on his own.  One morning he walked into my office, wearing his black shirt, the coffee cup from the 3rd floor cafe in his hands, the rainbow colored band that held his badge around his neck, and started telling me about a fundamental equation.  I joined him at my black board, handed him a piece of chalk, and asked him to start at the beginning, because I wanted to understand it too.

Together, David and I built the PHD program from 50 students, to nearly 200.  Just a couple of days ago, I was sitting in my office and thinking of ways to organize this year’s admission process.  I had a thought, and like always, I wanted to bounce it off David.  Does this idea make sense David?  If we do it this way, would it make a difference?  I was about to leave my chair and go up to the 7th floor, to find him in his office, but then I sat down.

One of the worst things about growing old is that you lose people that you love.  And so it is for me.

John Keats wrote: "A thing of beauty is a joy forever: its loveliness increases; it will never pass into nothingness."  And so it is with David.

## Friday, June 27, 2014

### Effort of movements in Parkinson’s disease

The very terms that we use to describe the motor symptoms of Parkinson’s disease (PD) imply a subjective scaling of time and space: bradykinesia (slowness of movement), tachyphemia (cluttering of speech), and micrographica (smallness of handwriting).  Although these symptoms are stable features of the disease, a remarkable property of PD is that under some conditions the symptoms can spontaneously improve.

In 1965, R.S. Schwab and I. Zieper, two neurologists at the Massachusetts General Hospital, described the case of a 62-year old male PD patient who exhibited severe tremor and severe rigidity and was totally dependent on his wife.  His wife would start her day by dressing him, laying out his breakfast, making his lunch, go to work, then come back in the afternoon to make his dinner and finally get him undressed and ready for bed.  One evening his wife had severe abdominal pain and had to be taken to the hospital for emergency surgery.  The next day she woke worried about her husband, and was surprised when the nurse told her that he had come to visit her.  He had dressed himself, made his own breakfast, and then took a taxi to the hospital.  At the hospital his neurologist noticed him and upon examination found that he was able to walk 50% faster than in past examinations.  “All his motor tests were improved in spite of the presence of the same amount of rigidity and tremor that had been present before.”

A second case was another elderly male with advanced stage PD with severe rigidity who was confined to a wheelchair, unable to walk alone, living on the first floor of his home in Providence, RI.  A hurricane approached the city and his wife left to get some supplies from the drugstore.  “As a result of the storm the harbor overflowed 10 feet into the street.  The patient, sitting in his wheelchair, suddenly saw the door blown in and a wall of water entered the house.  Exactly how he did it is not clear, but he managed to get out of his wheelchair and climbed the steps to safety on the second floor where he was found several hours later by his wife, the waters having subsided. She found him seated in a chair as helpless as he was before.”

While these examples are anecdotal, there are other more controlled instances in which the PD patients show marked improvements in their movements.  One example of this is in the movements that are made during sleep.  Although healthy people do not move during REM sleep, people with PD sometimes experience REM sleep behavior disorder (RBD).  Valerie Cochen De Cock and her colleagues studied movements made during sleep by PD patients and reported that the movements were “surprisingly fast, ample, coordinated and symmetrical, without obvious signs of parkinsonism”.  They found one patient singing a song with a “strong and sonorous voice, a wide smile on his face” (he used to sing before his PD), another “declaiming political speeches with a loud voice” (he used to give speeches at the town council), another “shouting and getting hold of a heavy oak table and throwing it across the room”, and another “fighting with an invisible foil, with great agility” (apparently to save his lady-love from an attacking knight).

The mechanisms with which the brain of a Parkinsonian patient produces these feats remain a complete mystery.  But these observations do hint that latent in the PD brain is the ability to make fairly normal movements.  Yet, the movements are apparently unavailable for expression except under extraordinary circumstances.  Why?

Neuroeconomics of movements

Pietro Mazzoni, Anna Hristova, and +John Krakauer studied this question by asking PD patients and healthy controls to reach with their dominant (and more affected) arm to a target.  Visual feedback for the hand was removed at reach onset, and at the end of each reach the volunteers were given feedback with regard to the speed and accuracy of their movement.  Crucially, the trial had to be repeated if the speed was outside the requested range.  The authors found that for a given reach velocity, the endpoint accuracy of the movements made by the PD patients was similar to controls.  This again illustrated the latent abilities of the patients.  However, the patients required many more attempts in order to produce a reach that was as fast as the requested speed.  That is, the patients were capable of producing movements of normal speed and accuracy, but it took them more trials to become motivated to make the fast movements.  The authors proposed that under normal conditions, the patients seem to lack the “motor motivation” that healthy people possessed in generating their movements.

I have suggested that one way to view this result is to consider the possibility that in the brain, each movement is a balance between two factors: the reward that one expects to acquire at the end of the movement, and the effort (or motor cost) that will be spent in generating that movement (Shadmehr et al., Journal of Neuroscience, 2010).  The reward that we expect to acquire represents the subjective value of the movement.  For example, if you see a dear friend, the subjective value for the steps that you are about to take toward your friend are higher than if you are walking to greet someone that you may not be so fond of.  As a result, you will walk faster toward the dear friend.  (I have often thought that to examine how my brain currently values people in my life, I should measure the speed at which I walk toward them.)

Indeed, humans and other animals tend to move faster toward things that they value more.  This was first illustrated by Okihide Hikosaka and his colleagues in saccadic eye movements of monkeys.  In these experiments, thirsty monkeys were trained to move their eyes to a location in exchange for a reward (juice).  In some blocks of trials, the juice volume was a little larger, and in some blocks the volume was a little smaller.  The peak velocity of the saccadic eye movements in blocks in which there was more juice at stake was larger.  That is, the monkey’s eye movements were faster when the subjective value of the movement was higher.

In the real world we do not make saccadic eye movements in exchange for juice.  Rather, we move our eyes to place the part of the visual scene that we are interested in examining on our fovea.  Do we make faster saccades to things that we value more?  In humans, this idea was first illustrated by my former student +Minnan Xu-Wilson.  She asked people to make a saccadic eye movement to spots of light, but after the saccade was completed she ‘rewarded’ them by showing them a picture of a face, an object, or simply a noisy picture.  She found that saccades that were made in anticipation of viewing a face were faster.

These experiments illustrate that one of the factors that influences the speed by which we move, that is the vigor of our movements, is the subjective value of the reward that we expect to attain at the end of the movement.  The higher this expected value, the faster the movement.

The second factor is the subjective cost of the effort that is required to make the movement.  If the subjective value of the reward associated with two potential movements is the same, people pick the movement that requires less effort.

Now suppose that we have to move a given distance.  How does the brain decide on the speed of the movement?  The faster the speed with which we move to cover that distance, the greater force we have to produce.  If effort is related to force (perhaps because of metabolic cost of generating force), then the subjective cost of effort will be higher for the faster movements as compared to the slower movements that cover the same distance.  So if we move slower, we will produce smaller forces with our muscles and have a lower subjective cost of effort.

However, the slow movement will bring us to our goal later.  Time discounts reward.  That is, it is better to arrive at a valuable state sooner rather than later.  So the subjective value of the movement drops if we arrive later at the destination, making it better to move fast so we get to our goal sooner.

In summary, the subjective cost of effort makes it better to move slow so we produce smaller forces, but passage of time makes reward less valuable.  These two factors compete and the movement that the brain produces appears to be one that is the best possible given these two competing factors.  That is, the speed at which we move is one that produces the smallest possible effort (encouraging us to move slow), while at the same time maximizing the subject value of the reward we hope to attain (encouraging us to move fast).

Dopamine disorders alter the neuroeconomics of movements

In Parkinson’s disease, some of the neurons in the substantia nigra, a nucleus in the basal ganglia, gradually degenerate and die.  These neurons provide dopamine to much of the brain, and in particular the striatum, another region of the basal ganglia.  Dopamine appears to play a critical role in regulating the two factors that control movements: subjective value of reward and subject cost of effort.

In the course of the last two decades, John Salamone and his colleagues have been investigating the effects that loss of dopamine has on behavior of rats.  When rats are offered a choice between pressing a lever a few times to obtain good food, vs. eating a less preferred food for which they do not have to press levers, they choose to spend the effort and press the lever to get the preferred food, but only if the lever pressing requires modest effort.  But when a drug is injected into their basal ganglia that acts as an antagonist to dopamine, the rats become less willing to press the lever and forego the better food, settling for the less effortful choice.  On the other hand, if a drug is injected that enhances action of dopamine, the animal becomes more willing to press the lever, even if it has to press it many times in order to earn the better food.

Therefore, it appears that when dopamine’s actions are disrupted, the balance between subjective value of reward and cost of effort shifts.  Loss of dopamine shifts the balance by increasing cost of effort and decreasing value of reward, whereas increase of dopamine shifts the balance by decreasing cost of effort and increasing the subjective value of reward.

In this framework, loss of dopamine in PD shifts the neuroeconomics of movements towards ones that have smaller effort costs, which include movements that are slow. This speculation would not explain why certain movements of the patients are better during REM sleep, but does provide a framework for understanding the paradoxically fast and able movements that they exhibit under extraordinary circumstances: perhaps under these conditions, a greater proportion of available dopamine is engaged, increasing the expected reward for the movement, countering the effort costs.