Keyboard Shortcuts?f

×
  • Next step
  • Previous step
  • Skip this slide
  • Previous slide
  • mShow slide thumbnails
  • nShow notes
  • hShow handout latex source
  • NShow talk notes latex source

Click here and press the right key for the next slide.

(This may not work on mobile or ipad. You can try using chrome or firefox, but even that may fail. Sorry.)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (or add '?notes' to the url before the #)

Press m or double tap to slide thumbnails (menu)

Press ? at any time to show the keyboard shortcuts

 

Preference vs Aversion: A Dissociation

What kinds of processes in

individual animals

guide actions?

1. two kinds of process -- habitual vs goal-directed

2. two kinds of motivational state -- primary vs preferences

done before

There are at least two kinds of motivational state,
which have distinct roles in explaining behaviour.

anecdote

Dickinson’s melons

motivational states

primary motivational states

linked to biological needs, can be unlearned

  • hunger, thirst
  • satiety
  • aversion
  • disgust
  • ...

preferences

changing, influenced by learning (and fashion, ...)
 

  • chocolate over rhubarb
  • lime over lemon
  • red over blue
  • ...

standard assumption

Primary motivational states are locked to preferences.

For example ..

if you are hungry for a food, you desire it; and

if you are averse to a food, you do not desire it.

1

significance

two motivational systems

When Xenon and I are tied together, there are two motivational systems that often pull in different directions ...

2

significance

‘a philosophical end: elucidation of the notions of subjective probability and subjective desirability or utility’

(Jeffrey, 1983, p. xi)

What kinds of processes in

individual animals

guide actions?

1. two kinds of process -- habitual vs instrumental

2. two kinds of motivational state -- primary vs preferences

There are at least two kinds of motivational state,
which have distinct roles in explaining behaviour.

step 1

primary motivational states sometimes influence behaviour

Almost too obvious to mention---postnatal infant crying when hungry---but you will see why this matters in a moment ...

Shettleworth (1975, p. figure 1)

CAPTION: ‘Hamsters performing some of the action .patterns discussed in the text and defined in Table 1. (Drawn by C. Blinick from photographs by M. Jackson.)’

Shettleworth (1975, p. figure 2)

‘HxE’ means there is a significant interaction of hunger and environment
[More eating when not hungry? I double checked the figure in Shettleworth. That’s what her figure says. I think because in the Hungry condition the hamsters had no access to food.]
This is just to show that primary motivational states (e.g. hunger) modulate behaviour, and that they do so in fairly complex ways (e.g. how hunger modifies behaviour depends on location)
CAPTION: ‘Effects of food deprivation (hunger) on various behaviors in the golden hamster. Hunger does not blindly energize activity, but selects be- haviors from an organized behavior system. (After Shettleworth, 1975.)’

Shettleworth (1975, p. figure 2)

Just the cases where hunger and environment interact.
This shows that hunger’s effect depends on the environment; unlike the postnatal baby crying, the effect of the hamster’s hunger on its behaviour depends on its environment.
‘open field’ was a box made to look a bit like open field.

step 2

primary motivational states do not always influence behaviour

I was trained to press the lever for a novel food,
which I retreived from a magazine.

I was trained while satiated
so never ate the food when hungry.

modified from Callahan and Terry (2015) figure 3

Here you can see the magazine. Rat sticks head in magazine to retrieve the food.
source is completely irrelevant so do not include bibtext

After training,
I was tested in extinction.

MEAN TOTAL RESPONSES 0 160 80 LOWHUNGER HIGHHUNGER MAGAZINE ENTRIES LOWHUNGER HIGHHUNGER LEVER PRESSES PRE PRE PRE PRE

redrawn from Balleine (1992, p. figure 1 (part))

crazy, no!? Hunger makes no difference to the lever pressing even tho’ I was trained to press the lever for a novel food.
If you let the rat eat the novel food WHILE HUNGRY, the picture is completely different. Now the hunger does drive the lever pressing.

the different actions tell incompatible stories ...

magazine entry

-> when hungry, does desire the novel food (or drink, Exp 1B)

lever pressing

-> when hungry, does NOT desire it

I was trained to press the lever for a novel food,
which I retreived from a magazine.

I never ate the food when hungry.

I never ate the food when satiated.

MEAN TOTAL RESPONSES 0 160 80 MAGAZINE ENTRIESLOWHUNGERHIGHHUNGER LEVER PRESSESLOWHUNGERHIGHHUNGER PREPRE PREPRE

redrawn from Balleine (1992, p. figure 3)

crazy, no!? Hunger makes no difference to the lever pressing even tho’ I was trained to press the lever for a novel food.
If you let the rat eat the novel food WHILE HUNGRY, the picture is completely different. Now the hunger does drive the lever pressing.

two actions, incompatible stories ...

magazine entry

-> when satiated, does NOT desire the novel food (or drink, Exp 1B)

lever pressing

-> when satiated, does desire it

Are the stories really incompatible? Not if you allow that there are two or more independent motivational systems, primary motivational states and preferences

same data but just the lever presses

MEAN TOTAL RESPONSES016080LEVER PRESSES ONLY SATIATED WHEN TRAINEDPRELOWHUNGER PREHIGHHUNGER PRELOWHUNGERHUNGRY WHEN TRAINED PREHIGHHUNGER

redrawn from Balleine (1992, p. figures 1 and 3)

step 2

primary motivational states do not always influence behaviour

step 3

Why are magazine entries and lever pressing different?

This is a rough-and-ready distinction. I do not think we can get very far with this and I can’t find quite this distinction in textbooks. E.g. Domjan (2010) has ‘elicited behaviours’ but that doesn’t get to this distinction as far as I can see.
But I think the distinction is fine because it’s essentially model-free vs model-based.

action-guiding processes

outcome driven

driven by expectations concerning how likely the action is to bring about an outcome

stimulus driven

driven by the presence of a stimulus

action—outcome

stimulus—action

e.g. lever pressing -> obtain food

includes goal-directed processes

e.g. enticing smell -> salivation

includes reflexes, habitual processes and more

step 3

Why are magazine entries and lever pressing different?

1. Primary motivational states directly influence
only stimulus-driven processes.

2. Lever-pressing is a consequence of outcome-driven processes.

3. Magazine entry is a consequence of ??? 🐀

Since the food is not actually in the magazine, there would *seem* to be no possibility that magazine entry is a consequence of a stimlus-driven process. But ...

conditioning

Pavlovian (classical)

Acquired through exposure to contingencies

Results in stimulus—stimulus links (e.g. bell-food)

The animal responds to the first stimulus as if the second were present

Subject to overshadowing and blocking (u.a.)

Overshadowing occurs when more than one stimulus is associated with an event: e.g. a light and a tone. The salience of the tone affects how much the animal learns about the correlation between light and event: a salient tone reduces the amount of learning. (31- 32)
Blocking : if an animal has previously learnt about a correlation B -> E2, it is much less likely to learn about a correlation A -> E2 when presented with a situation in which E2 is preceded by A and B. (48)

Operant

Acquired through being rewarded when acting in the presence of the stimulus.

Results in stimulus—action or action—outcome links.

The animal responds to the stimulus by performing the action.

step 3

1. Primary motivational states directly influence
only stimulus-driven processes.

2. Lever-pressing is a consequence of outcome-driven processes.

3. Magazine entry is a consequence of ??? 🐀stimulus-driven processes.

‘simple anticipatory approach to a food source, such as that involved in magazine entry, is primarily under the control of Pavlovian processes’

Balleine (1992, p. 248)

What exactly are Pavlovian processes? I’m assuming that they are stimulus-driven, not outcome driven.

step 3

Why are magazine entries and lever pressing different?

It is possible to
hunger for it but not want it; and
to want it but not hunger for it.

What kinds of processes in

individual animals

guide actions?

1. two kinds of process -- habitual vs instrumental

2. two kinds of motivational state -- primary vs preferences

There are at least two kinds of motivational state,
which have distinct roles in explaining behaviour.

motivational states

primary motivational states

linked to biological needs, can be unlearned

  • hunger
  • thirst
  • satiety
  • disgust
  • ...

preferences

changing, influenced by learning (and fashion, ...)

  • chocolate over rhubarb
  • lime over lemon
  • red over blue
  • ...

Q1: Can your primary motivational states diverge from your preferences?

For example, can hunger drive you to seek a novel food even tho you have no desire to eat it? And can satiety reign in your search for a food even though you desire to eat it?

For example, can sugar solution rank highly among your preferences even after you have become averse to it?

In thinking about this question, it is useful to recall some ideas from the previous lecture ...

 

You see a rat and a lever. The rat presses the lever occasionally. Now you start rewarding the rat: when it presses the lever it is rewarded with a particular kind of food. As a consequence, the rat presses the lever more often.

Devaluation - standard procedure:

Training: Rat is put in chamber with Lever; pressing Lever dispenses sucrose (novel food).

Devaluation: Rat is taken into another chamber, poisoned, and then exposed to sucrose.

Extinction Test: Rat returns to chamber with Lever; pressing Lever does nothing.

Dickinson, 1985 figure 3; Balleine & Dickinson, 1991 figure 1 (part)

‘Mean lever-press rates during the extinction (left-handpanel) and reacquisitiontests(right-handpanel) followingthe devaluation of either the contingent (group D-N) or non-contingentfood (group N-D).’
‘Experiment I: Mean number of lever presses ... during the extinction test session ... The various groups received either immediate (Groups IMM/SUC and IMM/ H20) or delayed (Groups DELjSUC and DEL/H2O) toxicosis [delayed did not cause aversion] and were re-exposed either to the sucrose solution (Groups IMM/SUC and DEL/SUC) or to water (Groups IMM/H2O and DEL/H20).’
[skip this in this sectino; may add a section specifically covering it]
Pavlovain conditioning, primary motivational states can have a direct effect on actions.
Now consider a question.

recap

What causes devaluation?

Devaluation is defined as a reversal in \emph{preference}.

In the standard procedure, the subjects are poisoned then re-exposed to the food.

Poisoning changes primary motivational state. Does it also change preferences?

Q2: What happens if we poison the subjects but do not re-expose them to the food?

Is just poisoning enough?

Hypothesis 1: Poisoning does directly influence preferences

Prediction: lever-pressing should reduce (as in the standard procedure)

Hypothesis 2: Poisoning does not directly influence preferences

Prediction: lever-pressing should not reduce (unlike in the standard procedure)

You already saw this

Balleine & Dickinson, 1991 figure 1 (part)

‘Experiment I: Mean number of lever presses ... during the extinction test session ... The various groups received either immediate (Groups IMM/SUC and IMM/ H20) or delayed (Groups DELjSUC and DEL/H2O) toxicosis [delayed did not cause aversion] and were re-exposed either to the sucrose solution (Groups IMM/SUC and DEL/SUC) or to water (Groups IMM/H2O and DEL/H20).’
‘lever presses during the re-acquisition session in 5-min blocks (right-hand panel)’ re-acquisition : i.e. you now have exposure to the sucrose solution

Toxicosis causes aversion to the novel food.

The aversion is manifest in magazine entries, and in reactions when re-encountering the novel food

Aversion to the food is consistent with desire for it.

Method: answer Q1 by answering Q2

Q1: Can your primary motivational states diverge from your preferences?

Q2: What happens if we poison the subjects but do not re-expose them to the food?

Q1: Can your primary motivational states diverge from your preferences?

For example, can hunger drive you to seek a novel food even tho you have no desire to eat it? And can satiety reign in your search for a food even though you desire to eat it?

For example, can sugar solution rank highly among your preferences even after you have become averse to it?

motivational states

primary motivational states

linked to biological needs, can be unlearned

  • hunger
  • thirst
  • satiety
  • disgust
  • ...

preferences

changing, influenced by learning (and fashion, ...)

  • chocolate over rhubarb
  • lime over lemon
  • red over blue
  • ...

What kinds of processes in

individual animals

guide actions?

1. two kinds of process -- habitual vs instrumental

2. two kinds of motivational state -- primary vs preferences

There are at least two kinds of motivational state,
which have distinct roles in explaining behaviour.

relation to earlier

We discussed responses to whether dual-process theory of instrumental action is consistent with using decision theory to characterise preferences

Suppose the dual-process theory of instrumental action is true.

Can we use decision theory to elucidate the notion of preference (and subjective probability)?

Maybe decision theory characterises only the goal-directed process?

habitual process

Action occurs in the presence of Stimulus.

Agent is rewarded [/punished] p(class=theCls)Stimulus-Action Link is strengthened [/weakened] due to reward [/punishment]

Given Stimulus, will Action occur? It depends on the strength of the Stimulus-Action Link.

‘goal-directed’ process

Action leads to Outcome.
 

Belief in Action-Outcome link is strengthened.

Agent has a Desire for the Outcome
 

Will Action occur? It depends on the Belief in the Action-Outcome Link and Agent’s Desire.

problem now is that preferences feature twice in the scheme of things!
Is the preference that dictates the reward the same or different?

Concerning the habitual process, what makes outcomes rewarding?

possibility 1:

the very system of preference that is involved in the goal-directed process

possibility 2:

not the system of preference that is involved in the goal-directed process 🐀

Each has potential problems.
possibility 2: now it looks like the two processes might pull in different directions. What if anything ensures that they are not fighting against each other in the way that people with different preferences might? (My habitual processes love bananas and hate chocolate ...)
Not saying for sure but maybe this is right.

interim conclusion

Philosophical and formal theories of action and joint action
assume a single system in which
belief, desire, intention and the rest
are normatively
and inferentially
integrated.

inferential integration

One instrumental action can involve multiple, dissociable
motivational states
and multiple, dissociable
goal-selection processes.

This gives rise to interface problems.

An interface problem ...

‘we should search in vain among the literature for a consensus about the psychological processes by which primary motivational states, such as hunger and thirst, regulate simple goal-directed [i.e. instrumental] acts

(Dickinson & Balleine, 1994, p. 1)

Dickinson & Balleine , 1994 p. 1

This is a very basic question. Why do you go to the kitchen and press the lever to get some water when you are thirsty?

two motivational systems

... an interface problem where there are two sets of preferences.

An Interface Problem:

How are non-accidental matches possible?

Primary motivational states guide some actions.

Preferences guide some actions.

Pursuing a single goal can involve both kinds of state.

As in the case of lever pressing then magazine entry to get the sugar solution. (Not demonstrated in this talk.)

Primary motivational states can differ from preferences.

Two motivational states match in a particular context just if, in that context, the actions one would cause and the actions the other would cause are not too different.