Goal-Directed and Habitual Processes

Click here and press the right key for the next slide.

(This may not work on mobile or ipad. You can try using chrome or firefox, but even that may fail. Sorry.)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (or add '?notes' to the url before the #)

Press m or double tap to slide thumbnails (menu)

Press ? at any time to show the keyboard shortcuts

Goal-Directed and Habitual Processes

Here is an alternative to the Simple Picture and the story about intention we were just telling ...

Start with a simple history ... this might explain why you press a button now.

outcomes:

press red -> electric shock -> pain experience

press green -> chocolate flake -> pleasure experience

stimulus--action associations:

buttons -> press red [becomes weaker]

buttons -> press green [becomes stronger]

stronger stimulus--action association -> action is more likely

no intentions here

‘instrumental behavior is controlled by two dissociable processes: a goal-directed and an habitual process’

‘an action is goal-directed if it is mediated by

the interaction of

a representation of the causal relationship between the action and outcome

and

a representation of the current incentive value, or utility, of the outcome

in a way that rationalizes the action’ Dickinson (2016, p. 177).

belief desire

stimulus -> action -> outcome

may be rewarding; if it is ...

When the action is performed in the presence of the simulus, the connection is strengthened (or ‘reinforced’) if the action is rewarded.

If the connection is strong enough, the presence of the stimulus will cause the action to occur.

habitual process

Action occurs in the presence of Stimulus.

Agent is rewarded [/punished]

Stimulus-Action Link is strengthened [/weakened] due to reward [/punishment]

Given Stimulus, will Action occur? It depends on the strength of the Stimulus-Action Link.

Let’s check we all understand the key terms here.

Action may be a complex, coordinated instrumental action, such as pressing a lever.

What is the relation between an instrumental action and the outcome or outcomes to which it is directed?

habitual

The outcome is related to the action via their history.

Action was caused by stimulus-action link

Stimulus-action link exists because of a rewarding Outcome in the past

Habitual processes exist to enable the agent to bring such Outcomes about.

conclusion

In conclusion, ...

What is the relation between an instrumental action and the outcome (or outcomes) to which it is directed?

Answer 1

Intention ...

The outcome (or outcomes) to which an instrumental action is directed is that outcome (or outcomes) specified by the intention (or intentions) which caused it.

Distinction: goal-directed vs habitual processes

Potential Objection

Some actions may be consequences of habitual processes only. If so, there is no role for intention in these.

next step: relate it to philosophical theories of action

appendix

habitual processes and desire/intention

There is a role for desire and intention in habitual processes, although these are not *required*.

outcomes:

press red -> electric shock -> pain experience

press green -> chocolate flake -> pleasure experience

stimulus--action associations:

buttons -> press red [becomes weaker]

buttons -> press green [becomes stronger]

You might desire to bring about the same outcome as in the past. I might desire, conversely, to bring about a different outcome (novelty seeking).

But it doesn’t matter what we desire when the process is habitual. All that matters is the strength of the stimulus-action associations.

preference can influence what is rewarding, and so which links are strengthened

During instrumental learning,

preferences can influence which experiences are pleasure and which pain,

and so which stimulus--action links are strengthened and weakened.

While acting,

habitual processes are entirely unaffected by your preferences.

habitual process

Action occurs in the presence of Stimulus.

Agent is rewarded [/punished]

Stimulus-Action Link is strengthened [/weakened] due to reward [/punishment]

Given Stimulus, will Action occur? It depends on the strength of the Stimulus-Action Link.

‘goal-directed’ process

Action leads to Outcome.

Belief in Action-Outcome link is strengthened.

Agent has a Desire for the Outcome

Will Action occur? It depends on the Belief in the Action-Outcome Link and Agent’s Desire.

✔︎

habitual processes and desire/intention

There is a role for desire and intention in habitual processes, although these are not *required*.

appendix

what are representations?

Look at all this talk about representations ...

‘an action is goal-directed if it is mediated by the interaction of a representation of the causal relationship between the action and outcome and a representation of the current incentive value, or utility, of the outcome in a way that rationalizes the action as instrumental for attaining the goal’ Dickinson (2016, p. 177).

On a widely accepted view, representations involve subjects having attitudes toward contents. Possible attitudes include believing, wanting, intending and knowing. The content is what distinguishes one belief from all others, or one desire from all others. The content is also what determines whether a belief is true or false, and whether a desire is satisfied or unsatisfied.

There are three main tasks in specifying a form of representation. The first task is to identify its subject (a person, perhaps; but not necessarily).

The second task is to characterise some attitudes. This typically involves specifying their distinctive functional and normative roles.[^ex-brat-vell] [^ex-brat-vell]: For examples, see Bratman (1987) on intention or Velleman (2000, p. chapter 11) on belief.

The third task is to find a scheme for specifying the contents of mental states. This typically involves one or another kind of proposition, although some have suggested other abstract entities including map-like representations.[^see-maps] [^see-maps]: See Braddon-Mitchell & Jackson (1996, p. 163): `what is inside our heads should be thought of as more like maps than sentences.’

Might need additional ingredients like format later. But we should be careful to justify postulating any further ingredients.

first potential problem.

‘As a first pass, representations are

‘‘mediating states of an intelligent system that carry information’’

(Markman and Dietrich 2001, p. 471).

They have two important features:

(1) they are physically realized, and so have causal powers;

(2) they are intentional, in other words, they have meaning or representational content.’

(Egan, 2014, p. 116)

Is this true of beliefs?

‘There is significant controversy about what can legitimately count as a representation.’

(Egan, 2014, p. 117)

But why does Dickinson (and most scientists) talk about representation instead of sticking to belief, desire and the rest?

Pretty sure he doesn’t care whether beliefs are physically realised.

a further problem

Here’s Ayesha and she’s about to act, which involves some kind of processes occurring in her imnd.

Ben want’s to predict Ayesha’s action, perhaps so he can coordinate his actions around hers. He is therefore having a think about what Ayesha might be up to.

Implicit in Ben’s thinking is a model of actions.

And along comes the philosopher and attempts to guess what is going on in Ben’s mind when he is thinking about Ayesha. The philosopher asks, in effect, What model of actions is implicit in Ben’s thinking?

And this, essentially, is the raw material for philosophical theories of actions.

Focus on Ben for a moment.

What mundane purposes does thinking about actions serve? Prediction and coordination; ethical (assigning responsibility, blame; living together); normative (he wants himself and others to live it out as much as to describe how things are; there may also be something about ‘understanding’ here). So it’s not all about accuracy; in fact, of these, only prediction and coordination even potentially requires that his model of actions is accurate.

Functions of Ben’s model of minds and actions:

ethical
legal
normative
predictive
Second, consider Ben’s concern with making predictions.
--- speed vs accuracy
Whenever you are making predictions about anything at all, you face a \textbf{trade-off between accuracy and speed}. Making more accurate predictions requires considering more information and integrating it in a more complex model of minds and actions. By contrast, making faster predictions requires narrowing the information you consider and using a less complex model of minds and actions. Since Ben often has to make predictions fast enough to actually coordinate his actions with Ayesha’s, and since making predictions consumes scarce cognitive resources, Ben is usually needs to trade accuracy for speed.
So Ben’s model of minds and actions is not built for accuracy.

So this is the model of minds and actions on which many philosophical theories are based ... they are cast as attempts to characterise this model.

two problems

If we rely on beliefs and desires as paradigm cases of representation, ...

... we may thereby be rejecting some widely accepted claims about what representations are; and

... we may be relying on an unspecified notion that is not optimal for explanation.

habitual process

Action occurs in the presence of Stimulus.

Agent is rewarded [/punished]

Stimulus-Action Link is strengthened [/weakened] due to reward [/punishment]

Given Stimulus, will Action occur? It depends on the strength of the Stimulus-Action Link.

‘goal-directed’ process

Action leads to Outcome.

Belief in Action-Outcome link is strengthened.

Agent has a Desire for the Outcome

Will Action occur? It depends on the Belief in the Action-Outcome Link and Agent’s Desire.

Implicit functional role for Dickinson’s representations (maybe we do not need to equate them with beliefs; and maybe we should not because that would imply that they are inferentially integrated with all beliefs)

✔︎

what are representations?

Keyboard Shortcuts?f

Goal-Directed and Habitual Processes

Keyboard Shortcuts`?f`