Preference memory
=================

The contents of working memory are determined by *preferences* for
augmentations.  Preferences are retrieved by matching recognition memory
against working memory.  Preferences are statements about the relative or
absolute worth of an augmentation's specific value in working memory. In
order for an augmentation to be added to working memory it must have an
*acceptable* or *require* preference. The acceptable preference states that
the object is a candidate for the value and will be considered in the
decision, while the require preference states that the object must be
selected if the goal is to be achieved.  If there is an acceptable preference
for a value of an augmentation, and there are no other competing values, a
working memory element will be created for that augmentation.

If there are acceptable preferences for alternative values for the same
identifier and attribute, the contents of working memory will be determined
by preferences that allow for comparisons of values.  For example, one value
can be better than another, and a third can be best.

Soar's preferences fall into five categories:

feasibility:     
      acceptable, reject 
     Feasibility preferences determine the candidates for the value of an
     augmentation.
     
exclusivity:     
      parallel
     Exclusivity preferences determine if more than one value for an
     attribute is allowed.
         
desirability:     
      best, better, indifferent, worse, worst
     Desirability preferences compare and rank the candidates for a value of
     an augmentation.
     
necessity:     
      require, prohibit
     Necessity preferences determine if a selection must or must not be made
     in order to achieve the goal.
     
termination:     
      reconsider
     Termination preferences signal that the current object can be displaced
     and a new decision can be made for the value of the augmentation.  This
     preference is only relevant for context slots.

Preferences are held in a separate *preference memory* and they cannot be
used as a cue for the productions in recognition memory.  The one exception
is that acceptable preferences for context slot objects are available in
working memory.  This is necessary because the preference scheme is used to
select problem spaces, states, and operators.  By making the acceptable
preferences for these objects available in working memory, the candidates can
be compared before they are selected.  Thus, deliberative selection, where
alternatives can be examined before selected, is possible for PSCM objects.

Just as all working memory elements must be linked to the goal context, so
must the preferences.  If the linkage is ever broken, so that an identifier
is not the value of another acceptable preference (or require preference) or
working memory element, then all preferences containing that identifier will
be removed from the preference memory.  The removal of these preferences can
then lead to changes in the contents of working memory.

The retrieval of preferences from production memory supports the PSCM
functions for proposing, comparing, and refining problem spaces, states, and
operators, as well as the termination of an operator application. Problem
space, state, and operator proposals are performed by the creation of
acceptable preferences for the appropriate context slot.  Similarly,
comparisons are performed by creating desirability preferences for potential
context objects.  Refinement is done by creating preferences for new
augmentations of context objects.  Termination occurs when a reconsider
preference is available during the decision for a context slot.

Once a preference has been retrieved by firing a production, it should not
remain available indefinitely.  A preference must be removed whenever it is
no longer relevant to the current situation.  As the problem solving unfolds,
states will change through operator application.  The top state will change
as the sensors detect changes in the environment over which the system has no
control.  For example, if a production creates an acceptable preference for
an operator to pick up a block based on augmentations of the current state,
that operator will not be appropriate if the block is later covered by a
second block, either by the system or by an outside agent.

Preferences for the PSCM functions mentioned above are held in preference
memory as long as the production instantiation that created them is still
valid, unless the augmentation is explicitly rejected during operator
application (see below).  The maintenance of such preferences is called
*instantiation support* or just *I-support*.  A preference can have more than
one source of I-support if more than one active production instantiation has
the preference as its action.  A preference loses I-support from an
instantiation if the instantiation no longer matches working memory, either
through the removal of a working memory element matched by a positive
condition, or the addition of a working memory element that matches a negated
condition.

If a preference loses all I-support (and does not have other support as
defined below), Soar *retracts* the preference.  Retraction simply means that
the preference is removed from preference memory and is no longer considered
in determining the contents of working memory.  Just as the addition of
preferences can change working memory, so can retraction, because removing
preferences can change decisions.  Through instantiation and retraction,
Soar's production memory implements a justification-based Truth-Maintenance
System.


Operator application differs from other PSCM functions in that the
changes made to working memory should persist, even after the
conditions for making the changes are no longer satisfied.  For
example, in placing a block on another, a condition is that the second
block be clear.  However, the action of placing the first block
removes the fact that the second block is clear, so the condition will
no longer be satisfied.  Such operator application productions do not
retract their actions, even if they no longer match working memory.
Preferences that participate in the application of operators are
maintained throughout the existence of the goal in which the operator is
applied.  This is called *operator support* or just *O-support*.
 If a preference receives O-support from a production
instantiation, it does not also have I-support from the same instantiation
(although it could receive I-support from another production instantiation).

The determination of whether or not a preference is part of an operator
application, and thus should receive O-support, is based on the structure of
the production instantiation that created the preference. The following
discussion gives an overview of the criteria used for determining support.
For the exact details of how support is computed for preferences, see
Appendix See Support of Preferences on page *Note Support of
Preferences::.

An operator application involves testing the current operator of a goal and
modifying the state.  Thus, a preference receives O-support if the preference
is for an augmentation of the current state or substructure of the state, and
the conditions of the instantiation that created it test the current
operator.  It may not be possible to make the determination of O-support
until the production is instantiated if only the state's substructure is
modified.

O-support is also given to preferences that instantiate an operator from the
state because that operator substructure must be maintained while the state
is changing during operator application.  For example, if an operator is to
move a block 20 inches from its current position, X, a production can fire
and augment the operator with the desired position: X + 20.  If retraction is
allowed, this value will be removed as soon as the block is moved a small
amount, thus leading to confusion during operator application. Therefore, a
preference for an augmentation of an acceptable operator or substructure of
an acceptable operator receives O-support if the conditions of the production
instantiation that created it test the current state or part of its
substructure.  In the case that an operator is proposed (through the creation
of an acceptable preference for the operator) and augmented within the
actions of a single production, those augmentations will receive O-support if
they are dependent on the state or part of its structure.  The acceptable
preference for the operator receives only I-support.

A preference loses O-support (and I-support) if an O-supported reject
preference is created for the augmentation, in which case, all preferences
for that augmentation created in that goal or in subgoals (including the
reject) are removed from preference memory.  An O-supported preference will
also be removed from preference memory if it becomes unlinked from the
context-stack.  Finally, both I-support and O-support are local to the goal
in which a preference was created.  Thus, a preferences loses O-support when
the goal in which it was created terminates (see Section *Note Subgoal
results:: on page See Subgoal results for details on the support for
subgoal results).

All attempts to make persistent changes to working memory should be made
through operator application.  Otherwise, there is a danger that productions
will "loop".  Looping occurs when the effect of a production firing causes
the production to be retracted, possibly by removing an augmentation from
working memory that is tested by the conditions.  When the production is
retracted, the action is undone, and the augmentation is added back to
working memory.  The production fires again, and the cycle continues.  For
example, a production might move one block onto a second block.  If the
production tests that the second block is clear, then it will retract when
the first block is on top of the second, only to fire again if the retraction
causes the change to be undone in working memory.  If the production is part
of an operator application, then its effects will not be undone upon
retraction.