F INDINGS IN P SYCHOLOGY R ESEARCH

3.3.1 General Findings Indicate Linear Judgment Policies

Cooksey (1996, p.79) quotes Hammond et al. (1975) as succinctly summarizing the result of policy capturing studies comparing judges:

“Empirical regularities include the following general conclusions: (1) people do not describe accurately and completely their judgment policies, (2) people are often inconsistent in applying their judgment policies, (3) only a small number of cues are used, (4) it is difficult to learn another person’s policy simply by observing his judgments or by listening to his explanations of them (5) cognitive aids can reduce conflict and increase learning, and (6) linear additive organizational principles (i.e., nonconfigural information integration) are often adequate to describe judgments.”

Other reviews are consistent with this and state that findings generally indicate that linear judgment models can describe most judges and that experts are limited in the same manner as novices (Einhorn 1971; Slovic et al. 1977; Shanteau and Stewart 1992; Brehmer 1994).

3.3.2 Extent of Configurality may be Underestimated

The validity of the linear model in representing human judgment processes has, however, been questioned for many decision-making tasks (Hogarth and Karelaia 2007). This is due to linear judgment policies being cognitively demanding to execute (Elrod et al. 2004; Hogarth and Karelaia 2007). Judges may therefore resort to simplifying heuristics when the amount of information increases (e.g., more than three cues) or when they find the trade-offs involved in linear judgment policies to be too difficult (cognitively or emotionally) (Hogarth and Karelaia 2007). Such simplifying heuristics may imply configural cue processing (e.g., judgment policies where judgments are based on only one cue and remaining cues are ignored).

Furthermore, Cooksey (1996, 183) cautions that the lack of findings supporting configurality may be due to judgment analysis research designing out configurality by way of cue selection (i.e., configurality is not appropriate or expected in solving the experimental tasks since the relationships between the criterion and the cues is linear). This view is consistent with Brown and Solomon’s (1990 and 1991) explanation for not finding configural auditor judgments; it was not appropriate for the given tasks (i.e., the tasks were not designed in a way that made configurality appropriate).

Finally, the accuracy of judgments depends on both the inherent predictability of the environment and the extent to which the weights humans attach to different cues (and their interactions) match those of the environment (Hogarth and Karelaia 2007). In judgment tasks where the relationship between cues and criterion is nonlinear, configural judgment policies should therefore be appropriate. It is not unreasonable to assume that experts over time learn to apply such configural judgment policies when appropriate to the environment.

Since most research has used tasks with linear cue-criterion relationships (Cooksey 1996, 183), the potential use of configural judgment policies in tasks with nonlinear cue-criterion relationships may have been overlooked.

Empirical psychology research has found some evidence of configurality. Examples include studies of stockbroker’s judgments (Slovic, 1969), of psychiatric medical professionals (Rorer et al. 1967), and of moral judgment (Leon at al. 1973) (Brown and Solomon 1990).

Studies focusing specifically on the use of noncompensatory models (e.g., Einhorn 1970 and

1971) have also found some supportive evidence, but further evidence is needed before definite conclusions can be reached (Cooksey 1996 p.185).²⁵

Cooksey’s (1996 p.183) refers to Slovic and Lichtenstein (1973) for an early and comprehensive review of configural cue usage and Stewart (1988) for further discussion. His own summary of findings and the state of research reads as follows (Cooksey 1996, 183-185):

“While one can frequently find support for the existence of configural cue usage (…) the contribution from such usage is typically quite small compared to the overall contribution of linear main effect cue usage (…) While current judgment research comparing linear judgment representations to configural or nonlinear representations is relatively rare, it is fair to say that the balance of the evidence favors the use of linear compensatory judgment models to capture judgment policies. Nonlinear and noncompensatory models may, for certain judges under certain specific conditions, yield marginally better predictive power for the sample of cases on which the model is constructed, yet these models will frequently not cross-validate as well as the more parsimonious linear model. Nevertheless, judgment analysis should routinely investigate such models rather than assuming apriori that they will not be appropriate.”

Cooksey therefore urges researchers to continue looking for configural judgment policies even though they historically have been found to be of little importance and sample specific.

Overall, psychology research gives an impression of unease about the general finding of linear judgment policies. This unease is reasonable given (1) experimental tasks generally including linear cue-criterion relationships, and (2) the extensive heuristics and biases literature, where explicit recognition is given to the limits of human information processing (i.e., use of simplifying heuristics that may involve configurality) (Hogarth and Karelaia 2007).

25 In a compensatory model the judgment, based on any cue, may be offset by considering one or more of the other cues. In a noncompensatory model, the judgment may be determined by the level of only one of the cues, irrespective of the level of other cues.

3.3.3 Development of Judgment Models

Parallel with the search for configural judgment policies, a related research stream has focused on developing mathematical models of judgment policies (Einhorn 1971; Cooksey 1996; Elrod et al. 2004). Valid mathematical models of judgments are important because they provide precise specifications of theory (Elrod et al. 2004). The availability of mathematical models also allow researchers to infer the unobserved judgment policy from the observed judgments, eliminating reliance on self-reports, protocol data or multiple observations of intermediate steps which are often unavailable or unreliable (Elrod et al.

2004).²⁶ Even though the models do not measure the actual mental process which produces the judgment (i.e., they only provide surface relationships between inputs and outputs), the implications of different types of models may be important in terms of the mental processes they suggest (Libby 1981, 44). Libby (1981, 44) suggests that the most relevant models in accounting research may be additive compensatory models with positive or negative interactions, and conjunctive and disjunctive models (these models will be defined under the theory development section of this dissertation).

3.3.4 Summary of Psychology Research

The general finding from psychology research is that linear models describe human judgment (Hogarth and Karalaia 2007). Furthermore, configurality is not beyond human judges, it is just not very typical of human judgment (Brehmer 1994; Cooksey 1996).

Overall, however, an impression of unease about this general picture exists. First, the heuristics and biases literature suggests widespread use of heuristics that may involve configural cue processing. Second, experimental tasks may have made linear judgment policies appropriate. Third, some evidence of configurality exists, especially for noncompensatory models.

In order to answer questions about whether there will be configural components in judgment models, consideration of the characteristics of the specific task is therefore needed (Brehmer 1994; Cooksey 1996; Stewart et al. 1997). Furthermore, nonlinear, noncompensatory models

26Self-reports may be unreliable because subjects may be unaware of their own judgment policies or unable to report them accurately. Methods for collecting protocol data may interfere with the decision process they measure (Elrod et al. 2004).

may be useful for describing potential judgment policies (Einhorn 1971; Libby 1981, 44;

Cooksey 1996; Elrod et al. 2004).