Judgment Policies and Judgment Models

Several psychology researchers have argued for the importance of judgment task characteristics for the form of the judgment policy, and for the importance of developing mathematical representations (i.e., models) of judgment policies (e.g., Einhorn 1970 and 1971; Brehmer 1994; Stewart et al. 1997; Elrod et al. 2004). A judgment policy is defined as the way in which cues are combined by the judge when making a judgment. A judgment model is a mathematical representation of the judgment policy. Valid mathematical models

of judgment policies are argued to be important because they provide a precise specification of theory (Elrod et al. 2004). A wide range of models have thus been developed.

Audit research has been limited to studying judgment tasks with independent, compensating or amplifying controls and searching for compensatory, linear judgment models with amplifying or compensatory form ordinal interactions (e.g., Brown and Solomon 1990 and 1991). Non-compensatory judgment policies have not been studied in internal control research, although they are known to audit literature (e.g., Libby 1981, 46).

This study proposes that the following models may be relevant for audit research. Assume that “y” represents the judgment and that “y” is a function of cues “ci”:

Let y = f(c1,c2), where ci = 0,1 and i=2, and

0 = 0/100 < f(c1,c2) < 100/100 = 1

This provides the following judgment models (see table 2 below):

Table 2: Mathematical Representation of Judgment Models

Model Mathematical Form Main effects and interactions 1. Conjunctive f(1,0) = f(0,1) = 0, and f(1,1) > 0 No main effects

Complete positive interaction 2. Amplifying 0 < f(0,1) + f(1,0) < f(1,1) Main effects and amplifying

form ordinal interaction 3. Linear 0 < f(0,1) + f(1,0) = f(1,1) Only main effects

No interactions

4. Compensatory 0 < f(1,1) < f(0,1) + f(1,0) Main effects and compensatory form ordinal interaction

5. Disjunctive 0 < f(1,0) = f(0,1) = f(1,1) Main effects and interactions that nullify all but one of the (equal) main effects

The mathematical models of judgment policies are similar to the mathematical functions of cue integration in judgment policies presented in section 3.5.3. The reason for this is that this dissertation proposes that the judge perceives the cue interrelationships in the environment and applies a form of cue integration in his judgment policy that mirrors these cue interrelationships in the environment. Although the math is parallel, it represents different constructs. In the environment, the model represents the relationship between cues and the criterion (e.g., the effect of controls on risk). In the judgment policy, the model represents the weight the judge places on the various cues and their interactions (i.e., the effects of the cues on the judgment). The models are discussed below:

Additive, compensatory, linear models (non configural)

The linear compensatory model is the simplest and most used model in judgment research. It has three main features: (1) it is additive in its attributes, which means that the judgment is obtained by simply summing the assessments of each cue considered individually, (2) it is compensatory, which implies that the judgment, based on any cue, may be offset by considering one or more of the other cues, and (3) it is linear, which means that all cues relate in a linear manner to the judgments (Elrod et al. 2004).

Additive, compensatory, nonlinear models (configural)

If the third feature is relaxed, the mathematical model can still remain additive and compensatory, but it is no longer linear. This is the case if one or more cues interact, or if the form of a cues’ relationship to the judgment criterion is of a quadratic, cubic or higher order polynomial form. The model can be represented as additive by forming product terms of the interacting cues. Even though interactions are formed by a multiplicative rather than additive organizing principle, the overall policy model is still additive and can be analyzed by ordinary regression procedures.⁴³ The two compensatory, additive nonlinear models most relevant to audit research are compensatory form ordinal and amplifying form ordinal:

Compensatory form ordinal models represent cues that contribute independently and interact in such a manner that the combined effect of both cues is smaller than the sum of their individual effects. In such models the main effects will be significant and interactions for compensating cues will be negative (i.e., ordinal form compensatory).

Amplifying form ordinal models represent cues that contribute independently and interact in such a manner that the combined effect of both cues is larger than the sum of their individual effects. In such models the main effects will be significant and interactions for amplifying cues will be positive (i.e., ordinal form amplifying).

Nonadditive, noncompensatory, nonlinear models (configural)

If the second feature is also relaxed, the model becomes noncompensatory. In a noncompensatory judgment policy, the judgment may be determined by the level of only one

43 The model remains additive by adding polynomial and interaction terms to the mathematical model.

of the cues, irrespective of the level of other cues. The two types of noncompensatory judgment policies that have received the greatest attention are conjunctive and disjunctive (Einhorn 1970, 1971; Libby 1981, 46).

In a conjunctive judgment policy it is necessary that each cue be above a criterion level (i.e., all cues are necessary), thus the judgment may be based solely on the lowest/worst cue (i.e., if it is below the criterion level (e.g., absent)).⁴⁴ It is a form of multiple cutoff judgment, where each cue is required to exceed a minimum level. An example of this would be a control process that is judged to have deficient controls unless each step in the process is above a certain cutoff criterion. This is equivalent to judging based on the worst cue: it does not help if all other cues are above the criterion as long as the worst cue is under.

Conjunctive models seem intuitively appropriate for binary judgments where all cues are necessary criteria for a given judgment; if one of the cues fail, the judgment is negative.

Furthermore, conjunctive judgments require configurality.

Conjunctive models that have binary cues (e.g., 0/1 or no/yes) can be represented mathematically by a model with no main effects and only the highest order interaction term.

In such a model, the interaction term would be zero unless all cues are positive (i.e., 1 or yes), thus it would be judged on the worst cue.

In a disjunctive judgment policy it is sufficient if at least one cue is above a criterion level (i.e., each cue is sufficient), thus the judgment can be based solely on the highest/best cue (i.e., if it is above the criterion level).⁴⁵ An example of this would be a judgment of control risk being below a certain acceptability level and where several alternative controls are assessed. As long as one of the controls reduces control risk sufficiently (e.g., the best control), the other controls do not matter (i.e., they cannot change the judgment). Disjunctive models seem intuitively appropriate for binary judgments where all cues are individually

44 An example of a high level conjunctive judgment task is the overall SOX 404 judgment (AS5.2 PCAOB 2007): “(…) because a company's internal control cannot be considered effective if one or more material weaknesses exist (…)”. Thus, if one or more material weaknesses exist, the other effective controls contribute nothing to the overall judgment.

45 ISA 330.70 (IFAC 2008) provides an example of a judgment criterion; “The auditor should conclude whether sufficient appropriate audit evidence has been obtained to reduce to an acceptably low level the risk of material misstatement in the financial statements”. The sufficient and appropriate evidence requirement is an example of a criterion where the audit evidence cues (e.g., control test results) may be assessed according to conjunctive or disjunctive judgment policies. See further discussion in the hypotheses development.

sufficient criteria for a given judgment; if one of the cues is acceptable, the judgment is positive. Disjunctive judgments require configural cue processing.

Disjunctive 2-cue models with binary cues (e.g, 0;1 or no;yes cues) can be represented by a model where an interaction term is formed in addition to the main effects. The purpose of the interaction term would be to remove one of the main effects if both cues are positive, so that only one main effect remains. The model would then represent a judgment where only one of the cues impacts the judgment, even though they are both positive. The cue weights would be expected to be equal in size, but positive for main effects and negative for the interaction. The model can be generalized to judgments with more than 2 cues by adding interaction effects that eliminate the effects of having more than one positive cue.

It can be noted that the use of a conjunctive versus a disjunctive model may depend on the framing of the judgment criterion: If the judgment criterion is framed as “is there a material weakness in internal controls”, then it is sufficient to find one material weakness and a conjunctive judgment policy would be appropriate. If the judgment criterion is framed as

“are controls free of material weaknesses”, then it is necessary that no controls are materially deficient and a disjunctive judgment policy would be appropriate.

Range of relevant models

The five models discussed above may all be potentially relevant for internal control judgments, as shown in the given examples of audit judgment tasks above and in previous research (e.g., Brown and Solomon 1990 and 1991). I refer to Elrod et al. (2004) for further generalization of mathematical models of noncompensatory judgments. The range of relevant judgment policies in internal control judgment tasks may therefore include:

• Linear

• Compensatory

• Amplifying

• Conjunctive

• Disjunctive