Analysis of Complexity in Cognitive Tasks Graeme S. Halford School of Psychology University of Queensland gsh@psy.uq.edu

1. Analysis of Complexity in Cognitive TasksGraeme S. HalfordSchool of PsychologyUniversity of Queenslandgsh@psy.uq.edu.au

2. Acknowledgments • This work has been developed in collaboration with Bill Wilson, Steve Phillips, Glenda Andrews, John Bain, Andrew Neal, Murray Maybery, Keith Holyoak, Thomas Suddendorf, Paula Irving, Christine Boag, Julie McCredden, Damian Birney, Geoffrey Goodwin, and many other people have contributed in some way. Their contributions are gratefully acknowledged. We also acknowledge the contribution of Air Services Australia to our analyses of air traffic control situations.

3. The problem of what makes tasks complex occurs in many areas of cognitive psychology: • cognitive development • mathematics and science education • animal cognition • industrial situations (e.g. air traffic control) • Comparison of task difficulties depends on solving this problem

4. Peter Tom Peter Tom Mark Tom Mark • A transitive inference problem such as the following might seem rather simple: • Tom is happier than Mark • Peter is happier than Tom • Who is happiest? • To answer this we might order the elements:

5. Peter Tom Peter Tom Mark Tom Mark • Integration of premises produces a measurable processing load effect. • What is the source of this? • Answer: We have to consider both premises to assign an element to a slot.

6. How do we quantify this load? • Premise integration entails assigning entities to three slots. • It relates three variables • The task entails processing a ternary relation. • We can quantify complexity of tasks in terms of the relations they entail. • More complex tasks entail processing more complex relations. • There are three requirements to establish its validity:

7. Requirements of relational complexity theory: • . Tasks with the same complexity should be similarly difficult. • . Tasks with lower levels of relational complexity should be simpler, while tasks with higher levels of relational complexity should be harder, given that materials, procedure and knowledge are constant over levels. • . Performance should be predicted by tasks of the same level of relational complexity from different domains.

8. Task difficulty can vary for many reasons: • declarative and procedural knowledge • method of presentation, and strategy adopted. • Transitivity is one of a set of tasks that are difficult for young children. • The causes of this have been controversial, but it is fair to say that, when allowance has been made for the many hypotheses in the literature, there is a component of the difficulty that requires to be explained. • Our research suggests that the complexity that is inherent in the task is one source of this difficulty.

9. Another task that has been similarly difficult is class inclusion. The classical question is to ask children: are there more blue things or more circles? Children under about 5 years typically say there are more circles. There are a number of known sources of false negatives, but when these are corrected (e.g. using the procedures of Hodkin, 1987 or Halford & Leitch, 1989) the task is still difficult for young children.

10. There is a superordinate class (blue), a subordinate class (circles) and a complementary subordinate class (triangles). • Class inclusion, like transitivity, is a ternary relation. • The three classes must be assigned to the appropriate slots in the hierarchy. • Blue is assigned to the superordinate slot because it includes triangles and circles. • This imposes a processing load, that has been verified by Halford & Leitch (1989). • What is the cause of this difficulty?

11. Transitivity and class inclusion are superficially different, yet both entail ternary relations: • Transitivity and class inclusion were both originally Piagetian tasks, but • Other paradigms that entail ternary relations and are similarly difficult for young children: E.g. concept of mind.

12. The the relation between a property of an object, and the person’s percept, is modulated by a third variable, the viewing condition. • The concept of mind task entails relating three variables, and is ternary relational. Consequently a structural complexity effect was predicted ahead of evidence (Halford, Children’s Understanding: The Development of Mental Models, 1993) and has since been confirmed.

13. Concerning the second requirement for relational complexity theory, if we can reduce the relational complexity of a task, holding materials and procedure constant, then difficulty should be correspondingly reduced. • We have done this successfully in several concept domains. • The procedure for transitive inference is shown in the following Figure:

14. Transitive Inference Premises blue red green yellow purple blue green red green green red red blue blue Binary Ternary

15. The ternary relational task is to first place green and blue. It has to be inferred that green is above red and red is above blue, therefore the order is green, red, blue. This is a ternary relation. Once green and blue are in place, the child is asked to place red. This is very easy if the correct order has already been mentally constructed, and serves as a check on guessing or other lower level strategies.

16. The binary relational task is to first place green and red. This can be inferred from the single premise green above red, and entails processing only a binary relation. Then the child is asked to place blue. This can be done by processing the binary relation, red above blue.

17. Notice that binary and ternary tasks entail placing the same three blocks, using the same set of premises. The percentages of children at each age succeeding at the .05 level are shown in the following figure:

18. Children’s performance: transitive inference

19. The comparatively small change in procedure produced a dramatic difference in performance, especially for the younger children (Andrews, 1997). Relational complexity produces powerful effects on performance. • Relational complexity can also be manipulated with classification tasks. • The category induction task (Gelman & Markman, 1987) is binary relational, because it entails a category and its complement. If we tell a child that (say) cats climb trees, and ask them whether the attribute tree-climbing applies to instances of a complementary category. The structures of category induction and class inclusion are in this figure:

20. Complementary Basic Basic A Complementary Subordinate Subordinate Cold water fish: have bones Sharks B Trout: Have dark flesh Salmon: Swim upstream Structures of category induction and class inclusion

21. Assessing Category Induction and Class Inclusion • To make them comparable, the tasks have to be assessed by the same method. Both can be assessed by property inference (Johnson, Scott & Mervis, 1997). • We used familiar categories, but unfamiliar attributes (Halford, Andrews & Jensen, 1998).

22. Cold water fish: have bones Sharks Trout: Have dark flesh Salmon: Swim upstream 1. Basic to complementary-basic: Do all sharks have bones? 2. Basic to basic: Would this cold-water fish have bones? 4. Subordinate to subordinate: Would this salmon swim upstream? 3. Subordinate to complementary-subordinate: Do all trout swim upstream? Within-level questions

23. Cold water fish: have bones Sharks Trout: Have dark flesh Salmon: Swim upstream 1. Basic to complementary-basic: Do all sharks have bones? 2. Basic to basic: Would this cold-water fish have bones? 4. Subordinate to subordinate: Would this salmon swim upstream? 3. Subordinate to complementary-subordinate: Do all trout swim upstream? Within-level questions

24. Relational complexity theory predicts that category induction should be easier than class inclusion. To test this hypothesis, the tasks must be assessed by procedures that are strictly comparable. This can be done by property inference: Predictions: Relational Complexity Theory

25. than of non-salmon (trout). Property Inference 1: An attribute should generalise more to instances of the same category than to instances of a complementary category (within level inference): e.g. swimming upstream is more likely to be true of other salmon……., Cold water fish: have bones Sharks Trout: Have dark flesh Salmon: Swim upstream Subordinate to subordinate: Would this salmon swim upstream? Subordinate to complementary-subordinate: Do all trout swim upstream? Predictions: Relational Complexity Theory • Getting both these questions right should indicate understanding the relation between a category and its complement - a binary relation.

26. Cold water fish: have bones Sharks Trout: Have dark flesh Salmon: Swim upstream 2. Subordinate to basic: Do all coldwater fish swim upstream? 1. Basic to subordinate: Do all salmon have bones? 3. Basic to complementary-subordinate: Do all trout have bones? 4. Complementary-subordinate to basic: Do all coldwater fish have dark flesh? Between - level questions

27. Cold water fish: have bones Sharks Trout: Have dark flesh Salmon: Swim upstream 2. Subordinate to basic: Do all coldwater fish swim upstream? 1. Basic to subordinate: Do all salmon have bones? 3. Basic to complementary-subordinate: Do all trout have bones? 4. Complementary-subordinate to basic: Do all coldwater fish have dark flesh? Between - level questions

28. Cold water fish: have bones Sharks Trout: Have dark flesh Salmon: Swim upstream 2. Subordinate to basic: Do all coldwater fish swim upstream? 1. Basic to subordinate: Do all salmon have bones? Predictions: Relational Complexity Theory • Property Inference 2: An attribute should generalise more to instances of a lower level category than to instances of a higher level category: e.g. attributes of coldwater fish should generalise to salmon. • But not necessarily the reverse. • Getting both these questions right should indicate understanding of relations between hierarchical categories. • This is a between-level inference, and corresponds to a ternary relation. The task is designed so questions are matched within and between levels.

29. The hypothesis that ternary relational reasoning would be harder than binary relational was confirmed in two experiments. • The third requirement for relational complexity theory is that performance should be predicted by tasks of the same relational complexity in other domains. • We have shown this to be true in numerous experiments. In the category experiment just outlined, we found ternary relational performance was predicted by transitivity and class inclusion, both ternary relational tasks, the first from a different domain. • In another study we obtained similar results for concept of mind, which is also ternary relational, as noted above. • In other experiments we have found that relational complexity scores in 6 domains loaded on a single factor that accounted for 80 percent of the age related variance in fluid intelligence (Andrews & Halford, submitted).

30. Relational Complexity Metric • The essence of our theory is that complexity of relations processed in the same decision (in parallel) is a good way to define cognitive complexity. • Relational complexity refers to the number of entities related. It corresponds to number of slots or arity of relations. • A binary relation has two slots: e.g. • Larger-than(_______, _______) has a slot for a larger entity and one for a smaller entity. Each slot can be filled in a variety of ways: Larger-than(elephant, mouse) Larger-than(mountain, molehill) Larger-than(ocean-liner, rowing-boat)

31. Complexity of relations can be defined by the number of slots: Unary relations have one slot: e.g. class membership, as in dog(Fido). Binary relations have two slots: e.g. larger(elephant, mouse). Ternary relations have three slots: e.g. addition(2,3,5). Quaternary relations: e.g. proportion (2/3 = 6/9). • A slot corresponds to a variable or dimension. • An N-ary relation is a set of points in N-dimensional space.

32. The relational complexity metric is very general. It has been applied to • child and adult cognition • higher animals such as chimpanzees • is currently being applied to dolphins. • industrial contexts, including air traffic control. • Relational complexity theory encompasses declarative and procedural knowledge • Processing capacity is an enabling factor, but concept acquisition is knowledge acquisition, and cognitive development is experience driven. • There is a broad correspondence between levels of relational complexity and the phenomena that Piaget attributed to stages • However, there is no suggestion that concepts of a given level are all acquired synchronously. Concept acquisition is a function of experience, given that the relevant processing capacity is available. • Relational complexity applies to explicitly represented relations. It does not apply to tasks that can be processed by perceptual or associative processes, such as transitivity of choice (Chalmers & McGonigle, 1984; Wynne, 1995).

33. Capacity limits • A quaternary relation is the most complex that adults can process in parallel • (though a minority of people can probably process quinary relations under optimal conditions) • This is a soft limit • Processing capacity is not all or none • Increased complexity produces increased errors and decision times, rather than sudden catastrophic failure. • In order to handle more complex concepts, mechanisms for reducing processing loads are required. These are:

34. Conceptual chunking involves recoding concepts into less complex relations. However there is a temporary loss of access to chunked relations. • For example, speed = distance/time, is a ternary relation, but speed can be recoded into a unary relation, speed(60kph) as when speed is indicated by the position of a pointer on a dial. • However the chunked representation does not permit us to answer questions such as “How does speed change if we cover the same distance in half the time?” • Segmentation entails breaking tasks into less complex steps, which can be processed serially. Strategies and algorithms are common ways of doing this: • e.g. adding one column at a time in multidigit addition. • Chunking and segmentation skills are important components of expertise.

35. Complexity estimatesare based on the mental models used to represent a concept, taking chunking and segmentation into account. • Variables cannot be chunked if relations between them must be considered. • Assumptions about chunking and segmentation are made explicit and applied consistently across domains. • A useful heuristic is that relational complexity cannot be reduced if the variables interact. This is analogous to analysis of variance: • Interacting variables must be interpreted jointly. • A mathematical procedure for determining effective relational complexity is described by Halford et al. (Behavioral & Brain Sciences, 1998, 21(6), 803-864, Section 3.4.3). If a relation can be decomposed into simpler relations, then recomposed without loss of information, effective complexity is equivalent to the less complex relation.

36. Applications of relational complexity theory • Now we consider some applications of the theory, beginning with work on air traffic control in collaboration with Christine Boag and Andrew Neal. • Two air traffic control situations are shown. The first has fewer aircraft, but was reported as a heavy load by the controller. The second contains a lot of aircraft, but is a simple situation for a controller. • The first situation entails more complex relations between aircraft.

37. MEL TJS KRD LAO TMK RDS Display A AERODROME A AERODROME D AERODROME B AERODROME C TO AERODROME E

38. LAT AERODROME A DJT KAL TAK AERODROME D AERODROME B WEF SDK AERODROME C TJT EWF TWE MTJ RMS TO AERODORME E Display B SRD

39. The theory has also been applied to data from chimpanzees. The relational match-to-sample task requires participants to discriminate the relation same/different between a pair of objects. Choice of YY is correct. Chimpanzees succeed, if they have been taught symbols for the relations. This indicates they can process binary relations. We have work in progress with dolphins on this task, in collaboration with Thomas Suddendorf and Paula Irving.

40. Cardinality • Understanding that counting yields the cardinal value of the set requires a mental model that incorporates the principles of counting.

41. Successor Model • In the successor model, numerals are assigned to objects.

42. Sets Model • In the sets model, numerals are assigned to sets, but it does not distinguish items that have been counted from those that have not.

43. Inclusion Model • In the inclusion model numbers are assigned to sets, which include (a) an item not previously counted, and (b) the set of all items previously counted (this set is empty on the first item).

44. The inclusion model enables recognition that the last number represents the cardinal value of the set - the cardinal word principle. • Also, because it distinguishes previously counted objects from the object currently counted, it makes the order in which objects are counted irrelevant.

45. Understanding cardinality was assessed using three question types: How many: Children should say the last number they mentioned in counting S, and should not recount. Show me X: Children should indicate all the objects in the set, not just the last object. Reverse count: Asking children how many there would be if counted in the opposite direction.

46. Fractions: Quaternary Relation • Fractions and proportions cause difficulty for young children • (for reasons that never appear to have been completely explained) • Proportion is a quaternary relation.

47. Fractions: Quaternary relation • The examples alongside show that comparison of fractions entails working with four dimensions, because a change in either numerator or either denominator can affect the comparison. • Comparison of integers requires working with only two dimensions. • The powerful effect of structural complexity has been largely overlooked.

48. In order not to be misunderstood: • There is nothing to prevent younger children from understanding fraction concepts that entail simpler relations. (e.g. half a pie entails only binary relations, and should be understandable by two year olds). • Notice that the predicted age is actually earlier than appears to have been observed. • A good relational complexity analysis frequently uncovers unrecognised potential; • e.g. it predicts that even two year olds should be able to make balance scale judgements based on either weight or distance, though not both. • This is more optimistic than current norms, and has been confirmed empirically (Halford& Dalton, 1995) • Relational Complexity Metric

50. The relational complexity metric forms a single scale from the most basic psychological process, elemental association, to the most complex level of which humans are capable. • Sample concepts at each level are shown, together with: • the corresponding Piagetian stage and: • the neural net architecture that is characteristic of each level. • There is a unique empirical indicator for each level, and • each level imposes requirements on neural net models, over and above requirements of lower levels. • This provides converging evidence that the levels are genuinely distinct.