This is third of nine sections.
1. Introduction.
2. The cognitive processing element.

Lisanne Bainbridge

3.1. Data acquisition

This subsection outlines how cognitive needs can be met by accessing data which is in the environment or in stored knowledge.

Figure 3.1, a fuller version of Figure 2.3.1 with the stepped implementation arrows completed, illustrates two of the three basic types of direct data acquisition : from the environment, and from a knowledge base. (In this paper, the word 'information' refers to evidence from the environment, the word 'knowledge' to evidence from the person's memory, and the word 'data' means either.) The third method of meeting cognitive needs by data acquisition, by reference to nonlocal working storage, is discussed in Section 4 and Section 5.

Figure 3.1 : Working method for reacting to an alarm signal, with references to knowledge base. (from Bainbridge, 1992, Figure 6). For knowledge base, see Figure 3.1.2.

Even in a modest example, to account for Table 2 (nuclear power example), it is too complicated to put everything in one diagram. In clarification :
Figure 2.3.1 describes the behaviour in terms of cognitive needs, and meeting cognitive needs by subsidiary 'routines'.
Figure 2.3.2 shows the cross referencing links in working storage.
Figure 3.1 shows that cognitive needs are met from the environment or from the knowledge base.
Figure 3.1.2 illustrates what is in the knowledge base.

3.1.1. Attention to an External Source

The data needed may be obtained by attending to the environment : by listening, by looking, or by reaching and feeling in the appropriate direction. In this case the stepped arrow implies transfer of control from the originating cognitive need to visual or other sensory search and interpretation processes (which might be done in parallel). There may be no problem with obtaining the external information : the source and interpretation of information in the environment may be obvious or familiar. Or finding and interpreting the information, before it can be used to meet the cognitive need, may themselves involve cognitive processes, which control the search, interpret what is found, and translate it into a form usable by the originating cognitive need. The focus of classical interface human factors/ ergonomics is on minimising the need for this additional cognitive processing, before interface information can be used in task-related cognitive processing.

3.1.2. Accessing the knowledge base

In this case, the data needed may be obtained from the person's memory, that is, by transferring control to search in the person's knowledge bases. This section will introduce some points about this, using the nuclear incident in Table 2 as an example.

Table 2 shows that when the high radiation alarm went off the team of nuclear operators did not immediately assume there was high radiation, but instead thought about other reasons for the alarm. In the cognitive processes in Table 2, and in the following 10 minutes, the operators considered three possible explanations for the alarm, all of which were inferences provided by their knowledge of the situation, rather than information given to them directly by the interface. In checking which of these three explanations was actually the case, they used knowledge of two attributes of the possible explanations : the likelihood of an explanation, i.e. how frequently it has been the reason, and other evidence which would confirm or disprove a given explanation. In Figure 3.1.2, this knowledge is represented by a network with different types of nodes : different attributes are represented by different typefaces. The knowledge could equally well be represented by a 'coloured' graph with different types of links, or by a table. There is no necessary commitment to a particular form of representation here.

Figure 3.1.2 : Knowledge base referred to by the working method in Figure 3.1. This knowledge links evidence to underlying causes (from Bainbridge, 1992, Figure 7).

What is important is a general principle that, when people are doing a cognitive task with which they are experienced, the appropriate knowledge appears to be accessed automatically, without using cognitive processing capacity or conscious awareness of doing cognitive work to search for the data. This suggests that access to the knowledge base, and the organisation of the knowledge in that base, have become mutually compatible through use. This is represented here by having the originating cognitive need (in Figure 3.1) and the relevant attribute needed (in Figure 3.1.2) in the same typeface. This notion of compatibility between access and storage is implicit in Bainbridge (1989), and explicit in Bainbridge (1992). Bainbridge (1992, 1993) and Section 8, part 8.2.2 discuss the possibility that knowledge bases can be specific to particular modules of processing.

This small example only raises simple issues about the representation of knowledge. Some more aspects which need to be considered, when representing the knowledge used in complex dynamic tasks, are discussed in Section 8.

3.2. 'Routines'

If the data required to meet a cognitive need is not immediately available from the environment or the knowledge base, then some processing needs to be done to find it. The approach to processing used here assumes that cognitive needs are progressively expanded into more detailed cognitive needs, until the point at which some basic mechanism is reached, such as comparing, calculating, or using the perceptual-motor system. (I have not determined the principles defining this lower limit. When simulating the verbal protocol, the limit was practical rather than principled - I did not go into further detail when there was no evidence in the protocol about what the detail might be.) Inversely, cognitive needs are grouped into 'routines' which meet a cognitive need at a higher level. this section outlines the nature of these 'routines'.

3.2.1. The properties of a 'routine'

Above the lower limit, any grouping of cognitive needs which (when executed) meets a higher level of cognitive need has been called a 'routine', and is represented in the figures by cognitive elements linked by a dashed line, as in Figure 2.3.1.

The account of the nuclear operators' cognitive processes, used so far as a simple example, was based on inference from the general analysis by Pew et al (1981). The examples of cognitive processes in the rest of this paper are based on more detailed data : a verbal protocol given by a steelworks operator controlling the allocation of electric power to steel melting furnaces. In developing the simulation of this protocol, the 'routines' were identified from repeated sections of the protocol. I divided the protocol phrases into groups in which the phrases appeared to be concerned with a common topic. Then I compared groups of phrases which appeared to be concerned with the same topic, and which were similar in content. The 'routines' were then developed to describe, or generate, these groups of phrases. Bainbridge (1972) contains a full account, Section 2, part 2.2 and Bainbridge (1974, 1990) contain a simple example of the analytic method.

The word 'routine' is in inverted commas, because this grouping of cognitive needs is more flexible than a routine in a conventional computer programming language. These 'routines' have at least five special properties.

1. The order in which the cognitive needs making up a 'routine' are met is not rigid. The cognitive needs are only met in a certain sequence if meeting some needs requires the results from another need, otherwise the protocol evidence suggests that they can be done in any order. This implies some sort of underlying parallel representation of the 'routine', which appears in sequential form only when it is actually carried out. The simplification into a serial form has just been used to produce a simple diagram. That is, sequential processing emerges as a property of behaviour when a working method is actually used, rather than being inherent in the underlying representation which generates the behaviour. This flexibility is indicated by using a dashed line to link the elements in a 'routine'.

2. The protocol also suggests a second way in which a 'routine' is not a rigid sequence. The person may be able to interrupt the 'routine' at any point defined by the completion of a cognitive need, go and do something else, and then return to this point. There is not much direct evidence on whether this interruption can occur in any level of 'routine', or only a 'higher' levels. This is discussed further in Section 5.

3. The three types of link between the processing elements in a 'routine' (directly between its elements, to a compatibly structured knowledge base, and the cross-references in working storage) make three interconnected and interdependent networks - of the cognitive needs in a routine, of the links to and within the knowledge base, and of the cross references in working storage. These three networks are mutually reinforcing.

4. The items seem to have an integral part-whole relation to each other, in making up the 'whole' of meeting the higher level need. There seems to be an inherent stopping rule in many aspects of human cognition, as pointed out by the Gestalt psychologists. There is some in-built recognition that items together make a whole which is greater than the sum of the parts. In some complexity theories, forms of larger organisation appear as emergent properties when large numbers of simple items are active together. As the type of element suggested in this paper has not been tested on a computer, it is not possible to say whether larger forms of organisation would emerge from these elements in action, via the properties outlined above, or whether their organisation into 'routines' can only be accounted for by adding another mechanism, such as one for recognising that given elements are sufficient to meet the need : 'it works'.

5. This will not be discussed fully until Section 6 and Section 7, but a group of elements making up a working method which meets a cognitive need (i.e. a 'routine') can have associated with it meta-knowledge about the general properties of this working method, such as how long it takes, how much effort it involves, and what general type of result can be obtained. This meta-knowledge is used in choosing between alternative methods for meeting a cognitive need. This meta-knowledge, and the independence of the cognitive e need and the 'routine' for meeting it (the link represented by the stepped arrow) are powerful factors which will be discussed more fully in Section 6.

These points apply to well established 'routines' in cognitive skill. The element described in this paper does not provide a full mechanism for learning, but it does lead to some interesting points about learning which will be discussed in Section 7.

3.2.2. Additional mechanisms in 'routines'

The 'routines' described so far consist of linked cognitive elements. Four more mechanisms can be involved in 'routines', but were not needed to account for the simple fragment of nuclear operator behaviour.

1. Alternative sequences. As mentioned above, in many routines it is necessary to do a particular group of behaviours to meet a cognitive need, but not to do these in any set order. For example, before calculating the time when a furnace stage will end, it is necessary to find out the time when the stage started, and the length of time the stage will take. But it does not matter in what order these two needs are met, and the protocol mentions them in different orders. In the diagrams a bracketing arrow is used to indicate this parallel storage and freedom of sequence, as in Figure 3.2.2.1. Here it is assumed that the choice of the order in which to do the items is random. There are more complex and more structured sequencing decisions at a higher level, as discussed in Section 4.

Figure 3.2.2.1 : A way of representing the freedom of sequence in which processes are done.

There are also three other symbols in Figure 3.2.2.1 which are not in Figure 2.1.

a. The representation of cross references in working storage has been simplified. A box containing a thin arrow indicates that its contents are found by reference to working storage earlier within this routine, i.e. to local working storage. A box containing a thick arrow indicates cross reference outside this routine, to data available elsewhere in the processing (for more on this see Section 4 and Section 5).

b. The underlined words (italic in some figures), e.g. read, describe basic actions.

c. The parts of the routine written in UPPER CASE letters are explicitly mentioned in the verbal protocol. The parts written in lower case letters have been added by the analyst, on the basis of the inference that these processes must have been done, or the operator would not have been able to say the things which he did say.

Figure 3.2.2.2 : An example of a conditional element (Bainbridge, 1972, Figure 7.2.6).

2. Conditionals. A choice between alternative activities is represented by a conditional version of the element. There is a simple example in Figure 3.2.2.2 : the conditional question is indicated by an oval shape. The cognitive need, tested in the condition, is met either by reference to working storage, as in Figure 3.2.2.2, or by carrying out a 'routine'. Once the data has been obtained, processing continues, depending on the result of the conditional test. (For more on conditional elements, see Section 4.)

3. Iteration. Sometimes the protocol includes processing which is carried out on several items. This iteration does not appear to require any unusual principles. It can be represented simply by a multiple box, as in Figure 3.2.2.3. (For more on multidimensional working storage, see Section 5, part 5.2)

Figure 3.2.2.3 : An example of iteration (Bainbridge, 1972, Figure 7.2.5). The furnace operator is checking which of the furnaces have already had an action made on them.

4. Two arrow box. In some 'routines' in the protocol, data needed for a given cognitive element was obtained sometimes by carrying out a 'routine', sometimes by cross reference to data obtained recently by other processing. This is represented in the diagrams by a 'two arrow' box, i.e. a box which both contains an arrow indicating cross reference, and has a stepped arrow for calling on a 'routine'. This mechanism means that working storage is referred to if the data is available, otherwise the 'routine' is carried. For an example, see Figure 3.2.2.3 (melt list). This mechanism is the basis for several important aspects of behaviour discussed later. In particular it means that either serial or parallel processing can emerge from the same mechanism, depending on circumstances.

Figure 3.2.2.3 represents a small 'routine' which uses all four of the additional mechanisms and symbols. The aim of this 'routine' is to make a list of the furnaces which are currently both melting (shown by a red light) and using less than 100% of the optimum electric power. If a furnace is charging, and so using no power, the operator notes important features of this stage. A flat static diagram is very inadequate as a presentation of the dynamic and adaptive nature of the actual processing in action.

3.2.3. The main 'routines' in industrial process operation

The previous subsection used some of the 'routines' identified from the steelworks protocol as illustrations. About a dozen main 'routines' were needed to describe the operator's behaviour. The cognitive needs which they met are listed on the left side of Table 3.2.3.1. A general survey, of detailed analyses of operator cognitive processes in several industries, has identified several general classes of main cognitive need or function which recur in process operation, though they are not necessarily all needed in all tasks (Bainbridge, 1992) :

infer/ review present process state

review/ predict events

predict state

review/ predict task goals

evaluate process state

review availability of compensatory events

review action availability and effects

choose best action

identify need for enabling action

plan future activities

monitor action effects

Table 3.2.3.1 :
Data Items found by Main Routines (A), and determining sequence of behaviour (B).
Adapted from Bainbridge, 1974, Table 1.

	A	B
	Basic Routines find :	Sequencer decisions use data about :
Process state	a) control error. b) step change.	a) size of control error. b) step change. c) time in half-hour.
Action availability	a) list of furnaces with power supply cut. b) furnace chosen for action. c) size of action to be made.	a) list of furnaces with power supply cut. b) furnace chosen for action. c) furnaces previously altered.
Stage lists	stages furnaces are in now, listed by type.	furnace stages now.
Predictions	a) for each furnace : 1) next stage. 2) time of change to next stage. 3) power usage change at that time. b) control state after next power changing event. c) action to be made after next power changing event.	a) predicted events causing : 1) power usage change. 2) correction of control error. b) future events in this half-hour. c) control state after next event. d) need for action after next event.

If there are many main cognitive needs, this raises the question : what determines the order in which the main cognitive needs are thought about, the overall organisation of the behaviour ? When operators first think about their task, for instance at the beginning of a shift or after an unexpected change in process state such as an alarm, there is at least partly a necessary sequence to meeting these main cognitive needs. Thinking about some needs must be done earlier to give the data needed in thinking about other topics. Figure 3.2.3 shows some possible dependencies. This suggests that the results of one part of the processing are available for reference by other parts. If the results of all the aspects of processing (main cognitive needs) are available, this would suggest the operator has a general overview of the state of the task, which can be referred to by any part of their cognitive processing (see example in Section 4).

Figure 3.2.3 : Main interdependencies between aspects of working storage (Bainbridge, 1992, Figure 1).

When a person first comes to a task, they carry out all the processing needed to build up the required overview. When they have been through this once or twice, and built up a relatively complete picture for reference, then they only need to update it. This build up has been seen in studies of cognitive processes in complex tasks, e.g. Bainbridge, 1972, Umbers, 1976. Observers have also frequently noticed that process operators may arrive quarter of an hour or more before they are due to take over control, so they can build up this picture of the overall situation before they have to make decisions about the process. And this need for extensive thinking about the situation, before a person has sufficient data to make informed decisions, has implications for the design of automated systems in which human beings are expected to take over control when the automatics fail (e.g. Bainbridge, 1983)

Once the operator has been through all the relevant main cognitive needs at least once, so they have an overview describing the state of the task, then the main cognitive needs may be thought about in a very varied order. Table 3.2.3.2 is part of a protocol from a gas-grid controller collected by Umbers (1976). The main text is the protocol. The bold comments are an analyst's inferences about the main cognitive need being met by the prior group of phrases. The reasons for thinking about the cognitive needs in this order are not immediately obvious.

Table 3.2.3.2 : A section of a gas-grid controller's protocol, illustrating the flexibility of sequencing (Bainbridge, 1992, Table 3).

I'll give you a quick resume of what the forecast was this morning,
that is forecast estimate,
and the estimate for this morning was 75 of MG [manufactured gas) and the NG[natural gas] send of 150 m.
predict state

the stock aim on both MG is 62.6 and NG 180.7
review goals

we have 3 plants at work, the 3b at C/hill, the 1c at Tipton, and No. 3 at W H,
this gives us a total cum make per hour. of 4.2 approx without B/ air
review present state

B/ air will give us something in the region of -
sorry, a correction to the total available without B/ air would be approx 4.4
and there's 0.8/hour. of B/ air,
so that we can in fact make 5.2 with full NG,
This is something in the region of 129-130 for the day.
review action availability and effect

we don't contemplate using this much
evaluate

but there is the facility for taking NG off the reforming and putting Naphtha in,
which will give us another increase of something in the region 0.30-0.35/hour.
review action availability and effect

so that we're not too badly off
evaluate

if the weather forecast which they give for tomorrow has been mostly sunny and warm,
with a max of 68 and a mean of 58 F.
predict goals

we shall cope with the send on MG with what we've got quite admirably.
evaluate

So, the requirement to process several main cognitive needs suggests the importance of two features : an overview in working storage of the overall state of the task, and a way of determining the sequence in which the main cognitive needs are thought about. It turns out that these two can be dealt with by the same mechanism. This is discussed more fully in Section 4 and Section 5.

Summary of main points in Section 3

* Cognitive needs may be met by direct data acquisition from the environment, from the working storage associated with other cognitive needs, or from a knowledge base.
* If the environment is badly designed, or the person is not experienced with the task, then finding and interpreting the data may require cognitive processing.
* For an experienced person, access to the reference knowledge, and the structure of that knowledge, are mutually compatible.

* Cognitive needs are otherwise met by processing 'routines', themselves made up of cognitive processing elements to some lower limit of basic processes. These 'routines' have different properties from those of a conventional programming language. (This distinction should become clearer in later sections.)
* The two-arrow box symbol means that if data is not available from working storage, it is met by carrying out a 'routine', i.e. this mechanism may produce serial or parallel processing depending on the circumstances.
* The processing element can be in a conditional form.
* The elements in a 'routine' have a part-whole relation to each other.
* The three types of link in a 'routine', between its elements, between the items in working storage, and to a compatibly structured knowledge base, make three interdependent and mutually reinforcing networks.
* The order in which the cognitive needs making up a 'routine' are met is not rigid, serial processing is an emergent property from a parallel representation.
* One 'routine' may be interrupted by another of higher priority.
* A 'routine' has associated meta-knowledge about its properties, which is used in choosing between alternative methods for meeting a cognitive need.

* A complex dynamic task involves several main classes of cognitive need.
* The results of meeting the main cognitive needs are maintained in working storage as an overview of the state of the task.
* The main cognitive needs may be thought about in a very varied order.

©1997 Lisanne Bainbridge