Chris Deacon, MAÊ ICT in Education, ( Research and Development module)
Using Situated Learning and Multimedia to Investigate Higher-Order Thinking by J.Herrington and R.Oliver - A Critical Review, August 2001
This research presents a problem in the form of a question: Do learners exhibit higher order thinking as they use a multimedia program representing a situated learning context ?
The motivation for Herrington and Oliverâs research lies within the general theoretical field of higher order thinking and a consideration of learning contexts which might facilitate it. The authors draw on Baker, Lewis and Smithâs definition of higher order thinking as the interrelationship of new information and stored memory which is then applied to a problem in the search for an answer. A situated learning situation is raised as a possible context to enable higher order thinking and the authors point out that there has been little research on this possible connection. The authors also refer to the perceived potential of multimedia resources in facilitating higher order thinking and comment that to date there has been little empirical evidence cited of this relationship in mainstream learning activities. The problematic that this paper is addressing therefore positions itself by focussing on the recognition of higher order thinking during the use of an interactive multimedia program which is identified as a situated learning framework. The lack of researchÊ ( situated learning) and lack of positive results ( multimedia ) with respect to higher order learning is cited as justification for this specific piece of empirical work.
The authors empirical setting involved a group of 8 pre service Mathematics teachers using a multimedia program focussing on assessment in schools. The overall operational structure reflected Baker, Lewis and Smiths guidance in that students were given a problem and structured information within the multi media program which could be assimilated and evaluated in order to arrive at a conclusion. Students worked in pairs at the keyboard and their talk was recorded and analysed as a representation of their thinking. In concluding their findings Herrington and Oliver state that during the use of the programÊ the whole spectrum of thinking ( social, procedural, lower- order and higher- order) was evident but the majority of responses were higher- order. They comment that their findings confirmed their expectations that a multimedia program based on a situated learning approach could promote substantial levels of higher-order thinking. The authors conclude by claiming that they have identified two enabling components frequently lacking in contemporary multimedia learning situations; the situated learning informed design of the multimedia resource, and the use of the resource in a relevant problem solving situation. By implication the authors are therefore suggesting a model for improving the design of multimedia resources so that they can be a more valuable learning tool.
In this consideration of whether the authors claims are justified it will be necessary to reflect on their research design from the initial theoretical discussions, through the statement of the problem, creation of the empirical setting, to the analysis of fieldwork records and presentation of conclusions. Three critical questions to reflect on during this evaluative process are :
Did the multimedia program represent a situated learning framework either in the terms specified in this paper or as described in the source literature ?
A situated learning context is viewed by the authors as a facilitator of higher- order thinking and they claim to have constructed a multimedia resource which matches this framework. The authors list 9 characteristics of match: authentic context, complex authentic activities, multiple perspectives, expert performances, coaching and scaffolding, authentic assessment, and opportunities for collaboration, reflection and articulation. The activities, assessment and task in the program are authentic in the sense that the users ( learners) were pre service teachers and the multimedia resources focussed on different aspects of the assessment practice that they would utilise as class teachers. The video clips of assessment strategies being used in the classroom , together with comments from teachers and students, would certainly engage the learner with the relevance of this simulated assessment task. However the task was fictional and elements associated with a real situation were missing. The authors refer to the program having multiple perspectives and they are partly alluding here to the representation of a community of practice which is a notion at the core of situated learning theory. The main interface of the program shows these perspectives to be the teacher, the student and the Îexpertâ. There is no representation of parent audience views, however, beyond the letter of complaint that initiates the task and this therefore limits the authors Îmultiple perspectiveâ claim.
The claim that there are opportunities for coaching and scaffolding in the program is justified and represented by the advice given within video clips and the representation of the strategy being used in the classroom. These would certainly assist the pre service teachers to start to develop their own ideas on the use of assessment and it could represent a peripheral sort of engagement. In a situated learning context learners are said to move from the periphery to the core of the community of practice by participation based on situated negotiation and renegotiation of meaning. This process implies active communication on the part of the learner as well as thinking skills and comprehension. The authors cite the characteristics of collaboration, reflection and articulation presumably as evidence of this communication and active engagement and there is an opportunity for learner input in the electronic notebook. On closer consideration however although there is space for student reflections here there is no facility for them to receive comment or become part of a discussion. Students therefore have no opportunity for renegotiation of meaning through a process of individualised interpretative support other than with their equally peripheral peer partner.
Collaboration, as listed in the characteristics of the learning environment, would therefore take place between the two students working together but not within the program or in the use of the program which is the implied intention if the multi media program represents the situated learning site. Watching video clips, looking at pupils work and writing reflections is a means of using the resources to widen the learners knowledge but could not be viewed as collaboration with the program. The investigations and problems included in the notebook might encourage higher-order thinking but to represent collaboration they would have to be sophisticated individualised packages where the route through the problem was determined by the learner. Interestingly the collaboration between the two learners could begin to resemble situated learning within this very small and semi-virtual community of practice because each would assimilate the knowledge from the program, discuss ideas with the other learner and, through discussion and back reference to the program, develop an understanding of the meaning of assessment. It would appear then that opportunity for the situatedness of this learning is centred at the keyboard rather than within the program and by implication the grouping arrangements at the keyboard would be as important as the program structure in facilitating the higher- order thinking potential.
For the use of this particular multimedia program to represent a sound operational decision it would have to be argued that not only did it facilitate learning ( with scope for higher-level) but also that it internally represented the situated learning environment claimed by the authors. Three aspects of Îsituatednessâ ( Lave and Wenger 1991) may be applied to this program önamely, the relational character of knowledge and learning, the negotiated character of meaning, and thirdly the engaged nature of the learning activity. Within the multimedia program there is a relational characteristic of knowledge and learning ( the program holds the knowledge of the community of practice and the user can access the knowledge in order to learn) but the facility for the learner to be actively engaged in renegotiating meaning and resolving dilemmas through discussion with members of the community ( teachers, children and experts) is not evident . The students seem to have been placed as observers of a community rather than participants in what should be a sociocultural learning practice. The claim therefore that the multimedia program was based on the situated learning framework is not fully justified although it did provide a stimulus for situated learning within the whole empirical setting of their investigation.
Is talk a valid indicator of thinking and specifically in identifying the balance of higher and lower order thinking ?
Herrington and Oliver presented the students with a complex problem. A wide variety of resources were available within the program and it was necessary for the students to devise an action plan of tasks in order to build up their knowledge and understanding of assessment. In linking situated learning and higher order thinking Herrington and Oliver imply that a constructivist learning style is at the core of the process. Before arriving at their findings and conclusions students would need to assimilate new knowledge, adapt their existing mental concept maps to accommodate any cognitive concepts arising and then externalise the knowledge by constructing generalisations. These sort of actions could promote higher- order thinking which might be observable in either the student conversations, during student interviews or discussions, or the finished report. The authors chose to concentrate on the first of these indicators and evidence of the validity of this measure needs to be considered.
The operational decision to record and classifyÊ student talk as an indicator of the existence and importance of higher- order thinking was a pivotal one. The authors noted the comments by writers such as Halliday and Young that in academic discourse the higher- order thoughts may not be externalised in speech whereas everyday exchanges are more fluent and therefore more complete. This possible imbalance clearly poses a problem for researchers who are intending to record the incidences of higher- order utterances as a proportion of the total conversation record with a view to assessing the importance of higher- order thinking as a percentage of all thinking. Transferring this issue into the specific empirical setting of Herrington and Oliverâs research it would mean that the higher- order thinking initiated by working through the task and presenting findings would be under-represented in the studentsâ conversation whilst the lower- order thinking would be over represented. Herrington and Oliver suggest that because their academic task and discourse is set in a situated learning framework,Ê this provides a social context whereby students articulate and share thoughts more easily and meaningfully implying that more of the higher- order thinking would be revealed in speech. They compare this with an alternative situation where the students could be asked to communicate their thoughts to a researcher and imply that the unnaturalness of the context would influence the conversation record. The work of Von Wright is cited as support for their stance with his view that social contexts elevate thinking to an observable status.
In considering Herrington and Oliverâs arguments around this operational decision a number of concerns become evident. Within the empirical setting, the multimedia program was said originally to represent the situated learning context whilst the two students working together provided a conversation record that could indicate externalised thought. However in the discussion on the use of talk as an indicator of thought, the authors state that the social context of the situated learning framework is provided by the collaboration of the students. This would suggest a lack of clarity with respect to where the community of practice is centred in their empirical setting. The results of group 3 would seem to add weight to the notion that the students are more central to the community of practice than originally suggested. The lack of sociability of group 3 students was said by the authors to explain their low score (27%) in higher- order thinking ( once the Uncertainty and Path of Action statements had been reclassified). These students were using the same multimedia program as the other groups and yet the same Îsituatednessâ of the program and the task did not result in a similar high proportion ofÊ higher- orderÊ thought score. The authors point to the weakness of the final report of group 3 as support of the conversation record but this would also further suggest that the social context of student groups was an important factor of the empirical environment and yet it is not clearly distinguished from the situated learning aspect.
A further operational decision related to the analysis of student talk was the adoption of a classification framework and Herrington and Oliver were clear about their specifications : it had to be in keeping with the contexts of education and new technologies and compatible with interpretive research methods.Ê Henriâs framework was devised to analyse student exchanges within a computer- mediated conferencing environment . Student exchanges were analysed in terms of content providing information on the students as learners and their way of dealing with a given topic. In terms of the transferability of this framework from Henriâs empirical setting to Herrington and Oliverâs there would seem to be a good match although the Î new technologiesâ specification is rather broad . In Henriâs research the student comments were mediated and recorded by the computer program whereas in this research the computer program allowed students to access resources and record comments in the notebook but the record analysed was of taped conversation. Furthermore Henriâs framework was devised for analysing a written record of thought and discourse and then applied to a spoken record that arose from an observation of simulated practice and problem solving task.
Henriâs framework classified the content of the students exchanges into five categories : participative, social, interactive, cognitive and metacognitive. Herrington and Oliverâs classification had four categories including social, lower order, procedural and higher order. In comparing these two classifications the difference in the empirical settings between the two researches becomes apparent with Henriâs conferencing categories of Îparticipativeâ and Î interactiveâ being replaced by Îproceduralâ and Îlower orderâ as more appropriate to a resource based learning task. The cognitive and metacognitive categories become Î higher- orderâ and as this relates directly to their problem the authors incorporated six specific indicators derived fromÊ Resnickâs work to add more recordable detail : Uncertainty, Judgement and interpretation, Multiple perspectives, Imposing meaning, Metacognition, Deciding on path of action. Resnickâs original work ( 1987) presented nine characteristics of higher- order thinking and Herrington and Oliverâs first five indicators are a close match with six of Resnickâs characteristics. The last indicator, Deciding on path of action, would appear to be a combination of Resnickâs Îcomplexâ and Îis effortfulâ and was one of the indicators removed from the higher-order category at a later stage together with Uncertainty. Clearly the translation of Resnickâs characteristics of thinking into indicators for analysing conversation was not totally consistent. Some characteristics were included as indicators without modification, some were freely interpreted and modified to fit in with the purpose of the study. This also relates to the general issue of whether speech is a valid representation of thought.
Within the Summary Chart of classification of student talk ( Table 2 ) the definition of Uncertainty is stated as Î Any student talk which involved deciding on an approach to adopt, suggesting a course of action, or any expression of dilemma or uncertaintyâ. An example of this type of student talk was ö ã OK do you want to start putting anything into our notebooksä ö as it stands this would seem to represent lower- order talk and therefore the authors decision to reclassify after comments on their initial analysis may suggest that they belatedly recognised a lack of integrity between Resnickâs concept of higher- order thinking and the derived indicators used in their empirical setting. The examples of talk representing metacognition seemed particularly suspect because although they showed evidence of students being aware of their own thinking and performance, there was no evidence of a comment in which this awareness was being used to reflect on and improve performance. The authors themselves commented on the low %s of metacognition recorded and it is probably the type of higher- order thinking that is most likely to remain internalised in thought. Interestingly group 3 recorded the highest percentage for metacognition.
One aspect of the analysis of talk that the authors paid particular attention to in their operational design was the unit of analysis. Herrington and Oliver considered three different Î grain sizesâ of talk ösingle utterances,Ê passages of dialogue, and units of meaning. Following the authorsâ arguments it is clear that the suitability of each conversation unit was considered in terms of reliablility of the grain size in recognising and recording the indicators derived from Resnickâs higher- order thinking characteristics. Categorisation by passage was tried and considered unsuitable because of the difficulties of defining the passage boundaries and the inclusion of a number of different categories of talk in one passage. In a further analysis of the conversation record each student utterance was used as the unit for classification but instances were still noted of students using more than one type of talk in one utterance. The authors recognised that if only one type of talk was recorded to represent the utterance then thought was not being fully indicated. In moving to units of meaning the authors gained operational precision in terms of their total record for analysis, however an element of subjectivity had now crept in with the classifier imposing thinking divisions within the talk by imposing their notion of a unit of meaning.
Did the qualitative analysis in this research represent a sound articulation between the results, the problem and the theoretical field ?
Herrington and Oliver describe their study as a qualitative one. Their findings were compiled using the NUD.IST package and presented in two forms : firstly group percentages of the 4 general types of talk ( social, lower- order, procedural, and higher- order) , and secondly charts for each group showing the proportion of each type of higher- order thinking (Uncertainty, Judgement, Multiple perspectives, Imposing meaning and Metacognition). The authors interpreted these findings in two very different ways. The more general group results were listed with the higher proportion of higher- order talk being matched against the lower proportions of the other categories. Occasionally additional information was introduced as interpretation such as the characteristics of the social talk of group 2 and the procedural talk of group 3 being linked to recurring computer equipment problems. In the discussion Herrington and Oliver comment that their findings Îshow that all the students used a substantial proportion of higher- order thinking in the situated environment where other studies have shown littleâ. This would seem reasonable as they are citing proportions of between 68 and 77% for higher- order thinking.
The group charts showing the breakdown of higher- order thinking are subject to much more, and at times quite liberal, interpretation. Tracking some of this interpretation through the groups for instance the authors comment that group 1 have a low proportion for Uncertainty and Path of action ( 43%) because the two students are comfortable working together. Group 2âs even lower Uncertainty and Plan of action ( 32%) together with a slightly higher Judgement record and Multiple perspectives was thought by the authors to indicate
Î that these students were forthright and confident in working out their path through the interactive multimedia program, and that they appeared to be very comfortable working together.â (Herrington and Oliver, 1999, p. 17)
Interestingly they appear to have interpreted the high proportion of multiple perspectives as indicating confidence when working out their path through an interactive multimedia program. In fairness to the authors they would have difficulty interpreting at this level of detail anyway because they had only collected evidence of the talk and therefore could only speculate on the characteristics of the learners.
In their conclusion Herrington and Oliver highlight the importance of social grouping and experience at collaboration in facilitating higher- order thinking and they refer in the discussion of the group charts toÊ Î tentative nature of the collaborationâ,
Î interactions often argumentativeâ, Îreluctance to challengeâ, Îstudents did not use the multiple perspectives they offered each other to inform the meaning of the taskâ. This indicates that they became aware when looking at the detail of the higher- order thinking that a wider socio-cultural context was informing the learning experience and not just the situated learning environment of the multimedia program. The results for group 3 would support this notion . Their active consultation of each other has been interpreted as Îtentativeâ collaboration and yet they were clearly learning on two levels ö as pre-service teachers learning about assessment, and as a new team learning to work together. Given the confusion throughout the article about where the situated learning community of practice was centred then it is easy to ascribe any high performance to engagement with the program and any under performance to weaknesses on the part of the individual learners.ÊÊ
The authors discussion of the classification scheme as an instrument to reflect cognition raises a number of points. Firstly they suggest that Framptonâs reservations(1994) on the use of classification schemes to identify cognitive responses to the organisation of media in a multimedia program are not pertinent because Henriâs framework that they adopted had been developed for use with multimedia. As I have mentioned earlier however it would appear that the framework was being applied in a slightly different way so that whilst they may be correct in discounting Framptonâs comments, they cannot be totally confident of the validity of Henriâs framework in their empirical setting. Herrington and Oliver suggest that their interpretation of higher- order thinking may be too liberal and certainly by moving Uncertainty and Path of Action into lower- order then the Summary Chart of student talk ( table 2 ) appears more accurate as an instrument of cognition. Once the figures for the groups are modified by this reclassification then we are left with proportions for higher- order thinking as follows : group 1Ê 44%, group 2Ê 47%, group 3Ê 27%, group 4Ê 44%. There is still an issue however with the representation of metacognition and the examples of talk given. The authors pointed out how little metacognitive talk was recorded and if it was of the type given in the examples then it would seem that this aspect of higher- order thinking is difficult to capture in talk and that some reclassification of comments in this category might be necessary too.
The multimedia resource at the centre of this study provided a valuable resource for the problem solving activity and there were some aspects of the program design, for example video clips of teacherâs comments, that Îsituatedâ the student activity in the context of the practice of assessing pupils progress. The program did not however create the full environment of a community of practice for a number of reasons. Firstly the exercise was very much a simulation with assessment practice packaged into defined and separate portions of the program, the cross links between them needing to be constructed by the student instead of arising from discussions with experienced practitioners. Secondly the perspective of parents in the assessment process was restricted rather artificially to the generation of the concern letter that prompted the problem solving task. Thirdly there were no opportunities within the program for individual discussions or interactions with practitioners so that the students could gain only limited insight in order to move from the periphery of the community. The collaborative nature of the activity however encouraged discussion at the keyboard, which allowed the students to construct ideas and concepts around the multimedia information in a socially situated context rather than a technologically situated one.
The recording and analysis of talk as an indicator of thinking was an area that the authors had obviously considered carefully. They maintained that the situated learning context for the activity would encourage the extension of higher order thinking into conversation. Certainly the research design decisions to make the task collaborative and a problem solving one were important as this wouldÊ encourage discussion of both the materials and the task, helping to keep the academic focus and facilitating higher order discourse. It is possible to envisage VygotskyâsÊ learning stages and Zone of Proximal Development applying in this process; the computer resources provide initial assistance through knowledge, student discussions and reflection during completion of the task facilitate internalisation of information, and the subsequent externalisation ( generalisation) of the key concepts would provide the learners with recommendations for the final report. Higher order thinking was in evidence in this multimedia research activityÊ developing from a constructivist learning process which engaged students collaboratively in finding out about an aspect of their future practice. If the students were highly motivated and able to discuss issues in detail and at depth then they would add value through their discourse to the multimedia resource and higher order thinking might then be said to arise from a situated learning context.
Herrington and Oliverâs choice of Henriâs framework to classify the student talk was an interesting one. It had been developed within a similar empirical setting that focussed on computer mediated communication and the identification of higher order thinking. The framework was devised however to analyse writing and it was recorded within a conferencing task where the collaboration would be asynchronous with no face to face contact. Discussion might appear more formal and academic in Henriâs empirical setting unlike the discussions at the keyboard in this study and this could influence its sensitivity as a general tool to analyse higher order thought. The incorporation of Resnickâs indicators into the Henri framework added the precision necessary to analyse the conversation record in terms of the chosen Îgrain sizeâ ö unit of meaning. There was, however, a slight lack of rigour evident in distinguishing lower order uncertainty from higher order uncertainty and the metacognition category was poorly defined and rarely recorded.
Herrington and Oliverâs research has merit and throws a clear light on an interesting learning situation involving a multimedia resource, collaborative learning and the identification of externalised higher order thinking. Inconsistencies and incoherence in translating theory into research design, particularly in relation to the nature of situated learning, do however limit their claim based on their findings. Their chosen analytical framework used to identify higher order statements also raised some issues of misclassification leading the authors to amend some of their results and reducing the proportion of higher order talk. The revised proportions for group 4 show higher order as 44% and combined lower order as 56%, so Herrington and Oliverâs claim that higher order thinking was in the majority now seems less certain.
Brown, A. & Dowling, P. (1998) Doing Research / Reading Research. A Mode of Interrogation for Education. London : Falmer Press.
Crook, C. (1990) Computers in the Zone of Proximal Development : Implications for Evaluation. Computers Education 17 (1), 81-91.
Herrington, J. & Oliver, R. (1999) Using Situated Learning and Multimedia to Investigate Higher-order Thinking. Journal of Interactive Learning research. 10 (1), 3-24.
Lave, J. & Wenger, E. (1991) Legitimate Peripheral Participation. Cambridge : CUP.
Scrimsaw, P. (1995) Language classrooms and computers. London : Routledge.
Vygotsky, L.S. (1978) Mind in Society. Cambridge, Massachusetts : Harvard University
ÊPress.