Philip Barker and Nigel Beacham

Human-Computer Interaction Laboratory

University of Teesside,

Middlesbrough, UK

A performance support system provides a mechanism by which human skill levels within a particular task domain can be improved. Various types of tool and support aid currently exist to facilitate this goal. These range in sophistication from simple, stand-alone tools through integrated toolsets to comprehensive, geographically distributed systems. This paper discusses the growing importance of distributed performance support systems and their use within the context of creating a ‘learning organisation’. The design and development of a distributed performance support system called ‘CLASS’ is described. Its use as a performance tool is then compared with conventional approaches to aid provision within the context of creating supportive environments to facilitate the use of computer-assisted learning within an organisational framework.

In their day to day activities, human beings are often required to perform a wide range of different tasks and jobs. These tasks vary quite considerably in their complexity. The successful execution of many of the more complex tasks often requires the use of some sort of tool, aiding facility or performance support system (PSS). The basic rationale underlying the use of a PSS is that individuals or groups of people (working together on a common project) are provided with appropriate tools and techniques to support the tasks that they have to perform. According to Barker and Hudson (1998), four major objectives of a performance support system are: (1) to achieve increases in productivity; (2) to improve the overall quality of task/job execution; (3) to improve the overall quality of the environments in which people work; and (4), from a human perspective, to reduce the complexity of the processes involved in executing a task.

Simple examples of performance support tools include: a typewriter, a telephone, a bicycle and an automobile. Increasingly, computer-based resources are being used to implement performance support systems. An electronic performance support system (EPSS) is therefore a facility which uses various types of computer technology to realise each of the four previously listed objectives within a given working environment or problem solving domain. Software packages such as word-processing systems and spreadsheets provide a good example of computer-based performance support systems.

Naturally, a fundamental requirement of an EPSS is that it should increase its users’ on-the-job performance within a given task domain. This usually involves improving skill levels, reducing task complexity and/or providing appropriate training. These requirements can be achieved in two basic ways. First, through the provision of ‘automation aids’; and, second, by providing various mechanisms to support ‘on-the-job’, ‘just-in-time’ (JIT) training which will enable users of any given system to ‘learn as they do’. This latter requirement is an important aspect of building a ‘learning organisation’ (Lassey, 1998).

According to Malhotra (1996), a learning organisation is one which has ‘an ingrained philosophy for anticipating, reacting and responding to change, complexity and uncertainty’. Because organisations must adapt dynamically to the forces generated by change agents, ongoing learning and training activities are a fundamental requirement to their success. As has been discussed elsewhere (Gery, 1991; Beacham, 1998; Barker, van Schaik and Hudson, 1998), performance support systems are one way of realising this requirement within an organisational context.

Many of the early developments in EPSS have been described by Gery (1991), McGraw (1994), Varnadoe and Barron (1994), Raybould (1995), Banerji (1995) and Barker (1995a). Currently, there are two important directions of development for EPSS facilities. First, the creation of integrated toolsets that are able to meet the performance support needs of particular application domains (Barker and Hudson, 1998; Hudson, 1998); and second, a growing requirement to facilitate group working at a distance through the incorporation of network technology and the principles of distributred computing environments (Beacham, 1998). Indeed, the advent of relatively low-cost network technologies and the widespread availability of facilities such as the Internet and the World Wide Web has meant that performance support systems can now become highly distributed in nature - that is, different components can reside (or be replicated) at, and be accessed from, different geographical locations.

As far as this paper is concerned, we therefore regard a distributed performance support system (DPSS) as an integrated and globally accessible collection of electronic tools and data that can be used (as and when required) at particular points of need (within a workgroup or an organisation) in order to improve human performance within a given task domain. Because of their future importance as a resource for developing learning organisations, the remainder of this paper describes and discusses the evolution and future potential utility of distributed performance support systems. The design and construction of a single-site, multiple-server distributed performance support facility called CLASS (‘Computer Learning and Support System’) is then described and an outline is given of some the ways in which we have been using it. An attempt has been made to measure the potential utility of the system from the perspective of supporting its end-users’ use of computer-assisted learning (CAL) packages. The findings from this work are briefly reviewed.

This section of the paper briefly discusses the background to the DPSS work that we have recently been undertaking. The implementation of the CLASS facility is then outlined. Finally, a description is given of the ways in which we have been using CLASS to support computer-assisted learning (CAL) and computer-based training (CBT) within the context of a learning organisation.

Some of our early work in developing EPSS facilities has been described by Banerji (1995) and Barker (1995a; 1995b). Banerji showed how the performance of students using a file transfer program (kermit) could be improved through the provision of PC-based, online help and interactive aiding and training facilities. Similarly, Barker (1995b) has shown how the emerging principles of performance support can be used to apply EPSS techniques to the design and development of integrated support tools for use by students and staff in an academic setting within a university environment. Each of these applications involved the development of stand-alone EPSS tools. Our first attempt at building a distributed performance support system involved using a computer network system (the Internet) in order to access different forms of subject expertise that resided at different geographical locations (Barker, Richards and Banerji, 1994). This work has recently been extended and enhanced through the use of ‘mobile computing’ environments (Barker, 1996)

Many of the presently available distributed performance support systems that are now in use have evolved from conventional stand-alone EPSS facilities (such as those described above) in two basic ways. First, by providing ‘dial-in’ access; second, by implementing systems that can be shared in an in-house fashion through the use of local area networks. Current research in this area now seeks to incorporate activities of this sort within a more holistic approach that also involves the utilisation of global trans-world networks such as the Internet and World Wide Web.

In an attempt to rationalise activity in this area, we have proposed a DPSS taxonomy that incorporates two basic classification dimensions: first, the number of information servers that are involved; and second, the number of interlinked sites that make up the network topology. These dimensions permit a basic, four-class taxonomy to be proposed (Beacham, 1998). The classes within this taxonomy are: single-site, single-server; multiple-site, single-server; single-site, multiple-server; and multiple-site, multiple-server. The CLASS facility, that is described in the following section, is an example of a single-site, multiple-server system.

The development of CLASS took place in three stages. The first stage involved conducting a needs analysis. Subsequently, in the second stage, a prototype system was built which was used for formative evaluation purposes. In the third stage, the prototype system was augmented and expanded into an operational DPSS. The needs analysis and implementation rationale are briefly described below.

The needs analysis was undertaken within the School of Business and Management (SBM) at the University of Teesside in the UK. Its main aims were: (1) to identify the main limitations (if any) of current course provision within the SBM; and (2) to explore how CAL and PSS techniques might be used to make any improvements that were thought to be necessary. Both staff (teaching, research and administrative) and student (full-time and part-time) perspectives were studied using questionnaire-based techniques. In some situations, ‘follow up’ interviews were conducted in order to clarify issues that were not clear or in order to obtain more information about particular aspects of the study.

The results of the SBM needs analysis were compared with two other related surveys (one internal and one external) in order to look for similarities and trends. The internal (campus-wide) study was conducted by the University of Teesside’s Library and Information Service (as part of its forward planning activities). The external survey was obtained from another UK university. As is discussed elsewhere (Beacham, 1998), the findings from both of these other surveys supported many of the findings of the SBM needs analysis.

The findings from the needs analysis were used to ‘steer’ the design of the CLASS facility and also set priorities with respect to the order in which computer-based resources and online facilities were created and made available. As was mentioned above, the initial design and development strategy involved building a prototype system. This was based on the use of an in-house intranet facility and incorporated a virtual university metaphor (Barker, 1999). This allowed staff and students to use virtual offices, a virtual library and virtual classrooms. HTML (Version 4) was used as the development language and Netscape Navigator (Version 4) was used as the access and delivery vehicle. Extended interactivity within the system was achieved through the use of CGI scripting techniques, JavaScript and Java applets. Appropriate use was also made of ‘helper’ applications (primarily for Microsoft’s PowerPoint, Word and Excel packages) and Netscape ‘plug-ins’.

This section of the paper briefly describes the main ways in which the CLASS facility has been used to support teaching, learning and administration activities within the SBM at the University of Teesside.

Once an intranet facility had been created within the SBM it became possible to use it to store, retrieve, deliver and maintain virtually all course-related documentation. By dispensing with ‘paper’ and providing electronic access to these materials through computer terminals, the productivity of staff and students increased substantially - in terms of ease of access to course-related information and the ease with which this could be modified and updated.

The basic rationale underlying our use of CLASS for the support of teaching and learning activities is similar to that outlined by Barker et al (1995) in their description of the use of a global performance support system for students and staff. However, the CLASS system goes much further, in that, it embeds a virtual university metaphor and allows various approaches to both synchronous and asynchronous teaching and learning activities. Some of the facilities that CLASS contains, or provides access to, include: online lecture notes; copies of lecture presentations (PowerPoint); problems and worked solutions; course-related CAL/CBT packages; details of project work; tutorial and workshop descriptions; email facilities; class schedules; conferencing facilities; electronic notice-boards; automatic marking of certain types of submitted work; and automated student self-assessment of progress. Although CLASS is by no means in its final form, the feedback that we have received with respects to its acceptance and utility provides significant motivation for its ongoing development. This feedback is discussed in the following section.

An extensive evaluation of the CLASS facility has been undertaken. Details of this are presented elsewhere (Beacham, 1998). This section of the paper provides an overview of the methods used and the results obtained.

A four-step evaluation strategy was used. This involved assessing both staff’s and students’ attitudes towards four different approaches to the provision of support to facilitate the teaching and learning of a particular topic X within a given module Y of a degree course Z. The four situations that were considered were: (1) minimal support; (2) conventional support with ‘add-on’ CAL; (3) conventional support with ‘ownership’ of CAL; and (4) the use of a DPSS. In case (1), a CAL package was used to teach the topic (X); no lectures were given and tutorials were organised in an ad-hoc basis, as and when they were needed. Case (2) involved a conventional teaching situation (with scheduled lectures and tutorials on topic X) with students using the CAL package as an ‘add on’ extra. Resource-wise, case (3) was similar to case (2) but involved re-organising the way in which the CAL package was used; in this approach the CAL program was used (by the lecturer) as a central resource (for demonstration and illustration purposes within lectures, tutorials and workshops) and by students (for the support of self-study activities). Case (4) was the same as case (3) but with the addition of a purpose-built DPSS facility (CLASS) to augment the course Z (in general) and the teaching of the module Y and topic X (in particular).

In each of the evaluation scenarios described above, questionnaires were used in order to obtain the opinions of staff and students with respect to pedagogic, technical, logistic, organisational and resource-based issues. Some of the results that were obtained are summarised in the following section.

As was mentioned above, four basic evaluation scenarios were employed. These were used to gather feedback about the different types of support environment (for using CAL packages) and, in particular, the potential utility of the CLASS facility as a support mechanism for teaching and learning within an organisational context. Based on the various types of questionnaire that were employed in the evaluation, a set of six ‘derived’ attributes were identified as a basis for comparing the different modes of support. The six attributes that were used are listed in table 1 along with the attribute values that have been derived from the questionnaire data. Overall, 330 student volunteers participated in the evaluation process. All students were involved in studying the topic of ‘Marginal Costing’ as part of a degree module entitled ‘Introduction to Financial Decision Making’ (the total enrolment for this course at the time the evaluations were conducted was 549). The distribution of students over the different evaluation scenarios listed in table 1 was as follows: Case 1 (61); Case 2 (183); Case 3 (43); and Case 4 (43). Some members of staff also participated in two of the evaluation scenarios (Cases 3 and 4). However, the data derived from the staff questionnaires is not included in table 1; this is presented and discussed elsewhere (Beacham, 1998).

As might be expected, the results shown in table 1 clearly reflect the apparent superiority of Cases 3 and 4 over Cases 1 and 2. Undoubtedly, this arises because the ‘ownership’ model of CAL usage and the integrated holistic approach provided by a DPSS (CLASS) each embed much richer teaching and learning strategies. Having said this, inspection of the results in row 1 of table 1 suggests that the topic knowledge transferred by CAL techniques using Case 3 seems to be less than that which takes place in Cases 1 and 2. This is indeed surprising since we would have anticipated that the knowledge transfer achieved using this approach to CAL package utilisation would be comparable with that observed in Case 4. We believe one possible reason for this anomaly is that the high level of knowledge transfer that took place in lectures and tutorials (using the ‘ownership’ model) reduced the need for students to make additional use of the CAL package themselves as a self-study resource.

As can be seen from table 1, the only approach which scores positively in all six attributes is Case 4. These results reflect that students felt that there were particular benefits to be gained by embedding a CAL package within the CLASS facility; namely, it was easier to use, it reduced the amount of difficulty experienced and provided a large number of sources from which to obtain information and practice skills. Bearing this in mind, we believe that these findings make a sound case for the more extensive use of the DPSS approach to the provision of aiding and support facilities.

Computers and electronic communications technologies are increasingly being used to provide mechanisms for the support of training, teaching and learning. These technologies make it possible to augment conventional approaches to instruction and, more importantly, provide mechanisms to facilitate ‘education on demand’ in a variety of different situations and contexts - such as, in the home, in the workplace, in libraries and in colleges. They also provide facilities for developing new approaches to skill and knowledge acquisition in the workplace through techniques such as ‘on-the-job’ learning and ‘just-in-time’ training. Within the context of creating ‘learning organisations’, an integrated, holistic approach to the provision of access to teaching/learning resources is now needed. We believe that this requirement can be fulfilled through the provision of distributed performance support systems similar to the CLASS facility that has been described in this paper.

Banerji, A.K., (1995). Designing Electronic Performance Support Systems, PhD Thesis, School of Computing and Mathematics, University of Teesside, Middlesbrough, UK.

Barker, P.G., (1995a). Electronic Performance Support Systems, Special Issue of Innovations in Education and Training International, 32(1), 1-73.

Barker, P.G., (1995b). Emerging Principles of Performance Support, 407-416 in Proceedings of ONLINE Information ‘95, Learned Information (Europe), Oxford.

Barker, P.G., (1996). Towards Real Information on Demand, 261-269 in Proceedings of ONLINE Information ‘96, Learned Information (Europe), Oxford.

Barker, P.G., (1999). Using Intranets to Support Teaching and Learning, Innovations in Education and Training International, 36(1), in press.

Barker, P.G., Banerji, A., Richards, S. and Tan, C.M., (1995). A Global Performance Support System for Students and Staff, Innovations in Education and Training International, 32(1), 35-44.

Barker, P.G. and Hudson, S.R.G., (1998). An Evolving Model for Multimedia Performance Support Systems, 60-73 in Trends in Communication: Part 4 Interface Technology - Enhancing the Quality of Life, edited by H. van Oostendorp and A.G. Arnold, Boom Publishers, The Netherlands.

Barker, P.G., Richards, S. and Banerji, A., (1994). Intelligent Approaches to Performance Support, ALT-J - Journal of the Association of Learning Technology, 2(1) 63-69.

Barker, P.G., van Schaik, P. and Hudson, S.R.G., (1998). Mental Models and Lifelong Learning, Innovations in Education and Training International, 35(4), in press.

Beacham, N.A., (1998). Distributed Performance Support Systems, PhD Thesis, School of Computing and Mathematics, University of Teesside, Middlesbrough, UK.

Gery, G., (1991), Electronic Performance Support Systems - How and Why to Remake the Workplace through the Strategic Application of Technology, Weingarten Publications, Boston, MA.

Hudson, S.R.G., (1998). Multimedia Performance Support Systems, Draft PhD Thesis, School of Computing and Mathematics, University of Teesside, Middlesbrough, UK.

Lassey, P., (1998). Developing a Learning Organisation, Kogan Page, London.

Malhotra, Y., (1996). Organisational Learning and Learning Organisations: An Overview, URL: http://www.brint.com/papers/orgling.htm.

McGraw, K.L., (1994). Performance Support Systems: Integrating AI, Hypermedia and CBT to Enhance User Performance, Journal of Artificial Intelligence in Education, 5(1), 3-26.

Raybould, B., (1995). Development Tools for EPSS, Journal of Interactive Instructional Development, 7(3), 7-10.

Varnadoe, S. and Barron, A.E., (1994). Designing Electronic Performance Support Systems, Journal of Interactive Instructional Development, 6(3), 12-17.