An education revolution appears to be looming. eLearning, or its myriad of other pseudonyms, is poised to become a universal solution for us to seemingly solve all the educational woes of our time. Should eLearning live up to the hype we can expect to see accessible, affordable, equitable, any-time/any-place, life-long learning permeate every aspect of our increasingly knowledge driven society. Yet, despite the growing ubiquity and impact that eLearning has had, it is interesting to note that – amidst the copious research articles, reports and discussion papers relating to improving the state web based education – “a consolidated evaluation methodology of e-learning applications does not yet exist” (Ardito, et al., 2006, p. 270). There are now established traditions of research into educational learning and human computer interactions (HCI), however these two fields of research seem to have little to do with one another (Squires & Preece, 1999, p. 463). Whilst literature on eLearning seems often focus on program design, delivery and facilitation there seems to be very little research on the actual interaction with content itself, an aspect of the online learning environment that often seems neglected. Due to the technical nature of the learning environment itself, as opposed to it’s content or use, the educator has very little control over it. The problem seems to be that educational facilitators and practitioners are not, at least in most cases, interface designers or computer programmers and, of course, the reverse is also true.
There is much documentation, particularly around the turn of the century, on how to best design and evaluate methods for computer mediated learning (Graham & Scarborough, 2001; Mayas & Fowler, 1999; Parlangeli, Marchigiani, & Bagnara, 1999; Squires & Preece, 1999; Squires D. , 1999), however there seems to be significantly less research in more recent years. Furthermore, it seems that what little research relating to best practices in usability and design in education are only ever a “a first step towards the definition of a methodology” (Ardito, et al., 2006, p. 270) – subsequent steps seldom seem to eventuate. However, there seems to be little scope, or desire, for educational institutions to influence the development of the systems they use. Rather than designing and building their own web based learning environments, an endeavour that would necessitate much time and cost, many educational institutions have outsourced their learning management systems to existing systems (Burgess, 2003, p. 7), such as the proprietary [Blackboard system] (http://www.blackboard.com). Therefore, educational institutions relinquish the design and development of their systems to professional organisations, so that they themselves may focus entirely on populating content and training faculty. This practice seems to be the norm for Colleges and Universities; Blackboard, a stock-exchange listed American initiative, interestingly seem to monopolise the facilitation of online learning - claiming to work with over 5,000 educational institutions around the world and millions of students ; there seem to be very few, if any, significant rivals in the higher-education domains. Although Blackboard have little to do with the development of content and learning materials delivered in their system it seems that, when compared with industry standard web development outside the field of education, the their interface designed for delivering such content seems to leave much to be desired. Blackboard facilitate many of the standard objects associated to asynchronous online learning – message systems, discussion boards, shared whiteboards and chat – however, these systems may often seem unintuitive and awkward to use. Whilst there are many articles relating to student satisfaction using online learning environments there seem to be very few that focus on the usability aspects. There are a number of usability testing tools employed by commerce oriented sites, such as [Loop11] (http://loop11.com/) and [Silverback] (http://silverbackapp.com/), however it is difficult to find such standardised user experience testing tools employed for educational sites (Ardito, et al., 2006). Perhaps this is because, as Squires & Preece (1999) argue, traditional measures of heuristics and usability cannot effectively be applied to educational evaluations as they may fail to address the specific challenges of learner centred interface design. However, it surely should be conceivable that a modified approach, integrating usability and learning, is necessary. It seems that research in this area has stagnated or is refusing to move forward – perhaps partially due to the apparent near-ubiquity and dominance of a package in the marketplace.
Without focussing specifically on Blackboard, or its predecessor WebCT, it seems worthwhile to investigate the importance of heuristics and usability of eLearning applications. It seems conceivable that aspects of usability play a significant role in the success and satisfaction of learners’ experiences but, presently, it seems that these aspects of online learning are largely outside of the realm of influence by the very educational facilities that employ them. By searching through books, reports and peer reviewed journals relating to computers and education available online, I will attempt to determine how significantly the literature determines the roles of usability and heuristics to be in facilitating effective online learning.
Ardito, C., Costabile, M. F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, et al. (2006).
An approach to usability evaluation of e-learning applications. Univ Access Inf Soc (4), 270–283.
In what is perhaps one of the most recent and most rigorous articles on the importance of usability in eLearning applications, Ardito, et al., (2006) discuss the need for a consolidated approach to evaluating accessibility and usability in the domain of eLearning. Building on the foundations of existing underpinning research, including Squires & Preece (1999), Ardito, et al., comprehensively define why evaluation of usability in education is a neglected, yet important, area of research. By defining the term “usability” from a technical international standard (p. 271), to discussing “pedagogical usability” (p. 272), the authors ensure there is no confusion about their topic of research or the significant role it plays for education. Without diminishing the importance of learning content and facilitation, the paper highlights simple truths about usability, such as “If the user interface is too rigid, slow and unpleasant, people feel frustrated, go away and forget about it” (p. 271).
The background of eLearning usability, and the literature surrounding it, is sufficiently discussed. There have clearly been attempts to create universal and simplified systems for approaching evaluation, however, it becomes apparent that there is no one-size-fits-all solution to evaluate eLearning. This can largely be attributed to the complexities of measuring both usability and pedagogical objective. It is technically possible for an educationally sound application to suffer from usability issues and likewise for a usable system to neglect the needs of the learner. Therefore, the authors propose the use of Systematic Usability Evaluation (SUE), a kind of broad-spectrum analysis that approaches evaluation from different, although specifically determine, abstract tasks (ATs). Subsequently, a small sample of Masters students (ten) were observed to determine suitable ATs for evaluating eLearning which identified four dimensions of analysis – Presentation, Hypermediality, Application proactivity and User activity. The choice of these four criteria are said to be founded on previous research and it seems reasonable that the small sample was used to legitimise, rather than determine, these criteria. Further discussing evaluative criteria and guideline for assessing the aforementioned analysis, and elaborating on other important aspects – such as measure of didactic evaluation, it is clear that Ardito, et al., understand that there are many peculiar aspects associated to evaluating eLearning. This is a field that the authors feel is underrepresented in research and, whilst not a comprehensive or authoritative resource in itself it is a useful indicator for reasearch that should follow.
Hilgard, E. (1969). Psychological Heuristics of Learning. N.A.S. Symposium , 63, 580-7.
Although falling well outside the predetermined criteria of articles published not more than a decade ago, it seems relevant to discuss Hilgard’s (1969) paper of the Psychological Heuristics of Learning. Although not specifically a research article itself, this related psychological article highlights the existing and persistent relevance of certain trains of thought by removing the ambiguities associated to the current application of heuristic on eLearning. It is, of course, necessary to exercise extra caution on the research and finding is such dated papers, however, despite the article being four decades of age, the relevance of Hilgard’s discussion can be directly linked or paralleled to matters of today.
Class sizes, are said by Hilgard, to have virtually no outcome over the success of a student, in an exam situation and so, the author subsequently qualifies that, “there is no reason why we should not try to introduce greater efficiency into it through some sort of cost benefit analysis of different kinds of teaching” (p. 580) – one of the driving factors for eLearning to date. Education has harnessed existing materials, such as books, tapes, lectures, etc.; to further advance learning but there is nothing quite so revolutionary as forthcoming computer assisted learning. The ideas laid out – that computers will offer flexible, customised, personalised learning that caters to the learners preferences, is the exactly the optimistic idioms that persist today. Hilgard breaks down a simple task, such as teaching a child to ride a bicycle, and how establishing the heuristics of such a task are inherently difficult- you just put him on the bike and let him learn. This highlights the limitations of psychology and learning, by noting the discourse between what happens in the laboratory versus in the field – “a monkey in a cage is not a student in a classroom” (p. 580). This can be said to parallel the heuristics of eLearning – as we tend to throw people into online environments and expect them to determine how to use them without much prior consideration to how they might use them. Noting that human life is “inescapably social” (p. 587), Hilgard has unknowingly pointed out exactly where online education is heading – providing more effective ways for people to communicate and collaborate with each other, but without needing to interact directly. This seems particularly relevant as we’re progressing ahead with computer technologies in previously dreamed of ways, perhaps without addressing underlying requirements that should have preceded them, such as analysing the very heuristics of how we should learn online. Although not a particularly scientific article, Hilgard makes the sound resolve that use of computers in education is, and always has been, well grounded (p. 587) however current literature still contrasts this to note that there is still much work to be done in the area.
Zaharias, P., Vassilopoulos, K., & Poulymenakoua, A. (2002).
Designing On-Line Learning Courses: Implications for Usability.
Retrieved November 5, 2009 from Scientific Journal of Applied Information Technology:
http://www.japit.org/vol1/issue1/zaharias_etal02.pdf
This paper has two distinct purposes – firstly authors review literature relating to usability achievement in learning applications and secondly draw on their own experience in web based environments. As with many articles, the authors begin by highlighting the potential impact that eLearning has, but are quick to point out the Human Computer Interactions pertaining to eLearning are or significant importance. This work shares many of the referenced works with Ardito, et al. (2006), perhaps most notably the work of Squires and Preece (1999), which is again underpinning the initial arguments. Learning is described as a complex task that cannot be dealt with in any simple manner; learning is also a back and forth communication rather than a simple one-way transmission of knowledge. The web then, for web based learning, can be said to facilitate this back-and-forth communication and it is important for it to act according to the users expectation.
Zaharias, Vassilopoulou, & Poulymenakoua’s (2002) literature review delves into many dimensions of usability effecting the web – navigation fidelity, meaningful representations, learner control, etc.- all of which should be familiar to anyone studying usability. This offers a well rounded approach to understand the area of discussion as the authors then relate the usability metrics to learning styles – relating such topics and learner-control and learner-centred constructivist approaches. The background on learning styles would be particularly useful as a foundation or refresher for HCI specialists outside of educational circles.
Subsequent to the review, the authors discuss the finding of their quatitative scale based study examining “the effects of a web-based testing application interface on learners’ attitude” (p. 7). The questions were developed using a combination of heuristics, including Nielson’s heuristics (which had notably been deemed as not directly applicable by Squires and Preece (1999) – not mentioned in the text) and Ravden and Johnson (1989, cited by Zaharias, Vassilopoulou, & Poulymenakoua, 2002). There is very little discussion on how they derived the questions, except having based them on related works.
Whilst the authors highlight the researches shortcomings, there is very little to be gained from this article except the prevailing notion to design software and applications that “make people more effective learners”.
Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses
Swan, K. (2001).
Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses.
Distance Education , 22 (2), 306-31.
The distinct difference that eLearning offers to other forms of learning, according to Swan (2001), is it’s heightened aspect of interactivity. Broken down into three categories – interaction with content, interaction with instructors, and interaction with classmates – this article demonstrates the importance of all three of these modes working while together for a student to have a perception of quality learning. Interestingly for this topic, Swan (p. 307) notes that interaction with the content has been the least examined aspect in the domain of eLearning. Citing (now somewhat dated, 1985-95) resources for good content creation the author is attempting to establish a motive for a holistic approach that considers many forms of interaction.
The central research of this article revolves around a satisfaction survey of some 1,400 students after completing their relevant course. There is good background on the method of research – the question asked, the time of day, explanation for lack of student responses, etc. Since thee could be no simple measure of student satisfaction with their interaction with course content – an arguably abstract concept in itself – students were asked to rank their perceived satisfaction with the course content compared to classroom interaction. Swan argues that this is relevant because online education offers the same opportunities and outcomes as face-to-face teaching – a point that would have been dubious to claim, particularly in 1999/2001. However, the findings of the survey demonstrate little difference, in almost all comparisons, between online learning and face-to-face learning. Critical reflections of the survey results suggest that online interaction is very much as social activity, requiring interaction with classmates and educators, and that interaction with the content is of equal importance. The article concludes, as do so many others, that support for students’ interactions in this area “clearly deserves the attention of online developers and instructors alike, and further investigation by the research community.”
Ilmberger, W., Schrepp, M., & Held, T. (2008). Cognitive Processes Causing the Relationship between Aesthetics and Usability. In A. Holzinger (Ed.), HCI and Usability for Education and Work (pp. 43-54). Heidelberg, Germany: Springer-Verlag Berlin.
Ilmberger, Schrepp, & Held (2008) being their relationship of cognitive processes with usability and aesthetics by citing an empirical study in which four functionally identical implementations of an ATM interface were assessed for their perceived ease of use. The outcome, predictably, demonstrated that “more attractive” interfaces were perceived as easier to use than less attractive ones – thus proving a correlation between aesthetics and perceived usability. The study is stated to have one fatal flaw; participants never actually interacted with the interface, but rather just studied them. The correlation is not shown to be true for all forms of interfaces, as demonstrated by five subsequent studies cited by the authors. In order to explain how, in some circumstances, aesthetics heavily influence perceived usability and not in other Ilmberger, Schrepp, & Held attempt to link cognitive processes with of user experience to find a more substantial relationship.
Firstly, the emotional state can be shown to influence the way in which a user approaches problems. Negative states can be said demonstrated an analytical way of dealing with problems, while a positive state “broadens the problem solving processes and facilitates creative thinking” (pp. 44-5). Secondly, social psychology gives an insight into a “what is beautiful is good” stereotype. Without interaction, a subject may be more likely to apply this stereotype – such as in the first cited example. To achieve accurate evaluations of applied processes Ilmberger, Schrepp, & Held note that interaction is a key element when applying usability measures, thus making considerations beyond simple aesthetics and proving a potential relationship between the two.
To best determine the relationship between aesthetics and usability the authors derive several hypotheses. The upshot of which is to determine whether it is cognitive state or mood that effects perception or whether the application of good heuristics itself effects the perception of aesthetics. The resulting study, on 72 student subjects, attempts to manipulate one dimension of perceived aesthetics – colour – to determine the perceived effect on aesthetics. At first, this might seem contestable, as colour has little to do with user experience. But on consideration it is the simplest measure, and easiest way of controlling a single variable, to determine the relationship between perceived user experience and aesthetics. A second test, comprised of manipulating simple user functionality and visual clues, was derived.
A purposely created web based shopping cart was employed as the usability test and a “The User Experience Questionnaire” was used to record participants perceptions, before an after actually interacting with the test. The foundation for using this form of questionnaire does not seem to be noted, and subsequent searching leads to an exclusively German based test. As far as a non-statistical reader can determine, the subsequent processing of information in this study seems well founded in statistical theory and consideration for confounding variable was given and appropriate manipulation checks are noted.
Although, as the authors point out, this is only a first step into examining the perceived relationship between aesthetics and mood, the reported outcomes do seem to show that the mood of a user influences the perceived usability of an interface and, given that a users mood might be influenced by their initial impressions of the interface, this may have considerable implications. For example, it could be inferred that poor usability may be compensated by good aesthetics, and vice-versa; however a combination of good aesthetics and good usability will improve the aesthetic perceptions of the user significantly. Therefore the assumption that “what is good is beautiful” holds more credibility than “what is beautiful is good”.
The authors frequently note the study’s shortcoming along the way, and demonstrate the areas where more study is required. Although, however, Ilmberger, Schrepp, & Held, highlight many underlying influences potentially effecting their research and whilst such research should not be considered universally applicable or acceptable it does provide some interesting insights. It is noted that such usability testing in controlled lab environments may not reflect the outcomes in real world circumstances but there is at least some credible evidence to at least suggest that “bad usability can not be compensated by a good visual design”.
- Ardito, C., Costabile, M. F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, et al. (2006). An approach to usability evaluation of e-learning applications. Univ Access Inf Soc (4), 270–283.
- Burgess, L. (2003). WebCT as an E-Learning Tool: A Study of Technology Students’ Perceptions. Journal of Technology Education , 15 (1), 6-15.
- Candy, P. C. (2004). Linking Thinking: Self-directed learning in the digital age. Commonwealth Goverment of Australia.
- Chiu, C., Hsu, M., Sun, S., Lin, T., & Sun, P. (2005). Usability, quality, value and e-learning continuance decisions. Computers & Education , 45, 399–416.
- Fresen, J. (2007). A Taxomony of Factors to Promote Quality Online Learning. International Journal of E-Learning , 6 (3), 351-62.
- Garrison, D., & Cleveland-Innes, M. (2005). Facilitating Cognitive Presence in Online Learning: Interaction Is Not Enough. The American Journal of Distance Education , 19 (3), 133–148.
- Graham, M., & Scarborough, H. (2001). Enhancing the learning environment for distance education students. Distance Education , 22 (2), 232-44.
- Hilgard, E. (1969). Psychological Heuristics of Learning. N.A.S. Symposium , 63, 580-7.
- Ilmberger, W., Schrepp, M., & Held, T. (2008). Cognitive Processes Causing the Relationship between Aesthetics and Usability. In A. Holzinger (Ed.), HCI and Usability for Education and Work (pp. 43-54). Heidelberg, Germany: Springer-Verlag Berlin.
- Knight, C., Gašević, D., & Richards, G. (2006). An Ontology-Based Framework for Bridging Learning Design and Learning Content. Educational Technology & Society , 9 (1), 23-37.
- LaPorte, J. (2003). Form, Function, Faucets, and Design. Journal of Technology Education , 15 (1), 2-5.
- Mayas, J., & Fowler, C. (1999). Learning technology and usability: a framework for understanding courseware. Interacting with Computers , 11, 485-97.
- NASDAQ. (2009, November 5). NASDQ. Retrieved November 5, 2009 from BBBB Stock Quote: http://quotes.nasdaq.com/asp/SummaryQuote.asp?symbol=BBBB&selected=BBBB
- Paechter, M., Maier, B., & Macher, D. (2010). Students’ expectations of, and experiences in e-learning: Their relation to learning achievements and course satisfaction. Computers & Education , 54, 222–229.
- Parlangeli, O., Marchigiani, E., & Bagnara, S. (1999). Multimedia systems in distance education: effects of usability on learning. Interacting with Computers , 12, 37–49.
- Squires, D. (1999, May). Usability and Educational Software Design: Special Issue of Interacting with Computers. Interacting with Computers , 11 (5), pp. 463-6.
- Squires, D., & Preece, J. (1999). Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with computers , 11 (5), 467-83.
- Swan, K. (2001). Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses. Distance Education , 22 (2), 306-31.
- van Merriënboer, J. J., & Ayres, P. (2005). Research on Cognitive Load Theory and Its Design Implications for E-Learning. Educational Technology Research and Development , 53 (3), 5-13.
- Zaharias, P., Vassilopoulou, K., & Poulymenakoua, A. (2002). Designing On-Line Learning Courses: Implications for Usability. Retrieved November 5, 2009 from Scientific Journal of Applied Information Technology: http://www.japit.org/vol1/issue1/zaharias_etal02.pdf