AIED2011 invited speakers

Stellan Ohlsson, University of Illinois at Chicago

Multiple Mechanisms for Deep Learning: Overcoming Diminishing Returns in Instructional Systems

Wednesday 29 June 2011

Stellan Ohlsson


The design of instructional materials in general and intelligent tutoring systems in particular should be guided by what is known about learning. The purpose of an instructional system is, after all, to supply the cognitive mechanisms in the learner’s mind with the information they need to create new knowledge. It is therefore imperative that the design of instruction is based on explicit models of those mechanisms. From this point of view, research has to date been characterized by two conceptual limitations. The first limitation is that systems are designed to teach to a narrow set of learning mechanisms, sometimes even a single one. There are signs that attempts to build intelligent tutoring systems that address a single mode of learning encounter diminishing returns, in terms of student improvement, with respect to implementation effort. The reason is that people learn in multiple ways. In this talk, I argue that there are approximately nine distinct modes of learning cognitive skills. To be maximally effective, instruction should support all nine modes of learning. This is the way to overcome the diminishing returns of tutoring systems with a narrow bandwidth. The second limitation is the traditional focus in both the science of learning and the practice of instruction on additive or monotonic learning: That is, learning in which the student extends his/her knowledge base without reformulating the knowledge he/she possessed at the outset. Additive extensions of a person’s knowledge are certainly real and important, but they do not exhaust the types of learning of which human beings are capable. In many learning scenarios, the learner must overcome or override the implications of prior knowledge in order to learn successfully. This requires cognitive mechanisms that transform or reject the prior knowledge, in addition to building new knowledge. In this talk, I provide an outline of the essential characteristics of such non-monotonic learning processes. I end the talk by spelling out some implications of the multiple-mechanisms and non-monotonicity principles for the future development of instructional systems.


Stellan Ohlsson is Professor of Psychology and Adjunct Professor of Computer Science at the University of Illinois at Chicago (UIC). He received his Ph.D. in psychology at the University of Stockholm in 1980. He joined the Learning Research and Development Center (LRDC) in Pittsburgh in 1985 and was promoted to Senior Scientist in 1990. He moved to his present position at UIC in 1996. Dr. Ohlsson has published extensively on computational models of cognitive change, including creative insight, cognitive skill acquisition and conceptual change. He invented the concept of Constraint-Based Modeling (CBM), one of the cornerstones of research on intelligent tutoring systems. He has held grants from the Office of Naval Research (ONR) and the National Science Foundation (NSF), among other agencies. He is one of the co-originators of the AIED conference series, and he co-chaired the 1987 and 1993 conferences. He has been a member of the editorial board of the International Journal for Artificial Intelligence in Education and other cognitive journals. In 2010, Dr. Ohlsson co-chaired the 32nd Annual Meeting of the Cognitive Science Society. Dr. Ohlsson recently completed Deep Learning: How the Mind Overrides Experience, a synthesis of his research, published by Cambridge University Press.

Stellan Ohlsson invited talk: mp4, 401MB (Right click -> Save link as)

Janet Metcalfe, Columbia University

Metacognitively Guided Study in the Region of Proximal Learning

Thursday, 30 June 2011

Powerpoint Slides (20.8MB)

Janet Metcalfe


Empirical data on people's metacognitively guided study time allocation-- data that resulted in the proposal that metacognitively astute people attempt to study in their own Region of Proximal Learning (RPL)-- will be reviewed. First, the most straightforward study-choice strategy that metacognitively sophisticated learners can use is to decline to study items that they know they already know. If an item has already been mastered, then further study is unnecessary. All theories, including the RPL model, agree on this strategy, and many, but not all, people use it. Its effective use depends on refined metaknowledge concerning the boundary between what is known and what is not known, as well as the implementation of a rule to decline study of items for which judgments of learning are very high. There are many situations in which people are overconfident, and if they are, they may miss studying items that are almost, but not quite, mastered. These items would yield excellent learning results with just a small amount of study, and so this failure to study almost-learned items has detrimental results. Data will be presented showing that young middle childhood children (7 to 9 year olds) tend not only to be overconfident—thinking they know things when they do not—but also to have an implementation deficit in using metacognitively-based item-choice strategies. One result is that many children at this age fail to use even this most obvious study strategy, even though it will be shown that it would benefit their learning. When the computer implements this learning strategy for the children their later performance improves. Second, with already-learned item eliminated, metacognitively sophisticated learners selectively study the items that are closest to being learned first, before turning to more distal items that will require more time and effort. This, as well as studying the materials that are within their cognitive reach, rather than items that are too difficult, is a strategy that conforms to the so-called “Goldilocks principle”—not too easy and not too difficult but just right. As will be detailed, while college-aged learners use this strategy, older middle childhood children (aged 9-11) do not. Children at this age are not without strategies, however. They do use the strategy of declining the easiest items (including the already-learned items). However, the older middle childhood children overgeneralize this strategy to selectively prefer the most difficult items. While their learning is negatively impacted by this, it is improved if the computer implements the Goldilocks principle on their behalf. Third, people use a stop rule that depends upon a dynamic metacognitive assessment of their own rate of learning. They discontinue study when they perceive that continued efforts are yielding little learning return.

This stop rule predicts that people will stop studying easy items when they are fully learned them (and the learning rate has reached an asymptote on ceiling). Study will also stop, however, if the item is too difficult to allow noticeable learning. This strategy keeps people from being trapped in laboring on very difficult items in vain. Finally, the value that each item is assigned on a criterion test, if known during study, influences which items metacognitively sophisticated people choose to study and for how long they continue to study them. Items worth many points on a test will be studied sooner, longer and more often, than items worth few points. But not all learners use these strategies to their advantage. To effectively use the strategies that the Region of Proximal Learning framework indicates are effective, the learners must both have adequate metacognitive knowledge and also exhibit good implementation skills. Both metaknowledge and implementation skills vary across people. Age differences, motivational style differences, and metacognitive expertise differences can result in strategies that vary considerably, and which can result in sizable differences in the effectiveness with which the individual is able obtain his or her learning goals.


Dr. Metcalfe is a full professor in the Department of Psychology at Columbia University. She has worked in many areas of cognitive and metacognitive research and intervention, and has experience with a broad range of research problems, populations, and settings. She is the editor of three books related to various aspects of metacognition—Metacognition: Knowing about Knowing; The Missing Link in Cognition: Origins of Self-Reflective Consciousness, and Metacognition of Agency and Joint Attention (forthcoming). She has also authored, with John Dunlosky, the first textbook on metacognitive processes: Metacognition. Her metacognitive research has been directed both at college-aged students and at school-aged children. She has received numerous awards from such agencies as NIMH (National Institute of Mental Health), NSERC, the National Science and Engineering Research Council of Canada, the Institute for Educational Science, in the Department of Education, and The James S. McDonnell Foundation, to investigate and develop computational models of human memory and metamemory, to study metacognition and control processes, to examine the mechanisms underlying human memory, and to seek ways to enhance human learning and memory. Her recent work has focused on theories of and methods to improve learning and to overcome errors. She has done breakthrough work on the hypercorrection paradigm, in which high confidence errors are shown to be more
easily updated than low confidence errors. She proposed and developed the Hot/Cool theory of delay of gratification. She has extensively researched people’s metacognition concerning their own agency. She has published seminal papers on metacognition and control processes, developing the Region of Proximal Learning model of effective metacognitively guided study time allocation.

Janet Metcalfe invited talk: mp4, 489MB (Right click -> Save link as)

John Sweller, University of New South Wales

Cognitive Load Theory and E-Learning

Friday 1 July 2011

Powerpoint Slides (82 KB)

John Sweller


Cognitive load theory (Sweller, Ayres, & Kalyuga, 2011) is an instructional theory based on some aspects of human cognition. It takes an evolutionary approach to cognition. The theory assumes two categories of knowledge: biologically primary and biologically secondary knowledge. Primary knowledge is knowledge we have evolved to acquire over many generations. Secondary knowledge is cultural knowledge that humans have required more recently and have not specifically evolved to acquire. Cognitive load theory applies to secondary rather than primary knowledge. With respect to secondary knowledge, the theory assumes that human cognition constitutes a natural information processing system that has evolved to mimic another natural information processing system, biological evolution, with both systems characterised by the same basic principles. These principles lead directly to the assumption that biologically secondary knowledge consists of a very large range of domain-specific knowledge structures and that the primary aim of instruction is to assist learners in the acquisition of that knowledge. There are two basic structures associated with human cognitive architecture that are critical to instructional design – working memory and long-term memory.
Cognitive load theory assumes a limited working memory used to process novel information and a large, long-term memory used to store knowledge that has been acquired for subsequent use. The purpose of instruction is to store information in long-term memory. That information consists of everything that has been learned, from isolated, rote-learned facts to complex, fully understood concepts and procedures. Learning is defined as a positive change in long-term memory. If nothing has changed in long-term memory, nothing has been learned.

The theory has been used to generate a wide range of instructional procedures. Each of the procedures is designed to reduce extraneous working memory load in order to facilitate the acquisition of knowledge in long-term memory. One such procedure is based on the transient information effect, an effect that is closely associated with the use of instructional technology to present information.

When technology is used to present information to learners, the modality and format of the presentation is frequently changed. For example, written information may be substituted by spoken information and the static graphics associated with hard copy may be replaced by animations. While instructional designers are usually highly cognizant of these changes, there is another, concomitant but less obvious change that occurs. Relatively transient forms of information such as speech or animations replace a relatively permanent form of information such as written text or visual graphics. Frequently, this change is treated as being incidental and is ignored. Cognitive load theory suggests that it may be critical. Limited human working memory results in transient, technology-based information having considerable instructional consequences, many of them negative. Theory and data associated with the transient information effect will be discussed in relation to e-learning.


John Sweller is an Emeritus Professor of Education at the University of New South Wales. His research is associated with cognitive load theory. The theory is acontributor to both research and debate on issues associated with human cognition, its links to evolution by natural selection, and the instructional design consequences that follow.

John Sweller invited talk: mp4, 425MB (Right click -> Save link as)