OVERVIEW
OF THE ARCADE PROJECT
The ARCADE (Automatized Reading Comprehension and Diagnostic
Evaluation) project is a 2-year feasibility study to develop advanced
artificial intelligence techniques and utilize research from the field of Cognitive
Psychology for the purposes of assessing reading comprehension of children in
elementary and junior high school. This feasibility study was funded by the
Information Technology Research (ITR) program within the National Science
Foundation (Award 0113369) under the project title: Development and Evaluation of an Automatized
Comprehension Assessment Tool.
Problems
with current tests of reading comprehension.
Typical tests of reading comprehension focus upon multiple
choice responses by students and attempt to determine which students have the ôbestö reading comprehension abilities. This is not a good
way to assess reading comprehension since we know from scientific research from
the field of Cognitive Psychology that reading comprehension consists of a
large variety of different skills (e.g., vocabulary, figuring out how different
ideas within the text are inter-related, and extracting the information in the
text which is personally important to the reader---not the test-maker).
Responses based upon multiple choice tests are good at assessing some of these
skills but are not very good at assessing high-level áreading
comprehension skills.
Assessment
of reading comprehension by reading specialists.
A very good way to assess reading comprehension is to have an EXPERT reading
specialist sit down with a child and have the child answer questions about what
they have read. The child should be encouraged to provide long complex answers
(in order to reveal their understanding of the text) and the child should be
encouraged to discussá things which are not
explicitly in the text (in order to reveal their ability to relate the text to
the real world). The two major problems with this methodology are that: (1) it
is expensive since it requires a highly trained educator to sit down and
interview each child individually, and (2) the reliability of the assessment
can considerably vary.
Automated reading
comprehension assessment: The ARCADE system.
We are developing a brand-new web-based artificial
intelligence system called ARCADE (Automatized Reading Comprehension and Diagnostic
Evaluation) which works in a manner similar to that of a human expert reading
specialist. The ARCADE system works by having
children logon to a particular web site. The children are asked to read
narrative stories and science texts and are asked to type answers to some
questions about the texts. The children are encouraged to type their answers in
any format they choose. Such "essay responses" are important since
they can reveal aspects of children's thinking styles which may not be
uncovered in standardized testing.á ARCADE then
automatically groups children with similar thinking styles together and
provides the classroom teacher with suggested customized group-specific
teaching strategies for each group. Thus, the ARCADE
system is designed to improve the quality of reading comprehension instruction
for all children in the classroom.
Multidisciplinary
research contributions. Moreover, this research project is not only
concerned with developing and evaluating new educational technology, but also
is concerned with developing and evaluating new ideas in the areas of
artificial intelligence, psychometric testing (the mathematics behind
standardized tests), and Experimental Cognitive Psychology.
The
ARCADE project team members. Dr.
Richard Golden heads the ARCADE System Development and Evaluation Team. Dr.
Richard Golden is Professor of Psychology and Cognitive Science in the School of Behavioral
and Brain Sciences at the University
of Texas at Dallas (UTD).
Dr. Golden is Program Head of the Cognitive Science undergraduate program and
Program Head of the Applied Cognition and Neuroscience program at UTD as well.
Dr. Susan Goldman, Distinguished College Professor of Psychology at the University of Illinois
at Chicago (UIC) heads the Assessment Materials Development Team for the
purpose of assessing the effectiveness of the ARCADE
system for improving classroom education. In addition, several schools and
school districts are working closely with the ARCADE project team as partners
to develop the ARCADE system. Graduate
students at UTD and UIC have actively participated in the ARCADE
project.