OVERVIEW OF THE ARCADE PROJECT

The ARCADE (Automatized Reading Comprehension and Diagnostic Evaluation) project is a 2-year feasibility study to develop advanced artificial intelligence techniques and utilize research from the field of Cognitive Psychology for the purposes of assessing reading comprehension of children in elementary and junior high school. This feasibility study was funded by the Information Technology Research (ITR) program within the National Science Foundation (Award 0113369) under the project title: Development and Evaluation of an Automatized Comprehension Assessment Tool.

 

Problems with current tests of reading comprehension.

Typical tests of reading comprehension focus upon multiple choice responses by students and attempt to determine which students have the ôbestö reading comprehension abilities. This is not a good way to assess reading comprehension since we know from scientific research from the field of Cognitive Psychology that reading comprehension consists of a large variety of different skills (e.g., vocabulary, figuring out how different ideas within the text are inter-related, and extracting the information in the text which is personally important to the reader---not the test-maker). Responses based upon multiple choice tests are good at assessing some of these skills but are not very good at assessing high-level áreading comprehension skills.

 

Assessment of reading comprehension by reading specialists.
A very good way to assess reading comprehension is to have an EXPERT reading specialist sit down with a child and have the child answer questions about what they have read. The child should be encouraged to provide long complex answers (in order to reveal their understanding of the text) and the child should be encouraged to discussá things which are not explicitly in the text (in order to reveal their ability to relate the text to the real world). The two major problems with this methodology are that: (1) it is expensive since it requires a highly trained educator to sit down and interview each child individually, and (2) the reliability of the assessment can considerably vary.

 

Automated reading comprehension assessment: The ARCADE system.

We are developing a brand-new web-based artificial intelligence system called ARCADE (Automatized Reading Comprehension and Diagnostic Evaluation) which works in a manner similar to that of a human expert reading specialist. The ARCADE system works by having children logon to a particular web site. The children are asked to read narrative stories and science texts and are asked to type answers to some questions about the texts. The children are encouraged to type their answers in any format they choose. Such "essay responses" are important since they can reveal aspects of children's thinking styles which may not be uncovered in standardized testing.á ARCADE then automatically groups children with similar thinking styles together and provides the classroom teacher with suggested customized group-specific teaching strategies for each group. Thus, the ARCADE system is designed to improve the quality of reading comprehension instruction for all children in the classroom.

 

Multidisciplinary research contributions. Moreover, this research project is not only concerned with developing and evaluating new educational technology, but also is concerned with developing and evaluating new ideas in the areas of artificial intelligence, psychometric testing (the mathematics behind standardized tests), and Experimental Cognitive Psychology.

 

The ARCADE project team members. Dr. Richard Golden heads the ARCADE System Development and Evaluation Team. Dr. Richard Golden is Professor of Psychology and Cognitive Science in the School of Behavioral and Brain Sciences at the University of Texas at Dallas (UTD). Dr. Golden is Program Head of the Cognitive Science undergraduate program and Program Head of the Applied Cognition and Neuroscience program at UTD as well. Dr. Susan Goldman, Distinguished College Professor of Psychology at the University of Illinois at Chicago (UIC) heads the Assessment Materials Development Team for the purpose of assessing the effectiveness of the ARCADE system for improving classroom education. In addition, several schools and school districts are working closely with the ARCADE project team as partners to develop the ARCADE system. Graduate students at UTD and UIC have actively participated in the ARCADE project.