Carnegie
Corporation
of New York
Fall 2007
< PREVIOUS
1 2
3 4
5 6 7
8 9
10
The Collegiate Learning Assessment
The Collegiate Learning Assessment, an outgrowth of RAND’s Value-Added
Assessment Initiative, is designed to measure higher order skills—critical
thinking, analytic reasoning, problem solving, and written communication—deemed
essential in the study of all academic disciplines. Indeed, says Richard
H. Hersh, co-director of the Collegiate Learning Assessment Project
and former president of Hobart and William Smith Colleges and Trinity
College, “What [the Collegiate Learning Assessment] measures is
core to everything else.” It is, he believes, a test worth “teaching
to.” Stephen P.Klein, recently retired senior research scientist
at RAND Corporation and currently director of research and development
for the Collegiate Learning Assessment, agrees. ”We made a strategic
decision,” Klein says, “to go after those things which are
common to all curriculums.” Our focus, he explains, was to be
on “measuring analysis, not memorization.” Asking students
to demonstrate what they remember about the requisite information and
skills pertaining to a task by filling in a line on a multiple-choice
test is not the same as asking them to perform that task. After we have
memorized the driver’s manual, we must ultimately drive the car;
after learning the scales, perform that sonata; and after memorizing
Grey’s Anatomy, perform surgery.
“We must remind ourselves that the value of a liberal arts education and education in general, is to enhance men’s and women’s powers of rational analysis, intellectual precision, and independent judgment,” observes Carnegie Corporation of New York president Vartan Gregorian. “After all, a proper and balanced education is neither a passive act nor an end in itself.” In the last analysis, he says, such an education is about doing.
According to Daniel Fallon, program director of higher education for Carnegie Corporation of New York, the Collegiate Learning Assessment literally “rose from the field,” reflecting the best creative thinking of the academic research and psychometric community. The fundamental objective of the developers of the Collegiate Learning Assessment, observes Fallon, was “to demonstrate the value of a liberal arts education—especially to liberal arts colleges.” The Corporation agreed to provide the lead seed funding for its development as well as grants to support pilot experiments. The results of these pilots, Fallon says, were, in a word, “knockouts.” In all, the project was awarded three grants by the Corporation totaling $1,150,000. The Ford Foundation, the Christian A. Johnson Endeavor Foundation, the William and Flora Hewlett Foundation, the Lumina Foundation, and the Teagle Foundation provided additional funding.
Available to colleges and universities since the spring of 2004, the Collegiate Learning Assessment is currently being used nationwide by 165 colleges and university systems as well as a number of consortia that use the assessment for benchmarking and comparative analyses. The Collegiate Learning Assessment is not a multiple-choice test, nor does it measure knowledge of course content. It is a ninety-minute, open-ended, computer-administered assessment that asks students to perform “real-life” tasks that they find engaging. For example, they might be asked to prepare a memorandum or policy recommendation after evaluating the credibility and relevance of a variety of sources, analyzing and synthesizing the information, drawing conclusions, and acknowledging that there might be alternative explanations and viewpoints. In addition, students are required to perform two analytic writing tasks (“make an argument” and “critique an argument”), which are evaluated holistically. Natural language processing software is used to score written communication tasks, and performance tasks are scored online by human raters using monitored and calibrated scoring. Within the next few years, it is expected that performance tasks will be scored by computer software.