Carnegie
Corporation
of New York
Fall 2007
The motivational issue of getting students to “volunteer” to take the Collegiate Learning Assessment is vexing. “If kids are not working as hard as they can,” saysCapaldi, the concern is that “[we] won’t get a good measure.” The students who do take it “think it’s terrific,” she says. In particular, they enjoy taking a test that is administered online. After two years, Arizona State University has conducted cross-sectional studies and would like to use it longitudinally. “We are interested in building on [the Collegiate Learning Assessment],” explains Capaldi.
While large public higher education institutions like Arizona State University and the University of Texas System have signed on to use the Collegiate Learning Assessment, the assessment was originally designed with small- to mid-size liberal arts colleges in mind. The Teagle Foundation has worked to make the Collegiate Learning Assessment accessible to these colleges through the funding of a number of consortiums nationwide. To elicit their ideas and concerns about student learning assessment, the Teagle Foundation invited representatives of small liberal arts colleges to a series of “listening” meetings. The message communicated by these colleges, according to Teagle Foundation president W. Robert Connor, was essentially “We need to explore what assessment really means to us. But we’re too small and don’t have the experts or necessary resources.” Consensus formed around the idea that for these colleges a collaborative approach might be the best to follow. Out of these “listenings,” the Kalamazoo Assessment Collaborative was formed, which included Kalamazoo, Colorado, and Earlham Colleges.
Paul Sotherland, professor of biology at Kalamazoo College, remembers the “listening” well, in particular, “the fiery talk” about the need for assessing liberal arts learning delivered by Richard Hersh. Out of these “listening” meetings, a proposal for a collaborative was developed. Entitled “A Value-Added Assessment Collaborative–A Catalyst for Cognizance and Change,” the proposal laid out the following goals: “To create and nurture friendly intra- and inter-campus environments for assessment; demonstrate in a compelling way the value added of a liberal arts education; collect and use ‘stories and data’ to articulate and strengthen each institution’s unique approach to the liberal arts; and in so doing, breathe life into our latent organizational sagas.” The Collegiate Learning Assessment was selected as the instrument of learning assessment; to provide additional data, the National Survey of Student Engagement and the Cooperative Institutional Research Project were also selected.
The Collegiate Learning Assessment was administered to freshmen and seniors during the 2005-06 academic year. Says Sotherland, “Kalamazoo blew the lid off the assessment after the fi rst administration.” Test results demonstrated that the “value added,” i.e., the mean senior Collegiate Learning Assessment score minus the fi rst year Collegiate Learning Assessment score, of a Kalamazoo College education was well above expected. While the Collegiate Learning Assessment results provided cause for Kalamazoo to celebrate (“Hey, maybe we’re doing something right!” Sotherland admits was their fi rst reaction), the results also raised questions. What was there about a Kalamazoo liberal arts education that could account for this overall performance differential? How did the variations in the students’ educational background (SAT scores, GPA averages) account for the differences? Why were some students likely to thrive in the Kalamazoo environment more than others, regardless of whether their academic ability was high?13
13 For a detailed discussion of Kalamazoo’s examination of these questions, see P. Sotherland, A. Dueweke, D. Cunningham, and B. Grossman, “Multiple Drafts of A College’s Narrative,” Peer Review, 2007, 9(2): 19-23.