wp home > junior year writing > sourcebook for instructors > dynamic criteria mapping
Dynamic Criteria Mapping
Another method of evaluation eschews the use of general standards and asks teachers to be more scrupulously candid about how they judge students’ writing as they read it. The method begins with real student papers and asks groups of teachers and/or students to discuss together, as honestly as they can, how they actually read those papers and what they really value and don’t value in them. It doesn’t do away with normative criteria; rather, it builds them up inductively and collaboratively. The method is called “Dynamic Criteria Mapping” (DCM) and derives from Bob Broad’s What We Really Value: Beyond Rubrics in Teaching and Assessing Writing (Logan: Utah State University Press, 2003).
In the classroom, the method entails reading diverse sample papers with students, talking openly together about them, and developing (under the teacher’s leadership) honest, rich, and specific criteria for evaluation. The idea is to give students more knowledge about the real challenges of writing well and information about how to succeed in meeting them. More than any other method examined here, DCM is a reader-based evaluation system.
DCM also has potential benefits for writing programs as a whole, whether first-year or junior-year, encouraging teachers to gather face to face with their colleagues, read student texts together, and debate their judgments of them. The resulting assessment is also likely to be more valid, drawing evaluative criteria directly from how teachers actually teach and assess students’ writing and thus “strengthening the links between what [they] tell students and the public . . . and what [they] really do” (122).
A program using Dynamic Criteria Mapping would undertake the following steps:
-
Selection of a diverse set of sample student papers, not shying away from ones productive of disagreement. Instructors then read the essays and make notes on what they value and do not value in each text.
-
Articulation in large groups: program-wide discussions meant to allow readers to share the criteria by which they were guided in evaluating each text, with the focus on listening to and understanding the full range of values at work.
-
Trios for “live” judgment, in which small groups of instructors read together actual papers from their own classes, keeping in mind the criteria developed above but now bringing in the special knowledge about students they possess.
-
Dynamic Criteria Mapping
-
Collecting data: Scribes record readers’ criteria during the articulation sessions, writing down the full range referenced, along with the passages pointed to in the sample texts. They resist reducing comments to one-word, abstract criteria.
-
Analyzing data: Afterwards, program participants look back at all the criteria recorded and try to develop “findings”: clusters of criteria and their interrelations. But they need to be true to the data and avoid importing familiar schemes to arrange them. The whole idea is to move “beyond what we think about how we value students’ writing and to discover what and how we really value” (132).
-
Debating and negotiating evaluative differences: Now that participants know “how they do value students’ writing, they need to undertake high-powered professional discussions regarding how they should value that writing” (133). The focus shifts from the descriptive to the normative.
-
Publicizing the map: The program should next publish the results of the process along with the sample texts used, making the results as simple as possible but still maintain “texture, nuance, detail and complexity.”
-
Revising the map: To be truly dynamic, the results of DCM need to “grow and change organically over time.” They are works in progress.
-
Classroom application: teachers should include students in the construction of evaluative schemes: they should be asked, “what does this instructor value in your work?” in order to ascertain the truth about the evaluative topography of the classroom.
Below is a sample map from Broad’s book. This one is focused on “textual qualities,” and is oriented towards the reading experience of actual readers in the program studied. Other maps focus on “textual features” (elements of texts, like grammar, punctuation, etc.) and “contextual criteria” (the standards of the course, assignment expectations, etc.).
For an example of DCM, as applied to Unit I of College Writing, click here (PDF).
To return to the list of systems for evaluating student writing, click here.
To return to the article on Evaluating Student Writing, click here.
Updated September 3, 2008
