Assessment

Assessment is a crucial, ongoing part of the FYW Program at EMU. In Adler-Kassner's view, without assessment how can a writing program represent what it does to others, and thus affect the discussions about writing on campus? Without assessment, how can a writing program see what it is doing well and see what it needs to improve upon? Everyone in the program--graduate students, adjuncts, lecturers, and professors--are engaged in assessment and encouraged to collaborate on how to interpret and act upon the results of this assessment. She believes that by involving the entire program in participatory assessment and redesign of the curriculum everyone feels they have a voice in the program. As she said, "you get huge buy in" ("Interview"). Adler-Kassner's approach to assessment was particularly influenced by Bob Broad's What We Really Value: Beyond Rubrics in Teaching and Assessing Writing and Brian Huot's (Re)Articulating Writing Assessment.

Assessment generally falls within these three categories:

Placement Assessment
All Freshman are required to take ENGL 121, and only about 10% manage to place out of the class based on their transcripts. Placement for ENGL 120 is "self-guided," meaning students choose on their own to take the class or not.

Portfolio (Course) Assessment
As mentioned in the Curriculum & Philosophy page, both ENGL120 and ENGL121 have end-of-course portfolios. Students select two major essays to revise and include in the portfolio. In addition, students write a reflection letter to accompany the portfolio. The FYWP has developed a common portfolio assessment rubric that teachers follow as they grade these portfolios individually (Portfolios are not graded together in a holistic grading setting). The program keeps an extensive number of past portfolios with grades, so teachers and students can think about the rubric and its standards illustrated by these writing examples. Since the FYWP has the portfolio as a common assessment instrument using the same criteria, these portfolios become a significant means for larger Program Assessment

Program Assessment
Adler-Kassner believes in being pro-active with assessment. As Dr. Adler-Kassner states, "I think that one thing that good WPAs do is look proactively for the horizon and think about how to position their program, their students, and their instructors in ways that are going to be helpful to students learning and instructors teaching" ("Personal Interview"). Since arriving at EMU in 2000, Adler-Kassner has led three major assessments of the FYWP. The following summary of these three assessment cycles reveal the ways in which Adler-Kassner has developed and positioned her program.

Cycle I--2002
This first assessment of the revised curriculum, implemented after Adler-Kassner and Heidi Estrem arrived, surveyed student confidence levels with the outcomes of the courses. This survey indicated that student confidence levels in ENGL121 significantly increased from the beginning to the end of the course. Based upon this feedback, changes were made to improve the reading curriculum in the courses.

Cycle II--2004-2006 The Campus-based Context Specific Assessment Project
This larger research project grew out of discussions with their Dean about the first assessment project and focused on generating an assessment rubric for the portfolios that reflected accepted standards at EMU. The research study centered on these two research questions:

* If qualities of good writing are context-specific, what are the qualities of good writing in our local (EMU) context?
* To what extent are these qualities of good writing evident in the final portfolios of students in English 121 (EMU's required first-year writing course)? ("First Year Writing Program (FYWP) Assessment")

To answer these questions, the researchers convened Focus Groups of students, staff, faculty, administrators from campus the community outside the FYWP. In addition, they held focus groups of faculty from within the FYWP. The research study followed these four phases:

1. Hearing about qualities of "good writing" in the EMU Context (May-June 2004)
   --Gather data from Focus Groups of all stakeholders inside and outside the program.
2. Identify qualities of "good writing" (June-Nov. 2004)
   --Transcribe and analyze the focus group discussions. Coding strategies and dynamic criteria mapping used.
3. Develop assessment instrument reflecting the qualities of "good writing" at EMU (Dec. 2004-April 2005)
   --Multiple drafts of this instrument were generated and reviewed.
4. Analyzing the extent to which qualities of "good writing" were evident in a sample of portfolios from students in ENGL121 (2006)
   --Two teams of raters assessed a sample of 110 final portfolios (roughly 10% of one semester's enrollment) using the new portfolio assessment instrument.

The results of this assessment of final portfolios using the newly created assessment instrument found areas where students addressed or exceeded the established criteria of "good writing," but also these portfolios demonstrated areas where the FYWP might devote more resources to improve the teaching of these research strategies within the course. These results informed revisions to the ENGL121 curriculum, particularly involving the research process. (For more information, view the September 2006 ENGL121 Assessment Preliminary Report.)

Cycle III--FYWP Assessment Plan 2007-2008

The third assessment undertaken by the FYWP under Adler-Kassner has sought to assess changes to the research components of ENGL121 curriculum instituted after the 2004-2006 assessment. These changes included:

• Incorporating information literacy outcomes more clearly into ENGL 121
• Developing a clearer articulation of research processes into the framework for ENGL 121
• Developing and/or identifying existing strategies to support each step of research processes throughout the entirety of ENGL 121 ("FYWP Assessment Plan 2007-08")

This new project assesses the impact of these curricular changes and addresses these three questions:

1. What do students do with sources in their ENGL121 classes?
2. To what extent do portfolios produced by students engaging in the revised curriculum demonstrate qualities of "good research?" ...
3. How do students' work with these qualities develop through students' WI [Writing Intensive] courses?
   ("FYWP Assessment Plan 2007-08")

To investigate these questions, this research study will collect and analyze three sets of data from a total sample of 42 portfolios.

Data Set #1 and #2 will constitute a comparison group to see if the changes in the new curriculum are having the desired effect. After a group of raters assess the portfolios, the results of this study will be used for additional improvements to the curriculum and teaching for ENGL 121.