Very Like a Whale: The Assessment of Writing Programs - Softcover

White, Edward M.; Elliot, Norbert; Peckham, Irvin

Softcover

ISBN 10: 087421985X ISBN 13: 9780874219852

Publisher: Utah State University Press, 2015

Synopsis

Written for those who design, redesign, and assess writing programs, Very Like a Whale is an intensive discussion of writing program assessment issues. Taking its title from Hamlet, the book explores the multifaceted forces that shape writing programs and the central role these programs can and should play in defining college education.Given the new era of assessment in higher education, writing programs must provide valid evidence that they are serving students, instructors, administrators, alumni, accreditors, and policymakers. This book introduces new conceptualizations associated with assessment, making them clear and available to those in the profession of rhetoric and composition/writing studies. It also offers strategies that aid in gathering information about the relative success of a writing program in achieving its identified goals.Philosophically and historically aligned with quantitative approaches, White, Elliot, and Peckham use case study and best-practice scholarship to demonstrate the applicability of their innovative approach, termed Design for Assessment (DFA). Well grounded in assessment theory, Very Like a Whale will be of practical use to new and seasoned writing program administrators alike, as well as to any educator involved with the accreditation process.

About this title

About the Author

Edward M. WhiteM/b> is emeritus professor of English who held positions at California State University, San Bernardino, and the University of Arizona. Norbert Elliot is Professor Emeritus of English at New Jersey Institute of Technology. Irvin Peckham is professor of rhetoric and composition at Drexel University, where he directs the writing program.

Excerpt. © Reprinted by permission. All rights reserved.

Very Like A Whale

The Assessment of Writing Programs

By Edward M. White, Norbert Elliot, Irvin Peckham

University Press of Colorado

Copyright © 2015 University Press of Colorado
All rights reserved.

ISBN: 978-0-87421-985-2

Contents

Introduction, 
1 Trends, 
2 Lessons, 
3 Foundations, 
4 Measurement, 
5 Design, 
Glossary, 
References, 
About the Authors, 
Index, 

CHAPTER 1

TRENDS

The national investment in composition instruction remains huge, despite recent budget cuts throughout the educational system. At the high-school level, we can count 358,136 students taking the advanced placement examination in English language and composition (College Board 2013, Table 9), the better-prepared tip of a population concerned with entry-level college writing. The National Center for Education Statistics (2012; Hussar and Bailey 2008, Fig. C) reports that total enrollment in degree-granting institutions increased 23 percent from 1992 to 2006. Between 2006 and 2017, a period of only eleven years, enrollment is projected to top at least 19.4 million students, almost all of whom will be enrolled at some level in college writing courses.

Such growth is accompanied by new challenges. Citing a 300 percent rise in average tuition during the past three decades, an average bachelor's degree debt of more than $29,400, and a graduation rate of only 58 percent for a four-year degree within six years, the Department of Education has launched federal initiatives to ensure quality in postsecondary institutions (White House 2013). Of special interest is the emerging Postsecondary Institution Ratings System (PIRS) that will be used to produce individual college scorecards linked to student financial-aid levels. Students attending colleges with higher ratings could, for instance, be eligible for larger Pell Grants and favorable rates on student loans. The opposite would be true for students at colleges with lower ratings (National Association of Student Financial Aid Administrators 2014).

In light of such developments, it is both reasonable and responsible for administrators and public officials to inquire into the effectiveness of the writing instruction programs in which students are enrolled, particularly at two crucial transfer points: between high school and college, and between graduation and career development. For over fifty years, composition researchers and assessment specialists have been attempting to provide reliable evidence — beyond the felt experience of teachers — that composition instruction leads to valuable categories of student improvement, including evidence that students are better writers. But the results have been infrequently laudable, often mixed, and sometimes disappointing.

WRITING PROGRAM ASSESSMENT: A DISCOURAGING HISTORY

Among the many reasons for this complex situation, perhaps the most important is the difficulty of defining writing and the objectives of writing instruction at the transfer points we have described. This complexity is deepened when assessment is understood as integral to the daily administration of writing programs and is made even more complex when program assessment is understood as a form of research. Even our leaders seem to be at a loss. For Rose and Weiser (1999), writing program research is taken to be "theoretically-informed, systematic, principled inquiry for the purpose of developing, sustaining, and leading a sound yet dynamic writing program" (ix). For Douglas Hesse (2012), that definition is an Aristotelian act of structuralism. Writing program research, he believes, most benefits by attention to the acts, purposes, and audiences of the programs themselves. Like Polonius echoing Hamlet's sarcasm, different observers see different things in shadowy shapes.

What we might call the definition problem is all the more significant because writing has been so simply defined. In 1963, Albert K. Kitzhaber measured writing performance by elaborate error count, thereby defining good writing as correct writing, a construct few would defend today Kitzhaber (1963). The Braddock, Lloyd-Jones, and Schoer (1963) study defined writing research in terms of limited empirical methodologies, thereby ruling out characteristics of writing not readily measurable. When some writing programs developed as separate entities from the English departments within which they customarily resided (marked by the founding of the Council of Writing Program Administrators [CWPA] in 1976), researchers began anew to define what such programs were seeking to attain beyond measures emphasizing knowledge of conventions. A team of researchers led by Michael Scriven (1981) produced twenty separate technical reports in a Carnegie Foundation-funded attempt to document the outcomes of the (then) Bay Area Writing Project from 1976 to 1979; the findings were inconclusive, though the revised volume provided what the authors called a "handbook" for those doing further research on the topic. The Fund for the Improvement of Post-Secondary Education funded a study culminating in an important volume in 1983 by Stephen Witte and Lester Faigley that focused primarily on a more expansive vision of what a college writing program could accomplish and found the usual empirical study inadequate because the assessment objectives were unclear: "We have suggested that the complexity of writing programs has done much to limit the development of adequate evaluative procedures and methods. Yet the sheer complexity of the thing evaluated is not the only reason the art of writing program evaluation remains in its infancy" (Witte and Faigley 1983, 66).

At the same time, a project led by Edward White and funded by the National Institute of Education employed empirical methods in a five-year effort to determine the writing program features most correlated with effective student writing. Published in a final report (White and Polin 1986), the results presented findings of interest but contained predominately descriptive information about the kinds of writing instruction then most in use. Although research — most notably the meta-analysis by Hillocks (1986) and the taxonomy by Stephen North (1987) — also appeared in the mid-1980s, the frustrating variety of definitions of good writing continued. And, as expected, equally absent were reports of convincing research on writing programs. Recent studies have expanded knowledge of program assessment by attempting to articulate the objectives of writing across the curriculum (Bazerman et al. 2005; Yancey and Huot 1997) and writing in the disciplines (Neff and Whithaus 2008; Poe, Lerner, and Craig 2010). Such specificity has yielded enormous benefits for the design of programmatic assessment, with recent scholarship focusing on the assessment of writing centers through attention to key elements of locally based assessment, including alignment of the program with institutional mission, establishing meaningful outcomes, and communicating assessment results (Schendel and Macauley 2012).

Yet we are not alone in our struggle. Interwoven with dissatisfaction about the inability of our field to come to terms with the success of its own instructional programs is a century's worth of similar expressions of doubt in the nation's educational system itself — especially its ability to produce reform through educational research (Lagemann 2000). If we seek a hallmark document, we could do no better than Edwin C. Broome's (1903)A Historical and Critical Discussion of College Admission Requirements. Analyzing admissions practices at Harvard, Yale, Princeton, Columbia, the University of Michigan, and Cornell, he cited "evils resulting from diversity in admissions requirements" (128). With its origin in 1887 and its aim to lessen such evils through standardization, the Middle States Association of Colleges and Schools Commission on Higher Education, along with other regional accreditation agencies, appeared to be part of the solution until the century's end. By 1991 the commission itself was under criticism from the federal government as it became increasingly apparent that standard gauge systemization was not the answer to valid assessment (Harcleroad 1980; Tedesco 1999). A proposed House bill to reauthorize the Higher Education Act of 1992 argued that state agencies should replace private accrediting groups such as Middle States, thus becoming the primary reviewers of institutions receiving Title IV funding (Parsons 1997). When the dust settled and the act was reauthorized, it was clear that the regional agencies would have to achieve new levels of accountability themselves if they were to continue to exist. The six regional accreditation agencies — Middle States, the New England Association of Colleges and Schools, the Commission on Institutions of Higher Education, the North Central Association of Colleges and Schools Higher Learning Commission, the Northwest Commission on Colleges and Universities, the Southern Association of Colleges and Schools Commission on Colleges, and the Western Association of Colleges and Schools Accrediting Commission of Senior Colleges and Universities — would now, by law, have to conduct reviews of institutions where the student-loan default rates exceeded 25 percent. Specific areas of institutional review were required. Writing program evaluation did not stand alone but was one element in the overall dissatisfaction with program accountability. If accountability were to be taken as a demonstration of accepted responsibility and wise resource use, the accreditation community would now be required to take a fresh look at its educational quality assurance system (Astin and Antonio 2012; Price et al. 2008).

Within the writing community, this fresh look took dramatic form on the active listserv associated with the WPA. In response to a query on the list in 1996, White suggested that it may have become time to articulate a consensus on the objectives of the first-year composition course. The overwhelming response to this proposal led to the formation of an outcomes group in 1997, which then led to multiple drafts (with about 240 colleagues taking part in 1998) of an Outcomes Statement approved by the WPA Executive Committee (at last giving this ad hoc volunteer group some formal standing). The Outcomes Statement was then published in WPA: Writing Program Administration (Outcomes Group 1999) and College English (Harrington et al. 2001), and became the focus of The Outcomes Book (Harrington et al. 2005). The widespread impact of the Outcomes Statement became clear in that book, and its continuing influence remains apparent in The WPA Outcomes Statement: A Decade Later (Behm et al. 2012). In 2014 the Outcomes Statement was updated to reflect changes in the field and current practices in first-year writing (Dryer et al. 2014). While an entry-level writing course is by itself only part of a college writing program — an argument we will make time and again — the ability to articulate a consensus on outcomes for the transition between high school and college became a major step in writing program assessment.

Further developments ensued. In 2011, the CWPA, in collaboration with the National Council of Teachers of English (NCTE) and the National Writing Project (NWP), issued the Framework for Success in Postsecondary Writing (CWPA, NCTE, and NWP 2011; O'Neill et al. 2012). While the WPA Outcomes Statement defined what was expected at the end of entry-level college composition, the Framework for Success focused on a broadened concept of writing ability so students could reach the intended outcomes. Alert to the ways that the Framework for Success could support kindergarten through twelfth-grade teachers, its authors added "habits of mind" as ways of approaching learning that would foster student success across disciplinary boundaries by means of attention to intrapersonal and interpersonal domains: curiosity, openness, engagement, creativity, persistence, responsibility, flexibility, and metacognition. These habits of mind, in turn, would be developed through reading, writing, and critical analysis experiences of the Outcomes Statement. The Framework has indeed generated some controversy, in part because the habits of mind — those intrapersonal and interpersonal domains — are more challenging to measure than the clarity of the cognitive domain expressed in the Outcomes Statement. In its insistence that questions such as What is writing? and How should we assess writing development? be answered, the Framework asks us to consider the writing construct in all its complexity.

While the Outcomes Statement has by no means brought the first-year composition course almost universally required in American colleges and universities into conformity (Isaacs and Knight 2012), it has provided a set of objectives and definitions that seem to apply widely enough to the course to suggest to textbook publishers a more outcomes-based approach to the field. Recent research by Emily J. Isaacs (2014) of 106 American Association of State Colleges and Universities Institutions reveals that 68 percent of writing program administrators conducted formal review of student writing in light of outcomes or objectives. To address such assessment needs, many publishers now link textbook features to the Outcomes Statement. One publisher — McGraw-Hill — has published a popular text whose theoretical underpinnings are directly related to the statement. The team of editors for the McGraw-Hill Guide used the statement as a framework for the book while at the same time using modern theories about writing as reflection (Brady and Schreiber 2013; Fox, White, and Tian 2014; Giles 2010; Inoue and Richmond, forthcoming; Smith and Yancey 2000; Yancey 1998) to foster assessment as part of the course (Kelly-Riley and Elliot 2014). Other publishers have followed this lead. As we write, most new textbooks for introductory writing courses include references or chapters citing the Outcomes Statement. The combination of portfolio-based writing assessment and outcomes-based curriculum derived from a national consensus document represents something new in the world of composition programs: a consensus model of the writing construct.

The history of writing program evaluation and its intersection with the history of accreditation in general suggests that a new era in writing program assessment is at hand. While we present both the risks and benefits of the Age of Accountability, our brief history of writing program assessment — which is not intended to be encyclopedic and which someone must surely write — suggests that informed, programmatic practices are rapidly becoming the order of the day. Developed locally, these practices can yield great benefits for specific institutional sites. Perhaps the best place to begin a programmatic approach to assessment is, as we have suggested above, with attention to definitions.

WRITING PROGRAMS: AN INTEGRATED FRAMEWORK

Many members of the public, as well as some in the university community, mistakenly believe that one or two courses in writing constitute an entire writing program capable of helping students to achieve the writing ability they — and future employers or graduate teachers — expect of college graduates. Among writing professionals, this is satirically referred to as inoculation theory — the metaphor stemming from the theory of McGuire and Papageorgis (1961) that resistance to harmful practices can be enhanced by exposure. In this metaphor, writing courses required early in the undergraduate curriculum serve as a polio shot, forever inoculating the student against future infections of comma faults, citation errors, faulty logic, and inadequate critical thinking. A moment's reflection is time enough for anyone who has done much writing, or has paid much attention to longitudinal research (Fishman 2012; Haswell 2000), to remember that the development of writing ability is a slow and laborious process that is, for many, a lifetime's work. While the entry-level composition class is an important part of every student's college learning, if it stands alone in the curriculum it becomes one more requirement students seek to "get out of the way" so they can get to more important matters, such as the requirements for their major course of study. One major practical incentive to maintaining writing-across-the-curriculum programs and writing-in-the- disciplines programs is to relieve that isolation of the first-year writing course and to increase opportunities for knowledge transfer (Holdstein 2014).

What has long been needed is an approach to writing program assessment that advances strategically planned integration instead of focusing on isolated assessment events. The Program Evaluation Standards (Yarbrough et al. 2011) provides a good definition of a program as "much more than just activities" (xxiii). Defined in detail, a program is a set of activities that uses managed resources to achieve specified, documentable outcomes (xxiv). Extending this definition to writing programs, we see that many universities have developed writing programs that contain all or most of the following components:

• a defined construct model that is used as the basis for the writing program;

• preenrollment assessment and placement to identify just-qualified students through tests, evaluation of transfer-course work, or directed student self-placement;

• required writing courses, often including basic writing for poorly prepared students and honors classes for well-prepared students;

• a writing center, where students seeking help with their writing can find tutors and other support services, directed by a writing center professional;

• a writing-across-the-curriculum (WAC) in-service program for faculty, with a WAC coordinator offering a range of support activities;

• writing intensive course requirements, usually designated and offered by departments or schools of the student's major discipline and overseen by a campus faculty committee committed to writing in the disciplines (WID);

• graduation writing requirements, which may include writing-proficiency examinations, portfolios, or capstone courses emphasizing writing in the major;

• writing requirements for entry into graduate programs, such as examinations, statements of purpose, or portfolios;

• thesis and dissertation requirements for graduate degrees;

• a writing program administrator (WPA) with graduate degrees in rhetoric and composition who speaks for writing on campus and has overall administrative responsibility for the entire writing program;

• a plan for sustainable financial support, with particular attention to salary and benefits (American Association of University Professors 2014);

• an articulated research agenda and program of research so the many composition pedagogies identified by Tate et al. (2013) — from basic writing through new media — can inform and be informed by research conducted within the program (Anson 2008); and

• an overall strategic plan for the program that establishes its mission, priorities, objectives, strategies, targets, metrics, accountability, and impact (Bryson 2011; Quinn 1980; Schendel and Macauley 2012; Smith 2011).

(Continues...)

Excerpted from Very Like A Whale by Edward M. White, Norbert Elliot, Irvin Peckham. Copyright © 2015 University Press of Colorado. Excerpted by permission of University Press of Colorado.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

Edition details

• Publisher: Utah State University Press
• Publication date: 2015
• Language: English
• ISBN 10: 087421985X
• ISBN 13: 9780874219852
• Binding: Paperback
• Number of pages: 208