When More Is Less

Much research suggests that more writing is associated with more learning, and that’s given more credence to the Writing Across the Curriculum movement, which promotes the importance of writing assignments everywhere, not just in composition classes. The landmark 2011 book Academically Adrift: Limited Learning on College Campuses, for example, says the one notable exception to the finding that students learn little after three semesters at college in terms of critical thinking and complex reasoning is among those students who write the most. But the research on which writing interventions are most helpful is less conclusive, and that’s caused some to doubt the effectiveness of pedagogies that promote a good deal of writing in fields beyond English. Supporters of such pedagogies, meanwhile, believe in writing to learn.

A new study sheds more light on the quality versus quantity issue. It seeks to clear up some of the outstanding questions about which writing interventions work best -- and whether more truly is better.

In short, the study says, it’s not.

“Effective writing practices are associated much more strongly than the amount of writing with greater student learning and development,” the study says. “There are undoubtedly instances where there is no student writing or so little that more would be salutary. However, the important lesson from our study is that quality matters -- that in many situations it would be better to place more emphasis on the design and use of the assignments than on the number or size of them.”

In a collaboration between the National Survey of Student Engagement and the Council of Writing Program Administrators, the study’s authors gathered with the help of colleagues a list of 27 writing practices that are said to be effective. They attached questions about those practices to the student engagement survey at 80 baccalaureate institutions, obtaining responses from 70,000 freshman and senior students.

The idea was to examine the relationship between the responses to the 27 writing practice-based questions and questions on the standard questionnaire regarding two sets of established survey constructs: participation in "deep approaches to learning," or more-than-surface-level understanding of content, and "perceived gains in learning and development." The latter means students’ self-reported intellectual growth and personal satisfaction over time.

Example practice-based questions included how many times during the current school year (mostly 2011) a writing assignment caused a student to brainstorm ideas before beginning a draft. How many times did you talk with your instructor to develop ideas before writing, another question asked. And did you explain in writing the meaning of numerical or statistical data?

The authors developed groupings -- essentially additional constructs -- for most of the questions: interactive writing processes, meaning-making writing tasks and clear writing expectations. The interactive construct assessed how students communicated with others, either orally or in writing, before submitting a final draft of an assignment. Meaning making assessed how students engaged in some form of integrative, critical or original thinking. And clear writing expectations explored whether students had a solid understanding of what was required of them upon receiving a writing assignment.

Using various models in a regression analysis, the authors compared those responses to those gauging deep approaches to learning and perceived gains in learning and development. The authors define deep approaches as those that lead to higher-order, integrative and reflective learning. Perceived gains include those in practical competence, personal and social development, and general education. The authors controlled for various student characteristics, such as gender, age, race, ethnicity, area of study and self-reported grades, among other factors.

How many pages students were asked to write appeared to have minimal impact. The bivariate correlations between writing quantity and deep approaches -- meaning the relationship gets stronger as the value approaches 1, from 0 -- was 0.15 to 0.27 for first-year students, and 0.11 to 0.22 for seniors.

The correlations between effective interventions and deep approaches, meanwhile, were 0.20 to 0.42 for first years and 0.19 to 0.41 for seniors. Meaning-making assignments seemed to have the biggest positive impact. The authors call the correlations “moderate,” but meaningful.

“Writing assignments and instructional practices represented by each of our three writing scales were associated with increased participation in deep approaches to learning,” the study says (although some of that relationship was shared by other forms of engagement). “First-year and senior students who reported that more of their writing assignments required meaning making were especially likely to report greater participation in all three forms of deep approaches to learning.” Students who reported that more of their writing assignments involved clearly explained expectations were more likely to report greater higher-order learning in the classroom.

Effective writing practices -- in particular interactive writing processes and clear expectations -- had a small but significant impact on students’ perceived gains in learning and development (the equivalent of about 5 percent of additional explained variance). Quantity had no impact on perceived gains.

Among interactive writing processes, students were most likely to talk with classmates and others about their ideas before drafting an assignment, and they were least likely to visit campus-based writing or tutoring centers for help. Among the meaning-making tasks, nine out of 10 seniors and first-year students said they were asked to analyze or evaluate something they read in least some of their assignments, while fewer students were asked to talk about data. Nine in 10 students said at least some of their instructors provided clear instructions, explained what they wanted students to learn and described the criteria they would use to grade an assignment.

The study, called “The Contributions of Writing to Learning and Development: Results From a Large-Scale Multi-Institutional Study,” was published in Research in the Teaching of English. It was written by a group of well-known writing scholars: Paul Anderson, the director of Writing Across the University and a professor of English at Elon University; Chris M. Anson, Distinguished University Professor and director of the Campus Writing and Speaking Program at North Carolina State University; and Charles Paine, a professor of English and the director of Rhetoric and Writing at the University of New Mexico. Their co-author, Robert M. Gonyea, is associate director of the Indiana University Center for Postsecondary Research and a research and reporting coordinator for the student engagement survey.

In a group email, the authors said their findings about which interventions are most effective aren’t just for writing instructors, and may even be especially useful for instructors in other fields, “many of whom feel that the more time they spend assigning and attending to their students’ writing, the less time they will have to spend on the primary focus of their courses.”

Gonyea said in an interview that the study has given way to a permanent new writing-based node based in the student engagement survey, pared down to 13 questions. Some 69 institutions elected to use it last year. Institutions like to compare their findings to the overall results, or compare year-over-year changes, he said.

“This has implications for the faculty, and how they structure their courses,” Gonyea added. “There are certain disciplines where students don’t do a lot of writing, and we’d encourage [instructors] not necessarily to assign longer papers, but to assign smaller writing tasks that go a long way in terms of student learning.”

The authors said their research supports calls made by professional organizations to promote evidence-based writing practices, including a push by the Association of American Colleges and Universities to establish a national framework for writing goals in baccalaureate education. The study also may be helpful, they say, to nonwriting specialists interested in accountability and improving educational quality.

Elaine Maimon, president of Governors State University and a founder of Writing Across the Curriculum, has helped shaped AAC&U's stance on high-impact practices. Maimon called Writing Across the Curriculum “the original” high-impact practice, and said she’d endorse the study's premise that quality of interventions matters more than number of pages assigned.

“Students should write a great deal to develop fluency, but they should ‘go public’” -- meaning submitting material for a grade, she said -- “only after carefully revising material, with interventions from the instructor and peers.”

Daniel Melzer, associate director of first-year composition at the University of California at Davis, is referenced in the new paper for his popular 2014 study suggesting that most writing assignments are poorly crafted (The authors agree, and say more assignments need to be based on the practices they’ve identified). He said he’s been following the authors’ preliminary research, and that it’s already made waves in the field.

Melzer said he agreed that getting faculty members to assign more writing has been a focus of the Writing Across the Curriculum program, but that it isn’t really a push for more volume as much as a response to the fact that first-year students on many campuses were writing very little outside of their first-year writing or writing-intensive courses. Like Maimon, he also distinguished between informal journals or “quick writes,” and more formal assignments. Writing Across the Curriculum emphasizes the former, he said.

“I've always followed the suggestions in this [new] article that it's better to assign less formal writing but have students engage in a deep, interactive process of writing and meaning making,” he said. “That said, my own research leads me to believe that in too many classes students only writing is short-answer exams, so more formal and extended writing may be called for in these cases.”

Richard Arum, chair of sociology and a professor of education at New York Univeristy, co-wrote Academically Adrift, which also is cited in the new paper. Arum said it's "exciting to see researchers working to improve measurement of writing instruction," and that he never asserted that more writing alone is most effective.

"It is hard for me to imagine that any thoughtful educator believes that increasing the quantity of assigned writing is the most effective pedagogical approach to improving the quality of student writing," he said, noting he was once a high school English teacher. Like the study's authors, Arum said that because the data rely solely on student self-reported learning measures, he'd be "eager to see the work extended to test the extent to which these constructs track with objective measures of actual student writing."