Care with discussing research

One of the points of doing more formal research is to be able to be more specific and accurate about what we claim — and what we do not claim.

So it’s important to have the reporting on that research be as clear as possible. As I noted at the beginning of my talk, I don’t have answers yet to my questions, only a small pool of initial data and a lot of additional questions. So, for instance, I can’t and didn’t make the claim that

“First-year male TA’s have difficulty balancing time demands....”

Instead, what I can and did say is that in one cohort of first year TAs at my university, the male TAs who participated in the first year of my study reported being strongly concerned with how they would balance time demands of responding to student writing when they started teaching for the first time the following year.

Similarly, I can and did say that second and third year TAs from my school who participated in the first year of my study were more likely than first-year TAs to report higher concerns about making assignment requirements clear to students and responding to pressures to raise grades — and that within that small sample, women were more likely than men to note those concerns.

Are these local or personal anomalies? are they more consistent trends? It’s tempting to make guesses based on our own experiences or anecdotal evidence, but the fact is that I don’t know for sure. That’s why my study is designed to run for at least three years at my university, and why I’m expanding to survey TAs at different university sites as well.

This kind of research into how people learn to do complex tasks like write and teach writing, and more specifically into what kinds of methodologies are most effective at helping the most people learn these tasks better, is necessarily slow; the claims from such research, especially early ones, are necessarily partial; and the need for replicating the research and then aggregating the data is high.

—Shelley Reid

Shelley Reid, at 10:00 am EST on December 28, 2007

The obvious way to determine whether writing instruction works is to read the writing that it produces. Kids either can or cannot write a correct sentence; they either can or cannot organize ideas in a coherent way.

It is all too easy to determine whether instruction is effective, or whether it sacrifices essential skills in favor of meaningless theory and research.

JBM, at 12:45 pm EST on December 28, 2007

I certainly agree that empirical research is needed in rhet/comp studies — particularly those studies that attempt to determine whether and how students benefit from particular kinds of conferences, commentary, or other writing assistance.

For example, in my own empirical study of online asynchronous and synchronous conferences between online instructors (in this case, Smarthinking e-structors)and first-year English students, I found that the students in the study did, indeed, make content (global) and sentence-based (local) changes to their writing in response to the conferences. Among other findings, they made their own decisions about what they would change, to include enlarging upon a suggested revision and/or choosing against such revision. In other words, students used the help, but remained autonomous. Interestingly, the research, which partially replicated and extended Anne Gere’s research of 1987, revealed a number of areas where the instructor commentary needed adjustment for the entirely textual nature online conferences. There were very specific kinds of comments (particularly “suggestions") that students seemed to find confusing and difficult to decipher into revision changes; I address those professional development needs in a book manuscript under review. Regarding publication, I have been fortunate to find interested publishers of the general research in Computers and Composition (23.1, 2006) and Readerly/Writerly Texts (Vol 11&12, No 1&2, 2004-2005). Other research that has led to publication includes that conducted by Christa Ehmann and myself in Preparing Educators for Online Writing Instruction: Principles and Processes (NCTE, 2004).

That said, I wonder if one thing that steers some scholars away from empirical research is the false impression that it reveals “truths” or certainties. Most people who conduct such research realize that there are relatively few “certain facts” that they can generalize to a broader population. But, given a contexualized understanding of the study, some sound suggestions usually can be made — and these are more valuable in rethinking our practices than anecdotal statements and lore.

Another thing that makes studies like the one Shelley Reid describes more challenging to do is that such work: (1) is very time consuming, (2) usually would benefit from more than one committed researcher, and (3) often is costly in materials other than person-hours. In other words, such research would benefit from being institutionally supported. An empirical study of almost any size is a big job and it takes a concerted effort to accomplish both the study and its analysis and ultimate reporting. So, while I agree with cga that grad studies need to support learning such research, so, too, do the departments and institutions in which the research would be conducted. It sounds like Shelley has that support.

This is a great topic to lead into a new year. Glad that Scott Jaschik has opened it up for us.

Beth Hewett

Beth Hewett, at 1:20 pm EST on December 28, 2007

“It is not as simple as the ability to create a grammatical sentence, nor even the ability simply to create a logical paragraph. That’s high school competency.”

Yes, of course it is. It is also competency that college students regularly lack.

JBM, at 2:55 pm EST on December 28, 2007

A comment from the trenches

I’ve been a community college comp teacher for more than 30 years.

Here’s why students are poor writers: They don’t read. Reading—not formal classroom instruction—is the way most of us learned spelling, punctuation, and the grammar of written English. Improving student writing is as simple as turning students into people who read for pleasure, not because they have to. Of course, that’s the trick. If I knew how to turn students into readers, I would have retired rich and famous years ago.

Researchers can research all they want and innovators can innovate until they’re blue in the face, but teaching academic English to students who don’t—or won’t—read is like teaching music to the tone deaf or painting to the color blind.

And, by the way, the complaint that kids today can’t write is nothing new. Pick a time in American history, and we teachers were saying the same thing.

Denis, at 4:20 pm EST on December 28, 2007

Writing if Crucial

I am not an educator nor part of any interest group other than a business man who assists high school students in choosing a college. I am excited for those of you who are venturing forth on all fronts of the “writing is critical” issue. My daughter teaches special education in the poorest district in our state. Her goal is to get third graders to write a sentence. How basic yet so crucial for their future. Please don’t let your efforts go unheard. Evangelize government, business, and your college presidents to treat this as seriously as medical and scientific research for if we can’t communicate effectively, we can’t move forward effectively. Thank you all.

Milt Eisenhardt, College Money, at 11:05 am EST on January 3, 2008

Reharding research in writing

The need is enormous. What is available is thin. I refer only to efficacy research, and my focus is K-12. The Smagorinsky research review, and the MacArthur, et al review are both terrific collections, but neither offers the critical point: “If you do these kinds of things with these kinds of students over this period of time, you increase the probability of getting these minds of writers. We have such efficacy evidence in reading, mathematics, piano, middle linebacking, sculpturing, and a host of other teaching/learning endeavors. The need in writing is desperate. We are mired in classroom management plans, in off-the-shelf diminutive lessons, in love affairs with this or that activity. We need evidence on what to teach, to whom, how, and for how long. We also need evidence relative to what to assess, how, and to what end. A bare start on the former appears in Action in Teacher Education (Summer 2007) and Journal of Basic Writing (Spring 2007). A response to the assessment question is out for review. We need as many advanced graduate students in writing doing basic research as there were in reading during the last third of the twentieth century. And they need to approach their research dispassionately.

Leif Fearn, Professor at San Diego State University, at 5:30 pm EST on January 10, 2008