In the past I’ve always found the methods sections of studies a little boring if I’m being honest before I understood the importance of them. I was one of those students who would read the introduction and the results and I wouldn’t pay that much attention to the often math-y middle bits.
A description of methods is important because they describe the context and interpretation of the study, and also give some insight into why the researchers discovered what they did. Without really understanding this information one runs the risk of grossly misinterpreting the results of a study.
I’ve tried much harder over the course of my studies in composition to pay much more attention to methods sections and to assess a greater meaning from reading them rather than treating them as a hurdle to jump before getting to the results of a study.
Specifically, I’m writing about what new understandings of methods I’ve come to recently based on the readings of my current composition class and how they affect my understanding of how to collect data during a classroom observation.
1. The subject(s) of the study can be and sometimes should be a co-researcher
I think traditionally speaking when it comes to studies we imagine that subjects are not involved in the collection of data and sometimes are not even aware of the goals of the study. This calls upon prior knowledge of studies adhering to the ideology of scientific neutrality. the idea being that all that matters is what the data reveals, and that interpretation or individual perspective is not to be trusted.
However, this kind of objectivity could make things difficult for researchers depending on what they are trying to understand. For example, in a study called “Double Binds and Consequential Transitions: Considering Matters of Identity During Moments of Rhetorical Challenge” by Elizabeth Wardle and Nicolette Mercer Clement, the subject of the study was made a co-researcher because of what the researchers were attempting to understand.
The study was meant to reveal what kinds of struggles a student goes through during a consequential transitions when transferring prior knowledge to a new learning setting. A consequential transition is one that causes a student, or person for that matter, to struggle with sense of identity or place in society. These are transitions that change someone’s perspective on the way that the world works and who they are or should be.
The problem with doing this kind of study without a participants involvement in interrupting data is that a researcher can’t see inside the brain of the participant. A researcher wouldn’t be able to look at the data and determine what the subject was feeling, or how they understood their own struggle. As mentioned in another study called “Dynamic Transfer in First-Year Writing and ‘Writing in the Disciplines” Settings which used interviews for the process and used this understanding as a reason for getting input from the students via interviews to help the researchers with interpreting the data.
2. Data points shouldn’t always be predetermined
This idea is not one that is new, but one that I’ve reflected on again recently. Studies are sometimes conducted with a concrete idea of what the researchers want to explore. Going into a study there might already be a system for coding data, and for determining what it means, but often it’s a good idea to allow the research to reveal data points.
As explained by Wardle in the previous cited study, “Predetermining data points […] entails making a number of faulty assumptions (164)”. Researchers cannot see how a student is reacting to material on a mental, emotional or social level and predetermining data points regarding problems of transferring writing knowledge from one context to another becomes problematic.
It doesn’t mean that researchers shouldn’t have a goal in mind, but it doesn’t help to make things to rigid and to collect only what the researchers assumes matters. In Wardle’s study it was discovered that a lot of difficulty regarding Clement’s consequential transitions were related to losing a particular social resource. Clement was used to support from her family, and due to the socially controversial material of the class she was involved in she lost a key facet of her support when it came to her writing which made the process more difficult.
This really leads to my final point:
3. Collecting classroom data during an observation involves understanding the emotions and thinking of the students
Understanding the way that students learn isn’t as easy as analyzing a text and determining how many grammar mistakes are present, or how many instances of misunderstanding how to communicate properly in an assignment there are points of data that cannot be tracked only by collecting handouts or completed assignments.
Researchers, depending on the task, need to be aware of how to collect data that is pertinent to their question. Some of that data may take a shape that is not easily collected and might require interviewing participants and otherwise involve them in the study. To understand learning we need to understand how learners feel about their own learning ability and what is taking pace when they are learning. What struggles they might be having internally while attempting to change their writing process may be unpredictable without the insight from those individuals.
Hayes,Hogan. Ferris, Dana R., and Whithaus, Carl. “Dynamic Transfer in First-Year Writing and ‘Writing in the Disciplines’ Settings.” Critical Transitions Ed. Anson, Chris. Moore, Jessie. Wac Clearing House and University Press of Colorado.
Wardle, Elizabeth. Clemet, Nicolette Mercer. “Double Blinds and the Consequential Transitions: Considering Matters of Identity During Moments of Rhetorical Challenge” Critical Transitions Ed. Anson, Chris. Moore, Jessie. Wac Clearing House and University Press of Colorado.