skip links
Chabot College Logo
envelope iconStudent Email | College Index Search Bar Left Corner
Chabot College Logo
 

Menu Gradient Bottom

Center for Teaching and Learning

Focused Inquiry Groups (FIGs) - Title III

Jumpstart - Assessment of Student's Reading Practices In Chabot's Developmental English Classes

Analysis of MARSI results
Jumpstart Program
Dennis Chowenhill
February 2010

GLOBAL READING STRATEGIES

Q.14: I decide what to read closely and what to ignore.
Q.22: I use typographical aids like boldface and italics to identify key information.


The responses to these two questions indicated the greatest increase in practice, of all the items in the MARSI survey, more than a full point of gain. I cannot account well for the causes of the change. Q.14 touches on a topic that was never discussed directly in class. We discussed and practiced finding key ideas and understanding how everything else in expository writing relates to them, by providing rewordings, examples, illustrations, researched evidence, and supporting and contradictory arguments. We also frequently discussed digressions, as Lauren Slater in Opening Skinner’s Box uses a lot of anecdote and personal experience that students often regarded as more digressive than supportive of her arguments. As students learned how Slater introduced chapters in her book, with descriptive narratives, they became rather casual about reading these passages and getting to her main idea. This could account for some of the difference in the responses to Q.14. The only times that we discussed the use of special graphics (Q.22) were when we looked at how textbooks are arranged, but this was a very small unit of the reading instruction.


Q.1: I have a purpose in mind when I read.
Q.23: I critically analyze and evaluate the information presented in the text.


The differences in the pre- and post-MARSI scores for these two questions were greater than +.73. This might reflect work that was done during class time, which included much direct instruction, modeling, and practice in both areas. Homework assignments routinely emphasized the importance of reading for a purpose, as each reading assignment specified something for the students to accomplish while reading: “Find at least two ways in which Rodriguez’s chapter on ‘Complexion’ reflects anxieties that he has expressed in previous chapters.” Several of these assignments required students to respond on Blackboard, where they would also see the responses of other students. By the middle of the term, as I was in midst of assigning a reading for homework, students would interrupt by asking, “What is it we are supposed to look for this time?” Q.23 states succinctly the purpose of most of the in-class work that we did with the texts we read, which was a major component of the class, taking up at least half of our class time.


Q.3: I think about what I know to help me understand what I read.
Q.7: I think about whether the content of the text fits my reading purpose.
Q.29: I check to see if my guesses about the text are right or wrong.


The scores in Q.3 and Q.7 were +.579 and +.578; Q. 29 was +.527. The topic of Q.7 is discussed above, in my comments about Q.1, regarding reading with purpose. Its higher score could be a natural consequence of the sort of work that students did that required them to focus on their purposes as they read. In their homework reading assignments, they learned to read more quickly the passages that did not address the questions that they were pursuing. I cannot match specific things done in class or as homework that address the concern of Q.3, aside from discussions that I conducted before students began a new chapter in either of their texts, when we would review what we know about the issues treated in the reading. These discussions were among the most casual in the class, however, and I did not speak directly about the importance of connecting one’s reading to what one knows. The students were inclined to relate most of the topics they were reading about to personal experiences of their own, and I chose to join them in this interest of theirs, acknowledging it but not co-opting it with direct reading instruction. More important to these discussions was the testing of the relevance of personal experience in the context of the texts we were reading. One of the skills that the students needed to develop was opening up their expectations of a text rather than limiting it to the extent that it matched what they already believed. This relates to Q. 4 (discussed below), about previewing texts, and Q.26 about guessing. Students seem to have learned in the schools to preview texts and relate their personal experiences to them, but often to a disadvantage, as students habitually could not get beyond their original expectations as they read texts. My instructional goal to address this was to get students to check for the accuracy of their predictions as they read, which was not an easy goal to attain. The difficulty of developing this reading practice might account for students still rating themselves slightly higher for the various types of guessing that they do (Q.26 = .336 at the end of the term) than checking their guesses (29 = .326 at the end of the term). There were many class discussions about checking their various guesses and predictions—both about the habits they had acquired regarding predicting what is in the texts they previewing and relating to their lives, and about how to check to see whether those first impressions were accurate, or if the texts might actually be saying something else. I am pleased that the increase for Q.29 (checking one’s guesses) is higher (+.527) than the increases for Q.4 and Q. 26. Previewing and guessing are not reliable reading strategies if students have not developed the habit of testing out their expectations (though they make students into active readers, which is positive attribute for readers).


Q.10: I skim the text first by noting characteristics like length and organization.
Q.19: I use context clues to help me better understand what I’m reading.
Q.26: I try to guess what the material is about when I read.


Q. 26 is discussed in the paragraph above. The difference in MARSI scores is +.368, which could be a consequence of class activities and reading assignments, and if class instruction was effective in this area, it makes sense that there would be a larger difference pre- and post MARSI scores for Q.29 (about checking one’s guesses), as there is.

Both Q.10 and Q.19 are within an increase of +.4 in the pre- and post- MARSI surveys. Class discussion often addressed organization, both to improve one’s understanding of how the parts of a text relate to each other, and to examine rhetorical strategies of writers. Skimming was also addressed as a method for finding specific information (e.g., while conducting research that requires one to examine many texts). It was not addressed, however, as a general pre-reading exercise. Context clues were required for students to answer many of the questions on quizzes. Typically, a question would ask what a specific sentence in a text meant, when finding the answer meant reading the text immediately before and after it.


Q.4: I preview the text to see what it’s about before reading it.
Q.17: I use tables, figures, and pictures in text to increase my understanding.


The differences in scores between the first and second administering of the MARSI were negligible, with Q.17 virtually unchanged, and Q.4 at -.05, and the score averages are in the range of low 3s. I address Q.4 in the paragraph preceding this. Regarding Q.17, none of the texts used in this English 102 uses graphic illustrations of any sort. There were two brief class discussion of the layout of formal textbooks, in which we noted the use of tables, pictures, and graphs. There were no exercises related to these discussions.


Q.25: I check my understanding when I come across conflicting information.

The pre-and post-MARSI responses to this question indicated the greatest drop in score, -.144. This is admittedly discouraging. Both texts assigned, Lauren Slater’s Opening Skinner’s Box and Richard Rodriguez’s Hunger of Memory contain conflicting information that was examined repeatedly in class discussions. Slater’s book presents arguments among psychologists who make conflicting claims, and Rodriguez’s book contains well-known conflicts in his own thinking. Both texts were selected partly because they provided opportunities for checking conflicting information. The Rodriguez text was especially pertinent to Q25, as the conflicts were not presented as conflicts in the book. Students discovered passages where Rodriguez apparently contradicted himself and had to check to see whether they had understood wrong. This was an exercise that I imposed on the students; when they discovered conflicts in Rodriguez’s thinking, they were not (at least during class discussions) inclined to doubt their interpretations. I would challenge their assertions of the contradictions and have them convince me that they were actually contradictions. These proved to be animated discussions, and productive. But I was not convinced at the end of the term that the students had developed the habit of checking their understanding of contradicting information. I am a little suspicious of the accuracy of students claiming that they do this at the 3.722 (beginning of term) or 3.578 (end of term) levels.


PROBLEM SOLVING STRATEGIES

The most salient feature of the pre- and post- MARSI scores in this category is the near evenness of how the scores increased slightly. Expressed as line graphs, the lines of the two surveys are close to parallel. The scores for Q.11: I try to get back on track when I lose concentration and Q.21: I try to picture or visualize information to help remember what I read have not changed significantly. The students try visualizing what they read (Q.21) more often than not (3.894), and this has not changed. Instruction in this class recognized the practice of visualizing, but did not emphasize it more than other reading strategies. Class discussions caused all of us to visualize information that we read, but I did not offer this as a reading strategy for students to apply independently. The higher score for Q.11 (4.473) is encouraging, indicating that students make an effort to maintain their concentration as they read, and this high number is likely a product of the students doing most of their reading to satisfy class assignments. Few of the students in this class read much outside of class assignments (as indicated through a reading practice survey that I administered at the beginning of the term), and the reading that they do habitually is relatively casual, demanding little of them as readers. Most of the students, thus, associate reading with being a student.


Q.13: I adjust my reading speed according to what I’m reading.
Q.30: I try to guess the meaning of unknown words or phrases.


These two items indicate the greatest increase within the Problem Solving Strategies category, both above +.4 increase. Q.30 (+.474) is the greatest increase of the category, which makes sense given the students’ other reading habits, and the instruction that they received. The students indicated during class discussions that they “read over” unfamiliar words, and this is consistent with their inclination to guess at general meanings and predict ahead what a text is about (without checking the accuracy of their predictions). Attention to the word was low generally among the students, and the first MARSI score for this item, 3.368, seems high to me. When students read passages aloud in class and came upon words that they did not know (and often could not pronounce), they commonly said something like, “Yeah, I don’t know that word.” When I asked if they had noticed the word when they read the assignment as homework, they admitted that they had seen the word and knew that they didn’t know what it meant. They had made no attempts to figure out what the word means. As they explained in class discussions, they would “skip” the word and go for the general meaning of the passage. When they could not understand the general meaning of the passage, they did not try to figure out the meanings of unknown words as a strategy for grasping the meaning of the text. Thus, the students were fairly active in getting meaning from their reading, but not at the word level. This is a characteristic of oral language use, where individuals do not inquire about the meanings of unknown words when they encounter them in conversations, but just stick with the conversation to get as much as they can from it. The attention that we paid to understanding individual words in text might account for the increase in the pre- and post-MARSI scores.

I do not know what accounts for the increase in scores for Q.13, though it might be expected after students have spent seventeen weeks completing homework assignments that gave them specific tasks to accomplish. In such a reading environment, students remain focused on their goals and learn to adjust their reading speed and energy, looking for the most relevant information on the page. (This can also have the effect of causing students to look up words (Q.30), as the tasks I assigned often required the students to understand the vocabulary used in the texts they were reading.)


Q.16: When text becomes difficult, I pay closer attention to what I am reading.

The increase in the pre- and post MARSI survey scores for this item (+.369) is not great, and the students rated themselves high for the category, above 4. Since students of this class do not do a lot of reading outside of their academic work, their high rating of themselves in this category is a surprise to me. Their behavior during the term indicated less conscientious reading habits than this. When texts became difficult, most of the students, as indicated through self reporting in class discussions, habitually put the text down and take a break, or pass up that section of the reading, hoping that the same idea will be covered later and be clearer to them then. This behavior will not enable them to succeed at homework completion or quiz taking, which might account for the score increasing as much as it did.


Q.8: I read slowly but carefully to be sure that I understand what I am reading.
Q.18: I stop from time to time and think about what I am reading.


The pre- and post- MARSI scores on these items increased only slightly, at +.26. The students rated themselves high in Q. 8, within the range of 4, and gave themselves high 3 scores for Q. 18. These scores are consistent with how students rated themselves in related categories: students rated themselves on Q. 16 (paying closer attention when the text is difficult) at 4; on Q. 13 (adjusting their reading speed) at 3; and on Q. 25 (the “global reading strategy” of checking one’s understanding) at 3.5. This internal consistency is encouraging, indicating that the scores represent the students’ reading habits, or at least their impressions of them, fairly accurately. My assessment, however, from observing the students reading, and from examining their skills by means of quizzes and tests and class discussions, is that they are less consistent about these reading skills than they think. But I also suspect that the questions are difficult to answer well. The terms of the questions, “slowly but carefully,” and “from time to time” are not precise—each indicates some measure of time, without expressing any unit for measuring it (minutes? seconds? fractions of seconds?), and any reader who does most of his reading to satisfy classroom assignments is going to have the impression that the task requires thought and reflection. My sense is that the students in my class acknowledge the importance of these reading practices and attempt them unevenly, mainly when pressured to do so. But their reading performances for this class indicated that they should be reading more “carefully,” “thinking about” what they are reading, “checking” their understanding, and paying “closer attention” than they are doing.


Q.27: When text becomes difficult, I reread to increase my understanding.

The pre- and post- MARSI scores on this question dipped, -.263, though the students rated themselves high in this category, between 4 and 4.5. This seems inconsistent with the results discussed in the paragraph above, regarding how self-conscious and reflective students are as they read. A positive interpretation is that students are finding that other reading strategies are bringing them enough success at achieving their reading goals that they do not feel compelled to reread. A less positive interpretation is that the class made them less inclined to go back over texts that they know they have not understood. In any case, students continue to rate themselves high as readers who reread difficult passages, and this is another skill that can be expected of students who do most of their reading for class work. They are, in other words, in terms of their reading, perceiving themselves as students facing tasks as they read.


SUPPORT READING STRATEGIES

Q.6: I summarize what I read to reflect on important information in the text.
Q.9: I discuss what I read with others to check my understanding.
Q.20: I paraphrase (restate ideas in my own words) to better understand what I read.


All of the “Support Reading Strategies” identified in the MARSI are strategies discussed directly in my class and practiced routinely throughout the semester. This might account for there being no drop in the students’ ratings of any of the items in this category. And three of the increases are greater than .5. It is also important to note, however, that in this general category, students rate themselves mainly in the mid-2s and 3s, which is low, with the exception of Q.20.

The largest increase in this category occurs in Q.6, with one full point of gain, from a 2.7 to a 3.7. This might have occurred because the language of this item is the most broad: “summary,” in the mind of a student at this level, can be associated with a variety of activities, including some of the activities specified in this category (“take notes” “discuss,” “paraphrase,” “find relationships”). Since it is a sort of umbrella category, one could expect a high rating. It is a positive indication that students increased their identification with this activity, though I would be more pleased if students rated themselves higher than 3.7 at the end of the eighteen weeks of instruction that they received. The averages of the MARSI scores, however, may be misleading. The combined averages for the pre- and post-MARSIs in this item have a range of 4 (which is true for most of the items in the MARSI), and for this item, more than a third of the students—in the combined calculations—rated themselves at 4 or 5.

The increase in the scores for Q.9 is also high (+.895), though it is not clear what these student responses mean. If the students are referring to their reading behavior during this particular semester, which is not the intent of the survey, it stands to reason that they would rate themselves much higher at the end of the term. Most of the reading that the students did during the eighteen weeks of this term was in my class, and we discussed all of it. Given the communities that these students live in, it is not likely that they discuss their reading with people outside of class, especially “to check . . . understanding,” so it might be misleading for them to assert at the end of the term, that, habitually as readers “I discuss what I read with others to check my understanding.”

Q.20, regarding the habitual use of paraphrase, had an increase of .632, with an average rating of just under 4 (3.94), which is likely a product of the work that students did in my class. Paraphrase was a common class activity, and occurred as a required skill in nearly all the quizzes and examinations. The average rating at the end of the term for this item (3.94) was the highest in its category.


Q.2: I take notes while reading to help me understand what I read.
Q.15: I use reference materials such as dictionaries to help me understand what I read.
Q.28: I ask myself questions I like to have answered in the text.


The increase for all these items was in the range of +.3, and all the scores are rather low. Q.2 remains at the end of the term below a rating of 3, and Q. 15 and Q.28 are both low 3. Q.2. and Q.15 refer to physical activities—the use of dictionaries and other reference materials, and taking notes. In my class, these were among a variety of options for being an active reader, and were not emphasized as necessarily being the best among them. It is fair to assume, however, that responsible students (as opposed to non-student readers) are using reference materials and also taking notes from their reading, so these low scores indicate to me a recommendation for my instruction: students in all the classes that I teach should be encouraged in these activities.

Q.28 echoes the issue in other MARSI items, regarding the global goal of reading with focus and goals. Most of the students who rated themselves at either 4 or 5 in Q.1 (I have a purpose in mind when I read), for instance, rated themselves in the same range for Q.28. Still, the overall average at the end of the term for Q.1 was 4.1, with an increase of +.737, while the overall average for Q.28 was 3.1, with an increase of .316, indicating that most of the students are not associating questioning with having a purpose, or have found other means of maintaining their sense of purpose as they read. The scores generally among this category of support reading strategies indicate that the more mechanical, physical means of maintaining purpose are not so common.


Q.5: When a text becomes difficult, I read aloud to help me understand what I read.
Q.12: I underline or circle information in the text to help me remember it.
Q.24: I go back and forth in the text to find relationships among ideas in it.


The increase in all these items was in the range of .2, which could reflect the peripheral nature in my classroom of the skills identified. Reading aloud and checking back and forth in a text are skills that were discussed and practiced routinely in class, but they were not recommended above other strategies. They were just always present. This could account for there being an increase in the ratings of both of these, and the increase being small. It is noteworthy that Q.5, regarding reading aloud, ended with an average rating of above mid 3, one of the high scorers in the category of support strategies. It is very likely that this comes from the daily classroom routine of reading aloud, which students occasionally observed was helping them to understand. I occasionally heard comments like “At home, I don’t get it, but when we start reading it in class it is easier.” It would be necessary to ask follow-up questions to see whether students have developed the habit of reading aloud when they are not in classrooms.


COMMENTS

Whether or not the students actually improved their practices in the areas for which they had increased scores at the end of the term, the students would be conscious of the importance of most of these practices, and aware that they had been applying the practices at least while in class. I suspect that this accounts for several of the increased numbers in the second MARSI. About a third of the students clearly demonstrated improved reading (assessed by quizzed, exams, and class discussions) by the end of the term; the others demonstrated either slight or no improvements. But all the students’ awareness of the skills addressed by the MARSI would be increased, and I suspect that some of their responses to the MARSI questions came from this increased awareness, more than increased practice as readers. This is of course a positive outcome, especially if the students remain in college. Whether or not a student now is actually reading aloud when encountering difficult passages, or taking notes when he is going to be held accountable for what he has read, being more conscious—and perhaps convinced—that these activities are useful is going to influence his behavior as a reader.

Generally, I distrust self-reporting of behavior. Research that I have conducted previously has demonstrated to me that people are often not keenly aware of their own routine practices, and tend to skew responses in surveys toward what they regard as “normal” or “learned” behavior. Students who have heard it repeated that one should “read with a purpose” are inclined to imagine that they do, whether or not they are that disciplined and self-controlled when they read. I know this more dramatically as a researcher in linguistics, where it is axiomatic that one cannot find out how an individual pronounces a word by showing the word to the informant on a card. Most informants will say the word the way that they imagine is “correct.” To hear how a subject actually pronounces a word, the researcher has to generate a conversation in which the word is likely to be used. My observations of my students as readers in the classroom indicate that they do not practice the skills identified in the MARSI as well as their personal ratings indicate. However, there is a clear and accurate pattern among the student by their skill level: the students with the lowest reading skills (as I assessed them over the eighteen weeks) rated themselves generally lower than the students did who have the highest reading skills. The general pattern of the data, in other words, seems fairly reflective of the students’ behaviors, regardless of how accurate the individual ratings are.

Another factor that likely skewed the MARSI results is that after the first MARSI, I discussed the skills identified in it, and the MARSI itself. The students’ consciousness of the reading skills identified in the MARSI, and their increased appreciation of the value of those reading habits could have two very different outcomes. For some of the students, the discussion of the MARSI could cause them to rate themselves more accurately the second time around, as they had come to understood better what the items referred to. On the other hand, my promotion of those specific reading habits, and the students’ applying them in class activities, could inflate their sense of how much they have actually adopted those practices independently of my classroom.

CONCLUSIONS

Were I to continue teaching the basic skills and college level English core curriculum, I would be influenced by what I have observed through examining these MARSI results. I would observe my students more carefully in terms of some of the discrete items on the MARSI, and I would have them assess many of these reading practices during the term, for the reading they do in my class as well as for the reading that they do in other environments (including other classes, and online reading).

Chowenhill's MARSI Workbook

 
Bookstore Icon Library Icon
Footer Left Corner