Center for Teaching and Learning

Focused Inquiry Groups (FIGs) - Title III

Jumpstart - Assessment of Student's Reading Practices In Chabot's Developmental English Classes

Analysis of MARSI results
Jumpstart Program
Alisa Klevens
Spring 2010

MARSI – Pre and Post

In all three categories – Global, Problem-Solving, and Support – students showed improvement.

Average increase <.44

At the beginning of the semester, students self-assessed in the “high” range in only the problem-solving category. By the end of the semester, they assessed in the “high” range in all 3 areas of reading. The largest gains were in the “Support” strategies category, with a gain of .77. This gain is significant in that the “support” category proves to be the lowest at the start of the semester. This trend has been observed across sections and across disciplines, when the MARSI was administered in sections participating in the Reading Apprenticeship Faculty Inquiry Group. Students indicate less frequent use of these support strategies and, it’s important to note, these are the strategies requiring “extra effort” on the part of the student. Therefore, such gains in student use in support strategies is quite an accomplishment.

For each strategy type, the items for which students indicated most gains, these items correlate to the strategies emphasized in class.

Items of greatest increase in each category:
<.80 Global: Question #23 “I critically analyze and evaluate the information presented in the text”
<.74 Global: Question #3 “I think about what I know to help me understand what I read”
<.70 Global: Question #7 “I think about whether the content of the text fits my reading purpose”
<.73 Global: Question #1 “I have purpose in mind when I read”
<.46 Problem: Question #18 “I stop from time to time and think about what I’m reading”
<1.40 Support: Question #12 “I underline and circle information in the text to help me remember it”
<1.07 Support: Question #2 “I take notes while reading to help me understand what I read”
<.93 Support: Question #6 “I summarize what I read to reflect on important information in the text”


Items of least gain OR reflecting a lower number than the 1st assessment – “Red-flag” items
<.27 Global Question #14 “I decide what to read closely and what to ignore”
<.34 Global Question #25 “I check my understanding when I come across conflicting information”
>.20 Problem Question #8 “I read slowly but carefully to be sure I understand what I’m reading”
<.14 Problem Question #11 “I try to get back on track when I lose concentration”
>.07 Problem Question #27 “When text becomes difficult, I reread to increase my understanding”
<.60 Support Question #9 “I discuss what I read with others to check my understanding”
<.47 Support Question #24 “I go back and forth in the text to find relationships among ideas in it.

Item # 14 was particularly important for our 2nd class text, which was filled with long explications that were demonstrating an abstract point, but could be identified as an example and skipped over. Students, at first, were getting too caught up in the detail of these explications and we practiced taking quick notes of what’s being described without taking the time to analyze the text.

Likewise, item #24 is something we practiced throughout the semester not only within texts, but between texts. The 3 texts were chosen, in part, in how they respond to each other. In fact, the last 2 texts reference similar data on IQ tests and students were prompted to connect these ideas.

The decrease in the “problem-solving” items needs to be understood within the context that students still identify their usage as “high” for each of these items. However, it’s disconcerting that the items which respond to students reading on their own (often the 1st read) are, by in large, not showing gains. This may be a “truer” response at the end of the semester. It certainly responds to my observations of students’ middling efforts in tackling the texts on their own, outside of class time.


Reading Survey – comparing Jumpstart 102 to English 101B
Throughout the semester, I was evaluating student engagement with their texts and their performance on assignments – reading quizzes, homework completion, and essays. For the 102 class, I was particularly concerned about the inconsistency in student performance. Despite efforts to encourage student engagement with their texts, the 102 students often came to class unprepared, having not completed the course reading assigned for that day. Often, students were conducting their 1st read in class. While students participated in whole class and small group discussions on the text, the end result of coming to class unprepared required taking additional class time for students to skim the book. Likewise, those who were unprepared could participate in the discussion, but their understanding of the text proved to be very superficial, resulting in lower performances on written indicators. As an instructor, one of my major concerns was that the books chosen were just too hard for the students to engage in on their own. However, when I performed a CERA assessment, students, in fact, were able to piece apart the most relevant information, even on a 1st read. Nevertheless, at the end of the course, I wanted to see how students ranked our texts and how they ranked their engagement with these texts outside of class time. I then compared these results to my 101B class, to see the differences in two classes, both set to advance into English 1A.

Background on the texts chosen. For each class, students read 3 texts. However, the amount and type of reading differed.

102 – Students read on average a total of 600+ pages over the course of the semester.
101B – Students read on average a total of 900+ pages over the course of the semester.

For the 102, students were assigned fewer pages of text to read for 2 reasons: 1) This is a combined course of 101A/B; therefore, students are having to progress in their skills faster than in the slower two semester sequence. Students are not expected to have had a semester of skill-building under their belts. 2) These texts were by in-large expository and not narrative based. Students tend to have a harder time understanding expository writing if it is not embedded in some type of story-line.

Rationale for Text Sequencing for 102 and 101B
102 – The texts moved from Surviving Justice, to Everything Bad is Good for You, to Outliers. The 1st book was mostly oral histories that followed individual narratives. However, for each assigned reading, students also read expositional footnotes related to the justice system. The 1st book I deemed to be the “easiest” of the three texts. I began with this text because of its exciting topic and because students could practice new reading strategies with an approachable/readable piece of writing. The 2nd book was entirely exposition. There was no narrative for students to follow, only argumentation. Likewise, students were challenged with quite a bit of difficult/technical vocabulary and having to navigate long explications, charts, and graphs. The book I deemed to be the “most difficult” of the three to read, but with the most relatable subject matter (i.e. popular culture). The 3rd book was a mixture of narrative and exposition. This book required students to put together both types of reading – following story and following argument. This book was more “readable” than the 2nd, but more difficult in its abstract subject matter (i.e. success). I put this book 3rd because the concepts were more sophisticated and required more critical analysis.

101B – The texts moved from A Long Way Gone, to Enrique’s Journey, to There are No Children Here. In my estimation, these texts were moving from “easiest” to “hardest.” The texts’ language grew increasingly sophisticated, the arguments became more complex, the need for background knowledge in decoding the texts was increased over time, and the arguments moving from a direct analysis of the narrative at hand to a more generalized argument extending beyond the particular character’s story, required students to engage in a more sophisticated and critical analysis.

Reading Survey (these are just 2 of the questions)

102 – 14 Responses

“From your experience this semester, the readings we covered were: easy, challenging, difficult, too difficult” (– apply a category to each book)

“Outside of class time, I read...”

102 Analysis of Survey
Students found the 2nd text the most difficult of the 3, with 8 rankings as “difficult or too difficult.” The third text was also given 8 rankings for “challenging,” therefore somewhat easier than the 2nd text. This confirms my own assessment of these tests in their order of difficulty. Likewise, it also indicates that very few students found the texts “too difficult,” and therefore, out of their range of competency. A question that I had was if I should switch the order of books two and three so that the texts progress in a more linear path of difficulty. However, they may have found the 3rd text slightly easier because they had book 2 as practice and; therefore, switching the order could produce the same result (book 2 seen as being more difficult).

The majority of students read outside of class time “half the time,” with a total of 13 responses, followed by “most of the time” with 12 responses. The “easiest” text was read independently the most with 11 responses for “most of the time or always,” as compared with 3 or 5 responses for the other two texts. Yet, for each book, between 2 and 4 students indicated having read independently “Not at all” or “Some of the time.” Therefore, in spite of ease or difficulty, students chose not to read on their own. Even with the extrinsic motivator of daily reading quizzes, the majority of the students chose to receive No Credit some of the time for having not completed the assigned homework reading.

101B – 17 Responses

From your experience this semester, the readings we covered were: easy, challenging, difficult, too difficult” (– apply a category to each book)

“Outside of class time, I read...”

101B Analysis of Survey
The majority of students ranked the order of difficulty as progressing from easy 10, to challenging 11, to difficult 8. This matches my assessment of these texts as increasing in difficulty over the course of the semester. Only the 3rd book was ranked as being “too difficult” and only by 1 student.

The vast majority of students read their texts outside of class “always” with a total response of 21. The 2nd highest ranking was “most of the time,” with 15 responses. For each book, 0 students reported reading the book independently “not at all.” I gave only a few reading quizzes in this class because it was clear that the students were coming to class prepared.

Comparative Analysis 102 and 101B Reading Surveys
By in large, both classes ranked their books “Challenging” – 19 times in 102 and 21 times in 101B. The other categories of ease or difficulty were indicated fewer times. This suggests these books were perceived to be, on average, within the students’ range of ability. Yet, while these figures equate nicely in both classes, the degree to which students are reading independently vastly differs on the spectrum. More students in 101B are more frequently reading on their own “always” and more students in 102 are reading on their own only “half the time.” Why? When the books are seemingly of comparable difficulty does one class read more often and more pages than the other? Despite the fact that they have approximately 300+ pages of more reading to complete, the 101B students are doing their work more regularly than the 102 students. Thus, quantity of reading is NOT a deterrent to independent reading. Perhaps reading content?

When asked “Did you like the three texts we read this semester? Be specific if you like some more than others, or if there was one you hated etc...and why” students responded:

102 (14 Responses)

101B (17 Responses)

Analysis: For both classes, the majority of the students “Liked” the books. Likewise, the middle books in both classes had more “neutral” responses than either the 1st or 3rd books. These “neutral” books were read “Always” less frequently, with the majority of responses being read “half the time” for 102 and “most of the time” for 101B. Therefore, a “neutral” interest does negatively impact independent reading, but more so for 102 students than 101B students. Why? Perhaps one explanation could be that 101B students are acclimated to college more and have a greater tolerance for a course progression over 18 weeks. They perhaps have a greater “stick-to-it-ness” than the 102 group, especially this “late start” cohort.

CLASS DATA COMPARISONS

Overall, I was very disappointed with my “success” results of my 102 Jumpstart cohort. My “success rate” was very low; although, a few additional students should have received “W’s” but didn’t. These were students who stopped attending after the “W” deadline. At the end of the semester, only 14 students were actively attending and of them, 8 received credit. That would change my success rate to 57%, but only retaining 14 students is pretty poor. I haven’t had such poor retention in years, and that was in an English 101A class. Whereas in the 101B, I still had 21 students actively attending, which would make the success rate 66%, above the general average of basic skills English courses. In conducting research into my 102, I was trying to explain the differences in student achievement and pinpoint any roadblocks that were impeding student progress. One of the persistent questions was whether the reading was too difficult. Yet, this analysis of reading ability indicates that students in the 102 are generally feeling successful when they read, and to a greater extent at the end of the course. Likewise, they indicate more competence in supporting themselves as readers. In addition, the general attitude of my 102 cohort was positive. Some students who knew they were not going pass the class continued to attend and participate. Participation was high and class discussions often very lively. They, by in large, enjoyed the class content and our focus on “False Promises.” Likewise, I enjoyed these students tremendously. Yet, good will on both ends was not enough to produce positive “success” results.

Success, in this instance, could possibly be measured in other ways. Students have progressed in their reading skills and it remains to be seen how these students persist and succeed at the college at large. Yet, the premise of funneling students into their basic skills English early and, in particular, into an accelerated curriculum, in order to maximize their likelihood of succeeding, did NOT manifest in my group. This group of “late deciders” often felt like an English 101A in that they seemed ungrounded and unmotivated to do college-level work, if it required time spent outside of class to get it done. It was not, in fact, a lack of skill that impeded these students. Most of these students could do the work, but chose not to and in order to succeed on the assignments, the work could not be done with middling effort. Likewise, as the curriculum increased in complexity, the students who were merely “getting by” in the 1st unit, were now drowning in the 2nd unit, and so on. In their self-reflections, students commented that “falling behind” was making it impossible to “catch up.” Despite countless lectures from me on the importance of doing the work and its direct correlation to success on papers, and despite opportunities for continual feedback on written homework assignments, and my reading and conferencing on rough drafts of essays, too many of these 102 students failed to “show up.” They continued to come to class, but not take the class. Nevertheless, I don’t think non-success was wasted effort for these students. For many of the non-success students, not taking English 102 again in this next semester could be the best thing for them. Some of these students need to find out why they’re in college and cultivate an intrinsic motivation to come to class and do the work. As an instructor, I also need to work on fostering that intrinsic motivation in students because it’s clear that extrinsic motivators will only fall short.

Klevens's MARSI Workbook

Case Studies:

• Weak Student
• Medium Student
• Strong Student