INTRODUCTIONIn the last fifteen
years two trends have gained prominence throughout higher education:
assessment and accountability. For various historical reasons, and
the source of considerable confusion, both are erroneously referred
to as "assessment." The first, "assessment for excellence," is an
information feedback process to guide individual students, faculty
members, programs, and schools in improving their effectiveness.
Assessment instruments are designed to answer a wide range of
self-evaluative questions related to one larger question: how well
are we accomplishing our mission?
trend, "assessment for accountability," is essentially a regulatory
process, designed to assure institutional conformity to specified
norms. Accountability advocates, including especially state
legisla-tures, to a considerable extent view colleges as factories
and higher education as a production process (Astin, 1993, p.17),
although there is widespread disagreement about what exactly they
are supposed to produce, and about how to measure it (Ewell, 1997).
Nevertheless, various performance measures, which attempt to measure
institutional effectiveness, particularly with regard to fiscal
efficiency and resource productivity, have been created and applied
to public universities and colleges throughout the country.
terms "assessment" and "accountability" are often used
interchangeably, they have important differences. In general, when
we assess our own performance, it's assessment; when others assess
our performance, it's accountability. That is, assessment is a set
of initiatives we take to monitor the results of our actions and
improve ourselves; accountability is a set of initiatives others
take to monitor the results of our actions, and to penalize or
reward us based on the outcomes. They have very different flavors.
Although assessment efforts over the past dozen years have been
largely focused on aggregate statistics for entire schools,
accreditation review boards recently have been increasing pressure
on institutions to actively engage departments and students in the
assessment-learning- change cycle (Gentemann, 1994). If learning is
our business, how well are we doing at all levels (assessment), and
how can we demonstrate that to others (accountability)?
increasing focus on assessment and accountability has powered a
shift away from prestige-based concepts of institutional excellence,
in which size of endowments, accomplishments or credentials of
faculty, or types of programs, for example, were as-sumed to be
indicators of institutional quality or effectiveness, and also away
from curriculum-based models that emphasize what is presented,
toward learning-based models which emphasize what students know and
can actually do. The emerging measure of institutional excellence is
how well institutions develop student talents and abilities, i.e.,
student learning outcomes (Astin, 1985, 1993, 1998). The purpose of
this paper is to provide an introduction to some of the
relationships among assessment, accountability, and student
learning, and to inform a discussion of these issues in the Western
community. Section I describes in more detail the relationships
between assessment and accountability; Section II discusses some of
the current thinking about student learning and how to improve it;
Section III discusses the role of assessment in improving student
learning, and provides some examples; Section IV suggests possible
directions here at Western for shifting institutional focus to
student learning, and offers two recommendations.
ACCOUNTABILITY AND ASSESSMENT
FOR EXCELLENCE"Assessment is not an
end in itself but a vehicle for educational improvement." (AAHE,
1992). As shown in Fig-ure 1, given the attributes of entering
students, measurement of an array of student outcomes provides
feedback about how well individual courses, programs, and the
university as a whole are accomplishing their stated missions and
goals. Assessment aims at the continuing improvement of student
development, and is generally consistent with a "value-added"
concept of education; note that the rationale for having better
programs is to ensure better student outcomes.
As shown in
Figure 1, the collection of assessment information is only the first
step in a four-part process. To be useful, it must be analyzed and
reflected upon by appropriate decision makers, and then used to
design and apply changes. In each iterative cycle, modified programs
are then reassessed and readjusted, continually improving
Even at the
departmental level, new guidelines for program reviews are shifting
the focus away from a preoccupation with departmental assets or
curricular structure and more toward "how resources are used, the
consequences of these uses, and the way in which students actually
experience the major" (Gentemann, 1994).
recent accreditation review indicated that the Office of
Institutional Assessment and Testing (OIAT) must do more to deliver
useful assessment information to academic units, and individual
academic units must do more to integrate assessment practices into
their programs. The bold arrows in Figure 1 show the current flow of
assessment data at Western; the dotted lines show the parts of the
feedback loop which need further development. These are discussed in
more detail in Section III.
FOR ACCOUNTABILITYAccountability measures
are an attempt to assert more direct public control over higher
education, as shown in Figure 2. They are primarily concerned with
resource allocation and fiscal efficiency. While it is completely
appropriate for those who pay the bills--taxpayers, parents, and
students--to evaluate critically what they get for their money from
public education, performance measures as they are currently defined
in Washington State remain problematical, for at least two reasons.
they are measured on arbitrary scales, their meanings are ambiguous.
Second, the measures themselves direct institutional goals to some
extent, rather than the other way around. Resulting University
policy is driven to achieve specific measurement targets, and these
may be at odds with the University's larger mission and goals,
including the enhancement of student learning.
performance measures which illustrate this point are fall to fall
retention of students and the graduation efficiency index; both are
commonly regarded as measures to be maximized. The rationale is that
for the sake of fiscal efficiency, a student should enter school,
stay enrolled, take only the courses necessary to graduate, and then
leave as soon as possible to make room for another student. This
kind of thinking assumes a factory model of education, in which the
measure of output is degree attainment, and the measure of cost is
time to degree.
Such a view
penalizes institutions for various kinds of normal student behavior
which make the numbers look bad, but which might serve students and
their educations very well--like taking a double major or taking
elective courses irrelevant to the major. Incentives are created for
institutions to eliminate these students, to narrow their
educational options, or to encourage them to go elsewhere for their
educations, all questionable goals from the standpoint of student
derives its legitimacy from the quality of its measurements; and
those being measured generally best know the area being assessed.
University mission statements ought to be the place to find out what
is important, and therefore what should be measured. Since student
learning figures prominently in most academic mission statements,
student learning outcomes may have special appeal as performance
C. THE ROLE
assessment-accountability waters are further muddied by developments
in academic technology. Manufacturers of computer software and
hardware have been heavily lobbying both legislators and academics
nationwide to substitute electronic and media technologies, such as
web-based distance learning, for more traditional, face-to- face
educational practices (Jacklet, 1998).
there is no reason to suppose that these computer technologies will
necessarily either improve learning or lower costs. Although there
are certainly ways in which such technologies can be applied
effectively to increase either faculty productivity or student
learning, or both (Chickering & Ehrmann, 1998), technology can
never entirely replace the face-to-face interactions among students
and between students and faculty which have shown demonstrated
importance in student development (Astin, 1993, 1998).
such lobbying is a persuasive distraction at all levels. Using
student learning outcomes and faculty productivity as the measures
of effectiveness of educational systems in general and of new
technologies in particular would help to assure that only those
technologies which are both cost-effective and learning- effective
RECONCILING ASSESSMENT AND ACCOUNTABILITYStudent learning
transcends facts and concepts, and includes the values, attitudes,
self-concepts, and world views students evolve in the interactive
intellectual and social environment which colleges foster. Grounding
accountability in student learning, with measures designed by the
units being measured, would provide the most rational basis to
measure university performance.
aims at improving fiscal efficiency, but is blind to issues of
educational quality. Assessment aims at improving the quality of
education, but is necessarily constrained by budgets. A focus on
student learning outcomes can be a bridge that links the two.
useful accountability measures should have two qualities. First,
they must be unambiguous, either monotonically increasing or
decreasing measures of either costs or benefits; i.e., we all agree
whether we want more less of whatever it is they measure. Second,
they must be linked in some way to indicators of quality.
It turns out
that student learning outcomes constitute useful measures of quality
in and of themselves. They are consistent with the stated missions
of higher education; improving them is a valid indicator of improved
institutional performance. Such indicators, when combined with cost
data, could also be used effectively as measures of changes in
institutional fiscal efficiency or overall performance over time.
measures based on student learning outcomes would be unambiguous;
they would tell us whether institutions are providing the same
levels of learning at lower cost, or providing improved levels of
learning for the same cost. Either type of measure satisfies the two
criteria for performance indicators, and gets more directly at the
tension between assessment and accountability: minimizing the cost
of a truly excellent education.
LEARNING OUTCOMESSharpening the focus of
higher education onto student learning outcomes goes beyond mere
tinkering with traditional structures and methods; it really
constitutes a paradigm shift in educational philosophy and practice.
An increasingly accepted view among educational scholars is that
traditional structures are dysfunctional and overdue for change
(Miller, 1998). To remedy this, "students and their learning should
become the focus of everything we do. . .from the instruction that
we provide, to the intellectual climate that we create, to the
policy decisions that we make" (Cross, 1998).
At this point,
it is useful to make some distinctions between "student outcomes"
and "student learning outcomes." Student outcomes generally refer to
aggregate statistics on groups of students, like graduation rates,
retention rates, transfer rates, and employment rates for an
entering class or a graduating class. These "student outcomes" are
actually institutional outcomes; they attempt to measure comparative
institutional performance, not changes in students themselves due to
their college experience. They have generally been associated with
student-outcomes statistics are often "output- only" measures
(Astin, 1993). That is, they are computed without regard to incoming
student differences and without regard to how different students
experienced the college environment. As a result, they do not
distinguish how much an observed measurement is the product of the
institution and its programs on students, and how much is due to
other factors, such as socioeconomic status, general intelligence,
or which high school was attended, for example, and can therefore be
learning outcomes," on the other hand, encompass a wide range of
student attributes and abilities, both cognitive and affective,
which are a measure of how their college experiences have supported
their development as individuals. Cognitive outcomes include
demonstrable acquisition of specific knowledge and skills, as in a
major; what do students know that they didn't know before, and what
can they do that they couldn't do before? Affective outcomes are
also of considerable interest; how has their college experience
impacted students' values, goals, attitudes, self-concepts, world
views, and behaviors? How has it developed their many potentials?
How has it enhanced their value to themselves, their families, and
essentially three threads which must be interwoven into a program
dedicated to the improvement of student learning: shifting
curricular focus to student learning; developing faculty as
effective teachers; and the integration of assessment into
curriculum at several levels. These are discussed in some detail in
the next several sections.
CURRICULAR FOCUSThere are thousands of
articles and hundreds of books on student learning; fortunately,
several scholars have painstakingly sifted through this material and
summarized important conclusions on which the studies are in general
agreement. Perhaps the best known is Chickering and Gamson's "Seven
Principles for Good Practice in Higher Education" (1987). The Seven
Principles provide a useful introduction to the thinking behind a
learning-based approach to higher education, and are listed below
(this annotated version is adapted from Ehrmann & Chickering,
Practice Encourages Contacts Between Students and
Frequent student-faculty contact in and out of class
is a most important factor in student motivation and involvement.
Faculty concern helps students get through rough times and keep on
working. Knowing a few faculty members well enhances students'
intellectual commitment and encourages them to think about their own
values and plans.
Practice Develops Reciprocity and Cooperation Among
Learning is enhanced when it is more like a team
effort than a solo race. Good learning, like good work, is
collaborative and social, not competitive and isolated. Working with
others often increases involvement in learning. Sharing one's ideas
and responding to others improves thinking and deepens
Practice Uses Active Learning Techniques
Learning is not a
spectator sport. Students do not learn much just sitting in classes
listening to teachers, memorizing prepackaged assignments, and
spitting out answers. They must talk about what they are learning,
write reflectively about it, relate it to past experiences, and
apply it to their daily lives. They must make what they learn part
Practice Gives Prompt Feedback
Knowing what you know and
don't know focuses your learning. In getting started, students need
help in assessing their existing knowledge and competence. Then, in
classes, students need frequent opportunities to perform and receive
feedback on their performance. At various points during college, and
at its end, students need chances to reflect on what they have
learned, what they still need to know, and how they might assess
Practice Emphasizes Time on Task
Time plus energy equals
learning. Learning to use one's time well is critical for students
and professionals alike. Allocating realistic amounts of time means
effective learning for students and effective teaching for faculty.
Practice Communicates High Expectations
Expect more and you
will get it. High expectations are important for everyone--for the
poorly prepared, for those unwilling to exert themselves, and for
the bright and well motivated. Expecting students to perform well
becomes a self-fulfilling prophecy.
Practice Respects Diverse Talents and Ways of Learning
roads lead to learning. Different students bring different talents
and styles to college. Brilliant students in a seminar might be all
thumbs in a lab or studio; students rich in hands-on experience may
not do so well with theory. Students need opportunities to show
their talents and learn in ways that work for them. Then they can be
pushed to learn in new ways that do not come so easily.
and unifying theme throughout the Seven Principles is student
involvement--with faculty, with other students, and especially with
their studies. These points resonate with Astin's (1993)
identification of student involvement as a major factor in student
talent development; increased levels of involvement, including high
levels of cultural diversity and community service, are strongly
associated with many measures of success after college. Therefore,
one logical direction for improving student learning outcomes is to
establish policies which encourage and enhance many types of student
involvement, including academic involvement; involvement with
faculty, student peers, and mentors; and involvement in work, both
on and off campus.
principles are also consistent with Marchese (1998), who has
recently reviewed at length the implications of recent developments
in neuroscience, anthropology, cognitive science, and evolutionary
studies for our understanding of human learning. These developments
form the impetus for new pedagogical approaches in higher education;
they demonstrate conclusively that student learning is a complex,
personally unique, and interactive process, and that traditional
approaches have many built-in shortcomings which can be greatly
improved upon. Ewell (1997) has condensed Marchese's discourse into
a summary list of ways to improve student learning, presented here
in edited form:
emphasizing application and experience.
like internships and service learning try to break down artificial
barriers between "academic" and "real-world" practice, while
effective curricular designs foster appropriate knowledge and skills
"just in time" for concrete application in current classwork or
faculty constructively model the learning
"Apprenticeship" models of teaching allow students
to directly watch and internalize expert practice. Such settings
also assign students consequential roles emphasizing correct
practices. The demonstrable effectiveness of undergraduate
participation in faculty research is a case in point, as are the
internship or practicum components of many existing practice
established concepts to new situations.
The best gains occur
when students are given both the conceptual "raw materials" with
which to create new applications and active cues about how to put
them together. For such approaches to work as advertised, though,
students must do the work themselves and faculty must assiduously
avoid "telling" them how to make these linkages.
stimulating interpersonal collaboration.
Research findings on
collaborative learning are over-whelmingly positive, with instances
of effective practice ranging from within-class study groups to
cross-curricular learning communities.
providing rich and frequent feedback on performance.
students are assessed powerfully affects how they study and learn.
Managing the frequency and consequences of such assessments, by
using weekly quizzes or ungraded practice assignments, for instance,
creates iterative opportunities for students to try out skills, to
examine small failures, and to receive advice about how to correct
consistently developing a limited set of clearly identified,
cross-disciplinary skills that are publicly held to be
Intentional and integrated "learning plans" can
affect learning powerfully. Needed integration must be both
"horizontal" (emphasizing the application of key skills in different
contexts) and "vertical" (fostering sequential vectors of
development) to be effective. And both depend critically on making
collective campus commitments about what should be learned in the
first place. If the Seven Principles to a large extent emphasize the
importance of different kinds of student involvement to enhance
learning, four additional principles of curricular design emerge
from these recent discoveries about human learning: 1) Learning is
enhanced by engaging the natural learning functions of the brain,
which involve processes of incremental and sequential integration.
Students form their own meanings from their interactive experiences
with new information, in ways that are per-sonally unique. What
works for one student may not work for another; 2) An appropriate
and continuing level of challenge stimulates student participation
and learning. Too much or too little discourages interest; 3)
Assessment procedures which provide frequent feedback are an
important part of learning. Entrenched practices of midterm, final,
and term paper--or less--may serve faculty as evaluative tools, but
deprive students of the rich learning engendered by ongoing
assessment and feedback practices; and 4) Ideas must be put into
practice and experienced in personal ways for students to embody and
deepen their learning. Teaching methods which emphasize application,
such as internships, service learning, experiential education,
apprenticeships, research, and other practices all help to transfer
abstract learning into concrete and measurable skills.
TEACHING EXCELLENCEAn increased emphasis
on student learning will have major impacts on the structure and
practice of teaching. An institutional commitment to student
learning could give faculty significantly increased responsibility
for real teaching excellence. Emphasis on learning demands a
different kind of teacher, and a different kind of teaching, from
the traditional model; it may no longer be enough that college
teachers are competent in their disciplines; they are likely to be
increasingly called upon to create, develop, and manage stimulating
learning environments, using a variety of resources, abilities, and
technologies, including assessment resources, in order to deepen and
enrich student learning.
In response to
these increasing demands, relatively more resources will be needed
to support the development of faculty. Such support could take many
forms. Western's recently created Center for Instructional
Innovation is already supporting faculty in the application of
information technologies to their courses. This Center, perhaps in
cooperation with other currently existing units, like the Faculty
Development Advisory Committee, which currently offers grants to
faculty for the development of teaching, might play an expanded role
in faculty development as teachers. Alternatively, various forms of
"out-sourcing," such as a continuing workshop series with leading
thinkers, could also stimulate faculty development as teachers.
form, a meaningful emphasis of student learning demands some kind of
serious program for faculty development as teachers. One excellent
example of a comprehensive support program is the Learning
Re-sources Unit (LRT) at British Columbia Institute of Tech-nology
(BCIT). "(The LRU) was established in 1988 as a key catalyst for
educational excellence at BCIT. Staffed with more than 25
instructional designers, technical writers, editors, graphic artists
and clerical personnel, its mandate is to improve the teaching and
learning process through faculty, curriculum, and learning-skills
development initiatives." (BCIT web page.)
provides workshops, teaching aids, and consultations with faculty
for course, syllabus, and curriculum development. It also provides
faculty with a number of confidential resources for development of
teaching skills, including instructional skills workshops, mid-term
student evaluations, videotaping, and one-on-one professional
classroom observation and feedback.
findings about learning and about teaching, a preliminary model of
institutional excellence emerges, as shown in Figure 3. Adopting the
best educational practices and structuring courses, curricula, and
university support programs to stimulate student involvement
enhances the conditions for learning and individual development.
Assessment of student learning outcomes then provides feedback which
guides further improvements in policy.
III. THE ROLE
ASSESSMENT LEARNING CYCLEAssessment will be a
fundamental and integral part of any curriculum based on student
learning outcomes. Basically the same assessment learning cycle,
shown in Figure 4, takes place at the levels of the student, the
course, the program, the college, and the university as a whole.
It is worth
emphasizing: assessment is not just the measurement of learning; it
is in itself an integral part of learning. Assessment is the first
step in a continual learning cycle which includes measurement,
feedback, reflection, and change. The purpose of assessment is not
merely to gather information; the purpose of assessment is to foster
improvement. Frequent assessment of students helps them to refine
concepts and deepen their understanding; it also conveys high
expectations, which further stimulate learning. "Students
overwhelmingly reported that the single most important ingredient
for making a course effective is getting rapid response" (Wiggins,
assessments of faculty teaching by students and faculty development
consultants help teachers to improve their teaching and course
organization. Program assessments tell departments and curriculum
committees how well programs are meeting their objectives; and
comprehensive university-level assessments provide feedback about
how effectively university policies are contributing to the
accomplishment of the university's mission and goals.
years beginning in 1988, a group of distinguished scholars met
regularly to share ideas and experiences and to formulate principles
for assessment. Their set of "Nine Principles of Good Practice for
Assessing Student Learning," (AAHE Assessment Forum, 1992) is
patterned after the learning principles discussed above, and
clarifies the linkages between assessment and student learning:
assessment of student learning begins with educational
We measure what is most important to our mission and
Assessment is most effective when it reflects an understanding of
learning as multidimensional, integrated, and revealed in
performance over time.
Learning entails not only what
students know but what they can do with what they know; it involves
not only knowledge and abilities but values, attitudes, and habits
of mind that affect both academic success and performance beyond the
Assessment works best when the programs it seeks to improve have
clear, explicitly stated purposes.
Assessment is a
goal-oriented process. Assessment as a process pushes a campus
toward clarity about where to aim and what standards to apply;
clear, shared, implementable goals are the cornerstone for
assess-ment that is focused and useful.
Assessment requires attention to outcomes but also and equally to
the experiences that lead to those outcomes.
outcomes, we need to know the curricula, teaching, and student
effort that lead to particular outcomes.
Assessment works best when it is ongoing, not
Though isolated, "one-shot" assessment can be
better than none, improvement is best fostered when assessment
entails a linked series of activities undertaken over time,
monitoring progress toward intended goals in a spirit of continuous
Assessment fosters wider improvement when representatives from
across the educational community are involved.
learning is a campus-wide responsibility; the aim over time is to
involve people from across the educational community. Assessment is
not a task for small groups of experts but a collaborative activity;
its aim is wider, better-informed attention to student learning by
all parties with a stake in its improvement.
Assessment makes a difference when it begins with issues of use and
illuminates questions that people really care about.
useful, information must be connected to issues or questions that
people really care about. The point of assessment is not to gather
data and return "results"; it is a process that starts with the
questions of decision-makers, that involves them in the gathering
and interpreting of data, and that informs and helps guide
Assessment is most likely to lead to improvement when it is part of
a larger set of conditions that promote change.
alone changes little. Its greatest contribution comes on campuses
where the quality of teaching and learning is visibly valued and
worked at, where information about learning outcomes is seen as an
integral part of decision-making
assessment, educators meet responsibilities to students and to the
Our deepest obligation--to ourselves, our students,
and society--is to improve. Those to whom educators are accountable
have a corresponding obligation to support such attempts at
assessment principle above, measure things that matter, accentuates
the important link that must exist between a unit's mission and its
assessment measures. "The mission of an institution is the answer to
the question, what do you do and for whom?. . .Colleges need to be
clear about whom they serve and how they serve them, and to measure
their results to determine how well they deliver on their promises"
(Miller, 1998). Put another way, "a strong institutional mission
statement provides an invaluable starting point for assessment. .
.assessment cannot and should not take place in the absence of a
clear sense as to what matters most at the institution" (Banta,
1996). Effective accomplishment of stated goals is the most
appropriate measure of institutional performance and effectiveness.
The same principle applies to all levels of assessment.
although Western's mission statement does contain language which
asserts that Western "nurtures the intellectual, ethical, social,
physical and emotional development" of its students, the statement
lacks the specificity necessary to form the basis of clear and
measurable performance criteria. Clarifying the mission of the
University in terms of specific performance objectives and
developmental goals for students is an essential prerequisite to an
integrated, learning-based academic pro-gram.
PROGRAMS BASED ON STUDENT LEARNING OUTCOMESPioneering efforts in
assessment and student learning have been made at several colleges.
While there may be little direct transferability between the paths
these schools have followed and the path that will be chosen at
Western, the experiences of these schools are nevertheless
instructive. They provide useful maps of approaches that work.
College is a small, independent, four-year liberal arts college for
women, located in Milwaukee, and is widely recognized for its
pioneering work in assessment. Over the last twenty-five years the
Alverno faculty has developed a highly sophisticated system of
assessment-through-the-curriculum, for which it has received
explicit recognition and considerable financial support from
numerous foundations (Alverno, 1994).
years of work, Alverno has defined eight measurable Abilities that a
successful liberal arts education should develop: Communication,
Analysis, Problem Solving, Valuing in Decision-Making, Social
Interaction, Global Perspectives, Effective Citizenship, and
Aesthetic Responsiveness. Each of these Abilities has in turn been
divided into eight developmental levels--generally ranging from
fundamental identification at the first level to integrated
application at the highest level.
three important characteristics. They are: integrated, involving an
integrated set of skills; developmental, implying an increasingly
complex hierarchy of processes; and transferable, broadly useful and
applicable across the student's future roles and settings. Every
course at Alverno defines two specific sets of learning objectives;
the first pertains to the levels of traditional knowledge and skills
associated with the course; the second pertains to the Abilities
addressed in the course.
pass a course, students must demonstrate not only appropriate
mastery of course material by doing something, they must also
demonstrate mastery of the Abilities by how they do it. For example,
a math course would not only have specific math skills students must
demonstrate, it might also have specific levels of Communication,
Analysis, and Problem Solving Abilities the student must demonstrate
as well, and which instructors have agreed to assess.
is a small, Catholic, liberal arts college. Its comprehensive
assessment program tracks the development in all students of a
series of transferable skills, derived from its mission statement,
which quite specifically articulates the College's responsibilities
for student development (King's, 1999).
King's has a
CORE Curriculum which "focuses in a deliberate and systematic manner
the skills of liberal learning: Critical Thinking, Effective
Writing, Effective Oral Communication, Library and Information
Literacy, Computer Competence, Creative Thinking and Problem
Solving, Quantitative Reasoning, and Moral Reasoning." In addition,
"each department. . .defines each transferable skill within the
context of the major and then divides the skill into specific
also incorporates two integrative projects into all student programs
of study: the Diagnostic Project, and the Senior Integrated
Sophomore-Junior Diagnostic Project: "Each department or program
designs a screening exercise to determine each student's ability to
transfer critical thinking and effective communication to an
appropriate project related to the major field of study. Faculty
interact with students throughout the project and share results with
them. If the proper level of skill is not apparent, the student is
referred to an appropriate office (such as the Learning Skills
Center) for assistance. The process also evaluates the student's
likelihood of success in the major."
Integrated Assessment: "Each department or program designs an
exercise, usually in the context of a required senior course, a
capstone seminar, or a project, to allow the faculty and student to
examine the latter's success in integrating learning in the major
with advanced levels of the transferable skills of liberal
California State University, Chico
adopted by the College of Behavioral and Social Sciences at
California State University at Chico provides an appealing example
of an approach to assessing student learning in an institution much
like Western. Its approach is rooted in two basic premises: first,
faculty in the unit have particularly high teaching loads, and
therefore no extra time; and second, faculty wanted to be able to
demonstrate the effectiveness of their own programs on their own
terms--given budgetary uncertainties (Jacob, 1998).
The plan was
divided into three stages. The first step was for each department to
engage in a dialogue about what it means to be a major in this
department, and what should a major in this department know; that
is, define departmental learning objectives. As obvious as these
questions are, what was learned in each department was often a
step asked faculty to link those learning objectives with the
learning processes and experiences which would lead to the desired
learning outcomes. This requires departments to address objectives
explicitly, and to consider dropping courses which meet no learning
The third step
was to identify and implement assessment procedures, which is still
in progress. The plan has resulted in the adoption of a wide range
of assessment tools and a spate of curricular reforms in nearly all
departments. While the search for better assessment tools continues,
the next step will involve exchanging ideas among units, perhaps
focusing discussions on identifying some set of "best practices" in
assessment. The experience at Chico State demonstrates that a simple
plan can be highly beneficial, and that program benefits begin to
accrue as soon as dialogue begins at the departmental level about
student learning, curricular goals, and assessment practices. The
all-important first step is to open a dialogue about student
learning and curricular objectives.
IV. TOWARD A
CURRICULUM BASED ON STUDENT LEARNING
BACKGROUNDSince the inception of
state accountability reporting requirements over ten years ago,
Western has created and maintained extensive databases and developed
analytical capabilities in assessment and in survey research. The
Office of Institutional Assessment and Testing (OIAT) and the Office
of Survey Research have generated scores of reports on student
attitudes, behaviors, and performance, and their relationship to
program effectiveness, in the form of entering and graduating
student profiles, alumni satisfaction surveys, employer satisfaction
surveys, and program reviews.
reporting requirements have recently been extended to include
additional performance targets, and recent communications from the
Washington State Higher Education Coordinating Board suggest that
additional reporting requirements regarding student learning
outcomes, at the level of academic units, will very likely also be
required in the near future.
At the same
time, the accreditation review process has also increased its
emphasis on assessment of student learning. Ewell (1997) has
suggested that, since nearly universal accountability requirements
duplicate many of the traditional elements of peer review,
accreditation review should narrow its focus to core academic
processes, which would or could include the integration of student
learning outcomes into curricula, and the incorporation of best
teaching and learning practices, such as the Seven Principles, into
academic programs. Western's recent Accreditation Review is evidence
that such a shift is already happening. Its recommendation that
Western's academic units must be more actively involved in
assessment is entirely consistent both with this new role for
accreditation and with a shift in mission and policy toward student
to Figures 1 and 2, what all of this means is that assessment to
date has been largely driven by the regulatory process; assessment
has been about accountability, fiscal efficiency, and resource
allocation. These trends are not going to go away, although they are
likely to continue to evolve. However, recent evidence suggests a
growing convergence of accountability requirements and accreditation
requirements regarding the central importance of student learning
outcomes as measures of institutional performance.
B. ROLE OF
THE OFFICE OF INSTITUTIONAL ASSESSMENT AND TESTING
extensive databases on student characteristics and student outcomes.
This includes data from the Student Tracking System maintained by
the Registrar's Office, for information about student backgrounds,
enrollment history, coursework, grades, and majors, and information
from a variety of student, alumni, faculty, and employer surveys.
Western has participated intermittently for many years in the
Cooperative Institutional Research Project (CIRP), a comprehensive
freshman survey developed and administered through the Higher
Education Research Institute at UCLA, and its corresponding senior
survey, the College Student Survey (CSS). Collectively, these
surveys generate detailed longitudinal information on student goals,
behaviors, activities, expectations, and values, both as they enter
Western and as they graduate.
In early 1998
OIAT assembled departmental information from a number of recent
survey instruments, including the CSS, and provided summaries of
this data to department chairs. Such reports could be made even more
valuable if they were constructed with substantial input from the
academic units themselves, and if databases were expanded to
facilitate analysis on a departmental level.
CIRP has been regularly administered to entering freshmen in recent
years, administration of the CSS has been limited, with sample sizes
too small to permit useful inferences about individual programs.
Beginning this academic year, however, OIAT plans to expand the CSS,
providing comparable entry and exit surveys on all native students,
so that a comprehensive longitudinal database can be formed, from
which to assess student development while at Western and beyond.
Development of a similar entering survey instrument for transfer
students is underway.
these capabilities is specifically designed to provide assessment
data to individual academic programs about the impacts of their
programs on student development. Academic units are invited to write
for these surveys a number of tailored questions which are of
particular interest to them about their students' experiences with
their programs. Applying a variety of statistical techniques
including frequency analysis, cross-tabulations, analysis of
variance, block regression, and factor analysis, OIAT will be able
to investigate an extensive array of impacts of Western and its
programs on student learning and development.
C. THE ROLE
OF ACADEMIC UNITS
In the future
the capability for department-specific reporting can be expanded,
and tailored to meet data needs of individual programs. First,
however, academic departments must examine their own missions with
regard to student learning objectives, and how they want to measure
their success at accomplishing them, so that appropriate data can be
essentially three kinds of questions academic units must
investigate. First, what kinds of affective and cognitive outcomes
are essential goals of their programs. ". . .(W)hat should their
graduates be able to know, think, do, believe, or value?" (Peterson
and Hayward, 1989.)
are those outcomes to be measured in ways that provide meaningful
feedback about program effectiveness? "It is not unusual for lofty
goals to be identified that are not really taught. Special attention
should be given to ways in which connections are made among goals
and elements of the curriculum." (Gentemann, 1994.)
And third, how
will the various academic units incorporate the best practices in
teaching and assessment into their programs in ways that enhance
student learning and that are truly valuable and useful, or
achievement defines significant intellectual accomplishment by
adults as construction of knowledge through disciplined inquiry to
produce discourse, products, or performances that have meaning or
value beyond success in school. . .but this 'real world' dimension
constitutes only one of three criteria for authentic intellectual
work; the other two insist on construction of knowledge through
disciplined inquiry--both of which pay significant attention to
students' basic knowledge and skills." (Newmann,
Alverno College have made it clear that their twenty-five year
pioneering struggle with these issues has been a difficult one.
However, they suggest (Alverno, 1998) that their program began
modestly, with a commitment to student learning as their common
goal. This commitment was reinforced by a President who provided and
enforced an action deadline for the inception of their new program,
ready or not. Then, through much dialogue over many years, they were
able to identify the seven Abilities, to define levels of those
Abilities they could all agree upon, and to reorganize their
academic programs and infrastructure around learning.
experiences have been reported wherever this inquiry has been
undertaken. This is the fundamental value of assessment in practice;
learning about a thing is the inevitable result of attending to it,
and improvement is the inevitable result of learning.
CONCLUSIONSThere is ample evidence
to suggest that reorienting Western's educational policies and
practices toward the improvement of student learning outcomes would,
over time, significantly improve the quality of education of Western
students and graduates.
reorientation would necessarily be an ongoing process; over time it
would likely constitute a quantum shift in our approach to
education. It would probably imply changes over time in our mission
and goals, in the structure of our curricula, in assessment
procedures from the classroom on up, in the responsibilities of
faculty, staff, and administrators, and in the organizational
structure of the University. However, all of these are the kinds of
changes which can evolve in an organic way specific to Western and
its community of students, faculty, staff, and administrators. The
important thing is to begin the process, and to allow it to develop.
RECOMMENDATIONSA true commitment to
student learning is a paradigm shift, but it doesn't have to happen
all at once. The first recommendation--the all-important first
step--is to initiate a campus-wide exploration and discussion
of whether and how to redefine Western's mission and goals to
reflect a commitment to excellence in student learning, and to
define strategies for achieving such goals. Faculty within academic
units must bear a particular responsibility for beginning a dialogue
about their own major programs, examining their willingness and
ability to restructure their programs, courses, and assessment
procedures to be consistent with improving learning outcomes. They
must be willing to ask the three questions posed at Chico State: 1)
What should our majors know; 2) How can they best learn these
things; and 3) How can we measure our success at teaching them?
recommendation is to establish some kind of "Faculty Development
Center," which would provide confidential consultations,
resource and technical support, and training to help faculty develop
as teachers. Such an office could be an extension of the new Center
for Instructional Innovation, or it could be modeled after the
Learning Resources Unit at BCIT mentioned in Section 2, which
provides a wide range of support services, including course
development, definition of course objectives, assessment
alternatives, and skills development. We should want to provide
explicit support to improve both the quality of teaching and also
the productivity of individual faculty, and to provide incentives
for teaching excellence.