What is a "direct measure" of student learning?
Direct measures assess student performance of identified learning outcomes, such as mastery of a lifelong skill. They require standards of performance. Examples of direct assessments are: pre/post test; course-embedded questions; standardized exams; portfolio evaluation; videotape/audiotape of performance; capstone course evaluation.
What is an "indirect measure" of student learning?
Indirect measures assess opinions or thoughts about student knowledge, skills, attitudes, learning experiences, and perceptions. Examples of indirect measures are: student surveys about instruction; focus groups; alumni surveys; employer surveys.
Examples of Direct Measures of Student Learning
- Faculty assigned to teach different sections of a ‘gatekeeper’ course in the general education – social and behavioral sciences program agree to assign a final paper that asks students for a comparative analysis of several sociological theories. With the students' permission, the faculty copy the final papers before marking on them, and then give the copies (or a representative or random sample) to the review committee. Three members of the committee score each paper according to guidelines developed by the program.
- Each final exam for every course in a general education – foreign language program requires that students translate selected passages they are unlikely to have seen before. At the end of the academic year, each translation is reviewed by at least two foreign language faculty using the same scoring guide to analyze points of strength and weakness in the translations for each academic class. As they accumulate student translations from year to year, the faculty are able to measure the improvement in the translations that individual students did in their first and second year language courses.
- Each year, twenty percent of the papers submitted by students in a program capstone course are randomly selected for assessment and sent to a panel of outside evaluators. The evaluators write comments about the overall quality of the sample, and then select ten papers to discuss in depth. As part of their discussion, the evaluators determine the extent to which each paper reflects the achievement of one or two of the program learning goals, which the program has previously selected, and score them according to guidelines previously developed by the program.
- Students in an occupational program are asked to collect and maintain a portfolio of materials throughout their occupational coursework, and to write a short summary for each course. Specific questions about learning styles and experiences are provided to help guide their reflections. They write a full paper about their experiences in the occupational program, tracing the main themes and content of their learning experience. In addition, each student is asked to design and carry out a research project on a topic that extends their learning experience. Three faculty independently rate the reflective and research papers, using guidelines and specific questions about how the students' performance reflects each of the program's knowledge or skill goals.
- The general education - math program have developed six outcomes for intermediate algebra and eight outcomes for college algebra. A portion of the final exam includes common problems that directly measure the students’ ability to perform the outcome. The problems are used in every intermediate and college algebra course offered. Common grading criteria have also been established and are outlined in the course syllabi.
- A general education - science program includes a set of common questions that measure the course outcomes in the final exam. Instructors are then able to determine overall if the students demonstrate competence, as well as identify areas in which students may be weak.
- An occupational program developed a competence checklist for speech and public speaking that evaluates two presentations students give in the program as well as their interaction in small group activities. Upon evaluating the students’ performance on designated assessment activities, the instructor indicates in the final grade roster – beside the final letter grade – whether or not the student has achieved the competence measured. The program is then able to calculate the percentage of students who were competent in the area assessed. This provides feedback to faculty in the program about where students have a high level of competence and where additional work may be needed.
- A general education – humanities program requires that students submit samples of their work from their courses. During ‘exam week’, faculty gathers and critiques the portfolios, looking not only at the quality of work in evidence, but at each student's improvement. The review committee then asks faculty to reflect on the trends they saw in their critiques, and what they might suggest about areas of strength and weakness in the program overall.
- A management/marketing program asks each student in the capstone course to develop a marketing plan for a product and to present it to a panel of faculty while being videotaped. Students have been told that the videotapes will be used for assessment and their permissions have been secured. The assessment committee then invites local marketing professionals to join them in developing guidelines by which to assess the presentations. They view a random sample of the presentations, and as a group, roughly divide them into groups of adequate, superior and inadequate. They then review the tapes for each group and come up with a five or six sentences describing the properties that make a presentation superior, adequate or inadequate. After inductively composing this scoring guide, the group divides in pairs, with each pair separately scoring every presentation in a larger sample. If the two scorers reach different conclusions, the presentation is sent to a third reviewer.
- Students in a Spanish language program are interviewed at the end of their second year. A panel of three faculty conduct the interviews in Spanish, and each rates each student according to a standard scoring guide that rates students' proficiency in Spanish as well as knowledge of Spanish literary, historical and cultural traditions. At the bottom of each score card, faculty note particular areas of strength and weakness within each category. After the interviews are completed, faculty gather to compare observations, analyze the scores, and discuss common areas of strength and weakness in the students' interviews.
- Students entering an occupational program take a test to determine their understanding of various concepts critical to the field of study. When the students complete their course sequences, they are given exit exams to determine their growth. Scores from the entering and exiting exams for a random sample of each year's graduates are compared as part of each year's outcomes assessment report.
Examples of Indirect Measures of Student Learning
Surveys - opinions, thoughts, reactions
- Student Surveys
- In Class Surveys
- Department Surveys
- Student Evaluation of Instruction
- Student Ratings
- Alumni Surveys
- Employer Surveys
- Faculty Surveys
Self Assessment / Reports
- Student/Alumni/Faculty self assessments or reports
Focus Groups
- Student groups
- Alumni groups
- Employer groups
Interviews
- Student interviews
- Exit interviews
- Employer interviews
Course Grades
Graduation / Completion Rates
Job Placement Data
Advisory Board Feedback / Evaluation
Course Content / Grade Correlations
