Career and Technical Programs

Assessment Methods

There are several different assessment methods available to use for your program. This is a listing of those methods and by selecting the method you can receive a detailed explantion including advantages and disadvantages and the when to use the method.

• Archival Records: Biographical, academic, or other file data available from the college or other agencies and institutions. Can be quick, easy, and cost-effective method, if data are available and accessible. Usually limited data quality but integral to valuable longitudinal comparisons. Should be a standard component of all assessment programs.

• Behavioral Observations: Measuring the frequency, duration, topology, etc. of student actions, usually in a natural setting with non-interactive methods. For example, formal or informal observations of a classroom. Observations are most often made by an individual and can be augmented by audio or videotape. This is the best way to know what students actually do, how they manifest their motives, attitudes and values. Special care and planning are required when studying sensitive issues, but in situations where the information derived from the observations is critical, it's usually worth it for highly valid, useful results.

• External Examiner: Using an export in the field from outside your program, usually from a similar program at another institution to conduct, evaluate, or supplement assessment of your students. Information can be obtained from external evaluators using many methods including surveys, interviews, etc. Best used as a supplement to your own assessment methods to enhance external validity, but not as the primary assessment option. Other benefits can be accrued from the cross-fertilization that often results from using external examiners.

• Focus Group: Typically conducted with 7-12 individuals who share certain characteristics that are related to a particular topic related to a research or evaluation question. Group discussions are conducted by a trained moderator with participants (several times, if possible) to identify trends/patterns in perceptions. Moderator's purpose is to provide direction and set the tone for the group discussion, encourage active participation from all group members, and manage time. Moderator must not allow own biases to enter, verbally or nonverbally. Careful and systematic analysis of the discussions provides information that can be used to evaluate and/or improve the desired outcome. Focus groups are a quick and, if locally done, inexpensive method of gathering information. They should be conducted by someone who has training and experience in conducting Focus Groups and analysis of Focus Group data. They are very useful for triangulation to support other assessment methods but they are not a substitute for systematic evaluation procedures. Focus Groups should meet the same rigor as other assessment methods and should be developed and analyzed according to sound qualitative practices.

• Exit Interviews: Asking individuals to share their perceptions of their own attitudes and/or behaviors or those of others. Evaluating student reports of their attitudes and/or behaviors in a face-to-face dialogue. Interviews provide opportunities to cover a broad range of content and to interact with respondents. Opportunities to follow-up response can be very valuable. Direct contact may be difficult to arrange, costly, and potentially threatening to respondents unless carefully planned.

• Local Exam: Objective and/or subjective tests designed by faculty of the program or course sequence being evaluated. Most useful for individual coursework or program evaluation, with careful adherence to measurement principles. Must be supplemented for external validity.

• Performance Appraisals: A competency-based method whereby abilities are measured in most direct, real-world approach. Systematic measurement of overt demonstration of acquired skills. Generally the most highly valued but costly form of student outcomes assessment. However, it is usually the most valid way to measure skill development.

• Portfolios: A competency-based method whereby abilities are measured in most direct, real-world approach. Systematic measurement of overt demonstration of acquired skills. Generally the most highly valued but costly form of student outcomes assessment. However, it is usually the most valid way to measure skill development.

• Oral Examination: As assessment of student knowledge levels through a face-to-face dialogue between the student and examiner – usually faculty. Oral exams can provide excellent results, but usually only with significant - perhaps prohibitive – additional cost. Definitely worth utilizing in programs with small numbers of students, and for the highest priority objectives in any program and local testing policies do not prohibit the testing method.

• Simulations: A competency based measure where a person's abilities are measured in a situation that approximates a "real world" setting. Simulation is primarily used when it is impractical to observe a person performing a task in a real world situation (e.g., o the job). An excellent of increasing the external and internal validity of skills assessment at minimal log-term costs.

• Written Surveys: Asking individuals to share their perceptions about the study target – e.g., their own or others skills/attitudes/behavior, or program/course qualities and attributes. A relatively inexpensive way to collect data on important evaluative topics from a large number of respondents. Must always to treated cautiously, however, since results only reflect what subjects are willing to report about their perception of their attitudes and/or behaviors.

• Standardized Exams: Used primarily on students in individual programs, courses or for a particular student cohort. Relatively quick and easy, but useful mostly where group-level performance and external comparisons or results are required. Not as useful for individual student or program evaluation. May not only be ideal, but many times the only alternative for benchmarking studies.