This paper initiates the AERA symposium on issues and methods in qualitative evaluation with a theoretical overview and critique of the state of the evaluation field in the United States. Some of the basic theoretical underpinnings to a qualitative approach to evaluation are presented. This type of approach complements and compensates for those aspects of educational practice that conventional approaches omit. (RC)
Topics: ERIC Archive, Evaluation, Evaluation Methods, Evaluation Needs
The purpose of this research is to provide an overall view of PACE, as seen by project directors. The discussion is concerned with characteristics of the existing projects; problems encountered; views toward state, local and federal agencies, criteria for evaluation and funding; and ideas for the future. The study points out basic facts about the projects and basic relationships between different types of projects. The survey instrument covers the six major areas: (A) Problems in project...
Topics: ERIC Archive, Evaluation Methods, Program Proposals, Research Proposals
The need for a more comprehensive approach to educational planning is obvious in education as projects (a) become more complex, (b) become larger, (c) call for greater allocations of money, and (d) continue for a longer period of time. PACE has served to dramatize the glaring need that exists in United States education for better planning--a need that is not necessarily confined to this nation. There has been some improvement in project quality as a result of the guideline requirements for...
Topics: ERIC Archive, Evaluation Methods, Program Proposals, Research Proposals
The advantages of the adversary evaluation model are argued. Adversary evaluation utilizes a judicial, argumentative approach and promotes the use of multiple types and sources of data. It grants an investigative license to dig out the data to a greater extent than more traditional approaches. Some of the payoffs of the adversary approach are: (1) the critical input of colleagues is considered at the most opportune time--prior to actual investigation; (2) the inclusion of provocative questions...
Topics: ERIC Archive, Evaluation Methods, Models, Program Evaluation, Questionnaires
This paper is addressed to administrators and evaluators in school districts which are initiating bilingual education programs in compliance with a U.S. Supreme Court decision. Major evaluation problems are described and remedies suggested. Each step in the sequence, from needs assessment through final report, is discussed and illustrated. The objective is to share Milwaukee's long experience (since 1969) in bilingual program evaluation with other districts, and to forearm by forewarning....
Topics: ERIC Archive, Bilingual Education, Evaluation Methods, Program Evaluation
The Using Evaluation Data Form (UEDF) represents a psychological lever for getting a program's decision maker to consider major evaluation findings. The form may be used at any point of the evaluation process when sufficient data exist to support a finding deserving of action or reaction by the project staff. By local policy, it is required for inclusion in all end-of-year evaluation reports. Findings are results of evaluation activities conducted throughout the year. The process of the form's...
Topics: ERIC Archive, Evaluation Methods, Evaluative Thinking, Program Evaluation
This article describes the main aspects of the use of it technologies in the economy of the metallurgical industry, the Author emphasized the relevance of the study based on the current state of the Russian economy and the prospects of this direction. The article reveals the problems of assessing the possibilities of using IT technologies in the economy. And offered some solutions, which is of practical value.
Topics: it-technologies, metallurgy, metallurgical industry, evaluation methods, industry.
Two major functions of teacher evaluation documentation are to serve to help rationalize the process it documents, and to serve an evidential role, justifying decisions concerning personnel, policy decisions, etc. In sum, teacher evaluation documents provide access to teacher evaluation practice, the notion of teacher competency behind it, and some effects of the institutional and bureaucratic context in which teacher evaluation takes place. The materials for this analysis comprise separate...
Topics: ERIC Archive, Evaluation Methods, Research Needs, Teacher Evaluation
Stroke is a disease in the central nervous system in which sudden ischemia or haemorrhage in blood vessels disrupts the smooth blood supply to brain tissues, resulting in a partial loss of brain function and consequent functional disorders. Aim and Scope: To improve upper limb function, current stroke interventions employ various treatment methods targeted at the nervous system. Interventions and studies are underway regarding continuous upper limb training programs, such as constraintinduced...
Topics: Stroke, Upper limb rehabilitation, Treatment methods, Evaluation methods
Project REVIS (Research and Evaluation of Vocational Instructional Systems), completed in June, 1971, had as its objectives such goals as: to assist in the design, development, and evaluation of selected instructional systems in the areas of career orientation, communication, and computation; to identify and evaluate commercially-prepared instructional materials and equipment; to develop and evaluate supplementary instructional materials; to develop a systematic procedure for the evaluation of...
Topics: ERIC Archive, Evaluation, Evaluation Methods, Instructional Systems, Vocational Education
An evaluation of the 4 x 4 block schedule in effect at Cleveland High School and East Side High School, both in the Cleveland, Mississippi School District was conducted during the 2001 spring semester at the request of the associate superintendent of the school district. Multiple sources of data were identified, and data were collected during the spring 2001 semester. These included archival records on student attendance, dropouts and grades, informal classroom observations, and survey results...
Topics: ERIC Archive, Evaluation Methods, Evaluators, Program Evaluation, McNeil, Keith
The Competency Development Scales (CDS) for teachers of exceptional children is described as an instrument to evaluate teacher skills and aid in subsequent planning of appropriate inservice training. The CDS consists of five major skill areas: general competence (such interpersonal skills as enthusiasm, responsibility, patience, and communication); assessment, objectives, and curriculum planning; teaching/learning situation (direct teaching and behavior management); evaluation and records; and...
Topics: ERIC Archive, Disabilities, Evaluation Methods, Teacher Evaluation, Teaching Skills
This thesaurus to the evaluation field is not restricted to educational evaluation or to program evaluation, but also refers to product, personnel, and proposal evaluation, as well as to quality control, the grading of work samples, and to all the other areas in which disciplined evaluation is practiced. It contains many suggestions, procedures, comments, criticisms, definitions and distinctions. Criteria for inclusion of an entry were: (1) at least a few participants in workshops or classes...
Topics: ERIC Archive, Definitions, Educational Assessment, Evaluation, Evaluation Methods, Thesauri
It was hypothesized that a middle school organization embracing grades 5-8 would overcome certain inadequacies of a typical elementary and junior high school organization and would better cope with the social and educational needs of older children, preadolescents, and early adolescents aged 10-14. Regarding data collection, measurement techniques, testing concerns, statistical methods, followup studies, attendance figures, teacher attitudes, etc., 19 ways were formulated to test the...
Topics: ERIC Archive, Evaluation Methods, Middle Schools, Hines, Vynce A.
On behalf of the Joint Committee on Standards for Educational Evaluation, the author reviews the Committee's recent efforts and discusses the needs for further work related to the "Standards for Evaluations of Educational Programs, Projects, and Materials." The Committee has developed a program to guide its work. The program addresses four areas of activity: provision of training and technical assistance; research, development, and training for interpretation and use of the Standards;...
Topics: ERIC Archive, Curriculum Evaluation, Evaluation Methods, Program Evaluation, Standards
This paper attempts to define evaluation use and manipulable variables related to it. Approaches to defining evaluation use have included typologies inventorying kinds of evaluation use, threshold criteria defining accepted levels of evidence, construct or operational definitions, and dependence on the perspective of the decision maker. None of these approaches is found to be satisfactory. Within models of evaluation use, the concept is conceived as a dependent variable, and much of the...
Topics: ERIC Archive, Evaluation Methods, Evaluation Utilization, Models, Research Utilization
This report describes an evaluation of Project Information Packages (PIPs), sets of manuals and other materials intended to help a school district adopt and implement an exemplary education project. Four PIPs were evaluated in a field test, each PIP describing a different bilingual project. It was concluded that the awareness materials produced few applications for PIPs. Field-test sites that received PIPs tended not to follow PIP guidelines closely, but to adapt them extensively, often with...
Topics: ERIC Archive, Bilingual Education, Diffusion, Evaluation Methods, Summative Evaluation
This is the first of a series of reports to be issued at irregular intervals to communicate the plans, progress, and findings of the evaluation of the Trainers of Teacher Trainers (TTT) Program. The first 12 pages of the report outline in three general categories the activities carried out to date: 1) visits to cluster meetings and individual projects and conversations with key people connected with TTT; 2) staff meetings to bring together the results of information collected during visits; 3)...
Topics: ERIC Archive, Evaluation Methods, Program Evaluation, Teacher Educator Education
A research design is used to find out how effective a given piece of research has been. This short course in the fundamentals of research design is intended to bring together a few typical examples and a few generally accepted principles of research in a practical kit. After a brief introduction on how to progress from data that is uncountable to data that is countable, stress is placed on coming up with data that is accountable. Since analysis of research design must be done quickly and...
Topics: ERIC Archive, Data Analysis, Evaluation Methods, Guidelines, Research Design
This report describes the system used to code each segment of Square One TV for content analysis of all four seasons of production. The analysis is intended to aid in the assessment of how well Square One is meeting its three goals: (1) to promote positive attitudes toward, and enthusiasm for, mathematics; (2) to encourage the use and application of problem-solving processes; and (3) to present sound mathematical content in an interesting, accessible, and meaningful manner. The information in...
Topics: ERIC Archive, Coding, Content Analysis, Evaluation Methods, Program Evaluation
The literature review was undertaken to establish the current status of the methodology for forecasting and assessing technology and for quantizing human resource parameters with respect to the impact of incoming technologies. The review of 140 selected documents applicable to the study was undertaken with emphasis on the identification of methods of assessment of the impact on human resource parameters of advances in technology. The review failed to provide a solution to the problem of...
Topics: ERIC Archive, Evaluation Methods, Human Resources, Literature Reviews, Technological Advancement
This paper discusses the three facets of evaluation which could be used in vocational education. The evaluation could fall into three dimensions: (1) the content dimension consisting of input, process, and output evaluation; (2) the operational dimension consisting of the following criteria: fakability, validity, desirability, probability, and reliability; and (3) the product dimension consisting of hypothesis, methodology, results, discussion, and recommendations. (Author/RC)
Topics: ERIC Archive, Evaluation Methods, Models, Vocational Education, Chuang, Ying C.
The preliminary experiences of a process evaluator contracted to evaluate a three year program to develop Alaskan Native leadership are presented. With hindsight, some characteristics of this particular project having implications for a process evaluator are noted and some general comments on process evaluation, based on the author's experiences, are generated. (RC)
Topics: ERIC Archive, Evaluation Methods, Formative Evaluation, Professional Personnel, Program Evaluation
Empirical literature on two types of hearings, and their applicability in evaluating educational programs, is reviewed. An introduction describes the use and nature of hearings and distinguishes adversary hearings, in which two parties argue opposing sides of an issue, from committee hearings, in which a variety of positions are presented to a panel of decision-makers. Adversary hearings, the type primarily used in evaluation, are the main focus of the paper. Following a brief overview of...
Topics: ERIC Archive, Committees, Evaluation Methods, Hearings, Models, Program Evaluation, Validity
This proposal describes a framework and strategy for evaluating the impact of World Bank Institute (WBI) programs. Each section details a relevant aspect of the evaluation strategy and framework, their goals and strategies, and how they relate to the broader goals and mission of the WBI and the World Bank. The evaluation proposal focuses on WBI client learning programs that have been implemented for at least 2 years and whose effects include impacts on individuals, countries, policies, laws,...
Topics: ERIC Archive, Evaluation Methods, Models, Program Evaluation, Bardini, Mark D.
Aggregation of data is, by definition, an obscuring of details for the sake of achieving a summary. It is, therefore, potentially harmful to accuracy. Attention should be given to the aggregation that scores undergo prior to statistical tests. Two familiar research designs where this is important are a) in two groups/one measure cases, and b) in two groups/pre-post measurement cases. Another problem for researchers develops when incomplete and missing data are encountered for identification...
Topics: ERIC Archive, Data Analysis, Educational Research, Evaluation Methods, Research Problems
The Children's English and Services Study was a project designed to assess the bilingual education needs of limited English speaking children in the United States. The submission of a draft final report prompted the present report from the sponsoring organization, in which various methodological procedures are questioned and recommendations are made for the revision of the final report. The three analytical issues involved are: (1) were the items selected for inclusion in the Language...
Topics: ERIC Archive, English (Second Language), Evaluation Methods, Language Proficiency, Testing
This document contains these presentations from the Enumerating Homeless Persons conference: (1) "Conference Welcome" (Pat Carlile); (2) "Conference Objectives" (Charles Jones); (3) "Deciding Where We Are" (Barbara Everitt Bryant, Cynthia M. Taeuber, and Tom Jones); (4) "Judging Where We Are Going and How to Do It" (Anna Kondratas); (5) "Facilitator Remarks" (Susan Miskura); (6) "Federal Data Collection Needs and Requirements" (Chris...
Topics: ERIC Archive, Census Figures, Evaluation, Evaluation Methods, Homeless People, Incidence
The Project on Secondary Analysis at Northwestern University is funded to (1) test and develop new methods of evaluating educational programs, and (2) reanalyze existing evaluation data to assure that estimates of program effects are as unbiased and unequivocal as possible. This paper examines the topic of secondary analysis and describes some of the strategies that have been applied to Project Cali data. Project Cali is an intervention project designed to evaluate the impact of increased...
Topics: ERIC Archive, Data Analysis, Evaluation Methods, Measurement Techniques, Program Evaluation
A model for evaluating educational products is presented which is based on a simple philosophy: decide in advance what the final product ought to look like; then use this conception to prescribe methods for developing and measuring it. In the preordinate model, five separate activities are identified which occur in approximately the following order: (1) selection of the product, (2) selection of the critical properties of the product, (3) making critical properties as operational as possible,...
Topics: ERIC Archive, Evaluation Methods, Models, Summative Evaluation, Clark, D. Cecil
The intent of formative evaluation is to improve programs as well as to justify their continuation. It is critical to separate clearly those functions of the evaluation which are political from those which may lead the way to instructional improvement. Data for formative evaluation should be gathered in an interpretable way at the level at which decisions will be implemented, usually at the classroom level. This forces data collection activities to a single planned incursion. This...
Topics: ERIC Archive, Data Collection, Evaluation Methods, Formative Evaluation, Instructional Improvement
Educational researchers often utilize analysis of covariance type techniques to assess the effects of innovative programs implemented in naturalistic settings. This paper delineates and describes analysis and reporting considerations for the application of analysis of covariance type techniques in public school settings, based primarily on a review and critique of the national Follow Through evaluation. The areas discussed include: relating specific research hypotheses, the results of the...
Topics: ERIC Archive, Analysis of Covariance, Evaluation, Evaluation Criteria, Evaluation Methods
In keeping with a model of intelligence that identifies at least 12 intelligence "talents," formal and informal intelligence or talent assessments have been developed. This paper presents some of these informal instruments that can be used to assess convergent and divergent forms of intelligence. These nontraditional instruments have been designed to be enjoyable. Twelve tests are included. (SLD)
Topics: ERIC Archive, Evaluation Methods, Humor, Intelligence, Intelligence Tests, Lemire, David
The general objectives of this study were to determine the overall influence and impact on education of a sample of terminated ESEA Title III Operational and Planning Grants. The five major areas in the evaluation instrument included: characteristics, project accomplishments, provisions for continuation, project design, and final appraisals. The sample consisted of 94 terminated planning grants and 43 terminated operational grants. The instrument developed for analysis of the reports covered...
Topics: ERIC Archive, Educational Planning, Evaluation Methods, Program Proposals, Research Proposals
This sixth, and final, report features reports by 17 special consultants. Most of them have been with the study for two years. In the course of its work, members of the two study teams have visited close to 300 PACE projects, studied over 1,000 proposals, attended several meetings on ESEA Title III, and read numerous materials. In this volume, each consultant views the future of PACE from the vantage point of his own specialty. Credit or criticism rests with each author. (JL)
Topics: ERIC Archive, Educational Planning, Evaluation Methods, Program Proposals, Research Proposals
It is common for supervisors to evaluate their supervisees with a rating form. Despite the importance of supervisor ratings to the training of counselors and therapists, very little attention has been devoted to the overall reliability (generalizability) of these ratings. This study examined the generalizability of supervisor ratings of counselors-in-training. Participants included 23 counselor trainees enrolled in a masters level prepracticum course and 9 doctoral-level counseling supervisors....
Topics: ERIC Archive, Counselors, Evaluation Methods, Rating Scales, Supervision, Supervisory Methods
The report documents the achievements of the SPAP (Speech-Language Pathology Assessment Project), which examined competency statements for beginning speech-language therapists. Noted are the development and revision of the SPAI (Speech-Language Pathology Assessment Instrument), a training conference for using the SPAI, practice assessment of speech-language pathologists in school, and the preparation of new materials for use in training SPAI data collectors. Material preparation consisted of...
Topics: ERIC Archive, Communication Disorders, Competence, Evaluation Methods, Speech Therapy, Therapists
A 2-day seminar, attended by 35 physicians and others concerned with graduate physician education, was held to review the state-of-the-art of evaluation, current techniques, and effectiveness of programs. Presentations during the opening session were: (1) "The State of the Art of Evaluation" by Ray Elling, (2) "Defining Objectives in Evaluation" by George E. Miller, (3) "Contemporary Activity in Program Evaluation" by John P. Hubbard, (4) "Judgement...
Topics: ERIC Archive, Evaluation Methods, Medical Education, Physicians, Program Effectiveness, Seminars
This final volume of the report on the training program in educational evaluation and development at the University of Colorado contains four appendixes: 1) "Vitae of Project Staff"; 2) "Progress Report and Evaluation of the Graduate Research Training Program and the Laboratory of Educational Research"; 3) "Sample Measurement Instruments," including course evaluation inventory, trainee attitude scale, and trainee questionnaire on program operations; and 4)...
Topics: ERIC Archive, Consortia, Educational Development, Educational Programs, Evaluation Methods, Teacher...
Educational and social programs often develop from a weak or imprecise conceptualization relating the program's system of input variables to its claimed outcomes. Evaluation personnel can contribute both to the final development of a program and to the fair evaluation of such programs by learning to formally characterize programs and to construct causal models of them. The evaluation effort represents an attempt to determine the correctness of the program's existing conceptualization, and if...
Topics: ERIC Archive, Evaluation Methods, Models, Preschool Education, Program Evaluation, Research...
The paper presents an evaluation design for measuring the effectiveness of special education programs for handicapped preschoolers. Reasons for not selecting traditional evaluation designs are noted, and the use of a "change-index" is supported instead. A value-added analysis was selected to determine the value added by the program beyond that which would have been expected (due to maturation) without the program. Factors such as sex, race/ethnic group, handicapping condition, and...
Topics: ERIC Archive, Disabilities, Evaluation Methods, Preschool Education, Program Effectiveness, Program...
This paper reviews graphical and nongraphical methods for estimating multivariate normality. Prior to exploring this methodology, a foundation is established by presenting ways to assess univariate and bivariate normality. A data set of three variables used by J. Stevens (1986) is analyzed using Q-Q plots, stem and leaf plots, histograms, skewness, and kurtosis coefficients, the Shapiro-Wilk statistic, and bivariate and multivariate scatterplots. Multivariate normality is explored in terms of...
Topics: ERIC Archive, Chi Square, Evaluation Methods, Multivariate Analysis, Ashcraft, Alyce S.
Naval Postgraduate School
10
10.0
Nov 19, 2020
11/20
by
Fleumer, Albert M.; Urrutia, Michael B.
texts
eye 10
favorite 0
comment 0
This thesis describes the Department of the Army (DA) Financial Management Quality Assurance Program, the Finance Information Network Evaluation System and the Finance and Accounting Monthly Operations Report System. Sample performance data from Finance and Accounting Offices (FAOs) are used to develop methodologies for identifying substandard performance; to determine the effect of the (DA) Quality Assurance assistance visits on FAO performance; and to develop a current profile of the...
Topics: Management, financial quality assurance, financial management, Army financial management evaluation...
The purpose of this technical paper is to outline the procedures used in designing the training programing in educational evaluation and development involving an elaborate collection of data from many sources and a drafting of a tentative plan which was then screened, reviewed, and critiqued by each of the consortium units. The data sources were existing manpower studies, U.S. Office of Education sources, group and individual meetings with consortium units, working papers drafted by several...
Topics: ERIC Archive, Consortia, Educational Development, Educational Programs, Evaluation Methods, Teacher...
This is a report on the third survey conducted on procedures for evaluating the performance of administrators and supervisors in local school systems. A questionnaire was sent to school systems enrolling 25,000 or more pupils, and results indicated that 84 of the 154 responding systems have formal evaluation procedures. Tables and discussions of the survey results cover the following topics: 1) probationary periods for administrators, 2) which personnel are evaluated and the frequency of...
Topics: ERIC Archive, Administrator Evaluation, Evaluation Methods, Personnel Evaluation, Principals,...
Viewing decision makers within the program as the primary audience for program evaluation information, a model for the evaluation of ESEA Title I projects is developed for program improvement as well as program assessment. The model incorporates four stages of assessment, with five evaluative criteria distinguished for each stage. Both negative and positive criteria define the evaluation cycle. A rationale is given for each phase of the evaluation process, including program definition, stage...
Topics: ERIC Archive, Evaluation Criteria, Evaluation Methods, Federal Legislation, Guidelines, Program...
This paper describes a relatively new approach to program evaluation, the "learning history" approach. The paper also presents preliminary results of a pilot test of the method and provides a preliminary account of how this new approach will be compared to the classic goals-based approach to evaluation. The learning history method was developed in the early 1990s by G. Roth and A. Kleiner. The technique was originally created as a means for generating organizational learning about...
Topics: ERIC Archive, Evaluation Methods, Learning, Pilot Projects, Program Evaluation, Clardy, Alan
This paper is part of a series of reports that examine the impact of No Child Left Behind, the newly revised Elementary and Secondary Education Act, on state policy and policymaking. It focuses on standards-based assessment. The first section explains that assessments play a pivotal role in standards based reform by: communicating goals that school systems, schools, teachers, and students are expected to achieve; providing targets for teaching and learning; and shaping the performance of...
Topics: ERIC Archive, Academic Standards, Elementary Secondary Education, Evaluation Methods, Student...
It is assumed that the delineation of values is critical to proper needs assessments. "Need" is defined as a requisite or desideratum generated as a discrepancy between a target state (T) and an actual state (A), if and only if the presence of the conditions defined by T can be shown significantly to harm, indispose, or constrain a subject (S). There are at least five points at which value judgments impinge upon needs determination. They are: (1) the identification of the domain of T;...
Topics: ERIC Archive, Educational Assessment, Evaluation Methods, Formative Evaluation, Needs Assessment,...
The digest presents an overview of program evaluation in gifted and talented education. Reasons for program evaluation are offered, including the need to base decision making on valid and reliable data and the demands of accountability. Eight purposes of program evaluation are briefly considered: (1) documentation of the need for a program; (2) documentation of the case for a particular approach; (3) documentation of the feasibility of a program; (4) documentation of program implementation; (5)...
Topics: ERIC Archive, Elementary Secondary Education, Evaluation Methods, Gifted, Program Evaluation, Talent