WorldCat Identities

Stecher, Brian M.

Works: 140 works in 388 publications in 1 language and 21,693 library holdings
Genres: Case studies 
Roles: Author, Editor
Publication Timeline
Most widely held works by Brian M Stecher
Toward a culture of consequences : performance-based accountability systems for public services by Brian M Stecher( )

19 editions published in 2010 in English and held by 3,644 WorldCat member libraries worldwide

Performance-based accountability systems (PBASs), which link incentives to measured performance as a means of improving services to the public, have gained popularity. While PBASs can vary widely across sectors, they share three main components: goals, incentives, and measures. Research suggests that PBASs influence provider behaviors, but little is known about PBAS effectiveness at achieving their performance goals or about government and agency experiences. This document summarizes a study that examined nine PBASs in five sectors: child care, education, health care, public health emergency preparedness, and transportation. In the right circumstances, a PBAS can be an effective strategy for improving service delivery. Optimum circumstances include having a widely shared goal, unambiguous observable measures, meaningful incentives for those with control over the relevant inputs and processes, few competing interests, and adequate resources to design, implement, and operate the PBAS. However, these conditions are rarely fully realized, so it is difficult to design and implement PBASs that are uniformly effective. PBASs represent a promising policy option for improving the quality of service-delivery activities in many contexts. The evidence supports continued experimentation with and adoption of this approach in appropriate circumstances. Even so, PBAS design and its prospects for success depend on the context in which it will operate. Also, ongoing system evaluation and monitoring are integral components of a PBAS; they inform refinements that improve system functioning over time
Organizational improvement and accountability : lessons for education from other sectors( )

14 editions published between 2003 and 2004 in English and held by 2,496 WorldCat member libraries worldwide

The No Child Left Behind Act (NCLB) is a performance-based accountability system built around student test results. The accountability system comprises explicit educational goals, assessments for measuring the attainment of goals and judging success, and consequences (rewards or sanctions). But the mechanisms through which the system is intended to work are not well understood. The authors examined five accountability models: two from the manufacturing sector (the Malcolm Baldrige National Quality Award Program and the Toyota Production System (TPS)), a performance incentive model used in the evaluation of job training programs for the poor, accountability in the legal sector, accountability in health care as shown by clinical practice guidelines, use of statistical risk-adjustment methods, and the public reporting of health performance measures. Although education faces unique challenges, the authors conclude that educators can learn much from these other sectors. The Baldrige, TPS, and the clinical practice guidelines suggest the importance of focused institutional self-assessment, understanding school and district operations as a production process, being able to develop and apply a knowledge base about effective practice, and empowering participants in the process to contribute to improvement efforts. The job training and risk-adjustment models and the legal and health care accountability models provide specific guidance on how to enhance system-wide accountability in education by broadening performance measures; making sure performance goals are fair to all students and schools; developing standards of practice in promising areas; and encouraging professional accountability
Making sense of test-based accountability in education( )

14 editions published in 2002 in English and held by 2,226 WorldCat member libraries worldwide

Test-based accountability systems that attach high stakes to standardized test results in the form of rewards and sanctions have raised a number of issues for those responsible for educational assessment and accountability. Do these high-stakes tests measure student achievement accurately? How can policymakers and educators attach the right consequences to the results of these tests? What kinds of educational tradeoffs do these testing policies introduce? This book was written in response to school policymaking's growing emphasis on high-stakes testing. It addresses several key areas, including how the tests are used within these systems, how to evaluate the technical quality and trustworthiness of the tests, how test-based accountability affects the practices of teachers and schools, and what effect political considerations have on the policy debate. The authors also provide some recommendations for developing more-effective test-based accountability systems
Reauthorizing No Child Left Behind : facts and recommendations by Brian M Stecher( )

9 editions published in 2010 in English and held by 2,179 WorldCat member libraries worldwide

This report synthesizes findings and draws lessons about the implementation and results of the "No Child Left Behind Act of 2001" ("nclb") based primarily on two longitudinal studies funded by the U.S. Department of Education. Progress to date suggests that "NCLB's" ambitious goal of having 100 percent of U.S. students proficient in reading and mathematics by 2014 will not be met. In addition, the flexibility provided to states by the law has resulted in the establishment of a different accountability system in every state, each with different academic standards, levels of student proficiency, and teacher requirements. Parents have not responded in great numbers either to school choice or to receiving supplemental educational services options. Should Congress reauthorize "nclb", the authors recommend that it consider making the following changes to the law: promote more-uniform academic standards and teacher qualification requirements across states, set more-appropriate improvement targets, broaden the measures of student learning beyond multiple-choice tests in reading and mathematics to include more subjects and tests of higher-thinking and problem-solving skills, focus improvement efforts on all schools while continuing to offer parental choice, and provide incentives for highly qualified teachers to teach in low-performing schools. Appendices include: (1) Data Sources for This Report; and (2) Abstracts of Reports. Individual chapters contain footnotes. (Contains 1 figure.) [This paper was written with Paul Steinberg. This paper was prepared for the U.S. Department of Education. For the companion report, "What Can We Learn from the Implementation of "No Child Left Behind"? Research Brief", see ed509398.]
Analyzing the operation of performance-based accountability systems for public services by Frank A Camm( )

11 editions published in 2010 in English and held by 1,881 WorldCat member libraries worldwide

Empirical evidence of the effects of performance-based public management is scarce. This report describes a framework used to organize available empirical information on one form of performance-based management, a performance-based accountability system (PBAS). Such a system identifies individuals or organizations that must change their behavior for the performance of an activity to improve, chooses an implicit or explicit incentive structure to motivate these organizations or individuals to change, and then chooses performance measures tailored to inform the incentive structure appropriately. The study focused on systems in the child-care, education, health-care, public health emergency preparedness, and transportation sectors, mainly in the United States. Analysts could use this framework to seek empirical information in other sectors and other parts of the world. Additional empirical information could help refine existing PBASs and, more broadly, improve decisions on where to initiate new PBASs, how to implement them, and then how to design, manage, and refine them over time
Pain and gain : implementing No Child Left Behind in three states, 2004-2006 by Brian M Stecher( )

9 editions published in 2008 in English and held by 1,858 WorldCat member libraries worldwide

The Implementing Standards-Based Accountability "ISBA" study was designed to examine the strategies that states, districts, and schools are using to implement standards-based accountability under the No Child Left Behind Act "NCLB" and how these strategies are associated with classroom practices and student achievement in mathematics and science. This monograph presents the final results of the ISBA project. It contains descriptive information regarding the implementation of NCLB in California, Georgia, and Pennsylvania from 2003-2004 through 2005-2006. It is a companion to MG-589-NSF, Standards-Based Accountability Under No Child Left Behind "2007", and updates those findings with an additional year of data, permitting further analyses of state-to-state differences and longer-term trends. This study suggests that school improvement efforts might be more effective if they were responsive to local conditions and customized to address the specific causes of failure and the capacity of the schools in question
How to focus an evaluation by Brian M Stecher( Book )

42 editions published between 1987 and 1996 in English and held by 966 WorldCat member libraries worldwide

Deregulating school aid in California : How districts responded to flexibility in Tier 3 categorical funds in 2010-2011( )

4 editions published in 2012 in English and held by 599 WorldCat member libraries worldwide

California's system of school finance is highly regulated and prescriptive. A large share of state funding is allocated through categorical programs, that is, programs whose funding is contingent upon districts using the money in a particular way or for a particular purpose. In 2008-09, the strings were taken off 40 of those programs, collectively known as the Tier 3 programs, as part of a budget deal that also reduced the funding for those programs. The authors conducted a survey of 350 California school district chief financial officers (CFOs) between April and August of 2011 to see how district leaders responded to this sudden, limited fiscal flexibility and the conditions that shaped their decisions
Measuring hard-to-measure student competencies : a research and development plan by Brian M Stecher( )

5 editions published in 2014 in English and held by 596 WorldCat member libraries worldwide

Efforts to prepare students for college, careers, and civic engagement have traditionally emphasized academic skills, but a growing body of research suggests that interpersonal and intrapersonal competencies, such as communication and resilience, are important predictors of postsecondary success and citizenship. One of the major challenges in designing educational interventions to support these outcomes is a lack of high-quality measures that could help educators, students, parents, and others understand how students perform and monitor their development over time. This report provides guidelines to promote thoughtful development of practical, high-quality measures of interpersonal and intrapersonal competencies that practitioners and policymakers can use to improve valued outcomes for students
Competency-based education in three pilot programs : examining implementation and outcomes by Jennifer L Steele( )

5 editions published in 2014 in English and held by 588 WorldCat member libraries worldwide

"In 2011, the Bill & Melinda Gates Foundation created the Project Mastery grant program to support competency-based education initiatives in large school systems that serve a high proportion of disadvantaged youth. Competency-based education meets students where they are academically, provides students with opportunities for choice, and awards credit for evidence of learning, not for the time students spend studying a subject. The Foundation asked RAND to evaluate these efforts in terms of implementation, students' experiences, and student performance. This report presents final results from that evaluation, offering an overview of competency-based education and the Project Mastery grant projects and describing the implementation of competency-based educational features under each project. The report concludes with six lessons for policy, partnerships, and practice."--"Abstract" on web page
Improving teaching effectiveness : implementation : the Intensive Partnerships for Effective Teaching through 2013-2014 by Brian M Stecher( )

4 editions published in 2016 in English and Undetermined and held by 559 WorldCat member libraries worldwide

"To improve the U.S. education system through more-effective classroom teaching, in school year 2009--2010, the Bill and Melinda Gates Foundation announced the Intensive Partnership for Effective Teaching sites. The Intensive Partnerships Initiative is based on the premise that efforts to improve instruction can benefit from high-quality measures of teaching effectiveness. The initiative seeks to determine whether a school can implement a high-quality measure of teaching effectiveness and use it to support and manage teachers in ways that improve student outcomes. This approach is consistent with broader national trends in which performance-based teacher evaluation is increasingly being mandated at state and local levels. To test the theory in practice, the foundation sought partnership sites. It selected three school districts -- Hillsborough County Public Schools in Florida, Shelby County Schools in Tennessee, and Pittsburgh Public Schools in Pennsylvania. The foundation also selected four charter management organizations -- Alliance College-Ready Public Schools, Aspire Public Schools, Green Dot Public Schools, and the Partnerships to Uplift Communities, all in California. To evaluate Intensive Partnership implementation, researchers from the RAND Corporation and the American Institutes for Research interviewed annually central-office staff at each site and teachers and other staff in a sample of schools for each site. They also used data from annual teacher and school-leader surveys and documents that the sites and the foundation provided. This report summarizes the implementation status of key reform elements at each site when the Intensive Partnerships initiative launched and five years later in the spring of 2014."--
New assessments, better instruction? : designing assessment systems to promote instructional improvement : research report by Susannah Faxon-Mills( )

3 editions published in 2013 in English and held by 547 WorldCat member libraries worldwide

This report reviews the literature on how assessment affects teaching practice and the conditions that moderate that relationship. The authors identify a wide variety of effects that testing might have on teachers' activities in the classroom and a number of conditions that affect the impactthat assessment may have on practice
Improving mathematics and science education : a longitudinal investigation of the relationship between reform-oriented instruction and student achievement by Vi-Nhuan Le( )

2 editions published in 2006 in English and held by 347 WorldCat member libraries worldwide

The term reform-oriented teaching describes a collection of instructional practices that are designed to engage students as active participants in their own learning and to enhance the development of complex cognitive skills and processes. This monograph presents the findings of a multiyear National Science Foundation (NSF)-funded study of the effectiveness of reform-oriented science and mathematics instruction. It builds on an earlier RAND study, called the Mosaic project, which found ₃a weak but positive relationship₄ between reform-oriented practices and student achievement. The present study, called Mosaic II, extends this earlier research in two important ways. First, it incorporates more-diverse indicators of student exposure to reform-oriented practices, including innovative, vignette-based measures. Second, it follows students for three years in order to measure the relationship after longer exposure to reform practices
The effects of the Washington State education reform on schools and classrooms : initial findings( Book )

12 editions published in 2000 in English and held by 306 WorldCat member libraries worldwide

A number of states are implementing standards-based school accountability systems in efforts to improve student achievement. Washington state's Education Reform Act is an example of such a reform. It mandated the creation of academic standards, called the Essential Academic Learning Requirements (EALRs), a state assessment system, called the Washington Assessment of Student Learning (WASL), and an accountability mechanism, which is still under development. Researchers from RAND and the University of Colorado, Boulder are studying the implementation and impact of the Washington reforms on school and classroom practices, focusing on the subjects of writing and mathematics. The surveys revealed a picture of districts and schools in transition, with many changes occurring but not uniformly across classrooms or schools. Also, not all aspects of the reform were equally salient. Classroom changes appeared to be consistent with the EALRs, although local educators appeared to be responding most to the highly-visible WASL scores and making curriculum changes primarily in the WASL-tested subjects. Classroom-based assessment was not as widely understood or endorsed as the EALRs and WASL. Finally, the surveys paint a mixed picture about the relationship between school practices and WASL scores. On the one hand, after controlling for student background factors, WASL scores were higher in schools where teachers reported alignment between their curriculum and the EALRS and where teachers reported that they understood the EALRs and WASL. some of the responses raised questions about the meaningfulness of WASL scores and score gains as indicators of attainment of the standards. Most teachers believed that test preparation was responsible for the majority of score gains. Many teachers thought that the WASL was not appropriate for the grade levels at which it was administered. These issues deserve further scrutiny as the accountability system is developed
Improving Perkins II performance measures and standards : lessons learned from early implementers in four states by Brian M Stecher( Book )

7 editions published between 1994 and 1995 in English and held by 262 WorldCat member libraries worldwide

The Carl D. Perkins Vocational and Applied Technology Education Act of 1990 required states to develop outcome-based accountability systems built around statewide performance measures and standards. States were given considerable flexibility in selecting outcomes to be measured, the measurement tools, and the standards; their systems were to be in place by the fall of 1992. To assess the progress states have made in the first three years, the authors examined the programs in four states that were among the first to begin implementation
Using alternative assessments in vocational education by Brian M Stecher( Book )

8 editions published between 1996 and 1997 in English and Undetermined and held by 204 WorldCat member libraries worldwide

This volume covers a study of the utility of alternative (i.e., non-multiple choice) forms of student assessment for vocational education, the primary objective of which was to describe various alternative assessments currently available and offer criteria for choosing among them. Six operational programs were used as case studies; each employs one or more constructed-response measures, and together they represent a range of assessment options. Each was critically reviewed based on descriptive materials, research literature, interviews, and, in four cases, site visits. The evaluation focused on comparing the different assessments' quality (reliability, validity, fairness), feasibility (cost, time, complexity, credibility), and potential usefulness for vocational educators. Design and implementation issues important to any decisions about using alternative assessments were determined (e.g., the need to clarify the assessment's purpose), as well as related issues (e.g., the relative advantages of low versus high stakes, voluntary versus mandatory participation). Illustrations are given of how vocational educators can use this review as an aid in determining the usefulness of the alternatives for a particular situation
Performance assessments in science : hands-on tasks and scoring guides by Brian M Stecher( Book )

6 editions published in 1996 in English and held by 181 WorldCat member libraries worldwide

Recent reforms of science curricula emphasize problem solving and conceptual understanding rather than knowledge of facts and formulas. Many educators argue for a corresponding change in testing. Specifically, multiple-choice tests, the traditional measures of achievement, should be replaced by assessment methods such as hands-on science exercises and other performance-oriented tasks that are better suited to measuring conceptual understanding and problem-solving ability. This report documents the hands-on science tasks and scoring guides that were developed as part of a RAND study of the technical quality and feasibility of science performance assessment in large-scale testing programs. The document contains a complete description of each of the 12 tasks used in the study, including the task blueprint (or "shell") from which the task was developed, the task booklet, photographs or copies of all the equipment and apparatus that accompanied the task, and the task scoring guide. The information should interest researchers who want to use these tasks in other studies and policymakers who want to understand more about using performance exercises for assessment at the local level
Issues in building an indicator system for mathematics and science education by Brian M Stecher( Book )

4 editions published in 1996 in English and held by 173 WorldCat member libraries worldwide

This report examines one option for developing an indicator system within the National Science Foundation (NSF) to monitor the success or failure of the education system in mathematics and science. This option, referred to as the "patchwork" model, was one of five examined in a precursor 1987 RAND study. The patchwork model is assessed to test the feasibility of a system that includes three key features of math and science education: (1) student achievement, (2) secondary curriculum, and (3) secondary teacher workforce. This study provides lessons about indicators, presents principles for an ideal indicator system, and helps shape recommendations to the NSF in its role as sponsor and user of educational indicators. The efficacy of the patchwork approach is questioned, with a recommendation that NSF adopt a "supplementary" approach that combines obtaining more complete data with commissioning periodic additional data collection to provide longitudinal measures, secondary sources, and in-depth data
External audiences for test-based accountability : the perspectives of journalists and foundations by Laura S Hamilton( Book )

5 editions published in 2004 in English and held by 144 WorldCat member libraries worldwide

The No Child Left Behind Act (NCLB) requires states to adopt test-based accountability systems designed to promote improved achievement by setting clear goals for student learning, measuring students' progress toward those goals, and holding educators accountable for this progress through the dissemination of results and the imposition of rewards, sanctions, and assistance. Although much has been written about the law, little has been written about the interaction of NCLB with external audiences that do not have formal connections with the schools. This study focuses on two key external audiences that are involved in public education and that frequently use data from schools and districts in their work--print journalists and foundation program officers. The authors use insights from conversations with journalists and program officers to identify ways to improve the utility of information from test-based accountability systems. They summarize the data needs of the two groups and the problems they encounter when trying to interpret test results and offer advice to the producers of performance reports and analyses (states, school districts, and researchers) and to the external consumers of those reports (journalists and program officers) on how to enhance the utility of the information
Working smarter to leave no child behind : practical insights for school leaders by Brian M Stecher( Book )

3 editions published in 2003 in English and held by 141 WorldCat member libraries worldwide

Accountability based on student test results is the heart of the No Child Left Behind Act, which assumes that schools can solve their problems if given proper incentives and technical assistance, and that parents should have educational alternatives if the schools fail to do so. This paper discusses evidence on accountability systems, describes the act's accountability model and its underlying assumptions, and offers educators specific recommendations for effective operation in an era of accountability
moreShow More Titles
fewerShow Fewer Titles
Audience Level
Audience Level
  Kids General Special  
Audience level: 0.14 (from 0.04 for Pain and g ... to 0.61 for External a ...)

Making sense of test-based accountability in education
Alternative Names
Stecher, Brian.

Stecher, Brian 1946-

English (184)

Organizational improvement and accountability : lessons for education from other sectorsMaking sense of test-based accountability in educationReauthorizing No Child Left Behind : facts and recommendationsAnalyzing the operation of performance-based accountability systems for public servicesPain and gain : implementing No Child Left Behind in three states, 2004-2006How to focus an evaluationImproving mathematics and science education : a longitudinal investigation of the relationship between reform-oriented instruction and student achievementImproving Perkins II performance measures and standards : lessons learned from early implementers in four states