The heart of NYU Wagner's programs is our faculty. An amalgam of full-time, clinical/research/visiting, and adjunct professors, they are outstanding teachers, expert researchers and committed practitioners.
Enter a search term below to find relevant videos, podcasts, publications and events
Ryan, Andrew M., Jan Blustein, Lawrence P. Casalino. Medicareâ€™s Flagship Test Of Pay-For-Performance Did Not Spur More Rapid Quality Improvement Among Low-Performing Hospitals. Health Affairs; 31(4):797-805.
Medicare’s flagship hospital pay-for-performance program, the Premier Hospital Quality Incentive Demonstration, began in 2003 but changed its incentive design in late 2006. The goals were to encourage greater quality improvement, particularly among lower-performing hospitals. However, we found no evidence that the change achieved these goals. Although the program changes were intended to provide strong incentives for improvement to the lowest-performing hospitals, we found that in practice the new incentive design resulted in the strongest incentives for hospitals that had already achieved quality performance ratings just above the median for the entire group of participating hospitals. Yet during the course of the program, these hospitals improved no more than others. Our findings raise questions about whether pay-for-performance strategies that reward improvement can generate greater improvement among lower performing providers. They also cast some doubt on the extent to which hospitals respond to the specific structure of economic incentives in pay-for-performance programs.
Aber, L., Brown, J.L, & Jones, S.M., Berg, J. & C. Torrente. School-based strategies to prevent violence, trauma and psychopathology: The challenges of going to scale. Development and Psychopathology, 23(2), 411-421.
Children's trauma-related mental health problems are widespread, largely untreated and constitute significant barriers to academic achievement and attainment. Translational research has begun to identify school-based interventions to prevent violence, trauma and psychopathology. We describe in detail the findings to date on research evaluating one such intervention, the Reading, Writing, Respect, and Resolution (4Rs) Program. The 4Rs Program has led to modest positive impacts on both classrooms and children after 1 year that appear to cascade to more impacts in other domains of children's development after 2 years. This research strives not only to translate research into practice but also translate practice into research. However, considerable challenges must be met for such research to inform prevention strategies at population scale.
Brown, J.L., Jones, S.M., LaRusso, M.D., & J.L. Aber. Improving Classroom Quality: Teacher Influences and Experimental Impacts of the 4Rs Program. Journal of Educational Psychology, 102(1), 153-167.
This study capitalizes on recent advances in the reliable and valid measurement of classroom-level social processes known to influence children's social–emotional and academic development and addresses a number of limitations in our current understanding of teacher- and intervention-related impacts on elementary school classroom processes. A cluster randomized controlled trial design was employed to (a) examine whether teacher social–emotional functioning forecasts differences in the quality of 3rd-grade classrooms, (b) test the experimental impact of a school-based social–emotional learning and literacy intervention on the quality of classroom processes controlling for teacher social–emotional functioning, and (c) examine whether intervention impacts on classroom quality are moderated by these teacher-related factors. Results indicated (a) positive effects of teachers' perceived emotional ability on classroom quality; (b) positive effects of the 4Rs Program on overall classroom quality, net of teacher social–emotional functioning indicators; and (c) intervention effects that are robust to differences in these teacher factors. These findings support and extend recent research examining intervention-induced changes in classroom-level social processes fundamental to positive youth development.
Jones, S.M., Brown, J.L, Hoglund, W.L.G., & J.L. Aber. A School-Randomized Clinical Trial of an Integrated Social-Emotional Learning and Literacy Intervention: Impacts on Third-Grade Outcomes. Journal of Consulting and Clinical Psychology, 78(6): 829-842.
Objective: To report experimental impacts of a universal, integrated school-based intervention in social–emotional learning and literacy development on change over 1 school year in 3rd-grade children's social–emotional, behavioral, and academic outcomes. Method: This study employed a school-randomized, experimental design and included 942 3rd-grade children (49% boys; 45.6% Hispanic/Latino, 41.1% Black/African American, 4.7% non-Hispanic White, and 8.6% other racial/ethnic groups, including Asian, Pacific Islander, Native American) in 18 New York City public elementary schools. Data on children's social–cognitive processes (e.g., hostile attribution biases), behavioral symptomatology (e.g., conduct problems), and literacy skills and academic achievement (e.g., reading achievement) were collected in the fall and spring of 1 school year. Results: There were main effects of the 4Rs Program after 1 year on only 2 of the 13 outcomes examined. These include children's self-reports of hostile attributional biases (Cohen's d = 0.20) and depression ( d = 0.24). As expected based on program and developmental theory, there were impacts of the intervention for those children identified by teachers at baseline with the highest levels of aggression ( d = 0.32–0.59) on 4 other outcomes: children's self-reports of aggressive fantasies, teacher reports of academic skills, reading achievement scaled scores, and children's attendance. Conclusions: This report of effects of the 4Rs intervention on individual children across domains of functioning after 1 school year represents an important first step in establishing a better understanding of what is achievable by a schoolwide intervention such as the 4Rs in its earliest stages of unfolding. The first-year impacts, combined with our knowledge of sustained and expanded effects after a second year, provide evidence that this intervention may be initiating positive developmental cascades both in the general population of students and among those at highest behavioral risk.
This study investigates the degree to which the racial composition of the school environment may influence the body mass index (BMI) of children aged 10 to 18 years. This research may be viewed as extending prior work that has found that the prevalence of risk behaviors among nonwhite adolescents is influenced by exposure to white adolescents.
This research used data from the Survey of Adults and Youth, which was conducted as part of the evaluation of the Robert Wood Johnson Foundation's Urban Health Initiative. The study population for this analysis is comprised of parent and child respondents in the 2004 to 2005 survey wave who lived in one of the five program cities: Baltimore, Detroit, Oakland, Philadelphia, and Richmond. We constructed two-level school random effects models and added school and census tract-level variables that describe the racial composition of the residential community and the school attended.
Black and Hispanic adolescent girls who attend schools with a mostly nonwhite student body have higher BMIs than do their white counterparts. However, black girls in predominately white schools do not have higher BMIs than white girls. Further, black and Hispanic girls whose schoolmates are predominately white have significantly lower BMIs than black and Hispanic girls in schools where fewer than half the students are white. These associations are not found among boys, and are net of a broad variety of individual, household, and group level characteristics.
Our findings suggest that the BMI of minority adolescent girls is influenced by the norms of the social environment.
Weitzman BC, Mijanovich T, Silver D, and C Brecher Finding the Impact in a Messy Intervention: Using an Integrated Design to Evaluate a Comprehensive Citywide Health Initiative. American Journal of Evaluation 30 (2009), pp. 495 - 514.
This article uses the evaluation of the Robert Wood Johnson Foundation’s (RWJF) Urban Health Initiative (UHI), a 10-year effort to improve health and safety outcomes in distressed cities, to demonstrate the strength of an evaluation design that integrates theory of change and quasiexperimental
approaches, including the use of comparison cities. This paper focuses on the later stages of implementation and, especially, our methods for estimating program impacts. While the theory of change was used to make preliminary identification of intended outcomes, we used the sites’ plans and early implementation to refine this list and revisit our strategy for estimating impacts. Using our integrated design, differences between program and comparison cities are considered impacts only if they were predicted by program theory, local plans for action, and early implementation. We find small, measurable changes in areas of greatest programmatic effort. We discuss the importance of the integrated design in identifying impacts.
Berry, C., Krutz, G.S., Langner, B. & Budetti, P. Jump-Starting Collaboration: The ABCD Initiative and the Provision of Child Development Services through Medicaid and Collaborators. Public Administration Review, May 2008, Vol. 68 Issue 3, p480-490, 11p.
Many policy problems require governmental leaders to forge vast networks beyond their own hierarchical institutions. This essay explores the challenges of implementation in a networked institutional setting and incentives to induce coordination between agencies and promote quality implementation. It describes the national evaluation of the Assuring Better Child Health and Development program, a state-based program intended to increase and enhance the delivery of child development services for low-income children through the health care sector, using Medicaid as its primary vehicle. Using qualitative evaluation methods, the authors found that all states implemented programs that addressed their stated goals and made changes in Medicaid policies, regulations, or reimbursement mechanisms. The program catalyzed interagency cooperation and coordination. The authors conclude that even a modest level of external support and technical assistance can stimulate significant programmatic change and interorganizational linkages within public agencies to enhance provision of child development services.
Blustein, J., Regenstein, M., Seigel, B. & Billings, J. Notes from the Field: Jumpstarting the IRB Approval Process in Multicenter Studies. Health Services Research, Volume 42, Number 4, August 2007 , pp. 1773-1782(10) Blackwell Publishing.
Objective. To identify strategies that facilitate readiness for local Institutional Review Board (IRB) review, in multicenter studies.
Study Setting. Eleven acute care hospitals, as they applied to participate in a foundation-sponsored quality improvement collaborative.
Study Design. Case series.
Data Collection/Extraction. Participant observation, supplemented with review of written and oral communications.
Principal Findings. Applicant hospitals responded positively to efforts to engage them in early planning for the IRB review process. Strategies that were particularly effective were the provisions of application templates, a modular approach to study description, and reliance on conference calls to collectively engage prospective investigators, local IRB members, and the evaluation/national program office teams. Together, these strategies allowed early identification of problems, clarification of intent, and relatively timely completion of the local IRB review process, once hospitals were selected to participate in the learning collaborative.
Conclusions. Engaging potential collaborators in planning for IRB review may help expedite and facilitate review, without compromising the fairness of the grant-making process or the integrity of human subjects protection.
Macinko, J. Guanais, F. & Souza, F. An Evaluation of the Impact of the Family Health Program on Infant Mortality in Brazil, 1990-2002. Journal of Epidemiology and Community Health, .
Objective: To use publicly available secondary data to assess the impact of Brazil's Family Health Program on state level infant mortality rates (IMR) during the 1990s.
Design: Longitudinal ecological analysis using panel data from secondary sources. Analyses controlled for state level measures of access to clean water and sanitation, average income, women's literacy and fertility, physicians and nurses per 10 000 population, and hospital beds per 1000 population. Additional analyses controlled for immunisation coverage and tested interactions between Family Health Program and proportionate mortality from diarrhoea and acute respiratory infections.
Setting: 13 years (1990-2002) of data from 27 Brazilian states.
Main results: From 1990 to 2002 IMR declined from 49.7 to 28.9 per 1000 live births. During the same period average Family Health Program coverage increased from 0% to 36%. A 10% increase in Family Health Program coverage was associated with a 4.5% decrease in IMR, controlling for all other health determinants (p<0.01). Access to clean water and hospital beds per 1000 were negatively associated with IMR, while female illiteracy, fertility rates, and mean income were positively associated with IMR. Examination of interactions between Family Health Program coverage and diarrhoea deaths suggests the programme may reduce IMR at least partly through reductions in diarrhoea deaths. Interactions with deaths from acute respiratory infections were ambiguous.
Conclusions: The Family Health Program is associated with reduced IMR, suggesting it is an important, although not unique, contributor to declining infant mortality in Brazil. Existing secondary datasets provide an important tool for evaluation of the effectiveness of health services in Brazil.
Blustein, J. Toward a More Public Discussion of the Ethics of Federal Social Program Evaluation. Journal of Policy Analysis and Management, Vol. 24, No. 4, pp 824-852.
Federal social program evaluation has blossomed over the past quarter century. Despite this growth, there has been little accompanying public debate on research ethics. This essay explores the origins and the implications of this relative silence on ethical matters. It reviews the federal regulations that generally govern research ethics, and recounts the history whereby the evaluation of federal programs was specifically exempted from the purview of those regulations. Through a discussion of a recent evaluation that raised ethical concerns, the essay poses - but does not answer - three questions: (1) Are there good reasons to hold federal social program evaluations to different standards than those that apply to other research?; (2) If so, what ethical standards should be used to access such evaluations?; and (3) Should a formal mechanism be developed to ensure that federal social program evaluations are conducted ethically?
Brecher, C., Silver, D. & Weitzman, B.C. Following the Money: Using Expenditure Analysis as an Evaluation Tool. American Journal of Evaluation, Volume 26, Number 2, 150-165.
This article describes the nature and utility of fiscal analysis in evaluating complex community interventions. Using New York University's evaluation of the Robert Wood Johnson Foundation's Urban Health Initiative as an example, the authors describe issues arising in defining and operationalizing constructs for fiscal analysis. The approach's utility is demonstrated in the use of interim findings to help redefine the program's goals for resource allocation, to modify its theory of change to include greater emphasis on state-level action, and to emphasize the importance of local public schools as resource centers and intervention targets. The fiscal analysis also provides new insights into the limitations of "preventive" versus "corrective" spending categories and helps make goals for such functional reallocation more realistic. The authors discuss limitations of fiscal analysis due to available data quality, the extent of cooperation needed from public officials to collect relevant data, and the level of expertise needed to interpret the data.
I argue for thinking of program evaluation as a decision problem. There are two steps. First, a counselor determines which program (treatment or control) each individualjoins,based for example on maximizing the probability of employment or expected earnings.
Second, the policymaker decides whether: to assign all individuals to treatment or to control, or to allow the counselor to choose.This framework has two advantages. Individualized assignment rules (known as profiling) can raise the average impact, improving cost effectiveness by exploiting treatment-impact heterogeneity. Second, it accounts systematically for inequality and uncertainty, and the policymaker's attitude toward these, in the evaluation.
Kaplan, S.A. & Garrett, K.E. The Use of Logic Models by Community-Based Initiatives. Evaluation and Program Planning 2005; 28:167-172.
Tuli, K. & Sansom, S., Purcell, D.W., Metsch, L.R., Latkin, C.A., Gourevitch, M.N. & Gomez, C.A. Economic Evaluation of an HIV Prevention Intervention for Seropositive Injection Drug Users. Journal of Public Health Management & Practice, Nov/Dec 2005, Vol. 11 Issue 6, p508-515, 8p.
To assess the cost-effectiveness of Intervention for HIV-Seropositive injection drug users-Research and Evaluation (INSPIRE), designed to reduce risky sexual and needle-sharing behaviors in research sites in four US cities (2001-2003). Methods: We collected data on program and participant costs. We used a mathematical model to estimate the number of sex partners of injection drug users expected to become infected with human immunodeficiency virus (HIV) (with and without intervention), cost of treatment for sex partners who became infected, and the effect of infection on partners' quality-adjusted life expectancy. We determined the minimum effect that INSPIRE must have on condom use among participants for the intervention to be cost-saving (intervention cost less than savings from averted HIV infections) or cost-effective (net cost per quality-adjusted life year saved less than $50,000). Results: The intervention cost was $870 per participant. It would be cost-saving if it led to 53 percent reduction in the proportion of participants who had any unprotected sex in 1 year and cost-effective with 17 percent reduction. If behavior change lasted 3 months, the cost-effectiveness threshold was 66 percent; if 3 years, the threshold was 6 percent. Conclusions: Although cost-saving thresholds may not be achievable by the intervention, we anticipate that cost-effectiveness thresholds will be attained.
Brecher, C., Searcy, C., Silver, D. & Weitzman, B.C. What Does Government Spend on Children? Evidence from Five Cities. Brookings Institute, Center on Urban and Metropolitan Policy, March, .
This paper examines public spending on children between 1997 and 2000 in five localities: Baltimore, Detroit, Oakland, Philadelphia, and Richmond. These cities participate in the Urban Health Initiative (UHI), a ten-year Robert Wood Johnson Foundation program aimed at improving health and safety for young people in these cities. The Center for Health and Public Service Research at New York University is evaluating the program. The evaluation seeks to determine whether collaborative efforts of interested organizations can develop and implement plans to change service delivery systems for children, and whether such changes result in better outcomes for children. A group of ten additional cities serves as a comparative benchmark.
Brown, J.L., Roderick, T., Lantieri, L. & Aber, J.L. The Resolving Conflict Creatively Program: A School-Based Social and Emotional Learning Program. In J.E. Zins, R.P. Weissberg, M.C. Wang, & H.J. Walberg (Eds.), Building academic success on social and emotional learning: What does the research say? (pp.151-169). New York, NY: Teachers College Press, .
The Resolving Conflict Creatively Program (RCCP) is one of the oldest and largest school-based conflict resolution programs in the United States. Beginning in 1994, we planned and implemented a rigorous scientific evaluation of the RCCP, involving over 350 teachers and 11,000 children from 15 public elementary schools in New York City. In this chapter, we describe the RCCP, explain the rationale for and design of the study, summarize the major results related to the program's impact on children's trajectories of social and emotional learning (SEL) and academic achievement, and discuss the implications of these findings for research, practice, and policy.
Aber, J.L., Brown, J.L., Gershoff, E.T., Jones, S.M. & Pedersen, S.F.A. Changing Children's Trajectories of Development: Two-Year Evidence for the Effectiveness of a School-Based Approach to Violence Prevention. National Center for Children in Poverty, New York, NY.
Awareness of youth violence has increased in recent years, resulting in more interest in programs that can prevent violent and aggressive behavior. Although overall rates of violence among young people have declined since the mid-1990s, rates of some forms of youth aggression, violence, and crime remain high. National data reveal that, each year, about 15 percent of high school students are involved in a physical fight at school and 8 percent are threatened or injured with a weapon. 1 Urban youth are at particular risk for exposure to violence and victimization.
This report describes one of the largest and longest running school-based violence prevention programs in the country-the Resolving Conflict Creatively Program (RCCP)-and discusses the results of a rigorous evaluation conducted by the National Center for Children in Poverty (NCCP) at Columbia University's Mailman School of Public Health. The evaluation provides concrete evidence that early, school-based prevention initiatives such as the RCCP can work and should be included in communities' efforts to prevent violence among children and youth.
This paper discusses the problem of evaluating and predicting the treatment impact of a program that is implemented at multiple sites. Two issues arise: is information from other sites relevant in estimating the impact at a given site? and how can we account for predictive uncertainty in the site effects? Using data from the Greater Avenues for Independence evaluation, I develop a hierarchical model for earnings which allows both for site effects and for smoothing the estimated impact across sites.
I show that the degree to which the estimated impact is smoothed across sites does not affect the estimate; i.e. most of the differences across sites are due to differences in the composition of participants. Second I show that predictive uncertainty regarding site effects is important; for example, when the predictive uncertainty regarding site effects is ignored, the treatment impact at the Riverside sites is significant, but when we consider predictive uncertainty, the impact for the Riverside sites is insignificant.
Third, I show that the hierarchical model is able to extrapolate site effects with reasonable accuracy when the site for which the prediction is being made does not differ substantially from the sites already observed. For example, the San Diego treatment effects could have been predicted based on observable site characteristics, but the Riverside effects are consistently underestimated.
Ellen, I.G. & Turner, M. What Have We Learned from HUDs Moving to Opportunity Program? In John M. Goering and Judith D. Feins, eds., Choosing a Better Life? A Social Experiment in Leaving Poverty Behind: Evaluation of the Moving to Opportunity Program. Washington, DC: Urban Institute Press, .
As the centerpiece of policymakers' efforts to "deconcentrate" poverty in urban America, the Moving to Opportunity (MTO) project gave roughly 4,600 volunteer families the chance to move out of public housing projects in deeply impoverished neighborhoods in five cities-Baltimore, Boston, Chicago, Los Angeles, and New York. Researchers wanted to find out to what extent moving out of a poor neighborhood into a better-off area would improve the lives of public housing families. Choosing a Better Life? is the first distillation of years of research on the MTO project, the largest rigorously designed social experiment to investigate the consequences of moving low-income public housing residents to low-poverty neighborhoods. In this book, leading social scientists and policy experts examine the legislative and political foundations of the project, analyze the effects of MTO on lives of the families involved, and explore lessons learned from this important piece of U.S. social policy.
Yoshikawa, H., Wilson, P.A., Hsueh, J., Rosman, E.A., Kim, J. & Chin, J.. What Frontline CBO Staff Can Tell Us About Culturally Anchored Theories of Change in HIV Prevention for Asian/Pacific Islanders. American Journal of Community Psychology,Volume 32, pp. 143-158.
Weitzman, B.C., Silver, D. & Dillman, K. Integrating a Comparison Group Design into a Theory of Change Evaluation: The Case of the Urban Health Initiative. American Journal of Evaluation 23:4 (Dec 2002), pp 371-385.
Fritzen, Scott, Stanely J. Final Evaluation: The Rural Infrastructure Development Program. UNDP and UNCDF.
Aber, J.L., Jones, S.M., Brown, J.L., Chaudry, N. & Samples, F. Resolving Conflict Creatively: Evaluating the Developmental Effects of a School-Based Violence Prevention Program in Neighborhood and Classroom Context. Development and Psychopathology, 10(2), 187-213.
Fritzen, Scott. Smallholder livelihoods in the balance: an evaluation of the Ba Che Natural Resource Management Program. Save the Children Federation, United Kingdom.
Aber, J.L., Brown, J.L., Chaudry, N., Jones, S.M. & Samples, F. The Evaluation of the Resolving Conflict Creatively Program: An Overview. American Journal of Preventive Medicine, Supplement to Volume 12(5), 82-90.
Sparrow, R. & McKnight, C.E., Chen, L.S. & Diggins, L. Evaluation of the New Jersey Department of Transportation. Transportation Management Association Grant Program. Report prepared for New Jersey DOT, prepared under a grant from the University Transportation Research Center. May.
|Alumni in Performance Measurement & Management Affinity Group Meeting||02/22/2012|
|Race and Savings with Darrick Hamilton and Caitlyn Brazill: Race and the Wealth Gap Series, Part 2||12/07/2011|
|Doctoral Colloquium - Fall 2011 - "Anatomy of Welfare Reform Evaluation: Announcement and Implementation Effects"||10/20/2011|
|The Power of Compassion: Book Discussion on THE LIFE YOU CAN SAVE and Interview with author Peter Singer||04/26/2010|
|UN Millennium Development Goals: 10 years after, 5 years to go||04/15/2010|
|Evaluating Leadership Development Programs: How to Clarify Results and Tell the Story of Program Impact||06/25/2009|
|P11.2171||Program Analysis and Evaluation|