Evidence-Based Management

What Passes and Fails as Health Policy and Management

What Passes and Fails as Health Policy and Management
Journal of Health Politics, Policy and Law, Vol. 39, No. 5, October 2014 DOI 10.1215/03616878-2813719

Rodwin, Victor G. and David Chinitz
07/18/2014

The field of health policy and management (HPAM) faces a gap between theory, policy and practice. Despite decades of efforts at reforming health policy and health care systems, prominent analysts state that the health system is ‘‘stuck’’ and that models for change remain ‘‘aspirational.’’ We discuss four reasons for the failure of current ideas and models for redesigning health care: (1) the dominance of microeconomic thinking; (2) the lack of comparative studies of health care organizations and the limits of health management theory in recognizing the importance of local contexts; (3) the separation of HPAM from the rank and file of health care, particularly physicians; and (4) the failure to expose medical students to issues of HPAM. We conclude with suggestions for rethinking how the field of HPAM might generate more promising policies for health care providers and managers by abandoning the illusion of context-free theories and, instead, seeking to facilitate the processes by which organizations can learn to improve their own performance.

Evidence-Based Management: Implications for Nonprofit Organizations

Evidence-Based Management: Implications for Nonprofit Organizations
Nonprofit Management and Leadership, Spring 2014, 24(3): 417–424. doi: 10.1002/nml.21097

Kovner, A. R.
01/09/2014

The article reviews evidence-based management and its implications for practice and teaching. My focus is on strategic decision making in nonprofit organizations. Evidence-based management is a process that includes framing the question, finding evidence, assuring accuracy, applicability, and actionability of evidence until the evidence is the best available.

Training Your Own: The Impact of New York City’s Aspiring Principals Program on Achievement

Training Your Own: The Impact of New York City’s Aspiring Principals Program on Achievement
Educational Evaluation and Policy Analysis, 34(2): 232-253.

Corcoran, S. P., Schwartz, A. E., & Weinstein
09/13/2013

The New York City Leadership Academy represents a unique experiment by a large urban school district to train and develop its own school leaders. Its 14-month Aspiring Principals Program (APP) selects and prepares aspiring principals to lead low-performing schools. This study provides the first systematic evaluation of achievement in APP-staffed schools after 3 or more years. We examine differences between APP principals and those advancing through other routes, the extent to which APP graduates serve and remain in schools, and their relative performance in mathematics and English language arts. On balance, we find that APP principals performed about as well as other new principals. If anything, they narrowed the gap with comparison schools in English language arts but lagged behind in mathematics.

Development of a predictive model to identify inpatients at risk of re-admission within 30 days of discharge (PARR-30)

Development of a predictive model to identify inpatients at risk of re-admission within 30 days of discharge (PARR-30)
BMJ Open 2012;2:e001667 doi:10.1136/bmjopen-2012-001667

John Billings, Ian Blunt, Adam Steventon, Theo Georghiou, Geraint Lewis, Martin Bardsley
08/10/2012

Objectives To develop an algorithm for identifying inpatients at high risk of re-admission to a National Health Service (NHS) hospital in England within 30 days of discharge using information that can either be obtained from hospital information systems or from the patient and their notes.

Design Multivariate statistical analysis of routinely collected hospital episode statistics (HES) data using logistic regression to build the predictive model. The model's performance was calculated using bootstrapping.

Setting HES data covering all NHS hospital admissions in England.

Participants The NHS patients were admitted to hospital between April 2008 and March 2009 (10% sample of all admissions, n=576 868).

Main outcome measures Area under the receiver operating characteristic curve for the algorithm, together with its positive predictive value and sensitivity for a range of risk score thresholds.

Results The algorithm produces a ‘risk score’ ranging (0–1) for each admitted patient, and the percentage of patients with a re-admission within 30 days and the mean re-admission costs of all patients are provided for 20 risk bands. At a risk score threshold of 0.5, the positive predictive value (ie, percentage of inpatients identified as high risk who were subsequently re-admitted within 30 days) was 59.2% (95% CI 58.0% to 60.5%); representing 5.4% (95% CI 5.2% to 5.6%) of all inpatients who would be re-admitted within 30 days (sensitivity). The area under the receiver operating characteristic curve was 0.70 (95% CI 0.69 to 0.70).

Conclusions We have developed a method of identifying inpatients at high risk of unplanned re-admission to NHS hospitals within 30 days of discharge. Though the models had a low sensitivity, we show how to identify subgroups of patients that contain a high proportion of patients who will be re-admitted within 30 days. Additional work is necessary to validate the model in practice.

Effect of telehealth on use of secondary care and mortality: findings from the Whole System Demonstrator cluster randomised trial

Effect of telehealth on use of secondary care and mortality: findings from the Whole System Demonstrator cluster randomised trial
BMJ 2012;344:e3874

Steventon A, Bardsely M, Billings J., et al
06/21/2012

Objective: To assess the effect of home based telehealth interventions on the use of secondary healthcare and mortality.

Design: Pragmatic, multisite, cluster randomised trial comparing telehealth with usual care, using data from routine administrative datasets. General practice was the unit of randomisation. We allocated practices using a minimisation algorithm, and did analyses by intention to treat.

Setting: 179 general practices in three areas in England.

Participants: 3230 people with diabetes, chronic obstructive pulmonary disease, or heart failure recruited from practices between May 2008 and November 2009.

Interventions: Telehealth involved remote exchange of data between patients and healthcare professionals as part of patients’ diagnosis and management. Usual care reflected the range of services available in the trial sites, excluding telehealth.

Main outcome measure: Proportion of patients admitted to hospital during 12 month trial period.

Results: Patient characteristics were similar at baseline. Compared with controls, the intervention group had a lower admission proportion within 12 month follow-up (odds ratio 0.82, 95% confidence interval 0.70 to 0.97, P=0.017). Mortality at 12 months was also lower for intervention patients than for controls (4.6% v 8.3%; odds ratio 0.54, 0.39 to 0.75, P<0.001). These differences in admissions and mortality remained significant after adjustment. The mean number of emergency admissions per head also differed between groups (crude rates, intervention 0.54 v control 0.68); these changes were significant in unadjusted comparisons (incidence rate ratio 0.81, 0.65 to 1.00, P=0.046) and after adjusting for a predictive risk score, but not after adjusting for baseline characteristics. Length of hospital stay was shorter for intervention patients than for controls (mean bed days per head 4.87 v 5.68; geometric mean difference −0.64 days, −1.14 to −0.10, P=0.023, which remained significant after adjustment). Observed differences in other forms of hospital use, including notional costs, were not significant in general. Differences in emergency admissions were greatest at the beginning of the trial, during which we observed a particularly large increase for the control group.

Conclusions: Telehealth is associated with lower mortality and emergency admission rates. The reasons for the short term increases in admissions for the control group are not clear, but the trial recruitment processes could have had an effect.

Medicare’s Flagship Test Of Pay-For-Performance Did Not Spur More Rapid Quality Improvement Among Low-Performing Hospitals

Medicare’s Flagship Test Of Pay-For-Performance Did Not Spur More Rapid Quality Improvement Among Low-Performing Hospitals
Health Affairs; 31(4):797-805.

Ryan, Andrew M., Jan Blustein, Lawrence P. Casalino.
04/01/2012

Medicare’s flagship hospital pay-for-performance program, the Premier Hospital Quality Incentive Demonstration, began in 2003 but changed its incentive design in late 2006. The goals were to encourage greater quality improvement, particularly among lower-performing hospitals. However, we found no evidence that the change achieved these goals. Although the program changes were intended to provide strong incentives for improvement to the lowest-performing hospitals, we found that in practice the new incentive design resulted in the strongest incentives for hospitals that had already achieved quality performance ratings just above the median for the entire group of participating hospitals. Yet during the course of the program, these hospitals improved no more than others. Our findings raise questions about whether pay-for-performance strategies that reward improvement can generate greater improvement among lower performing providers. They also cast some doubt on the extent to which hospitals respond to the specific structure of economic incentives in pay-for-performance programs.

The Role of Matched Controls In Building An Evidence Base For Hospital Avoidance Schemes: A Retrospective Evaluation

The Role of Matched Controls In Building An Evidence Base For Hospital Avoidance Schemes: A Retrospective Evaluation
Health Services Research, 47: 1679–1698. doi: 10.1111/j.1475-6773.2011.01367.x

Steventon, A., Bardsley, M., Billings, J., Georghiou, T. and Lewis, G. H.
01/06/2012

Objective
To test whether two hospital-avoidance interventions altered rates of hospital use: “intermediate care” and “integrated care teams.”

Data Sources/Study Setting
Linked administrative data for England covering the period 2004 to 2009.

Study Design
This study was commissioned after the interventions had been in place for several years. We developed a method based on retrospective analysis of person-level data comparing health care use of participants with that of prognostically matched controls.

Data Collection/Extraction Methods
Individuals were linked to administrative datasets through a trusted intermediary and a unique patient identifier.

Principal Findings
Participants who received the intermediate care intervention showed higher rates of unscheduled hospital admission than matched controls, whereas recipients of the integrated care team intervention showed no difference. Both intervention groups showed higher rates of mortality than did their matched controls.

Conclusions
These are potentially powerful techniques for assessing impacts on hospital activity. Neither intervention reduced admission rates. Although our analysis of hospital utilization controlled for a wide range of observable characteristics, the difference in mortality rates suggests that some residual confounding is likely. Evaluation is constrained when performed retrospectively, and careful interpretation is needed.

Efficient Funding: Auditing in the Nonprofit Sector

Efficient Funding: Auditing in the Nonprofit Sector
Manufacturing & Service Operations Management. 13(4) 471-488.

N. Privett and F. Erhun
04/01/2011

Nonprofit organizations are a critical part of society as well as a growing sector of the economy. For funders there is an increasing and pressing need to ensure that society reaps the most social benefit for their money while also developing the nonprofit sector as a whole. By routinely scrutinizing nonprofit reports in an effort to deduce whether a nonprofit organization is efficient, funders may believe that they are, in fact, giving responsibly. However, we find that these nonprofit reports are unreliable, supporting a myriad of empirical research and revealing that report-based funding methods do not facilitate efficient allocation of funds. In response, we develop audit contracts that put funders in a position to enact change. Auditing, perhaps obviously, supports funders; however, we find that it also benefits the population of nonprofits. Moreover, auditing results in improved efficiency for the nonprofit sector overall. Indeed, our conclusions indicate that nonprofits may want to work with funders to increase the use of auditing, consequently increasing efficiency for the sector overall and impacting society as a whole.

Improving Classroom Quality: Teacher Influences and Experimental Impacts of the 4Rs Program

Improving Classroom Quality: Teacher Influences and Experimental Impacts of the 4Rs Program
Journal of Educational Psychology, 102(1), 153-167

Brown, J.L., Jones, S.M., LaRusso, M.D., & J.L. Aber.
02/01/2010

This study capitalizes on recent advances in the reliable and valid measurement of classroom-level social processes known to influence children's social–emotional and academic development and addresses a number of limitations in our current understanding of teacher- and intervention-related impacts on elementary school classroom processes. A cluster randomized controlled trial design was employed to (a) examine whether teacher social–emotional functioning forecasts differences in the quality of 3rd-grade classrooms, (b) test the experimental impact of a school-based social–emotional learning and literacy intervention on the quality of classroom processes controlling for teacher social–emotional functioning, and (c) examine whether intervention impacts on classroom quality are moderated by these teacher-related factors. Results indicated (a) positive effects of teachers' perceived emotional ability on classroom quality; (b) positive effects of the 4Rs Program on overall classroom quality, net of teacher social–emotional functioning indicators; and (c) intervention effects that are robust to differences in these teacher factors. These findings support and extend recent research examining intervention-induced changes in classroom-level social processes fundamental to positive youth development.

Subscribe to Evidence-Based Management