Laura and John Arnold Foundation

ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

2015 Annual Report

FOUNDERS’ LETTER

T

his has been a productive year for the Laura and John Arnold Foundation (LJAF) and our team. We significantly expanded our work on evidencebased policy, extended our efforts in criminal justice, made highly strategic investments in education, and advanced important initiatives in various other areas, including healthcare and research integrity.

From LJAF’s inception, we have adhered to a single process for achieving transformative change, regardless of issue area: We study problems in great detail by surveying existing data and research, seek advice from leading experts, develop proposed solutions, rigorously test programs and interventions, and scale those that are proven to be effective.

As a young foundation, we are still at the early stages of this trajectory, but we have made great headway in 2015. Our Research Integrity team helped lead the effort to develop new guidelines for research transparency that have been endorsed by more than 500 scientific journals and organizations. Our Sustainable Public Finance team supported initiatives to improve retirement security for workers in several states. In education, we worked to advance the portfolio model of school governance, as more cities embrace this structure to address systemic inequities and improve student achievement. In criminal justice, we have increased the presence and impact of our Public Safety Assessment (PSA) tool while continuing to evaluate its efficacy. And in healthcare, we invested in efforts that are making it easier for doctors, insurance companies, and individuals to obtain objective information about how well pharmaceutical drugs work and for which patients.

In the coming year, we will continue to pursue our methodical approach to problem solving so that innovative ideas can be tested, proven, and widely adopted. In some areas, leaders have already expressed interest in interventions such as our PSA tool and the Pay for Success financing model. Our challenge is to promote the adoption of these tools while, at the same time, maintaining fealty to the very important concepts of rigor and evidence. Other areas will require investments in the development of an evidence base. Such is the case with our work in research integrity, where we fund replication studies to ensure that foundational knowledge is accurate and reliable, and in evidence-based policy, where we evaluate promising social programs to identify those that deliver significant results.

In addition, we will explore new areas where we see opportunities to apply our strategy for improving social good. This includes our recent investments to increase transparency and fight government corruption.

We plan to build on our commitment to furthering evidence-based practices across multiple disciplines in 2016, with the ultimate goal of fixing broken systems and producing measurable, lasting improvements in people’s lives.

Laura and John Arnold

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 3

EVIDENCE-BASED DECISION MAKING

T

he Laura and John Arnold Foundation (LJAF) works to solve some of society’s most pressing and persistent problems. Our team is focused on fixing the nation’s education and criminal justice systems, increasing government performance and accountability, improving the quality of scientific research, and addressing a range of other critical issues. At the heart of all of our work is a deep commitment to developing and identifying solutions that are grounded in reliable research and information. If we are to truly achieve transformational change in any aspect of society, we must have a clear understanding of the evidence.

In 2015, LJAF dramatically expanded efforts to advance evidencebased decision making in the public sector by launching a new initiative area specifically focused on evidence-based policy and innovation.

With team members based in our new office in Washington, D.C., as well as in Houston, this group supports initiatives that encourage governments and nonprofit organizations to help build the evidence base for social interventions and to consider reliable evidence as one of the primary factors in their decisions.

We believe that governments and nonprofits should continually analyze and evaluate the impact of their programs and try new approaches. Efforts that are found to be successful should be replicated and scaled, while those that are ineffective should be modified or eliminated.

All too often, public funds are allocated to projects and programs that do not deliver the intended results. When evaluated in scientifically rigorous studies, many programs are found to have little to no positive impact. In order to make significant progress in any area, we must accelerate the pace at which we learn what works and insist on services that produce measurable results.

4 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

RIGOROUS EVALUATIONS TO IDENTIFY PROMISING SOCIAL INTERVENTIONS A major portion of our work in evidence-based policy and innovation is focused on identifying the social interventions that produce the greatest benefit. One of the primary ways that we do this is by systematically monitoring all of the rigorous program evaluations that are conducted in every area of social policy. We then highlight the programs that meet, or nearly meet, a “top-tier” standard of evidence. These are interventions that have been shown in well-conducted randomized controlled trials (RCTs) to produce sizable, sustained benefits to the program participants or society as a whole.

Examples of highly effective interventions identified through our expert review process include an early childhood home visitation program for first-time mothers shown to reduce child abuse and neglect by up to 50 percent and to significantly improve other outcomes; and a low-cost, personalized job placement program that helps individuals receiving unemployment insurance return to the workforce and increase their earnings by 18 percent over two years.

We maintain a comprehensive, user-friendly database of these and other highly effective interventions spanning the full spectrum of social policy. The collection is one of the most frequently cited resources of its kind and is used by policymakers at all levels of government to inform more effective decision making. This review effort was initially launched in 2008 by the Coalition for EvidenceBased Policy, an organization that was integrated into LJAF this spring.

Another important way that we work to identify proven interventions is by directly funding low- or modest-cost evaluations, including well-conducted RCTs, that measure the effectiveness of programs and practices. RCTs, which are widely recognized as the gold standard in scientific research, make it possible to isolate the effect of a program apart from complicating factors. When programs are shown to have a significant impact, we fund more extensive evaluations to ensure that those impacts can be successfully reproduced and sustained over time and to identify the conditions under which the programs are most effective.

Key LJAF-funded RCTs include an evaluation of a diabetes prevention and management program that is administered by food pantries instead of doctors’ offices; a housing authority initiative intended to provide better support for tenants by restructuring rent rules to include financial incentives for steady employment; and an effort aimed at improving outcomes for college students by changing the way financial aid is distributed so that students are more likely to make wise financial management decisions and complete their studies.

CONDUCT EARLY-STAGE EVALUATIONS

CONDUCT RIGOROUS IMPACT EVALUATIONS

REPLICATE AND SCALE PROVEN APPROACHES

LJAF funds high-quality program evaluations to build credible evidence about what works to improve outcomes. If a program is shown to be successful in an initial study, we conduct rigorous impact evaluations to determine whether the program produces sizable, sustained benefits to participants or society. When programs produce the hoped-for effects, we support the widespread rollout of those proven interventions.

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 5

EFFECTIVE AND EFFICIENT GOVERNMENT

A

cross the nation, many state and local governments have expressed interest in using data and research to ensure that public services are delivered in an efficient and impactful way. Despite this strong desire to adopt an evidence-

based approach to decision making, jurisdictions often do not have the resources to translate data and research into sound public policy. Governments frequently lack the funding and expertise to rigorously evaluate the programs they provide. As a result, new programs may be implemented without a solid understanding of whether they will work as promised, and existing programs may remain in place without evidence to show that they are achieving their intended outcomes.

In 2015, we seeded policy labs to help governments conduct and use high-quality research to improve operations and the delivery of public services.

The Government Performance Lab at Harvard University’s John F. Kennedy School of Government is developing direct partnerships with a select number of state and local governments to help them carry out and evaluate a variety of projects. Key initiatives include establishing performance-based payments for social service providers, gauging the impact of current interventions, and implementing dashboards to track the efforts of government workers and improve the delivery of public programs. The lab also aims to create a network of innovative government leaders that is focused on using data to improve government performance across a broad range of areas.

A second project is the creation of the Rhode Island Innovative Policy Lab (RIIPL), a first-of-its-kind partnership between the state of Rhode Island and Brown University.

RIIPL is helping state leaders craft policies based on data and scientific studies in an effort to strengthen the state’s economy and provide meaningful improvements for Rhode Island families. Areas of interest include streamlining social services, improving public safety, and strengthening job training programs.

6 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

PAY FOR SUCCESS FINANCING TO IMPROVE LIVES AND SAVE TAX DOLLARS LJAF is working to advance Pay for Success (PFS), an innovative funding model that allows state and local governments to drive resources toward better, more effective programs.

Under the model, the government identifies a problem and specific target outcomes such as decreasing the number of days children spend in foster care or reducing the number of individuals who return to prison. Private investors cover the up-front costs of a program designed to meet those outcomes, and the government agrees to “pay for success.” The government should only repay the investors if an independent entity concludes that the program has achieved its predetermined goals. LJAF believes that rigorous, third-party evaluations are an essential component of all sound PFS projects.

Pay for Success Structure I N V E STOR S Fund projects up front and may receive a modest return if the program meets its predetermined goals

G OVERN M EN T Contracts to achieve certain outcomes and pays for success

I N T E R ME D I A RY Holds the contract and helps manage the project

S E RV I C E P ROV I D E R Administers services

E VA L UATOR Determines whether outcomes were achieved

L EGEND: Flow of up-front funding and project implementation

Potential flow of achieved outcomes and outcome payments

This unique approach to government contracting is helping to reorient public spending around outcomes and innovation by allowing governments to make low-risk investments in promising programs. There are roughly 30 PFS projects in various stages of development in the United States.

In order to support this demand, LJAF funds a number of groups that are providing direct technical assistance to help governments develop and execute PFS projects.

Our largest grant in this area supports the Urban Institute’s Pay for Success Initiative. The organization is creating resources to help governments determine whether a PFS project is the most effective and cost-efficient way to address a particular issue, as well as tools to structure deals, establish benchmarks for success, and identify a method to evaluate whether a project is successful.

In addition to funding technical assistance, LJAF supports the pilot and evaluation components of specific PFS projects and is a direct investor in two initiatives.

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 7

SUSTAINABLE PUBLIC FINANCE

O

ne of the most pressing issues threatening the fiscal health of cities and states is the unsustainable state of their public pension systems. These systems, which are supposed to protect workers’ long-term financial

interests, are now putting their retirement security at risk. In fact, conservative estimates indicate that cities and states currently owe workers more than a trillion dollars for retirement benefits they have already earned.

Many governments have been shortchanging workers for far too long. Political leaders have failed to make adequate payments into the pension systems and have relied on improbable investment return assumptions to make up the difference. This chronic underfunding has serious implications for both workers and taxpayers. The growing pension debt is forcing many localities to direct money away from critical public services such as schools and policing.

LJAF helps cities and states analyze the true magnitude of their pension problems and create systems that are affordable, sustainable, and fair.

We believe that responsible pension reform should include a plan for cities and states to pay down the pension debt in 20 years or less, and a requirement that governments pay their pension bill in full every year.

In addition to our work in public pension reform, we seek to improve retirement security for private-sector workers. Approximately 35 percent of private sector employees do not have access to a retirement savings plan through their employer.

One promising solution is a state-sponsored Secure Choice savings program that provides a low-risk, low-cost way for individuals to voluntarily save for retirement through automatic payroll deductions and to retain those savings even if they change jobs. LJAF supports Secure Choice plans in California and Connecticut by funding the research and analysis used by each state to develop a plan design that benefits all workers.

8 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

MINIMUM WAGE RESEARCH In an effort to improve the well-being of low-income workers and families, a number of cities have recently approved minimum wage increases of up to $5 an hour. Yet not much is known about the impact of such sizable increases. The existing research does not provide a comprehensive picture of how the effects will be spread across individuals and households, and how they will vary across industries, neighborhoods, and geographic areas. LJAF is funding several research projects that will study whether a higher minimum wage can promote economic opportunity and contribute to the long-term financial sustainability of workers and families. In particular, researchers will assess whether a higher minimum wage produces significant benefits for workers in areas such as health, food security, and overall earnings.

The researchers will also examine businesses to analyze changes in employment levels and operations, as well as low-income neighborhoods to determine the effects of residents’ increased spending power.

TAX POLICY As part of our effort to promote fiscal sustainability and the effective oversight of public funds, LJAF supports initiatives to help governments evaluate the impact of tax policies. Well-designed tax policies not only raise revenue to finance government programs and operations, they also play an important role in expanding opportunities, reducing income inequality, and supporting the nation’s long-term economic growth. However, poorly designed policies can have the opposite effect. In 2015, we funded research to assess the impact of minimum wage increases, the Earned Income Tax Credit, and how geographic location affects individuals’ ability to recover from economic downturns.

LJAF also supports The Pew Charitable Trusts’ Economic Development Tax Incentives project. In the past year, each of the six states participating in the project—Indiana, Maryland, Michigan, Oklahoma, Tennessee, and Virginia—has made progress toward implementing scalable solutions that would improve the way they evaluate the impact of economic development incentives, articulate policy goals, or use data in their incentive programs. In addition, Oklahoma and Tennessee approved laws requiring that tax incentives be regularly evaluated. The legislation will serve as an example for other states.

ANTI-CORRUPTION INITIATIVES To help ensure that government funds are spent wisely, LJAF supports initiatives to deter, identify, and combat corruption. We are funding the expansion of a research hub at Columbia Law School that is bringing investigators, prosecutors, oversight authorities, and ethics officials together to develop the alliances, tools, and insights needed to tackle corruption at all levels of government. Currently, investigators often rely on random audits and tips from whistleblowers to identify instances of manipulation of public data, bid-rigging, money laundering, and other abuses of public authority for private gain. A major portion of the center’s work is focused on building a data-driven framework that will allow investigators to proactively identify areas where problems are most likely to occur and to intervene before they become widespread.

We are also funding efforts to fight corruption in government contracting on a global scale. It is estimated that up to a quarter of procurement budgets worldwide may be influenced by corruption and fraud.

In addition to the obvious economic losses, there are significant human costs, such as when poorly constructed schools in earthquake zones have collapsed on students or when counterfeit medicines have resulted in patient deaths. The Open Contracting Partnership is working to improve transparency in public contracting, and we support its work to push for the standardized disclosure of data and information about the planning, procurement, and management of public contracts.

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 9

INFORMATION TRANSPARENCY

W

ell-informed citizens play a central role in a healthy democracy.

When individuals are armed with reliable information, they are able to vigorously debate important issues, evaluate the effectiveness of public and private institutions, demand accountability, expose injustice, and drive positive change. LJAF supports the development of technology and resources to make public information more readily available.

One such example is our funding for Digital Democracy, a web-based platform created by Cal Poly, San Luis Obispo that allows citizens to virtually access any state legislative hearing in California.

Individuals can enter a search term and find relevant video clips and contextual information about political issues and debates.

Users are able to watch videos and read scrolling transcripts that identify the speakers and link to profiles for the legislators and lobbyists who are participating in the discussion. In addition, individuals can access information about specific pieces of legislation, including summaries, the full text of the bill, and committee analyses.

SEARCH VIDEOS BY: - Keyword - Speaker - Committee location - Bill number - Date

SEARCH RETURNS DISPLAY speaker, date, location, and an excerpt from the transcript. Results can be sorted by date or relevance.

Digital Democracy is making California’s state government more open and transparent.

Images courtesy of Digital Democracy.

10 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

Digital Democracy has been recognized by journalists, state officials, and political activists as a potential game-changer for improving government transparency and state political coverage.

Since the launch of the platform in May, its automated transcription service has transcribed more than 700 hours of testimony from lawmakers, lobbyists, and members of the public. The Digital Democracy team has also developed a database of lobbyists and the clients they represent, and has created profiles for more than 8,600 people who spoke during public hearings in 2015.

In addition, LJAF supports the Internet Archive, the world’s largest public digital library, to ensure that there is a comprehensive, open record of the internet that is accessible to all. Studies show that, on average, a webpage is altered or deleted after only 100 days of being online. When pages are erased, edited, or abruptly moved, there can be wide-ranging consequences. For example, public figures can retract or modify statements after issuing them and annotations in scientific studies can be eliminated.

The Internet Archive has worked to address this issue by preserving more than 439 billion Web captures—including webpages, video, and images—through data donations, automated archiving of millions of websites, and collaboration with a thousand scholars and librarians. We are funding the development of an enhanced search engine that will enable users to perform more robust searches by entering a topic or keyword. This functionality will allow individuals to find an extensive selection of content tailored to their search criteria and will create unparalleled access to our digital history.

INVESTIGATIVE JOURNALISM LJAF recognizes the importance of high-quality journalism and the need for more thorough reporting on critical issues. In today’s competitive, fast-paced, digital world, many news organizations have shifted away from investigative reporting since these stories often require an extensive amount of time, significant resources, and in-depth analysis to produce.

We provide general operating support for organizations that create accurate, objective reports, including ProPublica, a nonprofit organization dedicated to investigative journalism in the public interest. ProPublica adheres to the strictest standards of journalistic impartiality and covers stories related to government, business, institutions, and other areas where the team’s reporting can make a major impact. Our funding also supports The Marshall Project, a nonprofit, online organization focused on issues related to criminal justice in the United States. The Marshall Project is committed to accuracy, fairness, and impartiality. It develops deep investigative projects, narratives, and profiles that put a human face on criminal justice, and produces explanatory and contextual pieces.

In addition, we fund the Center for Public Integrity’s (CPI) ongoing coverage of the flow of money in state politics in order to increase transparency in state-level government decision making. CPI tracked political contributions and advertising in all state races and ballot initiatives in the 2014 election cycle. It is now monitoring elected officials during at least the first two years of their terms.

Journalists are producing reports about officials’ voting records, analyzing policies that may have been influenced by donors, and creating profiles of national groups that are active in state politics.

The reporting, profiles, and other information are hosted on a Web platform that serves as a hub for the public and other news media.

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 11

CRIMINAL JUSTICE REFORM

I

n 2015, LJAF conducted the widespread rollout of the Public Safety Assessment (PSA). Considered one of the most important criminal justice reform initiatives in the United States, the PSA helps judges determine which defendants should be detained prior to trial and which can be safely released. A number of jurisdictions have adopted the pretrial risk assessment tool, and initial results indicate that it is helping to increase public safety, reduce jail populations, and free up millions of tax dollars for other government priorities.

PSA Sites Yakima Co., WA

Milwaukee, WI

Cook Co., IL

Toledo, OH New Jersey

Kane Co., IL

San Francisco, CA

McLean Co., IL

Santa Cruz, CA

Pittsburgh, PA Kentucky Mecklenburg Co., NC

Arizona

LJAF developed the PSA after research showed that many defendants who we as a society would expect to be detained while awaiting trial—the individuals who pose the greatest threat to public safety—are released. Meanwhile, low-risk, nonviolent defendants regularly spend extended periods behind bars before trial. This occurs because judges often do not have access to relevant information, such as a defendant’s criminal history, while making pretrial release and detention decisions, and frequently must rely on inflexible, charge-based rules, like bail schedules.

The PSA is helping to improve the way the system operates by providing judges with objective, reliable information about the likelihood that a defendant will engage in violence, commit a new crime, or fail to return to court if released before trial. Judges can then use that information—in concert with their professional discretion—to make more effective, evidence-based decisions.

Developed using the largest, most diverse set of pretrial records ever assembled—1.5 million cases from approximately 300 jurisdictions across the country—the PSA is based on factors that are raceand gender-neutral. These include a defendant’s criminal history, age, and current charge. The tool is easy to administer and will be adopted by other jurisdictions in the years to come.

Having now implemented the PSA in numerous jurisdictions of varying characteristics, we have engaged third-party researchers to conduct rigorous evaluations of the PSA. These evaluations include RCTs as well as qualitative evaluations, both of which will inform the scope and future use of the tool.

12 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

POLICING In response to recent incidents in Ferguson, Missouri, and other communities, LJAF is working to advance evidence-based policing practices. Initiatives in this area demonstrate our commitment to conducting rigorous empirical research and then translating it into actionable policy and practices.

One key initiative is focused on assessing the impact that police body-worn cameras have on public safety, policing strategies, and community perceptions of law enforcement. Currently, there is limited research about how the cameras can be most effective. We are funding a number of foundational research studies to address this and other critical questions.

MONETARY SANCTIONS This year, LJAF funded a series of research studies on the common practice of imposing monetary sanctions on individuals involved in the criminal justice system. Beyond anecdote and a handful of case studies, very little national research exists on important questions regarding the application of fines and fees, including their impact on public safety. To fill the gaps, we funded researchers to study the effects of court-imposed monetary sanctions and to assess the field of alternatives to fines.

CRIME PREVENTION LJAF supports a number of initiatives focused on preventing crime. One promising approach is the use of Cognitive Behavioral Therapy (CBT), which targets the dysfunctional thought and decisionmaking patterns that contribute to negative behavior. We are funding Roca, a nonprofit dedicated to disrupting the cycle of incarceration, and its partnership with Massachusetts General Hospital to create and evaluate a specialized CBT program for 17- to 24-year-old men, the population that is most likely to engage in criminal activity. To date, there has been no CBT program specifically designed for this cohort. If successful, the program will be scaled to serve as a national model.

FORENSIC SCIENCE LJAF is working closely with the forensic science community to ensure the reliability and integrity of evidence used in law enforcement investigations and criminal prosecutions. One of our major areas of interest is improving crime lab operations. In recent years, serious problems with oversight, policies, and procedures at a number of labs across the country have come to light. In 2015, we assembled a panel of experts to review and evaluate dozens of strategies to address these issues. We plan to develop the most promising concepts into specific projects.

DATA COLLECTION AND DISTRIBUTION Our Criminal Justice team is laying the foundation for best practices in data collection and distribution so that leaders can make decisions based on an objective understanding of the tactics and interventions that have the greatest impact on safety, efficiency, and fairness. This year, we funded The Governance Lab to develop ways to make it easier for criminal justice agencies to share data within and across jurisdictions. The initiative will place a fellow at the White House who will lead the effort to organize stakeholders in support of data-driven criminal justice innovation. It will also document the baseline of current data-sharing practices and communicate the benefits of more openly accessible data.

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 13

INNOVATION IN EDUCATION

O

ver the past ten years, students, teachers, administrators, and civic leaders in New Orleans have done what few thought was possible: They have transformed one of the lowest-performing school districts in the nation. Student performance in New Orleans is improving faster than it has in any other district in history.

The numbers speak for themselves: 62 percent of students are performing at grade level, compared to just 31 percent in 2004; the proportion of students attending low-performing schools has decreased by 47 percentage points; 73 percent of students are graduating on time, compared to 54 percent in 2004, and nearly two-thirds of these students are enrolling in college.

Scalable breakthroughs in education have been few and far between, but the New Orleans model is proving that opportunity and achievement can be dramatically increased across entire cities.

The model is built around a portfolio concept of school governance, where educators are empowered to make decisions at the campus level, families are no longer limited to their neighborhood school, and an independent entity evaluates student performance and works to promote equity throughout the system.

LJAF has invested heavily in New Orleans. We support efforts to create excellent schools, develop talented teachers, and build community engagement. We are proud to partner with all those who are committed to ensuring that every child is able to attend a high-quality school.

Inspired by the progress in New Orleans, we are also pursuing opportunities to help other cities, such as Memphis, Tennessee, and Camden, New Jersey, transform their school systems using elements of the portfolio model. We fund initiatives that promote a decentralized administration, more school choice, and greater flexibility and accountability for school leaders and educators.

In addition to this work, LJAF supports a number of education research programs and partnerships.

The Education Research Alliance for New Orleans at Tulane University is studying how the New Orleans reforms have influenced teaching and learning in the city’s schools and what those findings might mean for other communities. We also seeded the Houston Education Research Consortium at Rice University and the Dallas Education Research Consortium at University of Texas at Dallas.

Through these partnerships, researchers work with local school leaders in Houston and Dallas, respectively, to design and conduct studies that evaluate the impact of district policies and programs.

The long-term alliances are aimed at closing student achievement gaps by providing the evidence needed to support more effective decision making in K-12 education.

14 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

CUTTING-EDGE TECHNOLOGY TO IMPROVE STUDENT ACHIEVEMENT LJAF believes that technology can play an important role in helping to ensure that every child, no matter where he or she lives or attends school, can obtain a high-quality education. We are funding a digital personalized learning assistant (PLA) that has the potential to transform the educational experience for today’s students.

Developed by a team of scientists, engineers, and human learning experts at Rice University, the PLA combines the principles of cognitive science—what research has taught us about the way the brain works and how people learn—with the power of machine learning. It uses data-driven algorithms that adapt and react to an individual student’s needs.

The technology, which can be accessed for free from any device, assesses what a student understands and how he or she learns best. It then caters to that child’s learning style by delivering customized, interactive lessons. Teachers, parents, and administrators are able to track a student’s progress in real time. They can see which concepts the child comprehends as well as those that he or she finds more challenging. These unprecedented insights allow teachers to intervene at the first sign of trouble to support every child in mastering the subject material.

The PLA is currently being piloted for Advanced Placement physics and high school biology courses, and the Rice team will rigorously evaluate the PLA’s impact on student learning. If it is shown to be successful, the group will produce a replicable system that can be applied across subjects and grade levels in school districts throughout the United States.

In addition to funding efforts that promote high-quality educational opportunities at the K-12 level, we are supporting initiatives that leverage technology to make a college education more affordable. We fund edX, an interactive online platform that provides free, Massive Open Online Courses (MOOCs) from some of the world’s top universities. A 2015 grant will allow edX to expand its partnerships with colleges and universities. These partnerships will make it possible for more students to take classes on the edX platform and earn college credit for those courses at a fraction of the normal cost of tuition. Credits earned through the MOOCs are indistinguishable from those earned by students who take the classes on campus.

PERSONALIZED LEARNING ASSISTANT Adapts to individual learning preferences Delivers customized, interactive lessons Pinpoints areas where students need more help Reacts by delivering content to reinforce concepts

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 15

HIGH-VALUE HEALTHCARE

L

JAF has created a comprehensive plan to increase value in our nation’s healthcare system. We began by conducting a detailed analysis of every layer of the system, including the care provided by doctors and health systems; the coverage plans offered by employers and insurance companies; the quality and cost of medicines and treatments; and the decisions made by patients themselves. This extensive review allowed us to identify a number of high-leverage opportunities that we believe will help to improve the quality of care and reduce costs.

PAYMENT REFORM The healthcare payment system is currently designed as a fee-for-service model, where physicians and hospitals are incentivized to deliver services regardless of whether they are actually needed or will have the greatest impact on patient health. This structure does not support the best interests of individual patients and carries significant costs for the health of our nation and the economy. Our grantees are exploring ways to realign the payment system so that it is driven by the ultimate goals of keeping people healthy and providing patients with high-quality care at a sensible price.

As the single largest payer, Medicare exerts tremendous influence over healthcare pricing. It recently began experimenting with ways to transition away from the fee-for-service model to payment structures that emphasize lower spending and higher quality through models such as Accountable Care Organizations (ACOs). LJAF is funding an evaluation of ACOs conducted by the Harvard HealthCare Markets and Regulation Lab. We believe that this study will yield valuable information that decision makers can use to better understand and improve healthcare payment systems.

Employers can also play a central role in changing the system since they collectively purchase health insurance plans that serve 160 million Americans. Yet, to date, only a few of the nation’s largest employers have actively negotiated with insurance companies to make changes to the healthcare payment structure. We support Pacific Business Group on Health and its efforts to create a broad network of employer purchasers of healthcare that will evaluate new employer-led payment changes.

Those that are shown to have the greatest impact will be implemented and scaled throughout the network with the aim of reducing costs and improving patient health.

PATIENT DECISION MAKING The decisions that we make when choosing a doctor, hospital, or treatment plan are some of the most important considerations of our lives. Yet, we often have very little reliable information to help guide these decisions. It is nearly impossible for patients to effectively compare and evaluate the quality and cost of the care provided by doctors and medical centers. This has significant implications for people’s health and personal finances, and it plays a major role in the unsustainable cost of the healthcare system. LJAF is supporting a number of efforts that will empower patients to make informed decisions about their health.

16 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

We seeded the Center for Healthcare Transparency (CHT), which is developing solutions that will make it easier for individuals to compare healthcare options. CHT is establishing common protocols that will help regional data centers integrate various types of healthcare data—such as medical claims, clinical data, and patient surveys—that are extremely difficult to combine today.

They will then communicate that information in ways that benefit patients, employers, and public purchasers.

In addition, we are funding a Harvard HealthCare Markets and Regulation Lab study to determine whether providing cancer patients with transparent information about the cost of care influences their treatment decisions. People with cancer must often choose between treatment pathways when there is no definitive clinical recommendation for care. As a result, the “right” choice for a patient depends heavily on that individual’s preferences. Given that cancer care can be cripplingly expensive for patients and families, it may be critically important for patients to understand up front the likely health effects that the treatment will have as well as how much it will cost. Currently, most doctors are not able to provide patients with estimated out-of-pocket costs when discussing treatment options. Ensuring that individuals have financial information at the outset may help them to better assess the value of various treatment options relative to clinical benefits and to choose a plan that best meets their health and financial needs.

HOSPITAL AND PHYSICIAN BEST PRACTICES The way care is delivered to patients varies widely from hospital to hospital and even from physician to physician. This means some patients may receive treatment for a common condition such as diabetes or congestive heart failure that is less effective or more costly than other treatment approaches. While experts have demonstrated ways to improve value in certain settings, the lack of coordination across health systems and the absence of a framework for large-scale changes among different types of providers can lead to a significant gap between the time that breakthrough treatments or practices are identified and the point at which they are widely adopted. It is therefore essential to improve communication channels in order to help save lives and conserve resources.

Our grant to the High Value Healthcare Collaborative (HVHC) is funding the creation of a first-ofits-kind learning network for disseminating best practices across HVHC’s member health systems, which serve 20 percent of the U.S. population. The network will provide clinicians with access to the information, training, and resources needed to implement evidence-based treatment strategies that have been shown to make a difference in other hospitals.

PHARMACEUTICAL DRUG PRICING The skyrocketing price of pharmaceutical drugs poses a serious threat to the sustainability of the healthcare system. Drug prices have soared to new highs amid questions about their true value to patients. In an effort to address this urgent concern, LJAF is funding a program through the Institute for Clinical and Economic Review (ICER) that is designed to transform the way drugs are evaluated and priced. ICER is producing public reports on new drugs that have the potential to significantly impact patient care and health system budgets. Each report includes a thorough analysis of how well the drug works and for which patients, and how it compares to other similar drugs on the market. In addition, ICER uses transparent methods to calculate a price benchmark for each drug that is directly tied to the benefits it delivers to patients. The reports will help healthcare providers negotiate prices that reflect a treatment’s value to individuals and the healthcare system as a whole. This will support the design of new insurance benefit programs that more effectively manage patients’ health outcomes and the cost of treatment.

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 17

NUTRITION SCIENCE

O

besity and obesity-related diseases are costing the U.S. healthcare system more than a billion dollars each day, and the problem is only getting worse. In the almost four decades since the government first started

providing dietary advice, the percentage of Americans who are obese has more than doubled, and the number of people with diabetes, according to the Centers for Disease Control, has quadrupled. With such alarming increases, we can no longer pretend to know what constitutes a healthy diet.

LJAF has developed a three-pronged strategy to address this public health crisis.

First, we are funding rigorous nutrition research to help establish clear evidence about what is causing the obesity epidemic. Second, we are promoting the use of the limited high-quality information that does exist to inform nutrition policy in the near term. And finally, we are supporting efforts to develop a process that will help to ensure that, over the long term, all nutrition policies are crafted using highquality research.

RIGOROUS NUTRITION RESEARCH The vast majority of nutrition research that has been conducted to date is deeply flawed. It is difficult to discern any credible guidance from existing studies, which often rely on small sample sizes or demonstrate only correlation, not causation. Many studies also base their findings on unreliable observational research, such as the results of a survey in which people were asked to remember what they ate and write it down.

These and other substandard research methods and findings do not allow us to determine what is causing the obesity epidemic. Conclusive scientific evidence is needed, and LJAF is funding well-designed, highly controlled studies to help answer several critical questions: • Are all calories the same, or do calories from fat and carbohydrates have differing effects on weight gain?

• What is the most effective way to maintain a healthy weight: consume fewer calories, or change the amount of fat and carbohydrates in your diet?

• Can a sugar-free diet reverse the effects of Non-alcoholic fatty liver disease, the country’s most prevalent obesity-related disease?

These studies will be more reliable than those conducted previously and will help to establish what we should and should not eat in order to be healthy.

18 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

A research assistant works in the lab at Emory University, where scientists are conducting the first-ever RCT related to pediatric Nonalcoholic fatty liver disease. Photo courtesy of the Nutrition Science Initiative.

NEAR-TERM POLICY SOLUTIONS While more research is urgently needed, there is a growing body of scientific evidence that indicates that consuming too much sugar is harmful to one’s health and can lead to chronic diseases such as heart disease, liver disease, diabetes, and other conditions. We are sponsoring research to assess the impact that various policies can have on reducing sugar consumption and improving public health. These possible policy solutions include a tax on soda and sports drinks; a warning label on sugary products similar to the label that appears on cigarettes and other tobacco products; a limit on the amount of sugar included in food served in schools, daycare centers, and nursery schools; a restriction on the purchase of sugary drinks and incentives to purchase healthy food under the Supplemental Nutrition Assistance Program (SNAP); and the elimination of sugary food and drinks from kids’ menus in restaurants.

LONG-TERM POLICY SOLUTIONS Over the long term, we must develop a process that helps to ensure that nutrition policy is grounded in high-quality evidence. Earlier this year, the Dietary Guidelines Advisory Committee (DGAC) produced a report detailing recommendations for the government’s official dietary guidelines. The guidelines are used to influence a number of key government programs including SNAP and the National School Lunch Program. LJAF funded an analysis of the science that served as the basis for the DGAC’s report, and the independent review revealed numerous problems. In addition to undeclared conflicts of interest, much of the DGAC’s report is based on observational epidemiology that cannot show what sort of diet actually causes better health. The analysis indicated that the recommendations were developed after a select review of evidence that did not include highly relevant but contradictory studies. The committee failed to follow rigorous standards for reviewing evidence. A higher standard of rigor and transparency is needed in the scientific review process, and LJAF is continuing to pursue initiatives that will support the creation of policies that are based on reliable scientific evidence.

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 19

OPEN SCIENCE

L

JAF’s Research Integrity initiative is working to improve the reliability and validity of scientific research. As a society, we depend on research to produce scientific advancements, support the discovery of new medical treatments, inform sound public policy, and guide our personal decisions on everything from nutrition to parenting. Unfortunately, a substantial portion of research, including many studies published in academic journals, does not adhere to rigorous standards of quality.

In 2015, there were a number of major developments in our efforts to increase transparency and reproducibility in scientific research, including the release of the findings of an unprecedented replication initiative; the development of new publication standards for scientific journals; an expansion of our work to improve reporting and transparency in clinical trials; and the launch of a new center focused on improving the reliability of neuroscience research.

REPRODUCIBILITY PROJECT In August, our flagship Research Integrity grantee, the Center for Open Science, released the findings of the Reproducibility Project, the largest-ever replication effort in any scientific discipline.

The ongoing replication of important research is an essential component of scientific progress.

However, after attempting to reproduce the findings of 100 published studies in psychology, the Reproducibility Project found that less than half of the original results could be replicated by an independent lab. As The New York Times put it, the Reproducibility Project “confirmed the worst fears of scientists who have long worried that the field needed a strong correction.” The results of this initiative demonstrate the importance of rigorous standards in scientific research as well as the need for more replication studies to strengthen the body of scientific knowledge.

While the Reproducibility Project was focused on the field of psychology, we are funding a similar effort aimed at validating dozens of landmark cancer cell biology studies. Systematic validation of these studies could be crucial to developing future cancer drugs.

With funding from LJAF, the Center for Open Science is creating incentives and building free, open-source research tools that make it easier for scientists to conduct research in an open and transparent manner. The Center’s signature project, the Open Science Framework, allows scientists to share their datasets and research materials and to preregister their studies by submitting a description of the project before the analyses are performed.

20 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

PUBLICATION STANDARDS The researchers involved in the Reproducibility Project cited misaligned incentives as a problem that is undermining the credibility of science. Journals and funders frequently show a strong bias for provocative findings and statistically significant results. Researchers, who have a better chance of securing future funding when their previous studies have been featured in prominent journals, are incentivized to alter their research, manipulate results, and publish only their successes. By deemphasizing the importance of null, negative, or inconclusive findings, the research community is compromising the pursuit of scientific truth, which can only be achieved by considering all of the evidence.

One way that we are working to address this issue is by supporting efforts to improve publication standards for scientific studies. We sponsored the Transparency Openness and Promotion (TOP) Committee’s work to develop the TOP Guidelines. The guidelines call for scientists to preregister their studies by submitting a description of the project before the analyses are performed, so as to make it possible for others to assess whether the study’s findings truly confirm the stated hypothesis.

The guidelines also call on scientists to share their data and code and make their results publicly available. In July, the leading journal Science dedicated an entire issue to the topic of “self-correction in science.” It included the release of the TOP Guidelines, a related editorial from our Vice President of Research Integrity Stuart Buck, and a piece that Dr. Buck co-authored on “promoting an open research culture.” Since then, more than 500 journals and 49 organizations have pledged support for reproducibility and transparency by endorsing the TOP Guidelines.

CLINICAL TRIAL TRANSPARENCY Another major focus of our work in research integrity is improving reporting and transparency in clinical trials. A 2015 investigation published in The New England Journal of Medicine showed that most clinical trial results are hidden from public scrutiny. In fact, only 38 percent of trials complied with the federal law requiring the public disclosure of their results. Furthermore, the results of trials funded by the federal government were reported less frequently than the results of those funded by the pharmaceutical industry. This lack of transparency means that we do not have a true picture of the effectiveness of many drugs and medical treatments, which poses a significant threat to patient health.

In order to solve this problem, we support the AllTrials initiative and its campaign to require that all clinical trials—past and present—be registered with full disclosure of study methods and results.

In a related effort, we are funding the creation of Open Trials, an open, online database that will aggregate all data and documents on clinical trials throughout the world. Led by Ben Goldacre at Oxford’s Centre for Evidence-Based Medicine and Open Knowledge in the United Kingdom, Open Trials will help to automatically determine which clinical trial results have not been disclosed, making it possible to investigate their sponsors and researchers. For those clinical trials with published results, the Open Trials database will make it easier to double-check the findings by providing access to more detailed information than usually appears in the published literature.

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 21

NEUROSCIENCE RESEARCH LJAF has seeded several entities that are doing groundbreaking work to improve the quality of research in specific fields. In 2015, we provided seed funding for a new group that is focused on increasing transparency and improving reliability in neuroscience research. The Center for Reproducible Neuroscience (CRN) at Stanford University will create a free, online platform that will use supercomputers to provide scientists with cutting-edge tools that can help them better understand the root causes of serious mental illnesses such as Alzheimer’s disease, bipolar disorder, addiction, and depression. Many labs currently lack these computing capabilities because of financial or technical limitations. CRN will now make it possible for researchers across the country to perform in-depth analyses of large neuroimaging datasets as part of their efforts to study and help doctors diagnose brain disorders.

The first-of-its-kind platform developed by the Center for Reproducible Neuroscience allows researchers to conduct multiple data analyses simultaneously, gauge the reliability of their results, and identify the analysis methods that maximize reproducibility.

Photo courtesy of the Center for Reproducible Neuroscience.

SCIENCE AND TECHNOLOGY Recognizing that advances in science and technology are essential to solving many of the world’s biggest problems, LJAF launched a new division in 2015 that is specifically focused on driving innovation in these areas. Based in Washington, D.C., our new Science and Technology division is working to accelerate progress across the sciences by investing in high-risk, high-reward research; developing a network of objective entities that analyzes science policies and promotes those that are effective; and supporting the creation of technological infrastructure that has the potential to produce significant breakthroughs.

Occasionally, a tool, technique, or instrument is created that revolutionizes the way research is conducted. LJAF is currently exploring promising opportunities to help stimulate these important technological shifts.

We are driven by new ideas and are encouraging bold thinkers to take on major challenges. In the coming year, our Science and Technology division intends to pursue a number of strategies to catalyze and promote new scientific advancements.

22 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE

GRANTS This page includes an overview of LJAF’s grant commitments made between January 1, 2011, and October 31, 2015. A complete list is available on our website at www.arnoldfoundation.org.

Criminal Justice $38,668,242

Sustainable Public Finance $35,222,288

Research Integrity $40,656,588

Education

Nutrition

$190,576,959

$44,358,163

$489,781,150

Information Transparency $10,755,139

Healthcare $32,838,265

Evidence-Based Policy and Innovation $92,016,228

Emerging Areas of Interest $4,689,278

LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE | 23

P.O. Box 460809 Houston, Texas 77056-8809 (713) 554-1349 www.arnoldfoundation.org @LJA_Foundation

© 2015 Laura and John Arnold Foundation. All Rights Reserved.

24 | LJAF • ACHIEVING CHANGE THROUGH THE POWER OF EVIDENCE