Project Spotlight: Using Implementation Science and Human Centered Design to Change Human Services and Workforce Programs, Build Evidence, and Improve Lives
By Michelle Derr, Senior Fellow, Mathematica, Anna Mastri, Senior Researcher, Mathematica, Jonathan McCay, Lead Program Analyst, Mathematica
For the past twenty years, I have studied the implementation of programs designed to help people facing challenges to employment find and maintain jobs and get better outcomes for their families. Over time, I realized that many of these programs try innovative, evidence-based strategies. However, these changes often fail to stick, and most evaluations fail to find any impact of these programs on long-term family well-being.
We needed a change in the human services and workforce field. Working with my colleagues, we co-designed a process model known as Learn, Innovate, Improve (LI2) to support implementation of changes in human services and workforce programs. Rooted in implementation science principles and human-centered design (HCD) practices, LI2 is a systematic and analytic approach for adapting evidence-based practices and building organizational and leadership capabilities to implement and sustain them (see LI2 brief). We refined the process model with practitioners nationwide, to make sure it worked for them.
Mathematica’s Learn, Innovate, Improve Model
Over the years, we have used LI2 in dozens of communities to help people implement evidence-based and evidence-informed strategies. Recent projects include redesigning program orientation sessions in human services agencies to be more engaging and delivered remotely, standing up a transitional jobs programs in a workforce center by hiring jobseekers as temporary contact tracers for COVID-19, improving the customer experience by streamlining paperwork and processes (see Philadelphia redesign), and implementing a core practice model based on the science of self-regulation and goal pursuit (see Goal4 It!).
In September 2020, we brought representatives from a diverse set of human services programs together for a series of virtual sessions about their implementation challenges. We used some “go-to” HCD activities—Rose, Thorn, Bud; Problem Tree Analysis; and Visualize the Vote—to help teams Learn about the most pressing challenges they face in using evidence to improve services. Two central challenges emerged: (1) old habits die hard (i.e., changing organizational climate and culture is difficult); and (2) federal, state, and local policies sometimes pose challenges to using evidence-informed practices.
We then held design sessions to Innovate ways to address the challenges. We used “how might we…” statements to launch the design of what programs could do differently and how to infuse those changes into their current practice. We collaboratively defined intermediate and long-term measures of success. Now, teams are moving into the Improve phase, implementing and iteratively testing the changes. We call these road tests—a series of focused, analytic pilots. Later on, teams might scale or use a more rigorous test to measure the intervention’s effectiveness (see rapid-cycle evaluation).
More recently, our colleagues began using LI2 in other substantive areas including early childhood, child welfare, education, disability, and food assistance programs, among other areas. In addition, we are using LI2 in two large federal randomized control trial evaluations to prepare sites for evaluation, enhance the implementation study, and make course corrections when things go wrong such as low enrollment. Overall, we have found the use of LI2 to be an accessible way to bring researchers and practitioners together to strengthen programs, build evidence, and improve lives.
This article was featured in our monthly Implementation in Action bulletin! Want to receive our next issue? Subscribe here.