Last week, I had the opportunity to speak at the Chicago Legal Innovation and Technology Meetup at Northwestern University Pritzker School of Law. I spoke about how LegalRnD is training 21st Century lawyers, using a project assessing a pilot Eviction Diversion Program (EDP) that I led in the 54A District Court in Lansing, Michigan as an example. Because many people asked questions about the project after my talk, I’ve drafted this post to describe the project in greater detail.
Initiating the EDP
This project began in early 2017 when Judge Alderson reached out to Dan Linna, Director of LegalRnD and Professor of Law in Residence at MSU College of Law, to ask for LegalRnD’s help implementing and assessing an Eviction Diversion Pilot Program she planned to implement in her court. Professor Linna suggested bringing on two students and me (in my capacity as LegalRnD fellow and Innovation Counsel) to work on this project during the summer and fall of 2017. Nick Gamber, Andrew Sanders, and I completed this LegalRnD project and wrote a paper in connection with Professor Linna’s Quantitative Analysis for Lawyers class.
Bringing LegalRnD Skills to 54A
One of the great things about this project was that it allowed my team and me to utilize many of the core competencies we emphasize in the LegalRnD curriculum. We strive to replace legal industry anecdotes about what purportedly works and does not with data-driven conclusions, measuring the effectiveness of legal service delivery. We also start with identifying the customer and an understanding of what they value. Additionally, while we set big goals, we strive for continuous improvements, which may start small but will accumulate and compound over time. The following components of the EDP project show how we aimed to achieve this.
Project Management
To allow ourselves the freedom to iterate quickly, fail quickly, and learn quickly, we prefer an Agile project management style over a traditional waterfall method. However, there are key components that build the foundation of any successful project, such as: (1) communication with the stakeholders to understand the challenge and scope; (2) a project charter; and (3) a way to track the workflow. For the EDP, we instituted the following:
- Project Charter — a guiding document that addressed the scope of the project, a list of key stakeholders, project milestones, and constraints, assumptions, risks, and dependencies. This was an essential document that ensured our understanding of the project was in line with Judge Alderson’s expectations. We consulted and adjusted these key documents frequently as we moved through the project.
- Stakeholder Interviews — before the EDP began, we interviewed those who would be impacted by the program, including court personnel, landlords, and tenant groups.
- Kanban Board — as discussed in a previous post, Kanban boards are an incredibly useful tool to control workflow. For this project and others, I designed Kanban boards to track our action items, goals, and deliverables.
User Testing
A key component of the EDP was a flier that we sent out with the Summons and Complaint informing tenants of various ways to seek legal help, that they may be eligible for free legal aid and funding opportunities at court, and the consequences of failing to appear for their hearing. Our idea was that this would reduce defaults. But we did not just execute on our ideas. We did user testing to validate our assumptions and improve the flier. In doing so, we worked with Margaret Hagan, Founder of the Legal Design Lab at Stanford, to create a flier. Then, Professor Linna and I took that flier to the court before the EDP kickoff date and talked with tenants. We asked tenants open ended questions to better understand their first impressions and what actions they thought they would take upon receiving this flier. Margaret took this tenant feedback and created a revised draft of the flier.
Data Analysis & Visualization
Because we had been asked to assess the program as well as help implement it, data analysis and visualization was a large part of this project. After discussions with Judge Alderson, our team and I decided to focus on three key metrics: formal eviction rates (i.e., how many landlords came back for a final writ of eviction); default rate; and dismissal rate. Because only Judge Alderson piloted the EDP, we had a natural experiment. Finding differences between groups, we conducted tests to determine if the differences were statistically significant. In other words, the statistical tests allowed us to determine if the differences were large enough to reject the null hypothesis that the EDP had no effect. (We continue to gather and analyze data, but so far the differences have been statistically significant, supporting the alternative hypothesis that the EDP is reducing eviction rates and default rates.) Finally, I used Tableau to create data visualizations. Not only is Tableau an excellent tool for creating data visualizations, it also avoids data integrity issues that sometimes arise with other tools, including inadvertent changes to data.
It was a great privilege to be able to work on this project and see several LegalRnD core competencies at work. I look forward to sharing more in the future.