Supporting City of Austin caseworker decision-making process with AI
One of the City of Austin's organizations, Downtown Austin Community Court(DACC) runs a case management service to assist people experiencing homelessness to find stable housing. "Case workers" are clinical professionals who work with individuals to evaluate their situations and explore solutions. The critical details of client interactions are recorded as textual data called "case notes" which are used as references to make decisions. Caseworkers are often overloaded with several clients and deal with complex situations making it quite tedious. Hence, the goal of this work is to enhance caseworker's work by leveraging technology.
My key contribution
5 months (Jul - Nov 2022)
UX Research | Data Analysis
Provided a comprehensive understanding of the case note data to the team by generating a qualitative framework.
Used the insights to brainstorm design opportunities with the team
Conducted user interviews to evaluate the proposals and provided feature recommendations
Collaborated cross-functionally with academic researchers and non-profit organizations through the process
Designing a research study to identify effective ways to build the selected AI tools
Understand the risks and opportunities of AI for caseworkers in homelessness case management. Identify ways to build effective and responsible AI tools.
Step 1: Making sense of previous research and data shared
There is existing knowledge available from interview transcripts of previous collaborations in a different context. I derived relevant information by analyzing these transcripts to build a foundational understanding of the caseworker's day-to-day workflow.
Caseworkers refer to case notes in parallel with presented information by a client to make decisions.
Each case manager deals with multiple clients in a day needing to refresh the client profile each time.
Casenotes are referred to in real-time while dealing with a client.
Caseworkers need to input several other data along with writing the case notes for each encounter with a client.
Casenotes are considered the most reliable and detailed source of information by caseworkers.
Step 2: But what do these interactions between the client and case manager constitute?
We were provided a large corpus of case notes by DACC. To get an idea of what caseworker interactions with clients constitute and how they progress these case notes were rich resources. I first conducted descriptive statistics to understand their distribution with various parameters such as per client, over time, per case manager, etc.
How to make sense of the content though? There are 50,000 case notes!
After experimenting with a few NLP techniques, qualitative analysis proved essential to gain a structured understanding of the case note content. I started the qualitative analysis by selecting representative client samples. This was done by clustering clients based on their shared personal information, year, duration of the case, and the number of case notes. However, this was just to give the team a kickstart in getting into the data. Starting with around 5 clients from different clusters, I created an initial framework through qualitative analysis. I further modified the framework by revisiting clients and exploring new case notes using keywords or other parameters(such as date, and type of client interaction).
Broader categories identified:
Actions performed by the client or case worker as part of the interaction.
Ex: Starting a housing application, providing bus passes, sharing documents.
Information exchanged on task status, client's well-being, available services, etc.
Scheduling and tracking, there is a lot of back and forth while scheduling meetings with the client and following up on activities to perform.
Requests by client or caseworker for resources or task completion.
I presented the insights gained from the categorization to the team enabling a comprehensive understanding of the case management process. This led the team to brainstorm and propose 5 technology tools that can enhance the caseworker's job.
Wait! We need to validate these ideas before digging deep into how they should be developed.
The team created mockups of the 5 solutions and conducted stakeholder interviews to gauge the solutions. This helped us evaluate which tools would bring utility to the caseworkers.
Step 3: Going into details, talking to the actual users!
One of the preferred tools was a summarization tool that gives an overview of essential client information. To design this tool, we had to dig deep into what's required, what is to be considered "essential", how would it integrate into a caseworker's workflow, etc. I conducted 2 of 4 caseworker interviews and generated key findings and recommendations.
There is no common structure for writing case notes among caseworkers. The caseworkers adjust to the writing styles of others while referring to past case notes.
Caseworkers can have quite tangential ways of involving case notes in their decision-making process.
A lot of information exploration happens by searching for keywords/phrases in past case notes.
Sharing certain information can have a negative impact on decision-making. The information in the summary should not create a bias about the client.
Information in older case notes of a client is of low relevance unless there are some repeating patterns/trends. The latest case notes are the most referred.
Information relating to safety concerns is primarily desired in a summary.
Optional viewing of different sections of the summary to enable choice in accessing the details.
Keywords or phrases with links to the root case notes can be used as a way to easily skim through the summary.
Summarization models should consider the date of entry in content prioritization.
Step 4: Design and evaluation
I am creating sample summaries along with my team based on the interview feedback to evaluate with the caseworkers.
Interested in the work? Please reach out for a detailed walkthough!
Make the right mistakes, to find the best way to do things.
Due to the large complexity of data and the limited interactions with caseworkers, it was important to take the best approach to gather knowledge during user interactions. I learned to break things down into concise goals/tasks and iteratively generate an understanding of complex situations.
Documentation and journaling
It was essential to have structured access to the vast amount of knowledge generated throughout the work. I documented outcomes at multiple stages and maintained a journal on essential information/progress that later supported the team in making key decisions.
Communicating ideas to cross-functional teams
Leveraging instances and examples to communicate design ideas to users and other teams proved to be highly effective.