Human-AI Interaction
Supporting City of Austin caseworker workflows with technology

Overview
Context
One of the City of Austin's organizations, Downtown Austin Community Court(DACC) runs a case management service to assist people experiencing homelessness to find stable housing. "Case workers" are clinical professionals who work with individuals to evaluate their situations and explore solutions. The critical details of client interactions are recorded as textual data called "case notes" which are used as references to make decisions. Caseworkers are often overloaded with several clients and deal with complex situations making it quite tedious. Hence, the goal of this work is to enhance caseworker's work by leveraging technology.
My key contribution
Duration
5 months (Jul - Nov 2022)
UX Research | Data Analysis
-
Provided a comprehensive understanding of the case note data to the team by generating a qualitative framework.
-
Used the insights to brainstorm design opportunities with the team
-
Conducted user interviews to evaluate the proposals and provided feature recommendations
-
Collaborated cross-functionally with academic researchers and non-profit organizations through the process
-
Designing a research study to identify effective ways to build the selected AI tools
Research
Goal
Understand the risks and opportunities of AI for caseworkers in homelessness case management. Identify ways to build effective and responsible AI tools.
Step 1: Making sense of previous research and data shared
There is existing knowledge available from interview transcripts of previous collaborations in a different context. I derived relevant information by analyzing these transcripts to build a foundational understanding of the caseworker's day-to-day workflow.
Key points
-
Caseworkers refer to case notes in parallel with presented information by a client to make decisions.
-
Each case manager deals with multiple clients in a day needing to refresh the client profile each time.
-
Casenotes are referred to in real-time while dealing with a client.
-
Caseworkers need to input several other data along with writing the case notes for each encounter with a client.
-
Casenotes are considered the most reliable and detailed source of information by caseworkers.
Step 2: But what do these interactions between the client and case manager constitute?
We were provided a large corpus of case notes by DACC. To get an idea of what caseworker interactions with clients constitute and how they progress these case notes were rich resources. I first conducted descriptive statistics to understand their distribution with various parameters such as per client, over time, per case manager, etc.
How to make sense of the content though? There are 50,000 case notes!
After experimenting with a few NLP techniques, qualitative analysis proved essential to gain a structured understanding of the case note content. I started the qualitative analysis by selecting representative client samples. This was done by clustering clients based on their shared personal information, year, duration of the case, and the number of case notes. However, this was just to give the team a kickstart in getting into the data. Starting with around 5 clients from different clusters, I created an initial framework through qualitative analysis. I further modified the framework by revisiting clients and exploring new case notes using keywords or other parameters(such as date, and type of client interaction).
Broader categories identified in case note content:
-
Actions performed by the client or case worker as part of the interaction.
Ex: Starting a housing application, providing bus passes, sharing documents. -
Information exchanged on task status, client's well-being, available services, etc.
-
Scheduling and tracking, there is a lot of back and forth while scheduling meetings with the client and following up on activities to perform.
-
Requests by client or caseworker for resources or task completion.
I presented the insights gained from the categorization to the team enabling a comprehensive understanding of the case management process. This led the team to brainstorm and propose 5 technology tools that can enhance the caseworker's job.
Wait! We need to validate these ideas before digging deep into how they should be developed.
The team created mockups of the 5 solutions and conducted a stakeholder interview with the department head to gauge the solutions. This helped us evaluate which tools would bring utility to the caseworkers. (Note: During actual user interviews these ideas were briefly introduced to re-evaluate the perceived utility)
Step 3: Going into details, talking to the actual users!
One of the preferred tools was a summarization tool that gives an overview of essential client information. To design this tool, we had to dig deep into user needs:
-
What is the perceived value of a summary?
-
What is "essential" to be included in a summary?
-
How would the summary integrate into a caseworker's workflow?
I conducted 2 of 4 caseworker interviews and generated key findings and recommendations.
Key Findings:
General
-
There is no common structure for writing case notes among caseworkers. The caseworkers adjust to the writing styles of others while referring to past case notes.
-
A lot of information exploration happens by searching for keywords/phrases from past case notes.
Summarization tool
RQ1:
-
Summaries provide "holistic" information that informs the next steps.
-
Summaries provide access to critical client details in one place without the iterative search.
-
Summaries provide an overview of important events for a client that influence decision-making.
RQ2:
-
Sharing certain information can have a negative impact on decision-making. The information in the summary should not create a bias about the client.
-
Information in older case notes of a client is of low relevance unless there are some repeating patterns/trends. The latest case notes are the most referred.
-
Time-sensitive information and safety concerns are primary necessities from a summary.
RQ3:
-
Caseworkers can have tangential ways of involving case notes in their decision-making process. Some refer to it before a client interaction to avoid repetition of information while some refer to it post an interaction to avoid knowledge bias.
Preliminary recommendations:
-
Optional viewing of different sections of the summary to enable choice in accessing the details.
-
Keywords or phrases with links to the root case notes can be used as a way to easily skim through the summary.
-
Summarization models should consider the date of entry in content prioritization.
Step 4: Design and evaluation
I am currently working on summary content design through a co-design process with case workers and collaboration with NLP experts.
Interested in the work? Reach out for a detailed walkthough!
Learnings
Make the right mistakes, to find the best way to do things.
Due to the large complexity of data and the limited interactions with caseworkers, it was important to take the best approach to gather knowledge during user interactions. I learned to break things down into concise goals/tasks and iteratively generate an understanding of complex situations.
Documentation and journaling
It was essential to have structured access to the vast amount of knowledge generated throughout the work. I documented outcomes at multiple stages and maintained a journal on essential information/progress that later supported the team in making key decisions.
Communicating ideas to cross-functional teams
Leveraging instances and examples to communicate design ideas to users and other teams proved to be highly effective.