Generative AI Summaries
Designing LLM-Based Summaries for Homelessness Caseworkers
Overview
Context
-
The City of Austin strives to assist people experiencing homelessness by running a case management service.
-
Caseworkers, who are clinical professionals, handle numerous clients, recording vital information in free-text case notes for decision-making.
-
This work aims to improve caseworker efficiency in referencing client information from the large corpus of text using technology.
My Role
Duration
8 months (Jul 2022 - Feb 2023)
-
Analyzed case notes using statistical and qualitative methods.
-
Designed and conducted user interviews and co-design sessions.
-
Brainstormed design opportunities with the cross-functional team.
Tools
Zoom, Excel, Python.
Team
Team of 4 researchers and data scientists.
Process
.png)
Research
Goal
-
Identify opportunities to build tools for caseworkers to effectively utilize case notes.
-
Inform the design of effective and responsible AI tools.
Data Analysis
What information do casenotes representing caseworker-client interactions constitute?
The organization provided an extensive corpus of case notes, serving as a valuable resource to comprehend caseworker-client interactions and case note-writing practices.
How to make sense of the content though? There are 50,000 case notes!
Statistical Analysis
Descriptive statistics, topic modeling and clustering to identify patterns based on temporal, demographic and case attributes.
Qualitative Coding
Iterative analysis of representative case notes sampled from clustering.
The case note content involved six major themes.
Actions Performed
Activities conducted by the client or case worker during the interaction, such as providing documents, initiating a housing application.
Information Exchanged
Details regarding activity progress, client well-being, and available services.
Conversation Snippets
Emotional conversation details and client expressions.
Scheduling & Follow-Up
Scheduling and follow-ups on meetings or pending activities.
New Requests
Requests by the client for benefits or task assistance.
Next Steps
Potential next steps for case progress.
Insights from data analysis served as the foundation for steering the brainstorming of five technology tools that could enhance caseworker's work. We created mockups of proposed tools, to validate the concepts with the relevant stakeholders.
Based on utility and feasibility, the team decided to explore a text summarization tool to provide a snapshot of crucial client information
User Interviews
Digging deeper into user needs, talking to the actual users!
We conducted 5 in-depth user interviews to understand user needs, and validate and inform summarization tool. The interviews enabled us to explore the following:
What are the current challenges in referencing client case notes?
What is the perceived value of a summary to address these challenges?
How would the summary integrate into a caseworker's workflow?
Key challenges in referring casenotes for decisions:
Key concerns with summarization:
Inconsistencies in Information Format
Caseworkers had to adapt to diverse writing styles and note-taking structures, that required additional time and effort in navigating case note content.
Reliance on Memorable Keywords
Primary navigation through case notes involved caseworkers trying several keywords/phrases from past notes to locate specific information, impacting efficiency of information retrieval.
Temporal significance
Older case notes for a client held low relevance unless there were repeating patterns/trends. These patterns could be missed as caseworkers primarily referred to the latest notes.
Philosophical Diversity
Caseworkers practiced diverse approaches to decision-making, such as referencing case notes before or after client interactions. These led to varying information goals from a summary.
Information Bias
Caseworkers expressed concern that over-reliance on the summary could bias the interaction with a client.
AI Trust
Caseworkers expressed potential distrust in summaries generated by AI, leading them to revert to manually reviewing notes for assurance.
Co-Design
How would a caseworker approach creating a summary for case management?
We engaged in one-on-one co-design sessions with caseworkers, during which they crafted summaries for existing clients while reflecting on the information needs and reasons behind their choices. These sessions provided insights on the following:
What are the crucial details to be included in a summary?
What are the similarities or differences in caseworker expectations from a summary?
Summaries generated from co-design sessions were analysed on 3 aspects:
-
Variations in summaries based on goals.
-
Comparison of summaries for the same client by different caseworkers.
-
Comparison of summaries generated for caseworker's own and other's client.
Drawing from our comprehension of the user's cognitive processes, we developed a knowledge guide that distilled the intricacies into a set of relatively standardized rules. This document served as a cornerstone for guiding model training, prompt engineering, and output evaluation.
Designing LLM-based Summaries
The research insights generated from interviews and co-design sessions drove the LLM summary design to mitigate limitations in LLM outputs and effectively address user needs.
Design Considerations:
Reference Feature
Integrate original case note links into the summary, providing caseworkers with the ability to validate information or explore details when needed. This serves to address challenges related to accuracy and context in decision-making.
Time Representation
Prioritize content based on the temporal aspect, considering entry dates. This can be achieved through date markers or chronological order.
Summary Layout
Implement a structured layout with optional viewing of different sections in the summary to facilitate user choice in accessing specific details.
Prompt Engineering Considerations:
Annotation Guide
Reforming knowledge document to an annotation guide to ensure consistency among human annotators, and feed prompts for summary generation.
Goal-Based Prompts
Utilize the diverse goals identified during co-design sessions to prompt summaries, aligning with caseworker objectives and enhancing relevancy.
Section-wise Prompt Optimization
Optimize summary prompts for different categories identified. For instance, tailor prompts for housing services, medical services, etc., enhancing the specificity of the generated content.
Crafting LLM-based summaries for high-stakes domains like homeless services presents considerable challenges. Inaccuracies or omissions in these summaries can have profound consequences, potentially leading clients to miss crucial services that significantly impact their quality of life.
Interface design and a comprehensive framework representing information structure and goals play a pivotal role in ensuring the practicality and reliability of the tool.
Interested in the work? Reach out for a detailed walkthough!
Learnings
Leveraging secondary research
Given the intricate nature of the data and limited caseworker interactions, it was crucial to adopt an effective approach for knowledge gathering before user interactions. I learned to distill complex data through iterative analysis.
Interface, functional and strategic solutions
I honed my skills in discerning viable solutions at the interface, functional, and strategic levels to tackle diverse challenges. This required a grasp of design thinking, technical expertise, and organizational understanding.
Documentation and journaling
Structured access to the extensive knowledge produced during the project was crucial. I documented outcomes at every stage and maintained a journal highlighting essential information supporting the team in making critical decisions.