Concept Video Password: msdesignexpo
TICA: A CUI for Criminal Investigators
Microsoft invited my studio class at Carnegie Mellon University to participate in the 2016 Microsoft Design Expo. The design challenge was: "Achieving Symbiosis and the Conversational User Interface (CUI)." I teamed up with two other designers and the three of us conducted design research, prototyped, and developed a concept that would utilize CUI and human-computer symbiosis to help criminal investigators better document and process case-specific data in real time.
Developed research objectives, plans, and methods; recruited research participants; interviewed participants and facilitated co-design workshops; developed story boards; created design prototypes; presented progress, insights, and objectives; and acted in the concept video.
More detailed process documentation can be found here.
Defining the Territory
Since the project started with a very broad design challenge, my team worked together to determine what directions we were or were not passionate about. We considered who the users might be as well as the types of environments that CUIs could possibly be effective in. We used those parameters in a matrix to brainstorm some possible design foci.
Next, based on the ideas we produced, we put together a territory map that helped us hone down our specific interests. We layered social-focused environments (personal, shared, and public) with action-focused environments (research, collaborative, care, making, coordinated, and ad-hoc) and through a collaborative process, determined which area would best suit human-computer symbiosis, conversation, and our research constraints. In the end, we landed on field investigation.
Learning About The Stakeholders
Once we decided we wanted to focus on field investigation, I spoke with someone with detective experience to learn about who the stakeholders are and how they relate to each other. I created a concept map to illustrate how different stakeholders play a part in criminal investigation. Next I reached out to criminal investigation organizations. I spoke to the Pittsburgh Bureau of Police, the Pittsburgh Office of Municipal Investigations, and the Pittsburgh Federal Bureau of Investigation. Each of these stakeholders agreed to meet with us.
Making Research Plans
In preparation to meet with criminal investigators, we came up with learning objectives and research plans. Our objectives evolved with each visit to the investigators' facilities as we became more acquainted with the challenges, processes, and desires of investigators. Before each visit, we tested our research strategies and co-design workshops with students to make sure the research flow and instructions made sense.
First research session with investigators
Learn about work processes, communication flows, use of technology, challenges, and visions for the future.
We prepared a number of research materials, methods, and interview questions to achieve our learning objectives for our first round of research, including journey maps, network maps, card sorting, brainstorming about ideal futures and dot voting.
Collecting and sharing information is difficult. Investigators mentally take in enormous amounts of information, but have difficulty transferring all that knowledge to a document, report, or database. One investigator even exclaimed, "I need a brain scanner!" As a result of the information overload, investigators have to make decisions about which information is most important to document for a case. Often times they don't know what kind of information would solve a case. One investigator explained that something as detailed as knowing wether or not a victim's shoelaces were untied could make or break a case. However, how would an investigator know to document such detail?
The Second Research Session with investigators
Learn how human-computer symbiosis could be achieved in an investigative setting. Learn what qualities investigators look for in a symbiotic relationship. Learn what abilities would help investigators achieve better investigative work.
I came up with a method to learn what tasks investigators believe are inherently human or enjoyable and what tasks could be offloaded to a "magical helper". I asked them to use cards to write down tasks that are enjoyable, not enjoyable, and tasks that would be better if they had a magical ability. Next, I gave them two-column worksheet on which to place the cards they made. One column represented the tasks that they wished to do at work and the other column represented the tasks that they would delegate to a "magical helper" in order to achieve the most effective investigative work. At the bottom of the worksheet, I asked them to describe what kind of relationship they would have with their magical helper.
After this exercise, we gave the investigators blocks and people figurines and asked them to make a scene that shows how their magical helper would fit into their daily work. And lastly, we gave the investigators clay and asked them to build their magical helper and show us what it looked like.
We learned that the participants valued the time they spend collaborating and problem solving together, but not the administrative tasks related to email, phone calls, and meetings. They wished to have a consolidated, reliable data base to make sharing information easier. It also appeared that they desired a working relationship with technology—one where they feel that it "has their back" or can watch over them and jump in when they need help.
Based on the insights we collected from our initial research sessions with investigators, we put together several storyboards illustrating CUI human-computer symbiosis scenarios. We wanted to get a sense of which ideas resonated with investigators. From this research, we learned which could be helpful solutions and which would not.
We started to prototype CUI technologies using simple electronics, cell phones, tablets, ear buds, and roleplaying. This started to introduce us to the types of interaction design considerations we needed to keep in mind. We realized, for example, that a CUI alone was not enough. Visual communication is also an important way to quickly communicate time-sensitive information, so we decided to test a combination of CUI and GUI. We also learned more about the CUI and the investigator relationship—what aspects were useful and which were not.
Testing Prototypes with investigators
Next, we refined our low-fidelity prototypes and created three different prototypes to test with investigators. We called these prototypes Active Listener, Guided Reflection, and Sense Maker.
Active Listener Prototype
Active Listener was designed to help investigators become better listeners while interviewing witnesses, suspects, etc. The idea was that the CUI would listen to the conversation and take notes or transcribe the conversation for the investigator so that he or she could focus on the conversation. The CUI could then match notes from the conversation to data points in various police databases to find relevant information that could be helpful in the investigation.
Guided Reflection Prototype
This concept helps investigators think about what they did throughout the day, reflect, revise their reports, and share their ideas. It combines CUI with GUI, giving both audio and visual feedback.
Sense Maker Prototype
This concept also combined CUI and GUI to help investigators visualize data and organize it in ways that help them pull out important insights.
The investigators reacted positively to the Active Listener and Guided Reflection Prototypes. The Sense Maker Prototype was less appealing to them because of the complexity of the concept. They also gave great feedback relating to their concerns and excitement about the concepts. Their feedback helped us refine the concepts.
Refining and Communicating Our Ideas
Starting with a Video Sketch
With feedback from investigators, we made final refinements to our concepts including what kinds of artifacts would be involved and how the interactions work. Then we started storyboarding for the final concept video. We made a video sketch to make sure we communicated the main points of our concepts.
Video Sketch Password: msdesignexpo
Making the Final Video
Finally, we started putting together the final video. We made props for the artifacts, used special effects to show the GUI elements, and recorded our lines. The final video can be found at the top of this page.