Tori Breitling
UX/Product Designer

Reporting police conduct online

Context

The Office of Police Oversight is a new city office made possible by a new police contract negotiated in November of 2018. This office is the first civilian office in Austin’s history. The office’s goal is to improve, through monitoring and transparency, the relationship between the residents of Austin and the Austin Police department.

Challenge

One of the first goals of the office was to change the process by which residents could give feedback about their interactions with the police force. Our office was tasked with creating an online feedback form–the first of its kind in Austin–that residents could use either anonymously or not. The online aspect was in and of itself a leap. Previously, residents had to fill out a form describing what happened, along with a notarized affidavit, and turn it in to the office, which was in an out of the way office park in North Austin, and on the second floor, which required signing in at the front desk. Obviously, this requirement was not one that made it easy for less affluent or marginalized and at-risk residents to file complaints. There was no option for anonymity.

Objective

Requirements were largely driven by subject matter experts within the Office of Police Oversight. Each piece of information that was asked for was considered in the framework of what was required to conduct an initial investigation and to ensure that it was sufficient to use and triangulate with police records. Hard requirements were creating both a complaint and thank you form, in english and spanish.

My role

• Design lead, overseeing IxD, copy, and development

• Writing test plans, conducting 1-1 tests with English speaking residents

• Office of Oversight interaction 

The complaint form design

The idea of the complaint form was simple – allow the resident to enter information about what happened over a series of six steps. The reality of it was quite different, due to the inherent trust issues. The copy needed careful consideration, as well as the order in which questions were asked.

Our primary objective was creating a form that educated and instilled trust in the Office of Police Oversight. We knew from the research that residents had little trust that any city department was out of reach of the police department. This sentiment was also glaringly apparent as we attempted to test the form with residents.

As with any form, we needed to understand what information was actually required. This is directly related to the information the oversight office needed to conduct a viable initial investigation. Additionally, we wanted to be careful about how we asked for personal information so that residents felt secure and safe from retribution in telling their story.

Essential to this objective:

• How do we reinforce the notion of civilian oversight and anonymity?

• How do we structure the form in a way that supports the resident in reporting their experience, which may have been traumatic?

The team settled on a series of five steps, including a review page followed by a confirmation. Several steps used progressive disclosure to show additional inputs if the user selected

The team settled on a series of five steps, including a review page followed by a confirmation.
Several steps used progressive disclosure to show additional inputs if the user selected "yes".

Testing overview

We ran several rounds of 1-on-1 talk-aloud tests. One was conducted with English users, at a library which is frequented primarily by people of color. Another round was done with Spanish speakers. We also did a test with a blind user using a screen reader.

Our first test was with an early build. We set up at a library, and asked residents to “pretend” to report a negative incident with the police. Most had personal experiences they recalled, although we made sure they knew that they could manufacture details if they preferred. We tested on a laptop primarily in order to enable recording the session, though most refused to be recorded. For one test we were able to use a participant’s android device. We tested with six residents in total. Besides looking for any clues with regards to establishing trust, we primarily we were interested in usability of the location widget, the date and time widget, and the comprehension for field labels throughout.

Usability insights.

  • Residents were indeed suspicious. In fact, several residents refused to be videotaped for testing once they were told what the study entailed.

  • Overall the tests went well, but the map/location widget tested poorly. A few issues we discovered were:

  • Residents could either drag the map or enter an address as a way to input. Our first version was missing zoom in/out capabilities, which made it very onerous to work with if the incident was far away from the default location, and the user had chosen to drag instead of entering an address.

  • One resident somehow got into a 3d mode, which we later disabled.
    We needed the map to be locally biased, but still allow any address to be entered, ie, a police officer's misconduct may still be reported even if it takes place in another state.

  • There were a few other label changes we made, as well as some interaction idiosyncrasies on the review page. The USDS library utilized odd interactions with regards to editing from the review page. We modified it as much as we could, but some remain as they require too much reworking of the library to warrant fixing given our limited resources.

Screen reader modifications

The test with the blind reader turned up a couple of things:

  • The date picker widget didn't work at all.

  • The progress bar was completely lost on a blind user. Visually impaired users would have no way of knowing how long the form was; we added text descriptions to this image to fix it.

  • The way the review page was set up created unintentional false positives on answers because of the way the screen reader read the columns.


Spanish

  • One user reported that the form seemed so simple they felt it might be fra• Several users reported that the form seemed so simple they felt it might be fraudulent, and another commented about not seeing a city seal.

  • Several users commented that several of the words were uncommonly formal. We adjusted the translations to be more colloquial.

  • The map widget continued to be an issue for users. Scrolling seems to be more effective than typing. 

  • There are several places within in the form where the idea of anonymity could be reinforced or clarified.

A few changes:

Outcome

In the first three months, the Office of the Police Monitor received 71 submissions through the online form, compared with 10 in the old system. The chart below shows the increasing relevance of this work, especially with regards to the BLM protests during the summer of 2020. The APD response to these protests resulted in a public outcry, and as of this writing, the City Council proposed cutting the APD budget by one-third and redistributing those funds to social services.

Note: I don’t have official numbers, but about one-third of the complaints are submitted using the Spanish form.