Tori Breitling
UX/Product Designer

Office of Police Oversight

Creating Austin's first online method for residents to give feedback
about their interactions with the Austin Police Department

Context

The Office of Police Oversight is a new city office made possible by a new police contract negotiated in November of 2018. This office is the first civilian office in Austin's history. The office's goal is to improve, through monitoring and transparency, the relationship between the residents of Austin and the Austin Police department.

Challenge

One of the first goals of the office was to change the process by which residents could give feedback about their interactions with the police force. Our office was tasked with creating an online feedback form – the first of it's kind in Austin–that residents could use either anonymously or not. The online aspect was in and of itself a leap - previously residents had to fill out a form describing what happened, along with a notarized affidavit.

Obviously, this requirement was not one that made it easy for less affluent or marginalized and at-risk residents. There was no option for anonymity. In fact, historically, the majority of complaints filed were by white women.

Objective

Requirements were largely driven by subject matter experts within the city of Austin, community organizations, and research done previously by our Service Design Lab team.
Hard requirements were creating a complaint and thank you form, in both english and spanish.

Team

I'm part of a cross-disciplinary team that consists of:

• 3-5 content strategists

• UX designer (myself)

• Visual designer

• 3-5 developers

Concerned giraffe.

Concerned giraffe.

Out of Wireframes

I was vacationing in South Africa when this project came in. Because of a sense of top-down urgency, the team decided to use the United States Digital Services form library as their base using a mock up from our Service Design Lab team. Essentially this project went directly to high fidelity.

The library was fairly brittle in the end, as it was designed for simple intake, and USDS stopped maintaining it before we launched.

The complaint form design

Special considerations

The idea of the complaint form was simple – allow the resident to enter information about what happened over a series of six steps. The reality of it was quite different, due to the inherent trust issues. The copy needed careful consideration, as well as the order in which questions were asked.

Our primary objective was creating a form that educated and instilled trust in the Office of Police Oversight. We knew from the research that residents had little trust that any city department was out of reach of the police department.

  • What information is actually required? This is directly related to the information the office needed to conduct a viable initial investigation. Additionally, we wanted to be careful about how we asked for personal information so that residents felt secure in telling their story.
  • How do we reinforce the notion of civilian oversight and anonymity? We knew from the research that police retribution was a very real fear in the communities we wanted to reach.
  • How do we structure the form in a way that supports the resident in reporting their experience, which may have been traumatic?
The team settled on a series of five steps, including a review page followed by a confirmation. Several steps used progressive disclosure to show additional inputs if the user selected

The team settled on a series of five steps, including a review page followed by a confirmation.
Several steps used progressive disclosure to show additional inputs if the user selected "yes".

User Testing

One-on-one test with a resident.

One-on-one test with a resident.

Testing overview

We ran several rounds of tests, one with English users, at a library which is frequented primarily by people of color. Another round was done with Spanish speakers. We also did a test with a blind user using a screen reader.

Our first test was with an early build. We set up at a library, and asked residents to "pretend" to report a negative incident with the police. Most had personal experiences they recalled though we made sure they knew that they could manufacture details if they preferred. We tested on a laptop primarily in order to enable recording the session, though several refused to be recorded. For one test we were able to use a participant's android device. We tested with six residents in total. Besides looking for any clues with regards to establishing trust, we primarily we were interested in usability of the location widget, the date and time widget, and the comprehension for field labels throughout.

Usability insights.

Residents were indeed suspicious. In fact, several residents refused to be videotaped for testing once they were told what the study entailed.

Overall the tests went well, but the map/location widget tested poorly. A few issues we discovered were:
• Residents could either drag the map or enter an address as a way to input. Our first version was missing zoom in/out capabilities, which made it very onerous to work with if the incident was far away from the default location, and the user had chosen to drag instead of entering an address.
• One resident somehow got into a 3d mode, which we later disabled.
• We needed the map to be locally biased, but still allow any address to be entered, ie, a police officer's misconduct may still be reported even if it takes place in another state.

There were a few other label changes we made, as well as some interaction idiosyncrasies on the review page. The USDS library utilized odd interactions with regards to editing from the review page. We modified it as much as we could, but some remain as they require too much reworking of the library to warrant fixing given our limited resources.

Screen reader modifications

The test with the blind reader turned up a couple of things:

  • the date picker widget didn't work at all.
  • The progress bar was completely lost on a blind user. Visually impaired users would have no way of knowing how long the form was; we added text descriptions to this image to fix it.
  • the way the review page was set up created unintentional false positives on answers because of the way the screen reader read the columns.


Spanish

  • one user reported that the form seemed so simple they felt it might be fraudulent, and another commented about not seeing a city seal.
  • several users commented that several of the words were uncommonly formal. We adjusted the translations to be more colloquial.
Office of Police Oversight
Office of Police Oversight
Second iteration

Second iteration

Outcome

Reporting has definitely increased, and the Office of Police Oversight is thrilled with it's success. Over 140 complaints have been filed online since launch late last year. About one-third of those used the Spanish form. As community awareness about the form increases, no doubt the form will receive even greater traffic.

There are a number of improvements that could be made to the form still, but there's no budget for additional improvements at this time, and the team will not be using the USDS form library moving forward.