Aurich Lawson |
If you were suspected of a crime, would you trust a chatbot to explain exactly what happened?
Some police departments believe the technology is ready. And officers who have begun using chatbots to quickly complete their most dreaded task, writing police reports, apparently don't want to go back to spending hours each week doing their own paperwork.
In June, a police department in Frederick, Colorado, boasted that it was the “first law enforcement agency in the world to deploy Axon Draft One,” a new type of police technology that allows a chatbot to issue AI-generated police reports almost immediately after a body camera finishes recording a police operation.
Powered by OpenAI's GPT-4 model – which also powers ChatGPT – Draft One was first introduced to police departments around the world in April. Axon, a billion-dollar company known for its Tasers and body cameras, touted it as a “revolutionary new software product that creates high-quality police reports in seconds based on automatically transcribed audio from body-worn cameras.” And according to Axon, cops couldn't wait to try it out, with some departments eagerly participating in the testing.
Ars confirmed that the Frederick Police Department was the first agency to purchase the product in May, and that an unknown number of other agencies across the U.S. soon followed.
Draft One relies solely on body camera audio (not video) and essentially summarizes the key points of a recording, similar to how AI assistants summarize the audio of a Zoom meeting.
This may seem like an obvious application for AI, but legal and civil liberties experts warn that the humble police report is at the root of the entire justice system, and tampering with that report could have serious consequences. Police reports not only influence settlements, sentencing, evidence gathering, and trial outcomes, but also the way society holds police accountable.
“The mandatory function of writing down a reason, then swearing to its truth, and making that record available to other legal experts (prosecutors/judges) is a check on police power,” wrote legal expert Andrew Ferguson in the first law journal article analyzing the potential impact of Draft One compared to human reports. In addition, “police reports also serve as the factual basis for civil suits and insurance claims,” Ferguson noted.
By introducing chatbots known to hallucinate, mistake jokes for facts or arbitrarily add false information, police technology like Draft One could be used to legitimize wrongful arrests, increase police suspicion, mislead courts or even cover up police violence, experts warn.
Noah Spitzer-Williams, Axon's manager of AI products, told AP News that unlike ChatGPT, Draft One is less prone to hallucinations because Axon “has access to more knobs and dials than a real ChatGPT user.” Because Axon has turned down the “creativity dial” on Draft One, the AI tool is supposedly better able to avoid embellishments and stick to the facts, Spitzer-Williams claimed.
Axon is promoting Draft One as a way to save police officers time when writing police reports and is encouraging police departments to slowly learn how to use the AI assistants. In the press release, the company promises to “innovate responsibly.”
To minimize potential harm, early adopters like the Frederick Police Department were advised to limit the use of chatbots to generating reports on minor incidents and charges. Only once officers have gained enough experience “in effectively using the tool” on “low-severity reports initially” should they “expand to more serious reports,” Axon recommends in the press release.
Despite official advice to limit early use, Axon CEO Rick Smith publicly touted Draft One as a method that could put an end to the unnecessary work that exists in increasingly underfunded police departments around the world.
“Every single officer in the United States writes police reports, often every day and typically multiple times a day,” Smith said in the press release. “As we did with Draft One, harnessing the power of AI will prove to be one of the most impactful technological advances of our time to scale policing and revolutionize the way public safety works.”
Shortly after police departments began implementing Draft One, Matthew Guariglia, a senior policy analyst who tracks police use of AI for the digital rights group Electronic Frontier Foundation (EFF), wrote a blog post warning that the increasingly widespread use of Draft One urgently requires closer scrutiny.
“We just don’t know how it works yet,” Guariglia told Ars.