One California town found Flock's license plate cameras were wrong 71% of the time

Daniel Sims

Posts: 2,500   +76
Staff
In brief: The controversy surrounding Flock cameras primarily centers on privacy and fears of a surveillance state. However, the cloud-based license-plate-reading system's accuracy and effectiveness have also drawn scrutiny. In one particularly egregious case, an internal review uncovered more than 1,000 false positives over two years.

Police in Roseville, California, found that, out of 1,427 incidents in which the town's AI-powered license-plate cameras flagged a vehicle as stolen or used in a felony, the cameras misread the plate in 71% of cases.

Flock, the startup that supplies the cameras to Roseville and approximately 6,000 other communities across the US, attributed the errors to Roseville's atypical camera setup, but as Business Insider reports, the results could expose flaws in an already controversial technology.

Flock's cameras, typically placed on traffic light poles and roadsides, photograph passing vehicles and analyze their characteristics, including license plates, color, make, model, and other details via machine learning. Authorities can search the company's cloud database for vehicles using descriptions to track their movements, which they say has helped solve crimes and find missing persons.

The AI cameras process approximately 20 billion vehicle scans per month and have raised fierce debate across the US. Communities across the country have deactivated their cameras since last year out of concerns about state surveillance, but some towns, including Roseville, are also dealing with the system's blind spots.

There, a police supervisor joked about memorizing one driver's license plate because Flock's cameras misread it at least six times. In each case, the cameras mistook a 9 on his license plate for an 8. This is more likely to happen if a camera takes a blurry image or if the license plate is obscured by a tree or license plate frame.

Ironically, while Flock claims that the errors stem from how Roseville deployed the cameras, the suburb's unique implementation likely also prevented matters from worsening.

To avoid recording drivers' faces, Roseville configured the cameras to only capture vehicles from behind, which Flock says makes them less accurate. However, none of the false positives led to traffic stops, much less arrests, because Roseville officers must verify each report, which is not required in California.

Meanwhile, the LAPD recently let its contract with Flock expire after an internal audit found that 161 innocent people were pulled over in a two-month span. Outdated or inaccurate data from other jurisdictions was cited as a primary cause.

Critics have also pointed out that, despite license-plate reading being their primary function, the AI cameras can also analyze and track people on foot. Police have searched the database for people based on extremely vague descriptions, and numerous officers have been caught using the system to stalk ex-partners.

Permalink to story:

 
Something is off here.
OCR is nearly perfect already, I can't imagine how it could be wrong in 71% of the cases. Even 10 years ago failure rates were notably lower.

Misreading 71% of the plates in such a way that the wrongly read number matches a real number of a car involved in a crime - that's outright absurd. It simply can't be.

Anyway, if the cameras are wrong in 71% of the cases, we should forget our privacy concerns :)
 
Something is off here.
OCR is nearly perfect already, I can't imagine how it could be wrong in 71% of the cases. :)
Because the article has -- unsurprisingly -- lied to you. The system isn't wrong in 71% of cases, it was wrong in 71% of the automated false positives it sent to police. Those 1400 automated alerts are likely the result of tens or even hundreds of millions of plate reads. And (though you have to look at the underlying report to find this out) at least 6 of those came from just one vehicle owned by a man whose plate cover kept causing misreads.

Also left out of the reporting were these interesting facts:

"...Flock blamed Roseville's atypical deployment — older hardware, cameras mounted higher and farther from roadways than recommended, and rear-only plate capture ... Roseville says Flock has helped solve crimes and continues using the system....."
 
Because the article has -- unsurprisingly -- lied to you. The system isn't wrong in 71% of cases, it was wrong in 71% of the automated false positives it sent to police. Those 1400 automated alerts are likely the result of tens or even hundreds of millions of plate reads. And (though you have to look at the underlying report to find this out) at least 6 of those came from just one vehicle owned by a man whose plate cover kept causing misreads.

Also left out of the reporting were these interesting facts:

"...Flock blamed Roseville's atypical deployment — older hardware, cameras mounted higher and farther from roadways than recommended, and rear-only plate capture ... Roseville says Flock has helped solve crimes and continues using the system....."
It's wrong in 71% of cases where it flagged a plate for a felony. Even if it scans millions of plates, that means that when it DOES pick something up, it has an atrocious false positive rating.

At best, this means police departments get flooded with junk data that takes time and resources to sit through, and at worst, each one of those gals positives can ruin someone's life with court battles and legal proceedings from police forces that don't care about your civil rights.
 
It's wrong in 71% of cases where it flagged a plate for a felony.... it has an atrocious false positive rating.
It's wrong in 71% of the automated reports it sends ... the normal method police use this is to enter a tag number, and have the system output any matching hits, a mode where accuracy is higher.

One of the first things you learn when dealing with machine intelligence is the difference between accuracy and precision, and balancing false positives against false negatives. In this case, a false positive incurs little cost: less than 1000 flags in two years of operation, each of which takes an officer only seconds to manually examine, and see the tag was misread. Balance that against the 400+ felony hits it DID find automatically, exactly zero of which would have been located without this system.

Finally, you ignore the elephant in the room: the only reason this town is seeing such a high false positive rate is they're using older cameras and mounting them far from roadways, where visibility is bad. They can correct that easily enough ... or even easier, they can simply adjust the confidence factor in the software itself, lowering false positives (but dramatically increasing the number of actual felonies missed.


...at worst, each one of those gals positives can ruin someone's life with court battles and legal proceedings from police forces that don't care about your civil rights.
Yeah, that didn't happen. Police procedures -require- these hits to be verified. And if the police skip this, they can't "ruin someone's life with court battles and legal proceedings" .. because they have the wrong tag number. Did you honestly not think this through? If they get a report that, yesterday at 10:49 PM there was a stolen white Mercedes with tag # "ABC" at the corner of Lincoln and Elm -- but it was really you there, with tag number "ABD" instead ... how do you expect them to find and arrest you?
 
Last edited:
Of course the police will NEVER admit to any wrong doing when they pull people over who shouldn't be pulled over. Have you ever seen a law enforcement, have a press conference to admit they made a mistake about anything? I'm 68yrs old, and I've never seen it!
 
Of course the police will NEVER admit to any wrong doing when they pull people over who shouldn't be pulled over. Have you ever seen a law enforcement, have a press conference to admit they made a mistake about anything? I'm 68yrs old, and I've never seen it!
Shoot first. Ask later.
If nobody left there to answer, you were right.
Dead bodies do not tell tales.
 
...Have you ever seen a law enforcement, have a press conference to admit they made a mistake about anything? I'm 68yrs old, and I've never seen it!
MSN: "Police Chief Admits Mistakes After Misidentification Arrest"
News Now: "Henderson police admit telephone number error leading to veteran’s wrongful arrest"
AP: "Canon City Police admit error in DUI arrest of diabetic man"
Fox Chicago: "Bodycam footage shows an officer admitting, "I f***ed up," after wrongly detaining Derrick House...."
Yahoo News: "Police confirm that "clerical error" attributed fault to wrong driver in Kyle Shanahan crash"
Valley News: "Fargo Police Chief admits errors in mistaken arrest case"

Your post is particularly ironic, given the entire reason this story exists is because the Roseville police department issued a public statement on the high error rate.
 
Of course the police will NEVER admit to any wrong doing when they pull people over who shouldn't be pulled over. Have you ever seen a law enforcement, have a press conference to admit they made a mistake about anything? I'm 68yrs old, and I've never seen it!
Have you ever seen a modern day leftists admit mistake about anything in any forum. Much less a press conference.
 
Back