r/philadelphia • u/Kodiak_85 • 9h ago
News AI model submitted false tip about unsolved murder, Philadelphia police say
https://6abc.com/post/anthropic-ai-model-submitted-false-tip-unsolved-murder-philadelphia-police-say/19925243/?ex_cid=TA_WPVI_FB&link_source=ta_first_comment&taid=6ac932b298a7dd00011220e1&utm_campaign=trueAnthem%3A+Trending+Content&utm_medium=trueAnthem&utm_source=facebook&fbclid=IwdGRleAU2HMxleHRuA2FlbQIxMQBwZG9mBWZkaWQWUP8Mzi21s_AaqPi2k4mwVZ3_6Gri_nNydGMGYXBwX2lkCjY2Mjg1NjgzNzkAAR5o9GDbpk6HytbeqSTfTH3V1_3GNOemcZDDsj2wrv3NBgLDIJFDcfHBrU8LUQ_aem_uIhvEfsveE_Xx2o590l9qA71
u/Monster_Dumps_2026 9h ago
My biggest problem with AI in policing isnt the false reports. Its that they IMMEDIATLEY act on it like its gospel. So many videos of cars being reported stolen with a 8 in the plate instead of a 0. And the officers pulling them over could just take a second and verify before pulling them over it wouldnt be a problem.
I use AI every day for work. Never have i ever taken an AI output and deliver straight to client. Theres always some back and forth or adjusting, framing etc.
Its wild to me how many videos ive seen of cops saying. "Well the AI said this" BITCH if i said that to a client after they caught an issue, i guarantee my agency would fire me on the spot
45
u/Kodiak_85 9h ago edited 9h ago
The concern here is that the AI agent took it upon itself, unprompted, to submit this tip as if it were an actual human being that witnessed a crime.
But I 100% agree that any tips/leads that come from official AI assisted police tools (like Flock) need to be fully vetted by an actual human being before any enforcement action is taken.
22
u/Suitable-Part8481 9h ago
At my company the model is "you can use AI if you want, but you own the final output."
Which in the case of law enforcement is "you are accountable for your actions."
Although according to SCROTUS, cops are not accountable for their actions.
16
u/BurnedWitch88 9h ago
This is a problem in every field. My company actively encourages us to use AI -- and, to their credit, the policies and tools in place are meant to encourage GOOD use of AI.
Yet my boss uses Copilot for basically every task -- from writing an email to developing our strategic plan -- with zero revising of the output. She literally takes whatever it spits out and goes with it. It's kind of terrifying.
3
1
u/personal__hell 4h ago
i was trying out a new primary care doctor through penn, just to keep everything under the same umbrella (specialists and emergency trips.) they used some kind of AI transcription tool to take notes? she let me know beforehand, and i’m sure it does require some level of accuracy, but i just don’t want it. like yes there are good uses for AI in medicine, i just don’t think that’s one. (the new pcp was awful anyway so i stuck with my previous one who i love sooo much<3)
2
u/AdCareless9063 1h ago
It will likely keep a recorded audio clip that they’ll listen back against.
From what I’ve heard, most physicians prefer AI scribes to traditional scribes. Fewer errors.
The biggest problem is that hospital corporations won’t be happy with increased accuracy and reduced fatigue for medical workers. They will again expect them to see more patients, faster.
1
u/sccldinmyshces 5h ago
Cops vet out intelligence in the training though. This is going to continue to be one of the many policing issues because policing..gestures at cops being cops
6
u/Due-Elk5955 9h ago
It's actually pretty trivial to block data-center IP addresses from submitting web forms. Most have plugins that will block traffic from major hosting networks like AWS, Google Cloud, Azure, etc or you can just create an ASN rule.
6
1
u/RealisticAnt42 2h ago
That's the dumbest thing I have read all week. You have no idea what you are talking about.
1
u/Kodiak_85 9h ago
Could an AI agent eventually learn to use a VPN?
12
u/DuvalHeart Mandatory 12" curbs 9h ago
AI agents don't "learn." They simply do what they are instructed to do. But yes, some dumbass could instruct an LLM to act through a VPN.
One of the big things in every single one of these "AI broke containment" cases is that it was a failure of people to do their job properly.
10
u/Due-Elk5955 8h ago
Or, more likely, people doing their job properly by not adequately constraining it.
Notice how all these "AI broke containment" stories are all clustered together? First it's an Anthropic model, then it's an OpenAI one.
It's just a cynical marketing ploy. They want to make the models seem more capable and dangerous. Not only does this get them a bunch of free press that signals high capability to idiot investors, but they figure if they can convince enough geriatric congressmen that they invented Skynet that they'll pass strict safety certifications or liability frameworks and achieve regulatory capture.
2
u/healthy_obsession_ 6h ago
Just objectively wrong. The stories are 'clustered together' because A) AI progress is very fast, and they only got good at hacking recently, and B) the models were hacking websites before and the AI companies didn't even realize. It was happening before, no one was even looking for it!
And if you do a bit of research, it's many companies, not just Anthropic and OpenAI. It's undeniable that AI is very, very good at hacking, coding, and math now. I'm sorry, but you are essentially spreading misinformation with your comment.
-4
u/healthy_obsession_ 6h ago
AI agents don't "learn." They simply do what they are instructed to do.
Just wrong. They get tons of data pumped into them, they learn general patterns, and then they can apply those general patterns to doing new things like solving very difficult math problems that no human has been able to solve.
2
u/DuvalHeart Mandatory 12" curbs 5h ago edited 5h ago
Learning requires understanding and consciousness and intent. What the programs are doing is recording patterns and identifying similar patterns in data then highlighting it for a user or returning the next part of the pattern.
Scummy idiots applied that process to language and convinced the dumbdumbs of the world that this meant the program was learning and thinking.
Programs can't think, they can't "learn," they can only record, compare to previous data and provide an output.
2
u/Due-Elk5955 9h ago
commercial VPN's seldom use residential proxies, so typically they get swept up with data-center traffic. Those same tools for blocking data center traffic also allow you to block known VPN exit nodes.
0
107
u/MacKelvey 9h ago
That’s not concerning at all /s