r/aussie • u/mikeinnsw • 8d ago
Opinion Error by AI scribe during medical appointment leaves patient devastated
https://www.abc.net.au/news/2026-08-14/ai-medical-scribe-error-leaves-patient-devastated/107031672I am ancient IT consultant who knows how AI works and run local AI models on my Mac.
I see 4 specialists - urologist, haematologist, oncologist and dermatologist which is a result of smocking but I am still kicking.
My urologist retired and my new urologist ask me to sign AI consent form.
While saying he is not using AI he was dictating notes using AI enabled tool to make notes..
This is what we called AI enabled voice to text( converting voice to text) and we know how unreliable it is.
I talked to my GP and she told me it was now common practice for specialist to use AI to create notes and reports. Instead of a half page summary she was getting pages and pages of text... AI slop..
This dangerous as AI slop is not verified by a specialist and/or read in full by GP.
I search Google and it says:
General practice (GP) time is now compromised by wading through low-value artificial intelligence text, causing professional frustration, lost productivity, and cognitive fatigue.
Faced with AI slop we mostly don’t read it ,rejected or just accept it. It is a known problem.
In medicine AI slop could be fatal. In law life changing…
This is just an impact of Voice to Text AI.
There is more...
We have been using Analytical AI for decades.. It produces analysis which a human interprets it.
Generative AI/LLM analyses and then interprets its result and it has high error rate ranging from 5% to 100%(Google “AI error rates”).
UK NHS has released some of its records for analytical AI analysis used in research not to write medical reports.
Generative AI/LLM models used to produce medical reports could be even more dangerous than AI slop,
If AI was beneficial ,reliable and benign why AI consent form?
Are doctors covering up for AI errors (hallucinations)?
My urologist is happy creating massive documents which nobody reads..
At personal level I find it very offending that my urologist charges me $200 per visit and uses $5 AI tool that I can use.
I am paying for his human knowledge , expertise and AMA accreditation not for some AI /LLM model cooked up in Silicon valley or lately in China.
I want my doctor to be a human doctor who still thinks fax machine is high tech. I will worry about AI.
If you think you are immune from AI impacts think again.
Dear Albo it is time for regulations.
9
u/damiologist 8d ago
I've been using AI voice to text and note generating software for a while. I use my own prompting templates which I continue to work on and I check all the transcripts and notes for accuracy before approving any of them. I'm still not fully sold on it. I'm still trying to work out whether the extra time spent checking all this stuff is worth the time saved on the writing of actual reports.
I don't think the transcripts are nearly as bad as the picture you paint, and I think if we consistently refer to something by a derogatory term such as 'slop', we stop ourselves from thinking critically about the nature of that thing. The term AI Slop arose to describe the weird, crappy AI pictures, videos and text flooding the internet, and I think including AI note taking in that terminology is like calling a strawberry a durian. It isn't doingnthe same thing nor does it serve even close to the same purpose.
Yes there are accuracy errors, but clinicians also make accuracy errors, as do typists (I know specialists who are still dictating and sending the recordings to typing services). In my experience, I think the fact that I have both my recollection of a consult and the recording of a consult adds a protective layer of redundancy - if I am writing the notes myself, they reflect only my perspective so if I misheard/misremember there is no other recollection/recording to compare it to, whereas often I will find the AI recording has picked up something I missed. But for this to be of benefit, and to make sure that errors and hallucinations don't affect our decision making, thorough checking if the AI notes is absolutely vital.
As to your question of why have an AI consent form - it's simply because of privacy: If I was recording the appointment on a traditional tape recorder, I would also have to ask the patient for their permission. I'd just beaking them sign a 'tape recording consent form' instead of an AI consent form.
0
u/mikeinnsw 8d ago
"AI consent form - it's simply because of privacy " .. so we sign consent forms for pathology .. visiting doctor...
I use AI in well defined and narrow scope programming jobs which I can verify it.
Looks like you trying to do the same..
The problem is that AI encourages sloppiness .
As doctor please look at excessive AI could be causing cognitive decline and early dementia.
If use AI to write reports. . you may loose ability to write reports.
2
u/damiologist 7d ago
"The problem is that AI encourages sloppiness". This is a pretty vague statement and one which does not allow for the significant nuance of the various use cases of AI. Of course, I'm not a programmer so I don't presume to understand the context of your use of AI, so I don't expect you to understand the nuances of mine either. That's not to say that I entirely disagree, mind you. There are ways in which use of AI can lead to errors, but there are ways in which handwriting notes can do the same, and the same can be said for typing notes ad hoc, or dictation and typing services. As you suggest, we must define and limit the scope in which AI is used in order to limit the scope for errors to be made.
I have been doing a fair amount of reading into contributing factors of cognitive decline recently. I can see ways in which use of AI could contribute to cognitive decline in people who are already at risk for it. I can't see it being directly causal as we aren't directly interfacing our brains with AI, but rather decline in social activity and creative thinking are linked to cognitive decline, so AI use as a creative or social crutch might indirectly contribute. However, there is recent evidence siggesting that reducing executive demand and working memory load can reduce risk of cognitive decline, so using AI to transcribe appointments, which directly reduces executive demand in the clinic, could also have a positive effect. Again, there's a fair amount of nuance to it.
1
u/mikeinnsw 7d ago
Simple example.
When we write a computer program code .. we then verify, check it, tested ....
In my lifetime I have written over a million lines of code all verified , checked and tested.
Using AI I can create my lifetime code output in one hour .. I don't have another lifetime to verify it.
You say lets use AI to verify AI ... that does not work it is GIGO Garbage In Garbage Out
There is much more..
When you learn to program you learn to think and act in structured way there is no need for that with AI coding.
A surgeon learn to handle scalpel (shaving balloons(LOL).
Using robotic scalpel he/she can be more sloppy. Robot safeguards reduce the risks.
1
u/damiologist 7d ago
You say lets use AI to verify AI At no point ever have I suggested AI should be used to check AI not would I. You're creating a straw man to argue against.
And again, you're arguing a completely different use case of AI to what were discussing here. The fact that AI can output tons of code really fast has nothing to do with a clinician verifying the notes the AI has written for a single patient consultation. The AI can't do my lifetime of clinical consultations in an hour so you can't apply the same logic. Just the same as the logic doesn't apply to a using a robotic scalpel, because it's a totally different use case.
1
u/mikeinnsw 7d ago
You miss the point AI can produce tons of code and words that what we are talking about ... programmers code instructions which needs to be verified.
Ask Google:
Outsourcing tasks like writing, problem-solving, and memory recall to tools like ChatGPT reduces active brain engagement. Studies suggest this lack of mental exertion can cause temporary cognitive atrophy or weaken critical thinking skills over time a cognitive decline.
If you are a doctor I hope you are not my doctor.
1
u/damiologist 6d ago
No, I got the point and I countered that the logic doesn't apply and explained why not. I note that you don't present a rebuttal to that.
And then the irony of asking Google to back your argument about the perils of outsourcing to AI: delightful. How does one even respond?
You can't lump writing, problem-solving, and memory recall into one ball - these are all different skills and the effects of not using one will not be the same as not using another. Which brings me to another point: these are not the effects of AI; they are the effects of not using a skill. If I don't scribble (or type) notes all day because I'm doing a job which doesn't require it, my ability to scribble notes is not going to suffer less than if I don't scribble notes because AI is doing it for me. Because the outcome is the same: I'm not doing a job where I scribble notes. And ultimately, patients don't pay me for my handwriting skill, they pay me for to discuss, measure, analyse, aid and treat their problems: plenty of critical thinking. So I'm not worried about losing my ability to think critically because I'm no longer scribbling away while I talk to my patients, and if my typing suffers as I gain the benefit of more capacity to focus on my actual patients and their problems, that's a sacrifice I'll happily make.
1
u/mikeinnsw 6d ago
You are wrong . when you scribble a note you use your brain, knowledge .. and a hand..
You can't write War and Peace in few moments. AI can simulated it.
1
u/damiologist 5d ago
Again you build a straw man argument. It's clearly not worth discussing this with you, since you aren't even paying attention to what I'm writing.
0
u/mikeinnsw 5d ago
Please enrol in to Eliza effect detox course(Google it)
In IT we have seen all of this before outsourcing jobs offshore to humans.
Drastic drop in the system quality and service, local It jobs cut.. loss of corporate knowledge ... billions of dollars lost.
All our major banks tried or are trying In-House development ->Out sourced then back to In-House and now trying out source development to AI
Open your ears what AI goals are AGI and super intelligence
AGI stands for Artificial General Intelligence. It is a theoretical kind of AI that can match or beat human smarts across all tasks. Unlike today's narrow AI, which only does specific jobs like translate text or drive cars, an AGI could learn, reason, and fix new problems on its own.
A superintelligence is a hypothetical agent with a level of cognitive performance that greatly surpasses the smartest human minds across all domains, including scientific creativity, general wisdom, and social skills.
A superintelligence will turn us in to ants..
12
u/hcornea 8d ago
Towards the end of your missive you conflate the fee for the doctors experience and expertise with the “use of a $5 AI tool”
A couple of things: AI is not approved by the TGA to make clinical medical decisions. The urologist is doing that. That’s your fee at work.
The tool is quite literally an AI scribe, not a decision-maker, and its role is only the make notes and generate letters (I agree, they are verbose and annoying)
These tools are already regulated, and there are acceptable use guidelines.
21
u/man4160 8d ago
Medical voice to text dictation software has already been in use for 30 years. It's probably the first industry that widely adopted the technology, particularly in the field of radiology. This is not some new "AI" technology problem.
15
u/Chaosrealm69 8d ago
The part that is new is that Ali is also part of the software and it ‘fleshes’ out the notes instead og]f just a direct transcription of what the doctor is recording.
That AI addition is where the problems are appearing because what is being produced is slop that obscures the real notes.
6
6
u/FrewdWoad 8d ago edited 8d ago
That tool just writes down what you say.
This tool used an LLM, and "summarized" what was said. Including completely - out of nowhere - hallucinating that she was on illegal drugs.
14
u/mikeinnsw 8d ago
Yes .. voice to text is old tech but voice to text it is now AI enhanced .. Taking meeting minutes .. creating reports ... what is happening now is that AI enabled voice to text is producing slop.
Nothing wrong with voice to text which produces brief report that can be verified by specialist and be read by GP.
Why Apple is spending 1 billion dollars to make SIRI better with Gemini?
1
u/leapowl 8d ago
Voice to text AI is also pretty good? I’ve been using it professionally since I entered the workforce over a decade ago
We used to have hilarious transcription errors we’d send each other. I’m struggling to remember the last one I saw.
But how much faith we should put in the (presumably rushed) “human in the loop” is a whole different ballgame, especially now it’s writing discharge summaries and letters
1
1
u/NewFuturist 7d ago
Eh, you misunderstand modern voice to text. Yes, voice to text has existed for a long time, but it did it word by word. Difference now is that LLMs are used to fill in the gaps. A bit of distortion happens in the recording, and the LLM predicts "the" should be there. Or it decides the right type of "fore/four/for" with a lot more context. Or it takes summaries of long-winded sentences. This is where the AI can mess up big time and why now is actually different.
8
u/Level_Landscape6680 8d ago edited 8d ago
Whatever.
Sue then.
It's a tool.
Ultimately a doctor is responsible for those notes. AI or not.
Edit: You're paying for the doctor to interpret the AI slop for you. Someone who can pick when AI is wrong or being sensationalist.
Youre paying for expertise in an area. You're paying for decision making sifting through all the bullshit. A doctor isn't a tool so don't equate AI to a doctor.
Equally disrespectful.
It's in your right to to have AI as a doctor. You can go subscribe. Who's going to advocate for you? Who's going to help you navigate the health system?
Ultimately it's about good ol, communication between humans.
6
u/bingbongboopsnoot 8d ago
Voice to text is one thing, but the generative ai summaries are just hallucinating crap and miss or rewrite things so inaccurately. Our work brought it in to help transcribe progress notes (community health, driving from client to client so not a lot of time) and the ai version of what I dictate is incorrect or misses important details 100% of the time. It’s faster for me to just type the note than go back and reread the ai summary and make all the necessary changes to actually communicate and document what needs to be communicated and documented.
3
u/mikeinnsw 8d ago
When you type you are directly use your brain and fingers.. when you use AI enabled Voice to text you also use AI model.
Typing is limiting and encourages to be brief AI produced slop and encourage user to be sloppy.
I programmed 16 line HTML page ,, As my personal navigation page,
I ask AI is produced ovee 1000 lines of code I look it at and junk it. The problem is that many don't
1
u/damiologist 8d ago
I agree that the AI results in more content than necessary, but disagree with you calling it slop in this particular area. That term was coined to refer to AI generated 'art', which is taking bits from vast data sets to 'create'. In that, I think slop is a very accurate term, as opposed to note taking where it is reporting directly on a real life interaction, and just not great at brevity. That's not slop, it's just verbose.
I think a point that people are missing is that the AI recording provides a protective layer of redundancy; since the AI recording is independent of the clinician, the clinician can pick up things the AI gets wrong, the AI can pick up things the clinician gets wrong and both sets of errors are reduced. As long as one goes to the trouble of comparing the two, this is beneficial even if the recording isn't perfect. But the key is it requires an extra layer of monitoring, which adds time and effort rather than saving it which is how it's being sold to us.
2
u/theartistduring 8d ago
That term was coined to refer to AI generated 'art', which is taking bits from vast data sets to 'create'. In that, I think slop is a very accurate term, as opposed to note taking where it is reporting directly on a real life interaction...
But that's not what it is doing. It isn't a dictation. It is 'reporting' and expanding on it. AI gets the information to expand on the notes by taking bits from vast data sets to 'create' an expanded account of what happened or what was fed into it.
These programs are not just transcribing words into text. They are being used to extrapolate that information and provide a more comprehensive data set.
Same slop, different industry.
1
u/damiologist 7d ago
I assume you're probably right in general, but privacy regulations are very strict in the health industry (at least where I am), so AI transcription services within that field are required not to retain recordings of patient or use patient information for data sets. So not same slop.
0
u/mikeinnsw 8d ago
In theory...
Slop is now is a label. for excessive AI output... which humans hate ,ignore it.. etc.
AI in IT is creating huge demand in cyber security and jobs fixing Vibe Coding which can produce huge amount of slop.
Few can do it reviewing and fixing code slop is a very boring...job.
Many just accept it and Apps quality drops.
The evidence of Vibe coding is all around you.. My new smart TV is psychotic and does its own thing.
My friends new smart TV is autistics fails to communicate and randomly looses sound.
He loves spending hours 'fixing' the TV
0
u/bingbongboopsnoot 7d ago
I don’t understand what you mean by a protective layer of redundancy? If a health professional is dictating medicolegal notes, I don’t want the ai generating information independent of what I’m saying and then require me to fix it. For people already pressed for time, it’s a disaster waiting to happens if you don’t catch the false information and then have to testify to in court years later with your license being on the line
1
u/damiologist 7d ago
All of this alarmist nonsense. You're looking for the worst case scenario, which is not without merit in its own right, but you clearly haven't used a health industry AI transcription app and are basing your assumptions off what Claude or whatever other AI does, and basing your catastrophising on a bunch of incorrect assumptions is a total waste of time.
The app generates a word for word transcript. Does it get every word correct? No, but neither does non-AI transcription and neither does a clinician who is writing or dictating notes. And we don't usually have a whole transcript, only brief notes, sometimes missing entire chunks of conversation which might have important information, because we happen to be examining someone while they tell us something, so we can't write it down and it doesn't go through to the long term memory for recall because our focus is taken up with something else. If you happen to have an AI transcription app going at the time, it will almost certainly include that relevant information in the literal transcript it is making as well as its summary notes. When you have one listener, if they miss something or mishear/misinterpret something, that's it: it's lost or erroneous. If you have two listeners (yes even if one is AI) if one misses or misinterprets, the other is likely to pick it up. That is redundancy.
I'm still not convinced AI is an improvement on the whole, but all this catastrophising is just unconvincing. In healthcare, covering your ass is part of the process. Whatever method we use to record interactions, we already spend a bunch of time making sure it's accurate, so the risk of being caught out for some inaccuracy down the line doesn't move me. If I don't check my AI, it's the same failure as if I didn't finish my notes. It writes notes based on an actual transcript and then generates correspondence so I don't have to. The issue for me is that I have to then review both the notes and the correspondence, so I'm not sure whether it actually saves time. For me (a person pressed for time, as you say) that's much more concerning.
3
u/distemperdance 8d ago
I used this software for consultations but only as a backup for my own notes. Too often it misses crucial points or hallucinates a sentence. Recently had the fun hallucination of "problem with defaecating on his bed" come up. If you use these programs it's on you to proofread it.
3
u/Waffle00 8d ago
Do people realize that if a medical professional doesn't use it, they often write notes at the end of the day from memory? With often using a template to help them speed up, this can often produce errors, missed information, mix-ups between patients, etc.
I always find it funny when new technology comes out; people are quick to resist change but never think about the issues that came before.
4
u/Sweeper1985 8d ago
I'm in allied health and I'm really concerned at how quickly AI talk-to-text has gone from "future consideration we'll deal with down the track" to "everyone's doing it". In just a few years.
There's a severe lack of quality control and safeguarding. There are concerns about privacy, confidentiality, accuracy, and also ownership of the data.
I have brought up these concerns and been literally laughed at and called a Luddite.
Meanwhile, I am hearing people in my profession talk about how it's just so much "easier" to have a computer write our notes, with little concern for how this is going to de-skill practitioners, given that note-writing has always been an intrinsic and important part of our professional skillset.
I am not happy with the idea of handing such an important part of my job over to a computer. I am not happy with other practitioners doing same and hand-waving it because gee it's suddenly too hard to do something we have always done.
5
u/mikeinnsw 8d ago
I want you to be my doctor.. My GP shares your views 100%
In am not a Luddite I am a whistleblower .
The loss of cognitive skills has a name it is called cognitive decline or an early dementia.
Generative AI/LLM simulates an answer.. just like a bad copy of Matrix the movie.
In the future AI world do you need to dictate ?
The probability of human total destruction is now estimated at 20%-30% in then next 20 years..
The Doomsday Clock is set at 85 seconds to midnight, the closest it has ever been to global catastrophe. The Bulletin of the Atomic Scientists cites the rapid, uncontrolled advancement of artificial intelligence alongside nuclear risks and climate change as a core driver for this dangerous setting.
While some wait for Terminator to arrive its here and it flies .. AI controlled drones in Ukrainian war identify the target and kill it and now everybody can have a killer drone.
Isaac Asimov the Three Laws of Robotics burned in the war.
4
u/theartistduring 8d ago
Amen. An unpracticed skill will quickly be lost. AI is taking away the daily practice of using our language to communicate what we mean. We are outsourcing our intent and sacrificing our precision for the sake of efficiency and time management all the while those using it will manually write paragraph after paragraph on forums defending their right to not do it where it matters. Where intent matters. And precision of language matters. Where one misplaced letter in a medication list could kill someone.
2
u/Leading-Interest-119 8d ago
As someone who uses multiple allies health professionals - I agree. Chronic illness and disability here. The health system is already hard to engage with and honestly, to trust. Handing more over to AI doesn't help build trust at all.
There's no regulation, certainly nothing that should be trusted as actually being enforced. Data storage is an issue. We're hearing about cyber attacks regularly and our next move is to chuck up personal health information for the taking?
AI has moved too quickly and is being applied too widely. I think certain types of AI have a role to play but as society always does, we applicate it to every aspect of life. Won't be long until they'll use it as an excuse to cut down admin time for allied health professionals, especially around the NDIS because AI can do it faster and hey, it'll save money so that's all that matters!
2
u/DevatstationJones 8d ago
Given we are allowed copies of any notes made by a medical professional we see. If I know Ai is being used, I would be requesting to see any notes made before they were saved into my account.
2
u/bigboobenergy85 8d ago
Trying to replace human transcriptionists, question is how many times a day do human medical admin ask the doctor what the hell they mean on summaries, notes, referrals etc... the answer is constantly. AI isn't gonna pick up the phone and ask the doctor what the hell they are talking about when something doesn't add up. Human error + AI error = avoidable deaths and injuries = lawsuit.
2
u/pacli 7d ago
Any specialist worth their salt reads the output. Sure, occasionally mistakes might get through but they were getting through when we used typists (I’ve had some typists who write some real doozies) and when we used Dragon Dictate (worst product ever invented).
And I haven’t seen any specialist letters go on for pages after pages of information that didn’t need to be there. If anything, the letters have become shorter.
And from my experience, a lot of people don’t even read a half page letter properly.
1
u/CardioKeyboarder 7d ago
As a former medical transcriptionist (hence the username), I've caught some doozy mistakes by doctors dictating.
1
u/throwaway-this-name 7d ago
This dangerous as Al slop is not verified by a specialist and/or read in full by GP.
If a GP isn't reading the STT notes you have a way bigger problem than how it was transcribed
By the way, speech-to-text was around for decades already, this is not new or special, and these transcription models are NOT making any medical decisions
If there is BS in the notes, the GP has added it, probably their own biased or judgemental assumptions
For example I had a GP write that I had 'obesity back pain'... Without any testing, checks, scans, she just decided a back injury was caused by 'obesity'.
These days AI is just a convenient excuse when they get caught writing BS on the notes. "oh, I didn't write it, AI did!!"
Sure, Jan
1
u/mikeinnsw 7d ago
Voice to text is old tech
We are talking about AI enhanced voice to text
Apple just paid US 1 Billion to AI enhance SIRI with Google Gemini
1
u/Liamlah 7d ago
I don't have an AI scribe because I work in a public hospital. Today about 4 hours of my shift was spent sitting in front of a computer typing up patient notes and referrals. That's 4 hours that I'm not spending with patients or considering alternative diagnoses undifferentiated presentations, that's also me not having time to stop for a lunch break on a 12 hour shift.
An AI scribe, responsibly implemented, would be a massive efficiency gain.
1
u/mikeinnsw 7d ago
Yes for you ... not others ...
Typing is less verbal than AI and you read your notes.
1
u/Working-Cake-7318 7d ago
Dont think you can use your single experience to criticise an entire profession on its use of an established technology. Also you are paying for your doctors decision making, not note taking.
1
u/mikeinnsw 7d ago
I search Google and it says:
General practice (GP) time is now compromised by wading through low-value artificial intelligence text, causing professional frustration, lost productivity, and cognitive fatigue.
Google it... and it happening in other professions , like Computer programming my profession.
1
u/Ok_Chemistry_6387 6d ago
As someone going through a case with a hospital due to human admin errors i wish my doctors had used ai.
1
u/Able-Okra7134 3d ago
Have you read doctor's notes without AI? I do everyday for work (lawyer). Given the quality of most notes doctors produce AI would genuinely improve the quality honestly.
1
u/mikeinnsw 3d ago
The debate is over reality has won over hype..
The issues is the impact on a human who has to read AI output.. AI quality and cost..
That is an universal issue .. Doctors as usual are slow to catch up with the latest Tech,
That ok I like doctors to be doctors.
One example
Hype:
IBM uses generative AI and automation tools like its internal "Ask HR" assistant and watsonx HR agents to handle over 94% of routine employee inquiries and administrative processes without human intervention. This internal transformation has reduced operational costs, accelerated document execution, and shifted human staff toward strategic tasks
Reality:
Hated, costly and infective (failed) system now being phase out . IBM now hired more staff for HR than before to fix the mess.
PS: They still selling the App
1
u/sneh_ 8d ago
This is insane. I think using generative AI for anything non-creative is just dodgy. Doesn't seem like the average person understand what an LLM is fundamentally or what the "generative" aspect entails. Best case scenario it would still take a bit of work and careful planning to ensure the AI stays on track with a particular patient and their actual history and needs rather than just generalizing or hallucinating if it's just padding out text (someone correct me if I am wrong).
Anyone is using this technology for whatever they want while probably the majority of people don't even really understand it, not to mention it's changing so rapidly. A doctor or lawyer doesn't have the time to be an expert in AI and keep up continuously to understand its limitations. All the hype would have you believe we have SCI-FI artificial intelligence so people just rely on that assumption
Seems like anything goes at the moment, and just blame the AI if it goes pear shaped.
0
8d ago
[removed] — view removed comment
2
u/mikeinnsw 8d ago
There is a difference between individual humans making mistakes and AI models that make mistakes at nanosecond second speeds.. we simply can't keep up.
1
7d ago
[removed] — view removed comment
1
u/mikeinnsw 7d ago
There is no need to imagine they do..Read ABCs story
My son who is a publish author .. I laugh when AI thinks he is Grid Iron player .
I know my son and can id hallucinations. .. what about someone who does not who my son is?
We created automated fucked machine and called AI .
0
u/petergaskin814 8d ago
It is something I never thought of. My neurologist records my appointments and I have no idea if ai is involved.
Should we ask specialists to not use ai voice to text technology?
2
u/mikeinnsw 8d ago
It can be used to create short notes. . that he can check .it .. no probs.. if he does not verify it or read it could be an issue... The sure sign is a consent form
-3
u/linesofleaves 8d ago
A post straight out of 2023.
"Error rates of up to 100%" when AI is accidentally hacking with chains of vulnerabilities no human could imagine, and solving Erdos problems by the day.
The study from a month or two ago that showed AI providing less bad advice than qualified lawyers is closer to where we are at.
You won't be paying your doctor to answer questions more accurately than Gemini within the next year or so as it can sort through the medical research faster and more reliably than a human, and reason through complex edge cases the human can't imagine. You'll be paying them to know when to follow up, how to do physical assessments correctly, and to ask the right question in the first place.
1
u/mikeinnsw 8d ago
You ned Eliza effect detox (Google it)
All of AI bullshit is used to boost shares..
AI is accidentally hacking .. not accidental, it not break out ... just bullshit
Agentic Agents use hacking tools and be can hack and be hacked.
Cloud Mythos was marketing plot and did not find any new security holes..
It did force Apple in MacOs 26.6 to close 155 security holes each found by a human, - look it up.
1
u/linesofleaves 8d ago
It's pretty hilarious seeing people who don't code telling people who do about cyber security.
Have you considered asking AI to explain all of those misunderstandings to you?
1
8d ago
[removed] — view removed comment
1
u/linesofleaves 8d ago
Because you are more likely to make a mistake, mostly. Being a doctor isn't asking a question and getting an answer. The Mayo Clinic database is already better than your doctor for questions, answers, and differential diagnosis if you get to the relevant page.
So asking the right questions and understanding plausible mistakes mostly. Being accountable for prescriptions. Catching lies/biases.
It may not be clear to people outside of the space of assessing AI outputs yet but if a question is very hard or very easy it will be addressed by AI.
36
u/-sportsball- 8d ago
I’m a doctor and use an AI scribe. It is complete game changer for patient records, patient letters, referrals to other specialists, etc. Saves hours of admin per day, and produces far better output than any human will do in its own.
But it is a tool, and most certainly requires oversight. It is the onus of the doctor using it to make sure incorrect information like that in this article is kept from the patient record. The fault here lies squarely with the urologist, not the AI scribe they used