https://infosec.exchange/@malwaretech/114903901544041519
the article since there is so much confusion what we are actually talking about https://edition.cnn.com/2025/07/23/politics/fda-ai-elsa-drug-regulation-makary
Literal… I cannot stress this enough… Literal Idiocracy.
This is literally what happens in the film. Like the first 10 minutes.
Fuck.
I can’t remember, what happened in the film?
Luke Wilson’s character goes to the hospital where an AI misdiagnosis him.
For specific things like protein folding “Ai” has been useful but that’s not just a llm.
Yes, machine learning models trained to solve a specific problem can be very good at solving that problem. It’s artificial “general” intelligence we haven’t achieved but are trying to sell.
I’m pretty sure that undermining confidence in drug approvals is a feature, not a bug. The same people who were screeching about mRNA vaccines being secret poison that was rushed through approval are the ones doing this now, so when (not if) it does actually lead to dangerous drugs being approved and a collapse in confidence in the FDA, they’ll be the ones saying “We told you so” and getting their anti-medical way.
It’s the exact same playbook Republicans use in the rest of the government: Say Government doesn’t work, cry about government spending, and insist government regulation is crushing personal freedoms, then they actually do all of those things and when the next administration comes around they pass on the blame and say “I Told You So.”
The FDA need to get out of the way anyway. So much of what could be done isn’t done because they take their sweet time with decisions.
The average approval time for a new drug is about a decade mostly because the FDA just don’t do anything for the first 9 and 1/2 years. The covid vaccines were approved in a hot minute though and there was absolutely no issues with them despite what the conspiracy theorists thought. In fact they primarily based their conspiracy theory on the fact that normally the FDA takes forever and today in order to approve anything. Proving only that it doesn’t need to take that long in the first place.
That’s the point though. When data means nothing truth is lost. It’s far more sinister than people are aware it is. Why do you think it is literally being shoved into every little thing?
It is already making pictorial evidence worthless, which is a scary thought no justice system has even begun considering yet, even though it is literally already happening. Criminals all over the world rejoice, they can be caught doing the act on video, and it will be worthless. Of course this applies even more to large scale criminals like dictators. It will all be “fake news” from now on.
Capitalizing on a highly marketable hype bubble because the technology is specifically designed to deceive people into thinking it’s more capable than it is
lack of critical thinking is a feature in this administration
In this society, more like.
I’m constantly mystified at the huge gap between all these “new model obliterates all benchmarks/passes the bar exam/writes PhD thesis” stories and my actual experience with said model.
Likely those new models are varients trained specifically on the exact material needed to perform those tasks, essentially passing the bar exam as if it were open book.
Reminds me of a video that starts with the fact you can’t convince image generating AI to draw a wine glass filled to the brim. AI is great at replicating the patterns that it has seen and been trained on, like full wine glasses, but it doesn’t actually know why it works or how it works. It doesn’t know the things we humans know intuitively, like “filled to the brim means more liquid than full”. It knows the what but doesn’t get the why.
The same could apply to testing. AI knows how you solve test pages, but wouldn’t be that exact if you were to try adapting it into real life.
The real truth is just that standardized testing fucking sucks and always has
Right, I’m no expert (and very far from an AI fanboi), but not all “AI” are LLMs. I’ve heard there’s good use cases in protein folding, recognising diagnostic patterns in medical images.
It fits with my understanding that you could train a similar model on more constrained datasets than ‘all the English language text on the Internet’ and it might be good at certain jobs.
Am I wrong?
The problems with AI we talk of here is mostly with generative AI. Protein folding, diagnostic patterns and weather prediction works a bit differently than image making or text writing services.
You are correct. However, more often than not it’s just like the image describes and people are actually applying LLM’s en masse to random problems.
Hallucinating studies is however very on brand for LLM as opposed to other types of machine learning.
what ai, apart from language generators “makes up studies”
That’s because “AI” has come to mean anything with an algorithm and a training set. Technologies under this umbrella are vastly different, but nontechnical people (especially the press) don’t understand the difference.
Technically, LLMs as used in Generative AI fall under the umbrella term “machine learning”…except that until recently machine learning was mostly known for “the good stuff” you’re referring to (finding patterns in massive datasets, classifying data entries like images, machine vision, etc.). So I feel like continuing to use the term ML for the good stuff helps steer the conversation away from what is clearly awful about genAI.
There is no generative AI. It’s just progressively more complicated chatbots. The goal is to fool the human into believing it’s real.
Its what Frank Herbert was warning us all about in 1965.
Someone needs to to a test, when this AI launches, they need to try and get poison approved as a medication. Like straight up a lethal dose of cyanide or something.
What happens when people realise and it immediately become the most popular drug on the market?
Win-win?
Can you imagine how sad those LLMs will be if they make a mistake that winds up harming people?
About as sad as the CEO
Not at all, because they are not thinking nor feeling machines, merely algorithms that predict the likelyhood of words following other words and spit them out
This reminds me of how like a hundred or so years ago people found “miracle substances” and just put them in everything.
“Uranium piles can level or power a whole city through the power of Radiation, just imagine what good this radium will do inside your jawbone!”
So is this a situation where it’s kinda like asking chatgpt to make you drugs so it will go about any means necessary (making up studies) to complete the task?
Instead of reaching a wall and saying “I can’t do that because there isn’t enough data” I hope I’m wrong but if that’s the case then that is next level stupid.
no it’s supposed to help help the drug approval workers but everything it says has to be double checked so it ends up wasting time i put the article into the post now
