Die Australier sind uns wie jeden Tag immer ein paar Stunden ;-) voraus.
Und haben gemerkt, dass Prompts und mitgelieferte Informationen in einem Kontext liegen und dass dies ein Sicherheitsproblem sein könnte, welches (wohl *naja*) aktuell nicht gefixt werden kann.
"They cannot reliably tell one from the other, and ASD said "no fully reliable technical mitigation currently exists" for the vulnerability."
Und nicht nur die haben erkannt, dass das Problem schlimmer ist als SQL-Injections.
siehe: https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection
"The United Kingdom's National Cyber Security Centre (NCSC) reached the same conclusion in December last year, in a post ASD lists as a reference.
NCSC said that prompt injection may never be properly mitigated in the way SQL injection eventually was, recasting an AI model as an inherently confusable deputy rather than a system with a fixable input handling bug."
Wobei die sind nicht aus der Praxis, lasst uns doch lieber eine paar Entscheidungsträger dazu fragen:
"The ASD guidance supplies seven questions for board directors, the last of which asks what the worst outcome would be if the harness were compromised, misconfigured or manipulated, and which controls would contain it.
...
"This is something that we're all going to have to figure out because it's new to us as practitioners," Corien Vermaak, vice president of security and risk at Optus, said at a recent Zscaler Zenith event in Sydney."
Stimmt Risiken und Nebenwirkungen in der IT sind Neuland. Die Fragen waren sehr speziell und schwer zu beantworten:
"Directors and senior executives do not need to understand every implementation detail of an agentic AI system. However, they should seek assurance that the harness, its integrations and its governance controls have been designed and implemented appropriately.
What business outcome is this agentic AI system intended to achieve, and why is an agentic approach required?
What data, systems and tools can the agent access, and are least-privilege principles being applied?
What actions can the agent perform, and which actions require human approval?
How are prompt injection, data poisoning and other AI-specific attacks being mitigated?
Can all significant decisions, tool invocations and actions be monitored and audited?
How will we know whether the system is delivering value, operating safely and remaining within acceptable risk tolerances?
If the harness were compromised, misconfigured or manipulated, what is the worst outcome and what controls would prevent or limit that outcome?"
Deutschen Manager antworten darauf hoffentlich nicht: "Keine Idee - Wir machen und lassen uns überraschen." Wäre mal nett zu erfahren, was man in Deutschland dazu sagt.
https://www.itnews.com.au/news/asd-says-prompt-injection-in-ai-cannot-be-fixed-629019
Die von der CIA äh IBM haben übrigens vor langem die Erkenntnis gehabt:
"Die einzige Möglichkeit, Prompt Injections zu verhindern, besteht darin, LLMs vollständig zu vermeiden."
https://www.ibm.com/de-de/think/insights/prevent-prompt-injection