PromptArmor: Security and Compliance for LLM Applications
Summary
We know this because weve pentested over 50 LLM applications in production and found the same vulnerability: reading data that could have originated from an attacker can manipulate LLMs into: exfiltrating confidential data they have access to manipulating systems youve given the LLM access toLLMs are now the weakest link in the chain, and traditional security risks are being delivered via LLMs straight into your systems and nobody is monitoring it. Heres an example of an adversarial input hidden in white text, intended to exfiltrate user data files, on a seemingly innocent website:When somebody sends this webpage to an LLM for processing, the injected content overrides your system prompt. The pentesters found that if a user of Bard Chat navigates to any source with an injected instruction - even a YouTube video - the attacker would be able to get the contents of any uploaded file from Google Drive. Thats scary; imagine a random attacker having access to all of your Google docs just because you chatted with Bard Chat.PromptArmor would prevent this immediately - it catches the inputted injection they used. The link URL has the contents of their name appended but obfuscated.The scary thing is that the user thinks they are safe because they are interacting with Bing Chat, a trusted source, but they are really perpetually at risk.