The Wrong Way to Secure Generative AI
Do not make this mistake if you are securing GenAI systems
seem to be finally reaching the point where people are getting comfortable with Generative AI usage in their everyday lives
It seems that GenAI has now reached the point where people have accepted it as part of the new tech “normal”
Unfortunately Cybersecurity professionals are still in a tough spot
Despite the amazing abilities of GenAI that give us realistic photos and videos, or text written like a pro; these capabilities can also be misused to spread misinformation, commit fraud, or worse.
We’ve seen it happen before many many times as new tech gets adopted
So how do we make sure generative AI is developed responsibly as it goes mainstream?
Most Cybersecurity professionals talking about this focus on technical controls — things like input validation, data leakage access management, sandboxing, monitoring outputs, having humans review content, etc.
But here’s the deal — relying only on technical controls isn’t going to cut it with AI as complex as this.
We need to go beyond that and threat model these systems from the start.
Threat modeling basically means taking a hard look at generative AI and asking “What could go wrong here?” and “What vulnerabilities could bad actors exploit?”
Or more precisely the following 4 questions :
1. What are we working on?
2. What can go wrong?
3. What are we going to do about it?
4. Did we do a good enough job?
Then we prioritize addressing those high-risk threats as we design, build, and monitor the systems.
Some key risks to watch for include manipulated training data, tampered inputs trying to trick the AI, stealing proprietary models, and harmful use of generated content.
The point is, we have to threat model the risks unique to generative AI.
Technical controls are only part of the formula. Threat modeling gives us a more complete picture so we can direct security efforts at what matters most.
And we have to keep evaluating new threats as capabilities advance.
The benefits of doing threat modeling before implementing technical controls are many for GenAI:
It takes a broader, more proactive view of potential risks across the entire AI lifecycle — from development and training to deployment and monitoring. Technical controls tend to focus on implementation specifics.
Threat modeling encourages thinking from an adversary’s perspective to identify subtle vulnerabilities, not just obvious risks. This reveals threats that controls may miss.
It prioritizes the most urgent and high probability threats. This helps focus security efforts on areas that need them most. Technical controls are more generic.
Threat modeling considers social and ethical risks like misuse of outputs, not just technical exploits. So it frames security in the context of broader impacts.
It adapts to new threat information, so security evolves along with rapid AI advances. Technical controls may become outdated if designed for an old threat landscape.
A visual representation of the risks present in publicly hosted model of something like ChatGPT or MidJourney would look something like this:
We can use this to feed into a threat model following a methodology like STRIDE
If your company is implementing a localized GenAI model that the risks might look a bit different
So let’s keep this in mind as generative AI evolves.
Leverage both rigorous threat modeling and smart technical controls and we’ll develop these technologies safely while realizing their massive potential.
By fostering a threat-centric mindset, we can proactively address key security, privacy, and safety concerns, while still harnessing the capabilities offered by these tools.
I hope you found this useful. If you find the topic of GenAI security as interested as I do then check out the free preview of my new Udemy Course below :