AI Security is NOT the same as traditional Cybersecurity
Do not make the mistake of thinking AI can be secured in the same way as regular applications
early 2022 I published on AI governance and Cybersecurity on Amazon. It was a bit of niche subject and I thought it would be a good idea to create some awareness on this little known topic.
AI was known in cybersecurity but mostly around how it was being used in new types of products to detect malware, insider threats etc.
I honestly thought this topic was not being given enough attention so thought about writing a book to create some awareness on this little known topic ..
What an amazing difference just a few months make !
Once ChatGPT exploded onto the scene and became the fastest growing app in history .. AI has dominated the news and minds on the tech world everywhere
As companies race to adopt tools like AI and ChatGPT to gain strategic and competitive advantages in the industry; cyber-security teams are scrambling to mitigate the risks of these new technologies
At the same time, I see Cybersecurity professionals making a few key mistakes when they think about securing AI systems:
Mistake 1: Thinking ChatGPT is the be-all and end-all when it comes to AI risks.
Mistake 2: Thinking AI can be secured using traditional cybersecurity methods
Let me clear up about these two things:
ChatGPT is awesome but it is only a VERY small part of what AI can do and how it is being adopted.
Do not restrict yourself to just trying to mitigate ChatGPT risks as AI security is much much more
AI security is also far more than securing the underlying infrastructure, access control, configuration, patching , logging , alerting etc. All of the above are good and needed but AI introduces new security risks which will not get mitigated in the same way
This risk becomes even more dangerous when we realize that the vast majority of companies adopting AI and Machine learning have cyber-security teams who are unaware of these risks
Let us dive deep and see what these issues are !
Why AI security risks are different
AI systems use machine learning to build up their knowledge and make intelligent decisions in real time which can be applied to any use case
Cyber-Attacks on AI systems can happen in two ways :
Attack the infrastructure : The system itself can get compromised via its underlying technology infrastructure which hosts the AI model. The attacker is able to compromise the AI system via insecure configurations, missing access control, lack of patching etc. The attack is similar to how traditional cyber-attacks work and traditional security controls can be used to mitigate the same
Attack the AI model : In the second one, the attack manipulates the unique characteristics of how AI systems work to benefit his own malicious intentions i.e. attack the model’s decision making capability or poison the data being used to make these decisions. This is not just theoretical as many models have already been manipulated or tricked with numerous case studies listed here.
Like I mentioned .. AI and Machine Learning algorithms rely on their underlying models which analyze huge amounts of data to reach decisions.
What if an attacker was not interested in stealing the data but with tampering the decision making process ? Depending on the nature of decisions being made, the potential attack could be far more severe especially with the rising adoption of AI across a variety of high risk sectors.
Let us take a look at a few of these unique attacks
1 — Data Poisoning
Attacker can poison the training data that is being used to train the Machine Learning model. In most cases this data is taken from published sources and not created from scratch.
By contaminating this data source, the attacker can create a “backdoor” as he knows the model has been trained on faulty data and knows how to take advantage of it. This can can result in serious consequences if the decisions being made on this data have real-life consequences such as in the case of self-driving cars ( seen below )
2- Data Extraction ( inference )
Attacker can query the model and understand what training data was used in its learning. This can result in the compromise of sensitive data as the attacker can infer the data used in the model’s training and is especially dangerous if sensitive data was involved. This type of attack also called inference does not require access to the model’s functionality and can be done just by observing the model’s outputs. This can have serious consequences if the machine learning model is trained on data deemed to be sensitive. Let’s take a few examples of how this attack can happen:
● An attacker can query a model with a name or an identifier to find out if a person is on a patient list in a hospital or a sensitive medical list.
● An attacker could find out if a patient was being provided certain medication or not
● An attacker can provide images to a facial recognition model to find out if a particular face was used in the training or not.
3 — Model Evasion
Attacker tricks the model by providing a specific input which results in an incorrect decision being made. These inputs referred to as adversarial inputs are usually accomplished by observing the model in action and understanding how to bypass it.
For example, an attacker can attempt to trick AI-based anti-malware systems into not detecting their samples or bypass biometric verification systems. Even slight modifications which are not visible to the human eye can be enough to throw off AI systems that have not hardened against these types of attacks.
Below is an example of an adversarial input that facilitates model evasion for self driving cars by just slightly changing a few pixels in the image.
How to secure AI systems
The risks I have mentioned are just a few of the ones that AI introduces. If you are serious about mitigating these risks then a proper AI risk management framework should be setup.
The good news is that NIST has already released a great document on this which is available here
Apart from that, a few other key steps that you can do are :
Create an AI/machine learning security baseline : To make sure security is consistently applied across AI systems the company will need a minimum baseline of security controls to be applied. This has to cover both the security of the underlying infrastructure and the AI System itself .
Maintain an up-to-date inventory of all AI and Machine Learning systems : It is difficult to secure anything without knowing it exists in the first place! Identify all assets in your AI ecosystem as a fundamental step so you know how many AI systems are present and how they can be protected. Make sure the inventory captures the below:
Conduct detailed technical risk assessments of your AI systems : Based on the risk level of the AI systems identified in your inventory, detailed technical risk assessments must be carried out and documented with mitigations and timelines. This is a collaborative effort with the business and technical teams. The methodology you use is not important but what is critical is that this is a repeatable, standardized process that is consistently followed.
Create an awareness program about AI risks : Easily one of the biggest risks in AI systems is the overall lack of awareness amongst cyber-security professionals. As mentioned earlier the unique risks present in AI systems are either ignored or are treated like any other software rollout with hardening, patching, and configurations done without regard for the risk of the AI model. If you are serious about AI cyber-security, then it is crucial to up-skill your staff and make them aware of these risks ( check out my Udemy course below )
Update security testing to identify AI and Machine Learning vulnerabilities: Most companies have security assurance processes present as part of any application rollout in which a full security review of the application is carried out to capture security risks. You must make sure that AI-specific security testing is a part of this process. For example, using adversarial testing samples that simulate model evasion can be done during the model testing phase to assess its level of susceptibility to model evasion attacks as seen in the following diagram.
The good news is there are standards already available for simulating AI based attacks.
Security experts already available with public frameworks like MITRE ATT&CK will be happy to know that there is an AI based security framework available called MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems), which is described as
“a knowledge base of adversary tactics, techniques, and case studies for machine learning (ML) systems based on real-world observations, demonstrations from ML red teams and security groups, and the state of the possible from academic research”
ATLAS present at
https://atlas.mitre.org/
follows the same framework as MITRE, so it is very easy for cyber-security practitioners to study and adopt its techniques when they want to test their internal AI systems for vulnerabilities and security risks.
It also helps in creating awareness of these risks amidst the cyber-security community as they are presented in a format, they are already familiar with.
Wrapping up
Just like Application Security was a blind spot for companies in the early 2000s and software supply chain attacks blindsided everybody in recent times; AI has unique cyber-security risks and assessing them the traditional way will miss out on key vulnerabilities and leave the system exposed.
Start learning about these new risks today as we enter a new and exciting AI driven world !
Thanks for reading this. If the topic of AI security still interests you then be sure to check out my book and course on AI security also. Happy learning !