AI Can Now Take Over Your Computer... Is Your Cybersecurity Team Ready?

The Next Evolution Of AI Is Going To Be A Security Nightmare

Another day … another AI risk to talk about!

It seems that once the AI Pandora’s box was opened with GenAI .. Cybersecurity teams have a new risk to talk about every month or so

This time, a new alarm bell is ringing about AI’s ability to take over and control your computer.

Of course, I am talking about the recent release of Anthropic’s Claude 3.5 Sonnet, an AI model that can now execute actions on a computer with human-like interactions via its “Computer Use” API.

This feature enables the AI to interact with desktop applications, simulate keystrokes, and control mouse gestures, allowing it to perform a range of back-office functions autonomously.

Now, honestly, this feature is not THAT new if you have worked in Robotic Process Automation (RPA), but AI adds a whole new area to worry about

Anthropic’s vision of AI-powered agents will allow them to independently handle computer tasks, manage processes, and adapt to changing requirements.

This will add a new “action-execution layer” that allows AI to operate software and perform actions based on user inputs.

We are now moving beyond the back-and-forth prompt phase of AI and toward execution based AI.

This ability to “learn by doing” through this self-teaching approach is groundbreaking.

I have talked about Autonomous AI and its risks before highlighting the dangers and opportunities present.

It is great to see this concept move into practical reality, but what are some of the risks we can expect?

AI Agents And Their Risks

1 — Hacking AI Agents

Easily the number one risk on the top of every cybersecurity team’s risk register when it comes to Autonomous AI

What happens if these agents are compromised ??

CISOs already worry about service-based accounts in their networks being taken over, and this risk becomes even more amplified with AI agents.

Given these agents will have access to internal applications and systems .. an attacker gaining access could be game over for a company.

“Jailbreaking” AI via bypassing its built-in controls, will become a top priority for cyber criminals who know the potential payday that could happen.

Cybersecurity teams will have to think of new ways of securing this AI execution layer before it gets compromised.

Vendors will think of ways to make $$$ by introducing AI-specific firewalls.

Hardening Standards will have to be updated and reviewed

And so on...

2 — Data Leakage

Giving AI access to applications means you are also giving it access to the underlying data.

Cardholder information, banking details, social security numbers, etc., are data that an AI could potentially leak out if compromised.

Anthropic’s statements that its AI will retain screenshots for 30 days is also something that will make CISOs sit up and take notice.

This is what it says:

“When prompted by a User, computer use will process and collect screenshots from the computer’s display that Claude uses to interpret and interact with the interface, along with the user’s Prompts and Outputs. Anthropic will not collect any additional data from users’ computer interfaces.

Anthropic does not use your Prompts or Outputs to train our generative models, unless you explicitly report materials to us, or have otherwise opted in to training. Read more here.

By default, Anthropic will automatically delete all screenshots from our backend within 30 days, unless the Customer and Anthropic have agreed to different terms”

Legal teams are going to have a field day if these risks are not assessed from a data privacy and regulatory perspective

3 — Can you fire an AI Agent?

While some CEOs might already dream of a team of AI agents replacing human workers .. we are far from that reality.

Anthropic has admitted that Claude’s Computer Use API is still slow and prone to errors in tasks such as scrolling, zooming, and managing delayed notifications.

Even though Claude’s error rates in benchmark tests are improving, they remain well below the accuracy of human users.

This inconsistency can lead to unintended actions, particularly in sensitive applications.

AI has already been shown to be vulnerable to bias and data poisoning, so it might be a while before we see it take over mission-critical business processes.

How To Get Ready

Fully functional Autonomous AI is still a bit down the road, but we have reached a major milestone with Claude 3.5

CISOs and CIOs should start making action plans based on the following:

• Understand the built-in safety checks and features implemented by the AI providers. Anthropic has confirmed safety features in Claude 3.5 to prevent misuse, such as restricting the AI from taking high-risk actions, such as making purchases, posting on social media, or engaging with government websites without explicit direction.

• Upskill your Cybersecurity teams and ensure they are aware of the new types of risks that AI introduces and can identify them. Anthropic and other vendors have confirmed that these AI can be potentially vulnerable to attacks like prompt injections, so is your team ready to assess the same?

• Get in touch with specialist vendors who can help you create AI governance frameworks and assess AI tools for security risks. There is no shame in asking for help from the experts !

• Make sure that when these AI agents are implemented, manual approvals are still present to control any malicious actions that might be taken. This provides a safety check if the AI is compromised.

• Start by implementing standards like ISO 42001 and the NIST AI Risk Management Framework to create a culture of AI governance within your company.

Anthropic’s Claude 3.5 Sonnet is a giant leap forward, and AI-powered automation is right around the corner.

Cybersecurity teams must proactively assess the risks of such an AI-driven world before it becomes mainstream.

By understanding these challenge,s AI will become a massive enabler instead of something that just takes jobs and introduces new risks!