Securing the Artificial Intelligence Supply Chain May Require an Abundance of AI - AFCEA Signal Magazine
2024-01-01 07:20 (EST) - Diego Laje
The artificial intelligence (AI) safety rulebook is in a rewriting process by those delivering AI security. Application developers employ novel methods to protect the production chain’s three potential weak spots: training data, algorithms and hardware.
The approaches to protecting an AI supply chain are flexible and depend on the mission. Still, everything starts with data, and sometimes, critical information is a life-and-death issue.
Malicious actors targeting a capability that relies on AI can damage it by poisoning the inputs the model will run on.
“When I mobilized to go to Iraq, one of the challenges that was brought to me—and I was [Department of the Navy] CIO [chief information officer] at the time—was if somebody got into the blood supply database for my unit and modified our blood on record,” said Rob Carey, now president of Cloudera Government Solutions.
In this instance, altered records could have severely harmed or even killed warfighters under treatment. Carey dealt with this potential risk once it was brought to his attention, but the incident proved to be a lesson. When AI models become widespread, the corresponding data must receive special attention.
“The idea of the integrity of the data is absolutely paramount to the successful implementation of any decision-making that comes off of data-based decision-making,” said Carey, also a former principal deputy CIO for the Department of Defense (DoD).
Unvetted sources present one of the risks.
And there are two other less explored sources for models: one is for large language models (LLMs) and other chatbots that interact with humans to continue their training. The DoD has repeatedly mentioned leveraging these technologies for various uses in the military. This means those interactions could be poisoned deliberately to change the reliability of future outputs.
Another source is when actors use prompt language to obtain a result that the LLM itself should interdict.
These are prompt injection vulnerabilities, and these “involve crafty inputs leading to undetected manipulations. The impact ranges from data exposure to unauthorized actions, serving attacker’s goals,” according to a document by the Open Worldwide Application Security Project, a nongovernmental organization that works to improve software security.
An alternative approach is to avoid the problem of training data altogether when the mission allows. In the case of AI for preventing breaches, this may be the way.
“We don’t have an external training data set. I don’t learn attacks to predict attacks. I learn your environment,” said Marcus Fowler, CEO of Darktrace Federal, an AI company for cybersecurity.
