Data poisoning is a type of adversarial attack in which attackers try to manipulate training datasets of machine learning (ML) algorithms to produce unwanted and harmful outcomes.
In this definition...
What is a data poisoning attack?
Data poisoning attacks come mainly in two types: availability and integrity. With the availability attacks, the attackers inject malicious data into the ML system that makes the learning models useless. According to a report published by Cornell University, 3% of poisoning data may lead to errors ranging between 12% and 23%.
In integrity or backdoor attacks, the attackers access the ML system through malware that makes them capable of executing malicious activities. These attacks are harder to detect than availability attacks.
How does data poisoning work?
A data poisoning attack happens when the attackers inject faulty information into the ML model’s training datasets. It enables the system to learn these corrupted datasets and produce defective output that may lead to inaccurate insights. Availability attacks focus on simple settings of models, including logistic regression and support vector machines.
While making backdoor attacks, the intruders can install malware input to an ML’s algorithm without the owner’s knowledge. The backdoor allows hackers to create a string of malicious files that misclassify selected data targets.
Compared to availability attacks, backdoor attacks are more insidious, as there’s no prevalent drop in the performance of trained learning models that can be observed. It can only be noticed when the specific result appears on the model.
What are the impacts of data poisoning?
Machine learning is used to make insightful business decisions such as product pricing, campaign strategies, promotions, and more. Apart from businesses, it is used in healthcare anomalies detection, natural language processing, and computer vision. Therefore, false training in datasets can provide harmful outcomes and may cause potential damage to all industries, including finance, healthcare, law enforcement, education, and more.
For example, data poisoning attacks on the systems that power self-driving cars, healthcare devices, and military infrastructure could result in terrible consequences.
Examples of data poisoning attacks
Since the adoption of AI and ML, data poisoning attacks are common. Here are some of the prominent examples of data poisoning attacks:
Barni et al.
Barni et al. (2019) is an example of a Trojan horse attack, a variant of data poisoning attack where the attacker inserts certain malicious pattern files that are indistinguishable from the original dataset. Therefore, it’s difficult to detect the poisoned data and prevent the attack. However, the attacker does not aware of the victim’s model. To perform this attack, a backdoor signal adds to the target dataset on the Neural Network (NN). Then, the NN learns this dataset without knowing the presence of poisoned data and produces corrupted outcomes.
Rakin et al.
(2019) is another type of data poisoning attack, in which the attacker inserts Trojans into the NN to misclassify the training dataset. In contrast to the above attack, in Rakin et al., the attacker is well-aware of the NN architecture and parameters of a victim’s model. Therefore, the attacker can easily find the bits that need to be flipped. This attack can have a great impact on the output.
What can a business do to protect itself from data poisoning?
Identifying data poisoning attacks can be a tedious task for many businesses and organizations; However, businesses can adopt proactive strategies to prevent or mitigate the risks of data poisoning. Here are four basic cybersecurity principles outlined by CMMC of the Department of Defense to prevent data poisoning attacks:
Network protection
Setting up and updating strong firewalls, including human firewalls, may keep businesses out of internal and external threats. Limit the access to ML projects to those who are directly involved in the projects. Strong authentication methods like multi-factor authentication and two-factor authentications are also helpful to secure ML models.
Facility protection
Facility protection involves the physical protection of all systems of a business or organization. Businesses can restrict the unwanted entry of persons to the server room or data centers by using keycards or other similar methods.
Endpoint security
ML models are mainly using the data from IoT devices; therefore, malware attacks are highly prevalent among businesses due to the use of unsecured devices and technologies. The use of anti-malware software, data encryption, and other access control measures can provide endpoint security.
People protection
The users who have the access to ML databases should ensure the quality of training datasets and be aware of phishing attempts. Strong password management also protects businesses from data poisoning attacks.
Some other methods and tools, like regression testing, validity checking, and more, are able to detect and prevent anomalies by helping systems detect data drift in the performance of ML models.
