Securing machine learning models involves more than just protecting the code. Attackers might change inputs, tamper with the training data, add hidden backdoors, or use repeated queries to pull information from a model.
These attacks are often hard to detect. Your model might keep running as usual, without crashing or triggering any obvious security alerts. But it can corrupt your model behind the scenes, forcing it to give manipulated results.
In this blog, we’ll briefly explain OWASP machine learning risks and best practices to minimize them.
What Are the OWASP Machine Learning Top 10 Risks?
The OWASP Machine Learning Top 10 list covers the most common security risks faced by machine learning models, helping security teams better understand ML risks and serving as a practical starting point for finding and fixing weaknesses before attackers exploit them. It also introduces the specific risks covered below.
It helps your team see where attackers might target your ML systems, from the data you use for training to how you deploy, query, and maintain your models. The sections that follow show how these risks appear in practice.
However, the OWASP Top 10 ML risks differ from the OWASP Top 10 LLM & GenAI Security Risks. The latter is related to GenAI applications and LLM models. On the other hand, OWASP Top 10 ML risks are specifically related to machine learning models.
Traditional security frameworks usually focus on vulnerabilities in applications, infrastructure, and code. Machine learning systems expand the attack surface and they need special attention.These risks are very real, which means securing an ML system involves more than protecting the application code.
A model can be secure at the code level and still produce unreliable or manipulated results if the underlying data, training process, model, or deployment pipeline has been compromised.
ML01-2023: Input Manipulation Attack
Input manipulation, as the name suggests, is the process of changing a model’s inputs to affect its output. This targets inference time, and the change can be hard to notice but still enough to fool the model.
A 2025 Nature Medicine study clearly confirmed that even a small amount of poisoned data can impact the outcome of a medical LLM.
Researchers found that replacing just 0.001% of the training tokens with vaccine misinformation increased harmful vaccine-related outputs in their tests. This shows why testing models with modified and unusual inputs can help identify these weaknesses.
You can minimize input-manipulation risk by strengthening role-based access controls, verifying data provenance, validating labels, monitoring for unusual changes, and testing the model after retraining.
ML02-2023: Data Poisoning
Data poisoning refers to the risk of attackers sneaking incorrect or misleading data into the model training dataset. The model learns from this manipulated data, resulting in wrong output.
For example, if an attacker adds wrongly labeled traffic data to your training set, your model might start treating malicious traffic as normal. This could cause it to miss similar attacks in the future and give your security team a false sense of safety.
This is not a hypothetical risk and the threat is real. A 2025 study published in Nature Medicine found that poisoning even a small part of a medical language model’s training data increased harmful vaccine-related responses in the researchers’ tests.
To reduce this risk, teams should limit who can change it, check data sources, verify labels, and retest the model whenever the training data changes. An attacker does not always need to break into the model itself. Sometimes, changing what the model learns is enough, which is why the next section matters.
ML03-2023: Model Inversion Attack
Model inversion attacks occur when someone uses your model’s outputs to extract sensitive information from the LLM’s training data. This occurs by sending targeted queries and studying the responses; an attacker can sometimes reconstruct parts of the original data, but not the model itself.
Researchers have explicitly demonstrated the risks of model inversion attacks. Detailed confidence scores and unlimited queries can further make such attacks easier. The risk applies to models trained on medical, financial, or customer data, which is why the safeguards below matter.
You can minimize model inversion attack risk by limiting access, restricting repeated queries, avoiding detailed confidence scores when unnecessary, and monitoring unusual query patterns. Implementing privacy techniques such as differential privacy levels for different users can provide additional protection.
ML04-2023: Membership Inference Attack
Membership inference attacks occur when an attacker tries to determine whether a specific person’s data was used to train a model. They may not see the actual records, but even confirming that someone’s data was included can still be a privacy problem.
Researchers have shown that membership inference attacks pose risks against commercial image classification.
Attackers used carefully chosen queries to build substitute models that produced results similar to the original. This can support model theft, but it is a different risk from learning whether a person’s data was used.
For example, an attacker could try to determine whether a patient’s medical records were used to train a clinical AI system, even without seeing the records, which could reveal information about the person’s health or their connection to a healthcare provider.
Rate limiting, strong API authentication, monitoring query patterns, and limiting unnecessary details in API responses can make membership inference harder.
ML05-2023: Model Theft
Model theft, or model extraction, is the process of copying a machine learning model by seeing how it reacts to different inputs. Attackers send many queries to a model’s API, collect the responses, and use them to train a similar model.
If someone gets direct access to the model files, source code, or checkpoints, they can copy the model directly instead of recreating it through queries. That is a separate access risk from query-based model theft.
For example, suppose a company offers a fraud detection model through an API. A competitor might send thousands of chosen transactions to the API and record the results.
They could then use these inputs and outputs to train their own model that works much like the original, saving time and money compared to building it from the ground up. This shows how query access can lead to theft.
The December 2022 PyTorch dependency confusion attack demonstrated this danger when a rogue torchtriton PyPI package hijacked installs to exfiltrate SSH keys, environment variables, and system credentials across thousands of downstream machines. This shows how model theft risks can extend beyond queries and into the surrounding software supply chain.
The attack surface extended directly into model artifacts and orchestration runtimes, with vulnerabilities like CVE-2022-45907 (PyTorch RPC deserialization) and CVE-2024-34359.
Security teams can reduce model theft risk by implementing rate limiting, strong API authentication mechanism, and query monitoring, making it harder for someone to steal a model.
ML06-2023: AI Supply Chain Attack
An AI supply chain attack occurs when attackers tamper with or replace machine learning (ML) libraries, models, or other third-party components used by an AI system. These attacks often exploit trust in open-source packages, external repositories, or unverified dependencies to introduce malicious code into the ML development pipeline.
Once a compromised component is downloaded and integrated into an application, attackers may be able to execute malicious code, steal sensitive data, manipulate model behavior, or gain unauthorized access to the system.
Consider an organization that relies on a third-party ML library to build and deploy its AI applications. An attacker compromises the library and inserts malicious code into its package. When the organization downloads and integrates the modified version, the malicious code may execute within its environment, potentially exposing sensitive information or compromising its ML applications.
Real-World Context: In 2022, malicious packages targeting machine learning environments highlighted how attackers could exploit open-source dependencies and package repositories to compromise software development workflows. Such incidents demonstrate why organizations must verify package integrity, assess third-party dependencies, and monitor ML components for suspicious changes.
ML07-2023: Transfer Learning Attack
Transfer learning attack lets your organization build AI applications faster by adapting a pretrained model instead of starting from the scratch. But there’s a tradeoff: any weakness in the original model can carry over into your new application.
Researchers have shown that a poisoned pretrained model can carry a hidden backdoor that stays dormant during testing and only activates after deployment, once it hits a specific input.
This is why transfer learning needs careful checks before and after reuse, because the risk comes from inherited behavior.
ML08-2023: Model Skewing
Model skewing attacks refer to a mechanism in which attackers slowly change data patterns to alter how your model performs. Instead of causing a sudden failure, they introduce a bias bit by bit, allowing output quality to worsen over time.
Social media and recommendation platforms have already faced model skewing. Coordinated bot campaigns have artificially boosted engagement metrics, pushing algorithms to promote specific content disproportionately. This pattern can spread further.
In 2025, researchers showed that attackers could use reinforcement learning to generate synthetic user interactions and poison collaborative-filtering recommendation systems.
The attacks successfully manipulated recommendations in experiments using real-world datasets, showing how skewed feedback can alter model behavior and why monitoring matters.
Watching for data drift, checking feedback loops, and regularly reviewing model outputs help catch skewed behavior.
ML09-2023: Output Integrity Attack
Output integrity attacks can change a model’s output before it reaches users or downstream systems. The model itself is never compromised; its results are altered in transit. This is different from tampering with the model or its inputs.
Let’s consider a fraud detection system where an attacker gains access to the middleware connecting the model to a payment-processing system.
By modifying risk scores in transit, the attacker could get fraudulent transactions approved even though the model correctly flagged them as risky. This shows how transit tampering can undermine a correct model and why the safeguards below matter.
Securing communication channels, encrypting data in transit, and applying integrity checks keep model outputs accurate throughout the decision-making process.
ML10-2023: Model Poisoning
Model poisoning is the process of tampering with LLM models or their parameters to change their behavior. It can reduce the model’s overall accuracy or introduce a specific bias. Unlike data poisoning, which involves targeting the training data, model poisoning targets the model itself.
For example, consider a customer support system that uses an ML model to classify messages as positive, negative, or neutral. If an attacker gains access to the model parameters and changes them, the model could start classifying positive customer messages as negative.
An automated workflow might then send the wrong response, escalate harmless complaints, or route genuine customer issues to the wrong team. The impact depends on how the model is used. In a security system, poisoned parameters could cause malicious activity to be classified as safe.
In a financial system, it could affect risk scores or fraud decisions. Because the model may continue running normally, changes can be difficult to spot without comparing its behavior against known test cases.
You can help prevent this tampering by protecting your model files, restricting access to training and deployment systems, keeping trusted copies of your models, and testing model performance after updates.
How to Protect Machine Learning Systems from OWASP Top 10 Risks
Protecting your machine learning systems requires a sustained effort and t you need to make security part of every stage in your ML lifecycle. Below, we have presented some ways to secure ML systems:
A. Establish Strong Data Governance
Data is the foundation of machine learning, so its quality and integrity directly affect your model’s security. Validate and verify data, check for anomalies, and flag anything unusual that could signal poisoning.
B. Secure Access to Models and Data
Not everyone should have access to training data, model settings, or deployment steps. Role-based access controls can help prevent unwanted changes and keep sensitive information safe.
C. Continuously Monitor Model Behavior
Machine learning systems change over time. By watching prediction accuracy, data drift, strange results, and odd behaviors, you can catch attacks before they cause real harm.
D. Validate Third-Party Components
Open-source libraries, pretrained models, and outside datasets can all be attack points. You can reduce supply-chain risk by checking integrity, reviewing dependencies, and scanning regularly.
E. Incorporate Adversarial Testing
Machine learning systems should go through adversarial testing, including adversarial inputs, extraction attempts, poisoning, and privacy threats. Since attacks often start with exposed APIs, using authentication, rate limiting, logging, and response restrictions can help prevent abuse and reduce risks like model theft and information leakage.
F. Hire an LLM Security Partner
AI and ML security risks can be tough to spot with standard testing. A security partner can run an AI/ML security assessment to test your models, APIs, data, and deployment environment for weaknesses before attackers do.
Best Practices for Securing Machine Learning Models
Organizations that keep their machine learning systems secure consider security as an ongoing process, not just a one-time task.Some recommended practices include:
By making security part of every step in the AI lifecycle, organizations can greatly lower their risk from new machine learning threats.
Final Thoughts
Machine learning is changing how organizations work, make choices, and serve customers. But the same things that make AI effective also bring new security problems.
The OWASP Machine Learning Top 10 gives a useful guide to where AI systems are weak and how to fix those risks. Threats such as adversarial inputs, data poisoning, model theft, and supply chain attacks underscore the need to extend security beyond standard application protection.
Organizations that protect their data pipelines, monitor model behavior, vet third-party tools, and test for AI threats will be better able to build secure, reliable machine learning applications.
Looking for an AI & LLM security assessment provider to protect your models from attacks? Contact us now to learn more.
Frequently Asked Questions (FAQs)
The OWASP Machine Learning Top 10 is a security framework that identifies the most significant risks affecting machine learning systems, including attacks targeting training data, models, APIs, and AI supply chains.
The traditional OWASP Top 10 focuses on web application vulnerabilities such as injection attacks and broken access controls. The Machine Learning Top 10 addresses threats unique to AI systems, including data poisoning, model theft, and membership inference attacks.
Machine learning models depend on large datasets, complex algorithms, and continuous learning processes. Attackers can exploit these dependencies by manipulating data, extracting model information, or targeting third-party AI components.
Healthcare, financial services, insurance, e-commerce, government, manufacturing, and autonomous vehicle industries are particularly vulnerable because they rely heavily on machine learning for critical decision-making.
Traditional security tools remain important, but they are not sufficient on their own. Machine learning systems require additional controls, including adversarial testing, model monitoring, data validation, privacy assessments, and AI-specific security testing.
