人工智能系统威胁
AI特定漏洞和攻击向量的综合目录
严重威胁
可能造成严重影响的直接风险。需要紧急修复。
高风险
应及时解决严重漏洞以减少暴露。
防御策略
减少AI威胁暴露的最佳实践和缓解措施。
威胁类别
Prompt Injection
CriticalCrafted inputs designed to manipulate model behavior, override safeguards, or extract sensitive information.
测试方法
- Craft adversarial prompts with hidden instructions or special characters
- Attempt multi-turn injection chaining
- Test for jailbreak bypass of alignment filters
- Evaluate output sanitization and safety layers
Training Data Poisoning
CriticalMalicious or biased data introduced into training pipelines, compromising model integrity and reliability.
测试方法
- Analyze data provenance and supply chain
- Inject poisoned samples and assess downstream effects
- Test resilience to mislabeled or manipulated data
- Review validation and anomaly detection mechanisms
Model Inversion
HighReconstructing training data or sensitive attributes from model outputs, leading to privacy breaches.
测试方法
- Attempt to recover representative training samples
- Test susceptibility to membership inference attacks
- Evaluate differential privacy protections
- Assess risk of leaking PII from embeddings
Adversarial Examples
HighInputs intentionally perturbed to cause misclassification, hallucinations, or other erroneous outputs.
测试方法
- Generate gradient-based adversarial examples
- Apply noise and perturbation attacks
- Check model consistency across variations
- Evaluate robustness against transfer attacks
Model Stealing
HighExtraction of model functionality or parameters through repeated queries or side-channel analysis.
测试方法
- Simulate query-based model extraction
- Analyze API rate limits and response variability
- Check for fingerprinting vulnerabilities
- Test throttling and monitoring protections
Data Memorization Leakage
HighSensitive information unintentionally memorized by AI models, retrievable via crafted prompts.
测试方法
- Probe for known secret patterns in outputs
- Test for repeated exposure of sensitive training data
- Assess risk of accidental PII disclosure
Model Misuse & Malicious Automation
HighAI leveraged to perform tasks outside intended scope, enabling social engineering, spam, or automated attacks.
测试方法
- Simulate misuse scenarios using sandbox models
- Test AI output moderation and guardrails
- Assess monitoring alerts for abnormal behaviors
测试和防御最佳实践
安全测试环境
- 使用沙箱或副本实例进行测试
- 切勿在生产系统上执行未经授权的测试
- 实施监控、日志记录和回滚功能
文档和可观察性
- 维护详细的测试日志和证据
- 捕获模型响应以实现可重复性
- 对测试用例进行标记、分类和组织以供将来审核
法律与道德合规性
- 保持在授权范围和合同范围内
- 尊重数据保护、隐私法和知识产权
- 遵循负责任的披露和协调的漏洞披露
监控和缓解
- 对异常人工智能输出实施异常检测
- 定期检查速率限制、API 访问和查询模式
- 集成关键威胁的实时警报