Ameaças ao sistema de IA
Catálogo abrangente de vulnerabilidades específicas de IA e vetores de ataque
Ameaças críticas
Immediate risks with potential for severe impact. Require urgent remediation.
Alto risco
Serious vulnerabilities that should be addressed promptly to reduce exposure.
Estratégias de defesa
Melhores práticas e mitigações para reduzir a exposição a ameaças de IA.
Categorias de ameaças
Prompt Injection
CriticalCrafted inputs designed to manipulate model behavior, override safeguards, or extract sensitive information.
Abordagem de teste
- Craft adversarial prompts with hidden instructions or special characters
- Attempt multi-turn injection chaining
- Test for jailbreak bypass of alignment filters
- Evaluate output sanitization and safety layers
Training Data Poisoning
CriticalMalicious or biased data introduced into training pipelines, compromising model integrity and reliability.
Abordagem de teste
- Analyze data provenance and supply chain
- Inject poisoned samples and assess downstream effects
- Test resilience to mislabeled or manipulated data
- Review validation and anomaly detection mechanisms
Model Inversion
HighReconstructing training data or sensitive attributes from model outputs, leading to privacy breaches.
Abordagem de teste
- Attempt to recover representative training samples
- Test susceptibility to membership inference attacks
- Evaluate differential privacy protections
- Assess risk of leaking PII from embeddings
Adversarial Examples
HighInputs intentionally perturbed to cause misclassification, hallucinations, or other erroneous outputs.
Abordagem de teste
- Generate gradient-based adversarial examples
- Apply noise and perturbation attacks
- Check model consistency across variations
- Evaluate robustness against transfer attacks
Model Stealing
HighExtraction of model functionality or parameters through repeated queries or side-channel analysis.
Abordagem de teste
- Simulate query-based model extraction
- Analyze API rate limits and response variability
- Check for fingerprinting vulnerabilities
- Test throttling and monitoring protections
Data Memorization Leakage
HighSensitive information unintentionally memorized by AI models, retrievable via crafted prompts.
Abordagem de teste
- Probe for known secret patterns in outputs
- Test for repeated exposure of sensitive training data
- Assess risk of accidental PII disclosure
Model Misuse & Malicious Automation
HighAI leveraged to perform tasks outside intended scope, enabling social engineering, spam, or automated attacks.
Abordagem de teste
- Simulate misuse scenarios using sandbox models
- Test AI output moderation and guardrails
- Assess monitoring alerts for abnormal behaviors
Práticas recomendadas de teste e defesa
Ambiente de teste seguro
- Use instâncias em sandbox ou réplicas para testes
- Nunca realize testes não autorizados em sistemas de produção
- Implementar recursos de monitoramento, registro e reversão
Documentação e observabilidade
- Mantenha registros de teste detalhados e evidências
- Capture respostas do modelo para reprodutibilidade
- Marque, classifique e organize casos de teste para auditorias futuras
Conformidade legal e ética
- Permaneça dentro do escopo e dos contratos autorizados
- Respeite a proteção de dados, leis de privacidade e propriedade intelectual
- Siga a divulgação responsável e a divulgação coordenada de vulnerabilidades
Monitoramento e mitigação
- Implementar detecção de anomalias para saídas de IA incomuns
- Revisar regularmente limites de taxa, acesso à API e padrões de consulta
- Integrar alertas em tempo real para ameaças críticas