By Andrew Corrington, VP of Technical Solutions, eimagine

AI has changed the cyber chessboard on both offense and defense. In contact centers alone, deepfake voice fraud attempts rose more than 1,300% in 2024, with attempts now happening about every 46 seconds in U.S. call centers—an illustration of how quickly AI enabled threats scale. At the same time, AI is giving defenders leverage: in controlled studies, analysts using Microsoft Copilot for Security completed common tasks 22% faster and with 7% higher accuracy, while reporting improved confidence.

This blog lays out a public, vendor neutral view of AI cybersecurity—what it is, where it works, where it can fail, and how to adopt it responsibly using open frameworks and current evidence.

What Is “AI Cybersecurity”?

Definition.

AI cybersecurity applies machine learning (ML) and large language models (LLMs) to detect anomalies, automate routine response, enrich investigations, and generate decision quality context for analysts. In practice, this shows up as:

• Anomaly detection & behavioral analytics. Systems baseline “normal” user/entity activity and flag deviations (e.g., impossible travel, atypical API usage). “Impossible travel” is a standard risk detection in Microsoft Entra ID Protection and related UEBA stacks.
• Automated response. Playbooks isolate endpoints, revoke risky sessions, or file tickets; copilots summarize alerts, draft queries, and stitch timelines to reduce time-to-contain.
• Threat intelligence & knowledge grounding. Frameworks like MITRE ATLAS catalog tactics/techniques against AI systems (e.g., data poisoning, model theft, prompt injection), helping teams map detections and mitigations to adversary behavior.

Guardrails and governance.

Two public anchors are the NIST AI Risk Management Framework (and its Generative AI Profile, 2024) and NIST’s 2025 Adversarial ML taxonomy, which standardize the language for risks and mitigations across the AI lifecycle.

Enterprise Use Cases

1) Real‑Time Threat Detection in Cloud Identity & SaaS

Scenario. A newly registered OAuth app suddenly receives wide user consent (mail/files) followed by unusual Graph calls from unfamiliar networks. Individually, these are noisy signals; together, they form a clear incident narrative. Copilot style assistants have been shown to help analysts assemble context faster and author queries more accurately.

What to instrument.

• Risk detections such as impossible travel, anonymous IPs, unfamiliar sign-in properties, and token anomalies (via Entra ID Protection & SIEM).
• Narrative summaries and guided investigations in the SOC using copilots.

Why it matters. Identity centric attacks chain app consent, risky sign-ins, and data access; AI assistants compress triage time by correlating signals, not replacing human judgment.

2) AI Powered Phishing & Fraud Prevention

Attackers now mass produce AI written, polymorphic phish and voice deepfakes for account takeover, BEC, and vishing. Pindrop’s 2025 report documents >1,300% growth in deepfake fraud attempts, with sector specific spikes and attempts every 46 seconds in U.S. contact centers. Email security telemetry additionally shows AI features appearing in the majority of observed phish in late 2024/early 2025 snapshots, reflecting a sharp shift in attacker tooling.

Defensive moves.

Layer behavioral analytics, call center anti-spoofing, and identity protections that don’t rely on knowledgebase authentication, which AI easily defeats.

3) Insider Threat Monitoring & Behavioral Analytics

State of the art. Academic work in 2025 shows behavioral features + deep evidential clustering achieving ~94.7% accuracy with 38% fewer false positives on benchmark insider datasets—promising results for reducing alert fatigue when combined with identity/DLP context.

Practical guidance. Start with high signal telemetry (authentication, data access, exfiltration channels), then add ML models that explicitly model uncertainty to handle behavior drift over time.

Challenges & Risks (and how to talk about them)

• Adversarial attacks & model drift. NIST’s AI 100‑2 (2025) taxonomy catalogs evasion, poisoning, privacy, and misuse attacks across the AI lifecycle and outlines mitigations—use it to standardize terminology in policies, models, and audits.
• Ethical surveillance & privacy. The NIST AI RMF (2023) & Generative AI Profile (2024) emphasize transparency, privacy, and accountability—key when logging prompts, retrieved context, and model outputs for forensics.
• Automation vs. human oversight. Productivity gains from copilots occur with improved accuracy when analysts stay “on the loop,” reinforcing that AI should accelerate—not replace—expert judgment.

Implementation Best Practices

1) Start with pilots and observability.

Run limited scope pilots with explicit success metrics (MTTD/MTTR, case throughput). Instrument prompt/response logging and retrieval quality, with rollback paths if models drift.

2) Anchor to open frameworks.

• Zero Trust first for identity, continuous verification, and least privilege—including machine identities (service principals, workload identities).
• Map AI attack paths (prompt injection, model theft, data poisoning) to MITRE ATLAS and add detections/mitigations where gaps exist.
• Align risk controls to NIST AI RMF / GenAI Profile and incorporate NIST AI 100‑2 adversarial threats into design reviews and tabletop exercises.

3) Train the people who will use it.

Adopt the OWASP Top 10 for LLM Applications (now part of the OWASP GenAI Security Project) as your minimum bar for LLM safety—in prompts, retrieval, red teaming, and agent design.

4) Measure quality, not just volume.

Track the % of investigations materially accelerated by AI summaries/queries that led to confirmed findings; use those signals to refine prompts, retrieval, and playbooks.

What’s Next: Trends to Watch

• RAG enhanced detection & evaluation. Retrieval Augmented Generation grounds LLM answers in verifiable sources; new evaluators (e.g., Tavily’s Search Layer with LangGraph) build dynamic test sets for web grounded agents—useful when threats are evolving daily.
• Benchmarking the SOC’s real workloads. MLPerf Inference v5.0 (Apr 2025) reflects the industry shift toward generative workloads (e.g., Llama based tests), with submissions and performance surging—use these neutral scores when sizing AI infrastructure.
• Agentic RAG for semiautonomous defense. Early research (“CyberRAG”) shows LLM agents orchestrating classifiers, tools, and iterative retrieval to classify attacks with >94% accuracy and produce SOC ready narratives—promising but deploy with strong guardrails on data provenance and action authorization.

Closing Thought

AI won’t replace your SOC and a human in the loop is very important. Organizations that intentionally integrate AI into their security strategy, controls, and culture will out detect and out respond those who don’t. Begin with a small, well instrumented pilot, anchor to open frameworks, and measure real outcomes (MTTD/MTTR, analyst throughput) rather than novelty.

Sources & Further Reading

• Pindrop, 2025 Voice Intelligence & Security Report (press release)—deepfake surge, frequency of attempts.
• Microsoft, Copilot for Security GA details—22% faster, 7% more accurate.
• NIST, AI RMF: Generative AI Profile (2024)—governance & risk guidance.
• NIST, AI 100‑2e2025—Adversarial ML taxonomy & mitigations.
• MITRE, ATLAS—tactics/techniques against AI systems.
• OWASP, Top 10 for LLM Applications / GenAI Security Project—LLM risk patterns.
• KnowBe4, Phishing Threat Trends 2025—AI-powered phishing patterns.
• Microsoft Learn, Entra ID Protection risk detections—including “impossible travel.”
• MLCommons, MLPerf Inference v5.0 (Apr 2025)—generative AI benchmarking.
• Tavily Blog & LangChain, RAG evaluation & agentic RAG.
• arXiv, CyberRAG (Jul 2025)—agentic RAG for attack classification/reporting.

 

To learn more, please contact Andrew Corrington, VP of Technical Solutions

acorrington@eimagine.com