What Changed and Why It Matters
Sales and security both adopted AI copilots. The volume went up. The signal didn’t.
Outbound tools now let anyone generate “personalized” messages at scale. That raises noise—and attack surface. Meanwhile, attackers use the same AI to automate phishing and business email compromise.
“Better AI = more noise = less signal.”
Here’s the part most people miss: defenders are using copilots, too. The leading SOCs report sharp drops in false positives and faster triage, turning the same technology into a signal amplifier. This is the new equilibrium: AI-generated outreach and threats on one side, AI-driven filters and copilots on the other.
The Actual Move
Across the stack, the shift is clear:
- SOC copilots are reducing alert fatigue. VentureBeat reports teams seeing a 70% drop in false positives and reclaiming 40+ hours a week from manual triage.
- Security vendors are bundling AI into the core. Palo Alto Networks frames an AI-led push to simplify operations across products, while Cisco highlights the gap between expectations and reality for AI agents in networking and security.
- The threat side is scaling fast. Barracuda says half of inbox spam is now AI-generated. Security Boulevard reports AI phishing growth of 1,760% with claimed success rates near 60%—a sign that quality and frequency have both improved.
- Email and comms filters are getting smarter. Microsoft’s Exchange Online Protection and Defender for Office 365 now lean on AI to boost spam and phishing detection. Consumer tools like Cloaked use behavioral analysis to block calls and filter email.
- GTM copilots are flooding channels. Koncert’s guide shows how AI copilots automate outreach and enrichment. On the ground, operators observe more “personalization,” less relevance—which translates into inbox fatigue.
“SOCs are seeing false positive rates drop 70%, while shaving 40+ hrs a week of manual triage.”
“Half the spam in your inbox is generated by AI.”
“AI phishing attacks [are] rising 1760% and achieving a 60% success rate.”
The Why Behind the Move
AI raised the volume for everyone. Winners are shifting from scale to signal.
• Model
- Defenders: domain-tuned LLMs grounded in telemetry (alerts, identity, email, EDR), retrieval over knowledge bases, and reinforcement via analyst feedback loops.
- Attackers: prompt-driven automation for copy, spoofing, deepfakes, and adaptive lures—low-cost, high-iteration.
• Traction
- SOC copilots show measurable gains: fewer false positives, faster investigations, improved analyst capacity. Email and call filtering show continual improvement with behavior and content signals.
• Valuation / Funding
- Not the point. The durable value is in data access and workflow embed—where copilots become default UI for operations.
• Distribution
- Bundling wins. Microsoft 365, Palo Alto, and Cisco anchor copilots where the data already lives. That beats standalone tools fighting for integrations and permissions.
• Partnerships & Ecosystem Fit
- Strong alignment around identity, email, SIEM/SOAR, and EDR. Trust frameworks (DMARC, DKIM, BIMI), cryptographic attestation, and voice verification are becoming table stakes.
• Timing
- GenAI made outreach and phishing cheap at scale in 2024–2025. The defensive response in 2025 is about quality: risk scoring, correlation, and closed-loop learning.
• Competitive Dynamics
- It’s an arms race. Offense iterates on prompts and payloads; defense compounds signal with context and analyst feedback. The moat is the feedback flywheel across real incidents and real users—not the base model.
• Strategic Risks
- Over-automation can miss novel threats or bury legitimate messages.
- Hallucinations and false confidence from copilots.
- Privacy and governance gaps if copilots ingest sensitive content.
- Adversarial content (prompt injection in email/attachments) targeting the copilot itself.
What Builders Should Notice
- Measure signal, not volume. Lead with false-positive reduction, time-to-triage, and downstream outcomes.
- Distribution beats novelty. Ship copilots where the data and workflows already live.
- Design for trust. Human-in-the-loop, audit trails, explainability, and strong identity/authentication layers.
- Close the loop. Capture analyst feedback and real incident outcomes to continuously tune the model.
- Assume adversarial use. Instrument abuse monitoring and red-team your copilot’s inputs (email, links, files, voice).
Buildloop reflection
“In the AI era, scale is cheap. Signal is the strategy.”
Sources
- LinkedIn — Nate Taylor’s Post
- Adaptive Security — AI Copilots Increase Risk of Sophisticated Phishing
- Palo Alto Networks — Radically Simplifying Cybersecurity
- VentureBeat — How leading SOCs use AI copilots to fight signal overload …
- Barracuda — Half the spam in your inbox is generated by AI – Barracuda Blog
- Security Boulevard — How AI-Generated Content is Fueling Next-Gen Phishing …
- Cloaked — How Can AI-enabled Call Blockers and Spam Filters …
- CIAOPS Blog — Use AI to provide better spam protection and detection with …
- Cisco Blogs — AI Agents for Network and Security: Expectations vs Reality
- Koncert — The Ultimate Guide to AI Copilot for Sales Reps – Koncert
