There’s a shift underway in the implementation of AI in security operations. Tyler Martin, Senior Director of Security Engineering at FanDuel, has been working at the edge of this transformation, building custom AI agents that have changed the scaling laws for his team. Instead of the traditional tier 1-2-3 analyst model, they develop and maintain agents that autonomously handle what used to be manual triage work.
Tyler’s path from accidentally enumerating a healthcare database as a Java developer to leading one of the more innovative SecOps teams around provides good context for this conversation. His team has built multiple specialized agents, including “SAGE” (Security Analysis and Guided Escalation) for phishing, account takeover, and an incident response automation that runs entire IR workflows via Slack. The results are notable: a 90% reduction in incomplete post-incident review action items and engineers spending their time building rather than working on tickets.
This conversation gets past the AI hype to discuss the practical realities of building production-grade security agents. Tyler talks about everything from the “bronze-silver-gold” approach to automation maturity, to managing context rot in LLMs, to why the industry needs to start measuring different things. The most helpful part is understanding why starting with specific, high-volume use cases and gradually expanding works better than trying to automate everything at once.
Key Takeaways
Moving to All-Engineering Teams Enables Better AI Outcomes: By dropping the traditional tier 1-2-3 analyst model and staffing entirely with security engineers, FanDuel created a team that can build and maintain its own agentic AI systems. This change unlocked the ability to continuously improve automation rather than operate it, shifting from “working tickets” to “building systems that work tickets.”
Context Rot is the New Challenge in Agent Design: Just as analysts can be overwhelmed with too much information, AI agents suffer from “context rot” when given excessive data. The key is finding the right amount of information—enough signal for accurate decisions without overwhelming the model’s attention. This requires careful thought about what data enters the context window and in what order, similar to how you’d organize information for a human analyst.
The Bronze-Silver-Gold Maturity Model for AI Automation: Start with bronze (human-in-the-loop validation, no automated closures), move to silver (automated closures with some manual intervention), and eventually reach gold (fully autonomous triage). This phased approach lets teams build confidence, identify missing context, and add necessary integrations step by step rather than attempting full automation right away, which usually fails.
Runbooks are Now AI Agent Instructions, Not Human Documentation: The traditional detection runbook has evolved from documentation for human analysts to specific instructions for AI agents. While basic investigation steps should be in the agent’s system prompt, runbooks should focus on the context and investigation patterns unique to each detection. This is where prompt engineering becomes a critical security skill.
Incident Response Automation Through Slack Changes Things: FanDuel’s IR automation handles entire incident response workflows with simple Slack commands—automatically creating channels, inviting stakeholders, spinning up Zoom bridges with recording enabled, generating real-time documentation from transcripts, and assigning action items to specific team members in Confluence. This solved the problem of incomplete PIR action items and significantly reduced post-incident administrative work.
The transformation Tyler describes aligns with where Panther is going with our AI-powered triage capabilities. Our agent automatically handles the assessment layer Tyler discussed, gathering context and presenting risk-based summaries so your team can focus on investigation and response rather than manual enrichment. Learn more about Panther AI and how we’re helping teams make this transition.
