Last week, I spoke with Erik Bloch, VP of Security at Illumio, who brings two decades of security operations experience from working on security teams at Cisco, Salesforce, and Atlassian. Erik shared his contrarian take on the AI revolution in security: most organizations are nowhere near ready to deploy these tools effectively.
Erik's perspective comes from hard-won experience.
When Erik joined Cisco, he was immediately thrown into investigating source code theft. That baptism by fire led him to build Cisco's CERT team and spend most of his career in the trenches of security operations—building SOCs, leading IR teams, and even serving as a product manager for SIEM platforms.
This episode explores how to measure what matters in security operations, from team capacity utilization to business outcome dispositions, and why proper ticketing systems and actionable metrics are prerequisites for any advanced tooling to be effective.
Here are five notable takeaways from our conversation:
𝐅𝐮𝐧𝐝𝐚𝐦𝐞𝐧𝐭𝐚𝐥𝐬 𝐁𝐞𝐚𝐭 𝐒𝐡𝐢𝐧𝐲 𝐎𝐛𝐣𝐞𝐜𝐭𝐬: Erik's experience transforming underperforming security teams revealed a consistent pattern: highly-staffed SOCs missing SLAs 70-80% of the time while operating at 200% capacity. "A tool is not going to solve for a missing process or a broken process," Erik explains. "If your fundamentals aren't in place, it doesn't matter what tool you have; it's not going to fix the problem for you."
𝐘𝐨𝐮 𝐂𝐚𝐧'𝐭 𝐌𝐞𝐚𝐬𝐮𝐫𝐞 𝐀𝐈 𝐄𝐟𝐟𝐞𝐜𝐭𝐢𝐯𝐞𝐧𝐞𝐬𝐬 𝐖𝐢𝐭𝐡𝐨𝐮𝐭 𝐁𝐚𝐬𝐞𝐥𝐢𝐧𝐞𝐬: While companies successfully deploy AI tools across engineering and marketing, security operations face a unique challenge. "Until I have the metrics and measurements in place, until we have a process that works that's repeatable... there's no use bringing in a tool like that yet because I couldn't tell if it was effective or not."
𝐕𝐞𝐧𝐝𝐨𝐫 𝐌𝐞𝐭𝐫𝐢𝐜𝐬 𝐀𝐫𝐞 𝐌𝐞𝐚𝐧𝐢𝐧𝐠𝐥𝐞𝐬𝐬 𝐖𝐢𝐭𝐡𝐨𝐮𝐭 𝐂𝐨𝐧𝐭𝐞𝐱𝐭: Erik challenges AI security vendors on their effectiveness claims: "We're doing it 57% faster than a human—based on what? Some people say five minutes, some people say three hours. There's no baseline." He points out that speed comparisons are meaningless without knowing if you're comparing to "a two-person startup with one tool" or "a huge mega company with a thousand vendors in the ecosystem."
𝐀𝐈 𝐒𝐡𝐨𝐮𝐥𝐝 𝐄𝐥𝐢𝐦𝐢𝐧𝐚𝐭𝐞 "𝐃𝐢𝐬𝐡𝐞𝐬 𝐚𝐧𝐝 𝐋𝐚𝐮𝐧𝐝𝐫𝐲," 𝐍𝐨𝐭 𝐂𝐡𝐚𝐬𝐞 𝐀𝐏𝐓𝐬: Erik argues that 95% of security operations work is routine tasks—phishing emails, duplicate tickets, context switching between hundreds of tools. "How about the phishing emails that I do every single day? Crush those. That will impress me. Not the fact that you caught the latest Chinese APT." The goal should be freeing teams to work on the meaningful 5% that drew them to security in the first place.
𝐒𝐭𝐚𝐫𝐭 𝐖𝐢𝐭𝐡 𝐎𝐮𝐭𝐜𝐨𝐦𝐞𝐬, 𝐍𝐨𝐭 𝐓𝐨𝐨𝐥𝐬: Erik's methodology is simple but powerful—define the business outcomes you need to deliver, establish metrics to measure those outcomes, identify capability gaps, then find tools to address specific problems. "The outcomes not only help you get the tool you need, but they also help justify the budget." Without this foundation, even the most advanced AI becomes another expensive solution looking for a problem.
The episode is essential listening for security leaders who want to build sustainable, effective operations before adding AI to the mix. Erik's emphasis on fundamentals provides a clear roadmap for organizations serious about measurable security improvements.
Share this post