The containment era, and why ingest pricing was a horrible model

Shachar Hirshberg

Hey everyone, and welcome back to the Artemis podcast. Today we have a truly special guest with us, Doug Merritt. Doug is one of the most seasoned security leaders in the industry, and he's currently the chairman and CEO of Aviatrix. He previously led Splunk as their CEO, where he truly disrupted the category and took Splunk from hundreds of millions to billions in ARR. So first, thank you, Doug, so much for spending time with us today.

Doug Merritt

I'm excited to be here.

Shachar Hirshberg

To kick things off: you recently wrote that enterprises spend more money than ever on cyber, which is objectively true, but in practice they are less secure than ever. Curious on your take: why do you think that is the case?

Doug Merritt

Let me take a step back and frame the industry and the challenges it's dealing with. I think the simple answer is our landscapes are so fragmented, so many different generations of technologies, and different tools to cover each of them. The typical CISO is incredibly overwhelmed with too much advice, too many adversaries, too much input, and it's difficult to slow down and think about the foundations within cybersecurity. But at the core, why I think we're less secure is that life is emergent, and the cyber industry and adversaries are emergent. They're growing and iterating and evolving every single day.

I view the security industry as being characterized by two massive categories, or ways of thinking, that the defenders have been focused on. The first was the birth of cybersecurity. As we plugged the internet into data centers, people appropriately lost their minds: what the heck are you doing? This is a highway for six, seven billion people, whenever I go back into the tally, to come directly into my most trusted stuff. That whole wave, and these waves never go away, was primarily focused on prevention and primary defense. A real castle-and-moat orientation: let's keep these folks out. The birth of firewalling and IPS and IDS. But the adversaries are smart, and they keep morphing and changing tactics.

By the time I got to Splunk, we weren't really yet a SIEM vendor. When I got there, we wouldn't even allow Gartner to label us a SIEM. We were still a data lake. That's where Splunk started, I affectionately called it a garbage lake. It would take very messy, non-formatted logs. Don't try and clean them up; honor the lineage from where they came, so you could find signal in the noise of petabytes, or eventually exabytes, of data. And then the SIEM grew out of that. But my advocacy in 2014, '15, '16, seeing how the landscape changed, was: prevention is so good, don't stop focusing on that, but it's all about detection and response. The adversaries are inside. Well before Salt Typhoon and people admitting, hey, they're literally squatting on network infrastructure from third parties and sitting inside our networks, inside classic network vendors like AT&T and Verizon, for years. It was self-serving for Splunk, because we were this great data lake and then started to go to the SIEM category, but it was more authentic: you have to assume you just have to move faster than the adversaries. They're very clever, and you're going to have to detect and respond much more quickly.

And I think they've dramatically changed tactics again. I've been jumping up and down about, again, don't stop prevention. I believe in Artemis; I'm super excited about what you guys are doing. And don't stop the detect-and-respond piece. But you're going to have to add in a resilience orientation, which I characterize as: we're going into the containment era. They're going to be successful in penetrating through a variety of mechanisms, and ultimately your resilience will be determined by how far they can go and how much damage they can cause. So why we're more insecure: you have to think first principles, super foundational, about how systems are architected and how you effectively cover that landscape on a prevention, detection, rapid response, and containment basis. And it's overwhelming for many folks to think through that.

Shachar Hirshberg

Makes sense. And as you mentioned, we're seeing the threat landscape changing. Attacks that used to take months to plan and execute now take seconds. We recently saw what happened with OpenAI and Hugging Face, where a new model in development was able to break out of the OpenAI sandbox by finding a zero-day, and then understood that Hugging Face might have the answer it was looking for on the cyber benchmark. So it hacked into Hugging Face and got the answer it needed from there, while chaining together multiple zero-days and moving laterally through the network. There are not many people in the world who could do that, and it would have taken them a long time to achieve that level of execution against these two top-tier companies, but the latest model in development was able to do it autonomously. So attacks are changing, and this kind of frontier intelligence is going to be available to everyone. It's just a matter of time.

Doug Merritt

I mean, it already is. Again, people are so over-rotated on Mythos. It's a moment in time. Kimi K3 shipped what, five, six, seven days ago, and in some cases it beats Mythos. And obviously OpenAI's next model is even more capable than Mythos, and we're on this progression, and it's not stopping. Anyone that thinks they can control it just doesn't understand tech. The whole orientation was open-sourced by Google in 2015, '16, '17, and it's in the world. You've got millions of people taking those open-source initial papers and constructs, and now open weights, open models, and improving them. And they're going to keep improving them.

Shachar Hirshberg

Exactly, it's a new reality. And in this new reality, what does the defense that actually keeps up look like? Because to your earlier point, if it takes you hours to detect and stop something, or maybe days or weeks, it will probably be too late in an era where, like with OpenAI, execution is autonomous and happening within seconds. What does best-in-class, AI-native defense look like?

Doug Merritt

I'll go through it by timelines, because what most of the community has been focused on since the Mythos scare is what I view as the longest timeline, at least in terms of immediate impact. What everyone was screaming about was: we've got to do a better job of scanning our own code, finding different vulnerabilities, and increasing the rate of patching. Yep, we should. That's good basic hygiene. But given the size of the code base, the volume of new code entering, and the multiple dependencies and complexities, that's a four-to-eight-year problem to solve. And we'll get there. At some point in time, if these models continue, every piece of code that anyone touches will be absolutely hardened and foolproof, because it will be written by these genius models that understand all those vulnerabilities, and you'll be impenetrable, at least from a vulnerability perspective. But what do you do over the next four to eight years? You're living with systems, you still have people with COBOL systems, assembler-based systems. It's going to take some time.

The intermediate stage is where I think you guys and a lot of other people are focused, which is: you've got to be able to respond at a very different rate. We had humans attacking humans, and as you started with the Hugging Face incident, we've just democratized incredible skills. As you know deeply from your past, there were only a couple of thousand people with both the technical and cyber understanding skills that could do really sophisticated attacks. So we had a very limited audience. And now, with even 4.8, much less Mythos or K3, we've broadened that to millions of people. If you've got an interesting, nefarious idea and you're curious at all, you just play with these engines and they will help you. Guardrails help, but open-weight, open-source models do not have those guardrails, so they will help you. And then you automate the attacks. So it's not human against human. It's not even human-assisted-by-machine against human-assisted-by-machine. It's going to be machine against machine for a lot of this.

And then the most immediate, which, again, is going to sound self-serving if anyone looks up Aviatrix, but it just goes back to laws of physics and first principles and foundational thinking. It blows me away that this happened with OpenAI. I know the CISO that built that company, and he is a deep network domain expert and really understands systems. The fact that this agent could have broken out is not crazy, but with a network-level control, you should have been able to determine this thing is exiting, it's making queries. What does it mean to exit? It means you're pinging and interacting with things, and if it was really bounded and gated, it wouldn't have been able to do that, much less reach a commercial site. And then the receiving end, with Hugging Face, I'm sure it was a public call, but with the right immediate resiliency orientation and network controls, you should have been able to gate it. Maybe it got in through a legitimate channel, but it started quickly making calls to areas that, wherever it landed, should have been prohibited. And then the command and control back to that agent.

So the most urgent and immediate thing I have been advocating for almost a year now is: pay attention to the network. It's an incredibly important foundational control. Lock down as much as you can while you're rotating into great tools like Artemis is providing, so you can operate at machine speed. And then in the background, instead of the foreground, worry about hardening your systems. You can't be fast enough.

Shachar Hirshberg

Totally agree. There are certain truths in life, like taxes, and I think one of them is that patching servers is really hard. It will always be really hard, because it can impact the business in ways you just don't predict: you upgrade the version and then your e-commerce application goes down. So that will always take, to your point, years and years. So as a recap, it sounds like you'd do it in a three-layered approach: continue with the hygiene, add containment to limit the scope of potential impact as the immediate step, and also add detection and response, making sure that when someone is in the house, you're alerted about it as soon as possible, and ideally the machine already takes care of it on your behalf with human guidance.

Doug Merritt

Yeah. If you think about the data that's out there: CrowdStrike, six months ago I think, published a research report that was trusted at face value, 80, 82 percent, I think, of the successful breaches over the past 12 months carried valid credentials. Which means that initial detection is hard. You've got a valid entity trying to do something.

Shachar Hirshberg

That doesn't mean the detection is bad.

Doug Merritt

Correct. The valid credentials are eventually going to be misused, which should be the anomalous pattern that, if you've got machine-speed capability, you can begin to react to. But the game has changed really, really dramatically in how the adversaries are attacking. So outside the valid credentials, and I run a report every other week, there are three other areas they're intent on going after. The pattern is: I want something that likely has weak egress or ingress controls (if you're an ephemeral workload, you're generally weak within most environments, because there aren't static IP addresses and typical techniques to harden those) and high permissions and high reach. These guys are not idiots. So what fits that? The first thing is agentic AI infrastructure and the AI agents themselves. They tend to be serverless or Kubernetes-hosted pods, and they have the high permission and high reach. The second is security tools themselves. It's amazing how many security tools have been leveraged for the same reason: the whole purpose of a security tool is high permission and high reach across the landscape. And the third is the supply chain attacks we've been seeing, which are also crazy. You're getting in upstream so that you can infect a production code push to insert malware. These guys are smart.

Shachar Hirshberg

They are smart. It's the fun thing about cybersecurity: you always have innovation, because it's a constant battle between really smart people on both sides.

Doug Merritt

You're never out of business, and you're never out of innovating and curiosity, because it's changing all the time.

Shachar Hirshberg

Taking you back a bit to your SIEM days: you spent time disrupting that industry, and we're going through a big technological change in the industry right now. Knowing everything you know about the customers and how they use their SIEM, how would you start if you were starting in today's era, clean slate, AI-native architecture, in terms of the product itself?

Doug Merritt

I'll answer that more by rewinding: how did Splunk get there, and what were we trying to do at Splunk up until the time I left in early '22, as a foundation for how the SIEM was thought of back then, and then how different I think it is today. Again, Splunk grew up as this garbage lake, a great repository for very, very dirty data. The only reason we got dragged into the SOC and the SIEM space: we had started with IT ops as the key constituency. We were just trying to find people that understood machine data, had very complex, non-deterministic jobs, and could use this data lake effectively. IT ops was the first target, and through that customer base, they started talking to their SOC friends: hey, we've got this great result using this tool from Splunk, why don't you use it?

Our initial approach was: let's just go in and support ArcSight or QRadar or whatever the heck the SIEM was, because we would be both the more fine-grained, higher-fidelity source and the answer to the question. You get a bunch of alerts, you're trying to figure out which ones are false positives, false negatives, or true, so let me go into Splunk to get a source of truth. And the customers kept hounding us, saying: you've got the data, we use you more than we use the actual SIEM, why don't you just render it for us on top? Which was amazing, because that got us into a whole new category. Ultimately, by the time I left, over 90% of the deals that came in were from the security team, typically the SOC and the SIEM team, and would spread from there. So it was one of the key levers to our growth.

But there were things we were trying to correct. Because we moved quickly, instead of having separation of data concerns, given the complexity of what you're trying to do in a SIEM, we morphed the index, which was not built to be a multidimensional data warehouse store, or a TSDB, or an overall trace or metric store. To create a single user experience and integrity when we first launched, the engineering team just used the index. So a lot of the initiatives we were driving between 2015 and 2022 were: let's separate this back out. Let the index be the index, and introduce streaming technology and graph technology, data warehousing technology, this federated search piece we were trying to drive, so you could use the right data tool for the right job. So I would definitely take that into the SIEM: there are many, many different types of data and data sources, and how do you have a disciplined and effective strategy to use the right data store for the right job, while providing a contiguous, uniform experience, which is non-trivial from an access and rendering basis.

But the more interesting thing is: what are people doing with the SIEM? All the SIEMs today that I interact with were built for human users. And going forward, I think that's going to be less the main interface. It's going to be agents and machines, and how they digest information, which I think completely changes the way that people think about and build these things.

Shachar Hirshberg

Totally. And you spoke a lot about the growth you went through at Splunk. One of the key drivers of this growth was ingest-based pricing and the fact that customers saw value, so they just put in more and more, which translated into revenue. Security teams, on the other side, after some years with this model, decided that they don't always love it. Curious on your thoughts on the model itself: is ingest pricing still the way to go today, or is it something that was good for the past, but not anymore for this era?

Doug Merritt

Horrible pricing model. Horrible pricing model. It was invented back in, I frame it nicely, 2005 to 2007, when Splunk was just getting going. The big competition, the database giant, was Oracle, and Oracle was CPU-based. And no one likes any pricing model. So the two founders, Erik Swan and Rob Das, were trying to differentiate back then, and they came up with this super interesting thing: why don't we do data volume? And in their thinking back then, it was megabytes of data they were going to be dealing with, not even gigabytes yet. So it was super disruptive, and it was very friendly. By the time I got there in 2013, we could see our first petabyte coming. Customers weren't there yet, they were at tens of terabytes, but we could see they were going to a petabyte. And it just became the wrong metric.

Because when you go to something like cybersecurity, and this is what I kept arguing with our customers about, the core is the garbage lake. When you get breached, and you will be breached, going back to the detection and response era, you need to figure out what the heck happened. They kept saying: what's the valuable data? Because I don't want to pay for it all. It's like: you have no idea, and I have no idea. If you knew where you were going to get breached, you wouldn't be breached. It's going to come in some corner you didn't think about. I don't know if NetFlow logs could be the most important thing, or OS logs, it could be from anywhere. You need it all. And their response was: you're trying to increase my price.

So there were two reasons we moved to the cloud. One was, when you start to get to petabyte-plus scale, it's very difficult to manage an environment of that size. We could see our earliest, biggest customers getting there. They tend to be tech companies with incredible engineering skills that most of the rest of the companies didn't have. So we had to get to cloud to help them with the eventual size and management they were going to land on. But the other one, just as important for me, was trying to change contracting terms. When I got there, I think we were 3,800, 4,000, 4,200 customers, and by the time we made the decision, 10,000 customers. And the many contracts a customer will have, because it's not always one contract per customer, is unbelievably painful. No matter how much they complained, when a renewal came up it's: well, let's go through a whole different contract and a whole different pricing process. It would delay deals, and they would push back.

So the core driver of getting to cloud that I felt most attached to was giving the customers value-oriented pricing. It's really cheap to ingest and store logs. The value is when you query the logs. Getting to cloud and changing our pricing construct was my attempt, back in 2016, '17, '18, at getting more value-aligned. Because data volume was the barrier: your value comes from the diversity of data, and we were stopping you from doing that. And pricing is just such an important strategic decision, and it's so hard to reverse. It's a definite one-way door.

Shachar Hirshberg

Totally. And it's interesting today because, to your point, the diversity of data still matters a ton, but the main consumer of the data is not humans anymore, it's agents. They tend to be very hungry in terms of consuming data and tokens, but they also don't necessarily process information like a human would. For example, you switched from ingest-based pricing to compute-based pricing, and that aligns better with the queries customers are making. But a human might run an SPL query every 30 minutes; agents might run 3,000 queries in one minute. And a lot of these queries, by nature, will return nothing valuable, because most of the time nothing happens. So it's really interesting to think through, and I don't think anyone has solved it, how you continue to align the value customers get with the pricing, but in a way that is sustainable and not cost-prohibitive for customers.

Shachar Hirshberg

And to follow up on that: you went through this kind of rebirth for the company as you made the move from on-prem to cloud. And in every generation of new companies and new technologies, we see fragmentation and consolidation of tech companies. We're now in this new transformation, the AI wave, still early in that wave, but seeing some early indications of winners. I'm curious about your thoughts on who will be the winners of this age. Would it be the incumbents, the legacy platform companies? Would it be the hyperscalers, the model providers, or AI-native startups on the application layer?

Doug Merritt

Always hard to predict. My prediction is disruptive, focused, next-gen companies, and then who knows how the model companies or the hyperscalers pivot to take advantage of the work those companies have done. We saw all the hyperscalers pivot when Splunk started to get really successful and try to take us on. But I go all the way back to: what is the data layer of the new SIEM? I think the agents are so effective and productive at seeking data, transforming data, and querying data at the source that, when you go through a re-architecture, which I think we're seeing from a usage basis, from an output and response-time basis, from a data consumption and storage basis, it's really hard for the incumbents to do that. Because it invalidates everything: all the tech investment they've made, and it invalidates their pricing model. And going back to pricing, maybe consumption's not the model. Most people are still ingest-based. I'm not even sure what Splunk does with its pricing anymore; I thought we were trying to be innovative with consumption-based. But I think it's a perfect opportunity for some massive disruption, and customers need a very different approach.

Shachar Hirshberg

For sure, totally agree. So today you work with boards and companies, and obviously you're still the CEO of a company. I'm sure the audience would love to hear from you: maybe one piece of advice for the people back home on how to be effective leaders in the era of AI.

Doug Merritt

Being an operator, and I'm not sure we've compared enough notes on this, but we should compare notes on how you feel every day, being an operator in this environment is unlike anything I've ever felt before in my life. For people that haven't heard me say it before: we moved 11 times before I was in high school. I got into tech because I love change and I love learning and growth. But this cycle and this pace is insane. And it changes everything: how companies are built, what kind of employees you need, what the role of an employee is, how you motivate different people, how you pivot quickly enough to different competitive changes.

Where I've really centered, to the extent that I brought an author with this philosophy into the company to help all of us, is: try and ignore as much of the extraneous stuff as humanly possible, because the change rate is so overwhelming. When ChatGPT first launched, I went through a Yuval Harari depression cycle, as he was beginning to blog that this is going to be the end of humanity. Oh my God, is this really the end of our species? It took me six months to pull out of it. And I've landed on: who the heck knows? It's going to be a bunch of great stuff and potentially a bunch of bad stuff. Let's just enjoy it.

And I've tried to use that same mentality running a company. What we can control and focus on is our daily behaviors, our daily actions, the passion and vigor that you bring to trying to perfect your craft on a half-percent, quarter-percent, one-percent basis every single day. And try not to get too distracted, overwhelmed, fatigued, which, again, I battle every single day, by the big picture and how much is changing. For me, it's focused a lot more on the human contact.

Shachar Hirshberg

Love it. Thank you so much for coming today. It's great to see you. Love the advice and all the wisdom shared with the audience, and looking forward to seeing you again soon.

Doug Merritt

Oh, thank you. Me too. I really appreciate being here with you.