Building an AI Moat from the Inside

Welcome to this episode of Applied Intelligence. I am your co-host, Roderick Bates, along with Kelly Sarabyn. Today I am very excited to have Mark Koussa with us. Mark is with a company that has fifty-six years of experience, but is now all of a sudden facing AI innovation. And Mark is leading that as, I guess, what is it, the VP of internal AI at LexisNexis.

Yeah, thanks Roderick. really, really excited to be here. Honored that you'd asked me to be on. Yeah, I I lead internal AI innovation. So what that is, is how we as a company adopt AI, leverage its transform, and really more than anything, build out the way we work with AI as a competitive advantage, as a moat.

between us and anyone else in the market. So

And Mark for context, I know who LexisNexis is because I have a legal background and everyone in law knows who LexisNexis is. But for the audience, can you kinda talk a little bit about like how large your company is, what the different types of works and business units are within your company, just for context is the scope of of your role?

Yeah, absolutely. So there's LexisNexis Legal and there's LexisNexis Risk. And so I work in LexisNexis Legal and and that company is in all seven continents, many countries. We are nearly fifteen thousand

employees serving the legal industry and every aspect of the legal industry and we have been we have sales business units across Asia Pacific, across continental Europe, South Africa, UK, obviously all across the United States and North America as a whole. so yeah, we serve private and public. We serve

the corporate markets as well courts judges and so yeah we kind of span everything across the legal industry providing a whole series of tools as well as content we are one of the largest content providers in the world with billions of documents tens of billions of documents

So yeah, that's in a nutshell what we do.

It it's pretty fascinating. So there's there's actually a couple of pieces there that I wanna dive into, but one of them is obviously all these different regions. And here you are managing AI innovation with a company that obviously has to conform to whatever legal requirements might be for everything as you mentioned, everything from South Africa to Europe and North America. Are the policies consistent internally upon how AI is deployed across all these areas?

Yeah, that's a that's an interesting question. For the most part it is. There there's obviously some nuances around things like what

is allowed from a labor and employment standpoint because the different companies have different labor and employment laws. But for the most part, the technology is the technology. And we in every one of our markets wheel such a high duty of quality security content that is what our customers expect from us. And so that has to flow from everything we do internally to make sure

we are as secure as possible that the content is of high a quality as possible that we are not we're doing everything we can to remove risks to remove loopholes to enable reliability and accessibility and so in those regards it is important for us to take wherever the highest level of

security and duty to the customer is and everyone operates at that level. So we hold ourselves to a super high standard. And so in that regard, outside of the labor laws, it is largely the same right now.

Is there a situation that actually when you talk about you know this idea of security and quality and things like that, like can you actually deliver at that high level anymore without using AI internally?

It is quickly becoming unfeasible. I mean we can, we have built our we have built our reputation, we have built our company around that. But, you know, as we begin to I I think what it is, is there is an evolution at which AI is necessary to come along.

Us because now we can do more than ever, we can deliver more than ever, we can do it while also keeping the quality high. And it kind of gets to that point of AI as the ultimate enhancer, and so we can enhance everything we do. And so, yeah, it kind of becomes this self-fulfilling cycle where, well, we could do that with AI.

And so now we need to do that with AI.

And it's incredibly large scope that you have. I guess when you when you think about kind of what you're trying to drive with AI and maybe looking back on on where you started, right? This happened so fast. Were you kind of starting out looking for efficiency gains to current processes and workflows and then moved into trying to now drive better outcomes, higher quality work? and are there other ways that you're looking at it? For example,

repla replacing whole tasks or scopes that previously needed to be a a a human.

Absolutely. Yeah, that's spot on. I mean, we started by saying, well, let's deliver value. We can all move faster, right? Like start using ChatGPT or start using Copilot. And how can we even track dollars saved and we know we can work faster? What it has become now is that the primary goal of AI is value and benefit, period.

And when you now think about it from that lens, now it is interwoven into the fabric of every way that you as a company define success. So, for example, sure, working faster is absolutely one of the aspects. So is improved quality, so is the consistency of that quality. And then you shift internally, so is employee satisfaction, and so is some of the excitement.

that comes with using it. And so then it becomes, it started out as some tasks. Some help me with writing email.

Now it is, well, how do we transform workflows? And I have kind of this maturity framework that we laid out internally that says you begin by exploring. And it's kind of based off this quote I love from Ben Mann over at Anthropic, and he said, people who use the new tools like they were the old tools tend not to succeed.

you have to fully reimagine what you do. And so we start, we took that and went, well yeah, we're kind of, we're kind of just taking it and and using it like old tools to say that we could do this stuff faster. And instead we go from exploring how to use these tools.

Yeah.

To weaving them into your day-to-day, to then starting to say, as I can right now, I can do three times the work I did. I can go into new territories and new areas that I never could before. So now my actual objectives and my targets and what I can accomplish are things I couldn't even conceive of before. And so we reimagine workflows, we reimagine goals, and we reimagine the outcomes that we can actually deliver.

So the goal is really end-to-end workflows. And the last piece I'll say, because you've really hit on something that's near and dear to me, is that we also start to move from individuals creating this transformation to the company benefiting from that transformation being democratized to everyone. And that's, I think, one of the key last areas, which is you move from that. I have done

something super helpful. Two, we all benefit from this thing now helping all of us.

Th that's something I was actually really interested to find out about with your company, you know, the as you develop these workflow focused processes that are leveraging AI, are those ones where the people that are actually engaged in the workflow are developing those processes and then it's democratized from there? Or is there a central point in which

say there's a discovery process, you identify a workflow, you create an agent that specifically works on that problem, and then it's spread out across the company. You know, what's sort of the the order of operations of discovery and implementation?

It is a hundred percent both because what you have to do is

There are so many people who have so many creative ideas on what they can be doing. And the last thing you want to do is stifle them. You want to give them the platform. And actually, what I've come to accept very much very quickly is messiness and kind of like, I would rather a thousand great ideas and a thousand bad ones to make sure we have those nuggets of great ideas. And so the way I approach this.

With the internal AI innovation, is there is I have teams who actually do pure product development, product management, discovery, scope out problem spaces, build and deliver solutions for organizations. We're essentially following a forward deployed engineering model where we embed ourselves within different business units, study every aspect of the way that

business unit works, identify problem spaces, deliver value to them. And we have a separate arm that is entirely around empowerment, saying to all of you who also have an idea, go as far as yeah. Yeah, 100%. Yeah. I I have an enablement arm to my organization and I have a development arm to my organization.

so it really is a hundred percent both then. You're not lying. That's that's part of your formal strategy.

And what does the empowerment side of that look like in terms of kind of opening it up to the floor to say anybody who wants to come to the table with ideas, we're here to hear them? Like what does that process actually look like in real life?

It is trying to be everywhere they are at its highest level. So what we do is I think about it kind of as like four tiers. So we provide two employees to start.

The fundamental building blocks of how you use AI. And we will partner and have office hours where people bring questions and we train people and we bring in companies like OpenAI to help lead these office hours and Microsoft to lead these office hours. So we provide them the training. And then we have multiple forums. So we have an AI champions group that is very robust and has hundreds of champions.

Champions across the organization, and they are the front lines for their teams. Peer to peer learning is the single most important and effective piece of AI adoption that I have personally seen. I could train people all day. I could show them the coolest things all day, but when they see their peers using it and understanding how, it's almost like they've exactly.

They don't want to miss out, right? I mean Yeah.

They don't want to miss out and then it triggers this,

Well, I could do that too. So we have the champions program. We have the you know, we can build custom GPTs and we have a marketplace to share them. We have a kind of internal social media site where we have people talk about how they've used it, what questions they have, and the whole company is part of that. We have thousands of people in that Viva Engage community that we leverage. And then we bring this up at town halls and we recommend.

So I I just have one question, something I I know, particularly my co host Kelly, really is is mindful of is that you need to have some metrics, right? Some KPIs and things like that. And I'm curious on how you implement that within the context of both of these tracks, particularly the on the enablement side, you know, that's even more challenging, I would think.

Yeah.

The enablement side ultimately kind of it sits on me to in my team to demonstrate how we are driving adoption. So we look at things like usage and how often people are using these tools. We look at things like the manner in which we can quantify the different projects that are being run across the organization. You know, we have what's called an innovation office, and that innovation office works with every different business unit.

To say what are the things you are doing to deliver value with AI and what value is that to the earlier question you had, Kelly. And we quantify that in the different ways. And so we actually track that. And then when we have champions join our AI champions group, I actually have them take on some level of demonstration of impact to their own groups, how often they're doing trainings, how many people are attending.

some kind of looser KPOs for them because they are doing this in their spare time. But most of it sits with me and my team in those areas. So adoption, the value, the usage.

Sounds like you're coming d from a perspective of of leaning more into this concept of generating excitement, driving employees to see the value versus another approach, right, which was would be more on the negative. This is now part of your job. If you don't adopt it, we will no longer have this role, which I think you know, partly that's probably a maturity cup. S some organizations get to a point where it's just so embedded in every role. You you have to be

moving it but as we're sort of on the cusp of of implementing a lot of these things, it sounds like you've really focused on, hey, how can I motivate people to want to come drink from from from the cup instead of instead of making it seem like a chore or something that is just like being imposed imposed top down on

Yeah.

We're we're absolutely trying to do that because this should be so exciting. Now, I should add at the same time though, we do expect managers and people leaders to demonstrate really clear expectations of what they expect from their teams. And so I don't I I want to make sure to really highlight that piece because it's absolutely necessary. If it's only about this is going to be great, everybody, then that's not enough. And we actually got feedback to that extent.

Extent of I need to know what the expectations are of me and what the goals are around me. And so that sits out of my team. I work with the different leaders, but we do expect every organization and their managers to know what the right rigor is at which to set that for their team. Cause we can't have anyone not using anymore.

Yeah, that makes sense. And I guess more strategically, like how do you think about the trade-off between, you know, this technology is is rapidly evolving, especially when you think about the agentic use cases, right? Versus just like the LLM j generation of of information. That trade-off between speed, experimentation, and implementation, do you have a framework? Because I think everybody's trying to work through that, right? Like how fast should I go?

How deep should I go into experimenting and implementing that across the org when it's not yet proven? Do you have like a mental model that helps you make those decisions?

Yeah, do you mean on the on the internal side with my team, the model that we

Yeah, when you think about like implementing different workflows or agentic U cases internally, when the technology may or may not be foolproof and some of the particular use cases may be fully not be fully tested anywhere, thinking about that trade-off between, hey, we're gonna ask everybody to engage in this new process, we're gonna run it out.

Yeah.

And maybe it will pay off, maybe it won't, versus going a little bit slower to say, hey, let's test this in a pocket or let's let make sure the technology's greater certainty of delivering our own.

Mm-hmm. Yeah, absolutely. With since we are internally focused, we have the luxury of being able to be

a little rougher around the edges in what we deliver, you know, mate, with things where we're we're at such a higher standard with our customers because, you know, they're this affects their professional licenses. they they ha this has to be of a certain quality. So I will admit we have some more flexibility, but at my core through and through, I'm a product manager at heart. And so for me, what it always comes down to and what I think AI is pushing actually more than ever

Is value to the customer. My our customers are our employees. So whether or not something is a little raw, do they say, yeah, this delivers value? Like this, this helped me move X units faster. This helped me deliver this many more things. This my quality is higher. And is that demonstrated by

the typical metrics we would use with external customers, average engaged days, monthly engaged users, are they returning? Are they using? And so I follow the same, very much the same methods of we have really quick subject matter expert tests and prove-outs.

Of concepts we are building. We're prototyping so rapidly that we need people to just look. We have subject matter experts where we will dissect a problem with them. We will then determine the value of that to them. We will build out rapid prototypes, go to our internal subject matter experts, say, Does this look right? And they say.

Can you describe these prototypes? I'm actually curious. You know, are these functional ones? mock ups in the traditional sense? Are these like, hey, these actually work. They're just a little like you say, rough around the edges.

They start as you can kind of get in and play with them. They're some they're further than a wireframe and less than alpha code. So they are

You know, we we leverage things like Cloud Code and Codex very heavily. Same with GitHub Copilot. And so our product managers, even our UXers, as well as our engineers, can within the course of a couple hours actually fully stand something up that we can demo live.

So these are sort of vibe coded prototypes then in some ways.

Very much so. Yeah. Yeah. And so what that does is it allows the unit of code to get to that prototype stage.

Becomes so much less costly that it's fine if we scrap it. And we're very comfortable with the keep kill decision point being at multiple points. We could talk to the subject matter expert and they could go, that's not helpful. And we go, okay, cool. We lost three hours of time, but we learned something. That's fine. And then after the subject matter expert gets to a point where they say, This looks great, then we actually start discovery with the company, but really rapid discovery.

discovery to validate it with a pilot group and then we go ahead and launch after that.

Historically one of the hu biggest blockers to developing anything, whether it be internal and externally, is is dev resources, right? And this is now we're in this period where suddenly developer resources to produce this code is n is no longer really the the blocker or fastly not l becoming the the blocker because the resource is so readily available. have you do you have thoughts on like what this ramifications are for the product manager role?

for the designer and and also for like orchestrating across, right? Across different units. So in a world where everybody can develop so quickly, who who keeps everything orchestrated? Or do you have parts of the org or parts of a a product suite, for example, if it's external, being developed very quickly here and these are being very quick quickly developed? is is that something you have a a framework for or how you think about the role of of a product team changing?

Yeah, the way I think about it with my team and the way we talk about it is we're starting to see what I believe is a decoupling of the skills from the profession. By that I mean traditionally product managers did ABC, engineers did.

EFG. UX did this. And what's happening is the walls between the professions are starting to come down. And the handoffs and accountability is also starting to come down. So what I work with with my team is: hey,

We're all accountable to this shared KPO. The engineer is accountable just as much as the PM is, just as much as the UX person is. And it sounds like a really minor discussion, but what that means is the KPO is now shared for the outcome to the customer. And everyone takes on.

As many pieces of that as possible, meaning our PMs should help do some coding, not just wait on the engineer. They're not going to do the final code, but they should help with some of that. Our engineer should be helping to attend some discovery sessions with the customer and start to flesh out the business value, not just say, Hey PM, what do you think it is? And so we all now are kind of one team, one dream for

If you want to think about it that way. Though I will say right now, the PM still very much is the orchestrator. And the PM very much is expected to, at a rate they have not been able to before, understand how to deliver the non-functional requirements essentially and break barriers and get from zero to one.

Are you seeing people fully willing to sort of take on or give up or maybe I don't want to say give up scope, but perhaps share scope within that context? Yeah, it's like are are developers welcoming people from PMs trying to write more code, for instance?

Yes and no. yeah, Kelly, I think you already think you're already there. yes and no. obviously within my teams, my teams are designed this way. And so I'm seeing a lot of openness. I don't think what I'm seeing as much is resistance as what I would define as

a desire for clarity. There's a certain level of ambiguity that's really throwing people off. And they're like, well, i if you're telling me I'm not the one that entirely has to do this, then how much am I supposed to do? Basically, what does good

Well, that's a personality trait, right? Like some people do not like that type of ambiguity and some people thrive in it.

Very much. Very much. I listened to a fascinating podcast with Tony Robbins recently, where where he was talking about how critical the level of uncertainty that people are comfortable with is involved in these periods of transformation and disruption. And I see that

on almost a daily basis because like this this thing I can't st stop thinking about is how much this AI evolution is at its core deeply a human evolution. Like it is it is such a human story because this thing of like, well j I need you to tell me what good looks like for me now. And I need you to tell me what this process looks like for me now and who is actually accountable. And so there's a lot of that

determination of clarity and where I see that the most is in areas that have a lot of layers of decision making and a lot of really entrenched process versus areas that are kind of more like with my teams for example, we're kind of loose. We're like, hey, let let's just deliver outcomes and we could be a little messy along the way.

Do you see regional differences? You know, I I imagine some of the teams you're working with are in other countries, other cultures, things like that. You know, do you see different levels of comfort with that type of ambiguity?

I would have thought so. Yet at every turn, everything that I thought would kind of play itself out in certain buckets and easy categories has just been debunked. Like I can't, there is no like our our team in Paris and Continental Europe is doing some.

Fascinating stuff with how they approach adoption. And so is our Asia Pacific team has someone who heads up AI transformation and he's been doing these advanced, completely redesigned workflows leveraging things like N8N. So the short answer is I actually haven't seen that much.

It's more of an individual thing than than cultural in some ways then.

Hundred percent. Yeah, absolutely.

I like that. And w another question though, I'm curious with all of this, you know, you mentioned though if something doesn't work, we throw it you throw it away, right? Because it you know, the amount of investment isn't that big. But I'm curious also once you really do develop something that is whether either from your pipeline or from the enablement one that is integrated into a full workflow, you know, what does the maintenance look like for for something like that? I mean, have we gotten to the point yet where you're starting to maintain some of these AI developed processes yet?

We are just barely starting to get to that point. And that actually is the to be honest, that is the question that keeps me up at night a little bit. Of I mean, if we're building agents upon agents upon agents, we are quickly going to have hundreds of agents. And we are, you know, the way we're approaching this is we we have this agentic platform.

Yeah.

That is maintained by our embedded innovation team. And then these verticals that are stood up on that platform, drawing from the data layer as well as the agents, as well as the MCPs we're building. How they are maintained is something that actually will need to be adjusted. Right now they are just maintained by my team, but we really have to keep an eye on OpEx costs there.

Yeah, 'cause it's exploding. I mean that's the thing is all of sudden people realize like this is starting to turn into real money here.

Yeah. Yeah. And so

We're trying to explore a lot of different models. I I wish I could say I had the answer. I know what we're doing now and what we're thinking about in the future. And what we're doing now is we are expecting the teams to own it as much as they possibly can, the team that built it. We also have

Our own internal business systems teams who maintain business systems. And they will also help maintain everything that sits within a business system. And this is what they have done, this is what they are built to do. And so they help with that maintenance. We're also thinking about and by and by thinking about, I mean literally this weekend I was starting to write up what this looks like, which is

Do subject matter experts and do AI leads in the different business units we support help maintain those agents? Do they update the content? Do they update accuracy? Do they help with evaluations as our team continues to build more and more and more?

It could be a lot of work, right? I mean it's

Yeah huh.

It could, and I think this gets to starting to scratch the surface of what jobs are being created. We've talked so much about what jobs are probably going away, but what jobs are being created, and it is the level of work needed. If you really want to create a differentiated, like moat type of

Internal AI system, something that allows your company to thrive and succeed. There's a lot involved. There's a lot of maintenance. There's a lot, there are so many AI lead roles we are hiring within business units, within commercial areas, within marketing, within every single team.

And it is both a combination of evolving old roles into this as well as hiring new roles. And this is what I'm seeing immediately in: hey, there's there's an expansion of roles that is also going on to own and maintain these different systems that are being built.

I mean it makes sense, right? You think about it like just how rapidly also the technologies evolve, you know, you you have this agent you did two years ago and like I would imagine something that someone developed today in twenty twenty eight is gonna look positively ancient, in its functionality and its efficiency with regards to use to credits and things like that.

I mean it could look ancient by the end of 2026 at this point. Like I don't I mean the the amount almost every time I do a an AI presentation, I have to scrap like half of it by the time I do it six months later. Like it's crazy.

Yeah.

How can you talk about the kind of skills for these AI leads of the future in terms of how technical you think they need to be? Because on the one hand, AI is making it easier than ever to code without deep technical expertise. On the other hand, a lot of what we're talking about here is like the importance of orchestration and systems and and the future of like agent to agent and other agents directing maybe a team of agents. So like when you think about the role, is it important that they're a systems

thinker and operational minded or do you think they should also be d deeply technical and and have have that experience to apply?

I think where I sit right now, and I always have to give the caveat of where I sit right now, cause in two months I could sit somewhere completely differently. Like I was listening to this podcast of one of the leads of Anthropic being interviewed, and she had this video that went viral, so one of the heads of UX, and she's being interviewed three months later, and she said, It's kinda embarrassing how outdated that is. So sorry. All that is to say

Ha ha ha.

Yeah.

Right now, I think being system-minded is the most important. What I refer to it as is zero-to-one people. The people who look at a problem and want to own.

Every element of that to think about how we drive to that solution and what is every aspect that is needed in order to deliver on that solution. Because we aren't at a point where, for example, I would tell a PM, hey, I expect you to code end-to-end. I expect you to know deep architecture. Will that happen? Very possibly.

in the future it will. But right, today as I sit here, I hire for curiosity, I hire for strategic thinking, I hire for exactly what you said, which is a systems thinker that says, Hey, if this is gonna actually be delivered, like we we need to think about taxonomy too. Cause wait, so we have a taxonomy team, we need to go talk to them and you hit all of those 'cause otherwise you stumble over them.

And so I think that's really key, as well as enough technical understanding to be able to walk in any circle and be able to keep up with the conversation and maybe even bring some ideas, but you don't actually have to do the level of execution yet.

sense. We'd love to hear your thoughts on the AI tech stack, right? It's like rapidly evolving. You you mentioned like the Frontier Labs and their and their work and GitHub, but I assume in your role you've just thought about this immensely is the terms of all the different AI tools that are available and how it's gonna work with you know what is called legacy software, which is now introducing agents within their software, AI native software.

And then th the tools that are built on top of, you know, these frontier lab tools and then what the frontier labs are building themselves, which is getting more and more extensive by the day. So yeah, what are you thinking? How do you see the market today? How do you see it evolving?

I have so many thoughts on this. I guess we could we could do 45 minutes at least just on this. so I have a very unsexy answer, which is that I actually think what I press people on is there are so many exciting tools that either can or claim they can do everything, and there is a new shiny tool everywhere you look.

Ha ha.

Yeah.

What is the least that you possibly need in order to accomplish as much as the transformation will allow? And by that, for example, we have access to, for example, Gemini and Claude and OpenAI or and ChatGPT and Micro and Copilot. Pick one, maybe pick two. That really will get you to do what you need. Because otherwise, what we are very quickly having.

And it is a real concern is tool fatigue. When do I use this tool? I have another tool I need to learn. What are the differences between these tools? And when should I use this versus this versus this? What I actually do is at the outset for a lot of people, I say, you know what, just pick one. Like as far as the frontier models, just pick one. You don't need to use all of them. And see how far you can get with that until you actually can explain why you're hitting a wall with that.

And then when we go beyond the tech stack, that is where this systems thinking comes into play. Because you have to think, okay, this tool over here is also awesome. But does it actually fit in the workflow? Or do I need to go separately over here? Because what we started to find was when everyone dove all in, we're like, we need this tool and we need this tool and we need this tool. They're like,

Okay, but now I go over here for one, and then I go over here for another, and then I go over here for another, and suddenly this efficiency has completely broken down. So think very deliberately about the smallest number of tools that are needed to get you across your workflow and the number of people who can use that tool because we go back to that peer-to-peer learning.

It is more powerful when more members of your as many members of your team as possible are using that one tool. And so, yeah, I mean, it's you need a foundational model, you need a prototyping tool. And it really depends on what profession you're in. You need something for collaboration, et cetera, et cetera.

Like how this is really f fulfilling the maximum of, you know, perfect is the enemy of good. And you r bring up a really good point, this idea of the peer to peer learning, you know, having a simple tool set.

is great. You're like, someone doesn't now have to apply to get a license to something. They're like everyone already has it. I mean that makes a huge amount of sense. I I think there's a lot of logic there. And but I'm curious though, that Kelly mentioned something I thought I'd be interested to hear more about is just the your current enterprise tools that you may have that are embedding AI in them. you know, is that something that

Yeah. Mm-hmm.

you look for in some of your software vendors or is that not really where you see the big efficiency jumps happening?

We do so long, I mean, there's a world where that's ideal, but again, it gets back to we know what we know certain tools we will absolutely use. We know there will be some level of ChatGPT, some level of Claude, some level of these foundational models.

that you use. And as we've been seeing, these models are covering more and more surface area very, very effectively. And so we have to really go back to the vendors and say, do you cover this well enough? Like are are you actually going to innovate at the pace these other companies will? god, yes. Yes.

Which is hard to do 'cause those other companies are throwing money at innovation right now.

Yes. And they are flying. Go ahead. Sorry.

In I was gonna say, yeah, I think I think that's the core question, but also the domain expertise of those systems, right? So Claude can't be the domain expert across t you know, design, marketing, sales, finance, all all all these different areas. So I feel like if the vendors can leverage that domain expertise while innovating quickly enough to make it agentic, then it makes a ton of sense for businesses to use that.

Less so if they can't keep up with the innovation and what Claude has is then good enough.

Yeah, couldn't agree more. I mean, some of these companies are doing great. Like there's you know, I I talk a lot with Miro, for example, and I thought they were just a whiteboard tool. And they really started to reinvent themselves in a way that is both innovative and leverages the innovation of other companies via, you know, their MCP selection. I MCP is everyone's friend right now with this.

Ha ha ha

And you know, there's a lot of these legacy companies that are genuinely reinventing themselves. And it is more beneficial to be able to keep them because there's more knowledge around them. But there's there's a value decision. There's constant value decisions that are going on right now.

that gets to this idea of the cost associated with AI and it at least from where I'm sitting, it seems like the cost for internal AI has been escalating pretty quickly. And it's gone from, you know, a point where people are just

I wanna see people using tokens and using credits 'cause that's a good thing to all of a sudden now we're like, okay, we need to be a little bit careful about what people are doing and how much they're actually spending. Are we at that point where you're starting to to watch the meter a bit on on what people are doing and and or are you still encouraging unfettered exploration?

We are very much shifting into the phase of starting to think about we want exploration absolutely, but how much is it gonna cost and how much is it delivering? Because it and and it's correlating actually with the move a lot of these companies are doing to go from a per seat model to the consumption model.

The consumption model is completely changing the game right now because supply and demand industry-wide now is all around the tokens. And so we aren't saying don't do it. I mean, we we are ex we are understanding and expecting that these costs will go up, but we are actively having discussions around how do we

make the balance because we didn't need to until probably a couple months ago really think much about it. And that and that's how fast this is all changing. Yeah. And and that's where this all changes is cause we could use when it was a per seat model, you use the most advanced model. You use the heavy thinking model all the time and you just go to town and that's fine because that's not our problem. And then all of a sudden the companies are saying it it kind of is your problem now. And so

I know, that's what it seems like. All of a sudden the cost just seems to have really started to go up.

And now it is your problem. Yeah.

What we actually want to do is there's a lot of small wins that you can have in this by thinking of, well, how do we how do we administratively impact

When you use, for example, a thinking model versus a quick model. For if we think about Claude, for example, how often do you how often do we want you using Sonnet versus Opus? and so you can put little administrative things in place for that. Or maybe you put caps in place. And but if you put caps in place, then we we create and we and we did kind of put some caps in place, but then we had to make an outlet where you could still rethink.

Request that cap to be removed because there's some folks who, yeah, it's awesome when they do $5,000 in a month, $10,000. Cause that, yeah, because they're there the way they're doing it is so advanced. But then there's other people who might run some kind of cron job that is unnecessarily just blowing through tokens. And and and it doesn't all have to be AI, some of it can just be more traditional automation.

Yeah, you don't want to limit them. Yeah.

So it's it's around education and it's around these administrative tasks and it's around right now exploring and then reevaluating and reevaluating and reevaluating. We have a position on it, which I've said is kind of the cap based with with outlets and the ability to go more, but we have to keep reevaluating it every month or two.

Mm-hmm.

I guess that's the nature of where we are in the context of AI right now. It's every month or so you're reevaluating it, whether it's new technology, new workflows, new costs, new skills that are required.

Yeah, I I think that underlies everything about my role, which is tomorrow I will say, well, I'll work with my counterparts and say, here's the rule. And I know in my mind, I have no idea how long that is the rule. Because that could change in a week, that could change in a month, or I could hold for the rest of the year. But we we cannot hold tightly to all of the rules.

Or we will fight ourselves.

like how you have a company that on one size delivering a product that absolutely has to be yeah, sort of it's conservative, stayed, it's legal, right? You know, it's it's predictable, it's all those things. And then you have these internal processes that are are hell bent for leather. you know, innovating and and applying the lightest technology.

I feel so lucky in that regard. I mean, there's an aspect there was an element in me when I got this job where I was like, really? Like we really get to do this? And they're like, Yeah, go for it. And at every turn I keep thinking they're gonna go, no, just kidding. But no, they are Yeah, they're all in because I think they realize that to be competitive with our

Ha ha ha.

they can't do that now.

products, we have to be equally as competitive or more with how we do things internally and what we're allowed to leverage and the way we're exploring the true disruption that's going on.

Mark, this has been a great conversation. really it made me think very differently, about a few things and also some of the challenges that are lurking on the horizon that I hadn't considered.

Great. Thank you so much. I mean, it just the discussion was actually super valuable for me because it made me start thinking about how I'm approaching things and how I need to. so yeah, it's it's been really great. Thank you both so much. Thanks. Bye.

Thank you for joining us.