Welcome to another edition of the Always Be Testing podcast with your host, Tye De Grange. Get a guided tour of the world of growth, performance marketing, customer acquisition, paid media, and affiliate marketing.

We talk with industry experts and discuss experiments and their learnings in growth, marketing, and life. Time to nerd out, check your biases at the door, and have some fun talking about data-driven growth and lessons learned.

I am absolutely thrilled to have Michael Kaminsky on as our guest. Always Be Testing podcast, where we talk about growth, performance marketing, testing, experimentation, learning, partner marketing, and all those things. And yeah, I'm thrilled to have Michael on today.

Michael Kaminsky, the CEO of Recast, a modern approach to multimedia mixed modeling and all things attribution and incrementality. Welcome. Thanks for having me, Ty.

I am very excited to be here, excited for our discussion today. Absolutely, absolutely. It's going to be a good one and I'm excited to dive in.

This is, I feel like, such a hot button topic. For years, people have debated the topics of media mixed modeling, incrementality. It's a hot button topic.

It's a topic that comes up for growth people, for data science nerds, for paid marketers, for affiliate and partner marketers. And you are the guy to talk to because you've launched a business that solely focuses on this. So let's just jump in.

What the heck is media mixed modeling? Why is it so hot right now? What's going on? Yeah, great question. So I think everyone who's listening to this podcast can be familiar with a bunch of the changes that have been happening in the industry over the last couple of years, largely related to reduced ability to track people across the internet because of changes with app tracking transparency from Apple that rolled out with iOS 14.5, more changes to the ability to track that are coming with iOS 17, plus increased use of ad blockers, privacy regulations like GDPR. It has made the ability to track people across the internet consistently much, much, much harder.

And so that means that a lot of the tools that a lot of marketers, and especially digital-focused marketers, got used to using over the last five or 10 years, all of the digital tracking tools, first-touch attribution, last-touch attribution, all of those sorts of tools, they're now less reliable. And marketers know that, and they're starting to see the flaws in those methodologies that have really been made much more acute in the last couple of years with the changes to tracking. And so marketers have started to get really interested in, what are the alternative ways of measuring marketing performance that doesn't just rely on digital tracking? And marketing mix modeling is one of those forms of measuring marketing performance.

It's basically an econometric model that looks at patterns in the data to try to understand the true incrementality of marketing performance. And because marketers no longer feel that they can just rely on digital tracking, it's becoming a thing that a lot of people are talking about. That being said, because marketing mix modeling, a lot of the old ways of doing it were not really built for modern digital marketing, there's been a lot of technologies that have been developed in the last couple of years of people trying to figure out, how do we make marketing mix modeling actually work for the problems that marketers, modern marketers face today? And that's what we're doing at Recast.

And that's why everyone is sort of talking about this and trying to figure out how do we actually make this tool work for us with all of the problems that we're facing today in 2023 as marketers. That's awesome. Yeah, it certainly seems to be insane demand for the right type of brand, the right type of situation.

Just thinking back, when did this all start? The notion of media mix modeling has been around for quite a long time. Yeah, that's exactly right. So it's a really interesting history.

So media mix modeling, marketing mix modeling has been around almost as long as computers have been around. If you think back to pre-e-commerce, pre-internet days, and you're the CMO of a CPG company like Pepsi or Gillette or whatever, every year you need to make decisions about how you're going to allocate your marketing budget across the different ways that you can spend it. And that might be with doing promotional activity or running TV ads or running print ads or running radio ads.

And there's no digital tracking at all. People don't buy online. The internet doesn't really exist, at least not the way that it exists today.

And so how are you going to make that decision? And the way that CMOs made that decision historically, especially in fast moving consumer goods industries, was through these econometric projects. Effectively, you would hire a statistician or an econometrician. They would look at your historical data.

At times when you spent more on TV, how many additional Pepsis or Razors did you sell? At times when you spend more on radio, how many additional Pepsis or Razors do you sell? And so these econometricians would do these research projects once a year. They were very expensive. These were highly trained PhDs.

They were based on consulting projects. They would come in. They would do this analysis.

Lots and lots of research and investigation goes into it. They produce a report. And then the CMO looks at it and says, OK, great.

We're going to allocate $10 million to TV and $15 million to radio and $5 million to print. And then the marketers would go out and buy that media at upfronts once a year. And then the marketing team would go out and sort of execute on that plan with all of the media that they had bought.

And so that's where this idea came from, is how do we measure marketing effectiveness when we don't track anyone at all? We don't have any digital tracking to rely on. And so MMM, media mixed modeling, has been around for 50 or 60 years as these types of fast-moving consumer good brands have needed a way to measure marketing effectiveness without being able to digitally track someone. Over the last 10 or 15 years, it sort of fell out of favor a little bit, especially with the rise of e-commerce, where we could track people.

But now that our ability to track is again reduced, it's coming back into favor. But we all recognize that the old way of doing it, where you hire a bunch of statisticians and they produce a report once every six months or once a year, doesn't really match the way that modern marketing actually works. That's amazing.

Yeah, it's almost come full circle, it sounds like, right? Yeah, totally. Well, I mean, what's old is new again, right? And this is the way that marketing gets to work. But I think the key thing here is that we have to figure out what are we actually trying to do with this technology? And then how can we get it into the hands of more marketers so they can actually use it to make real decisions? That's what the core thing is that we're trying to do here, is how do we actually drive businesses forward by helping them accurately measure their marketing effectiveness? That's the most important thing.

And when we're thinking about MarTech tools, any sort of attribution methodology, we always want to be thinking about how are we using this to drive the business forward? How are we using it to actually estimate incrementality so that we can spend our dollars where they're most effective? And is incrementality like the kind of big value that really is derived from MMM? So in my view, incrementality should be really the only thing that matters for marketers. Marketers should be focused on how are we measuring incrementality for our business? And let's maybe take a step back for the listeners. What do we mean when we say incrementality? For me, incrementality means if I spend an additional dollar in this marketing channel, how much additional revenue is that dollar going to drive? Or if I pull a dollar out of some marketing channel, how much revenue are we going to lose from having pulled that dollar out of that channel? That's what incrementality means to us.

It is the causal relationship between the marketing activity that we're doing and then the business results on the other side. And that's the thing that we want to understand. Because if we understand that, if we understand true incrementality, then we can actually optimize our marketing budget.

Because we can take our whole marketing budget and then allocate the dollars to where they're all most effective. And that's the best that we can possibly do as marketers. Of course, there's a bunch of complications that go around that.

But in the ideal scenario, that's what we're aiming for when we're talking about doing marketing measurement. And so incrementality is the thing. Everyone should always only be thinking about incrementality.

In all of these different measurement methods, we should be thinking about is this getting us closer to incrementality or not? Or this measurement method works in this case, but in this other case, it's not actually good at measuring incrementality. And so that's what I want marketers to be thinking about is how is this measuring incrementality and how is it getting us closer to that ideal state of being able to really understand the true causal relationships between our marketing activity and our business performance. So incrementality, hugely important concept.

MMM is one way of measuring incrementality. And if you do MMM right and you're thinking about it the right way, you should be trying to use MMM to measure incrementality the same way that you should be using experimentation to try to measure incrementality. The same way that you should be thinking about how does our last touch attribution not measure incrementality? And in what cases should we not be relying on this tracking methodology to actually understand the true incrementality of what we're doing? All of those are the sorts of questions that I think that marketers should really be focused on.

Yeah, I love that. And I love the concept of like, hey, if this campaign marketing lever were to be removed, would you have received that conversion event or value or revenue without it? Exactly. It's a good kind of framework as well to build on your definition of incrementality.

And honestly, I'm really excited to talk with you, Tai, because I know that you're an expert in like the affiliate marketing space. And the affiliate marketing space is one, I think like a lot of people don't really understand the full range of affiliate marketing and what's possible with affiliate marketing. But also, it's sort of famously a really difficult channel to measure the incrementality of.

This is a thing that we've spent a lot of time thinking about at Recast. I know you have spent a lot of time thinking about it. So I'm really excited to have this conversation today because I want to ask you about what are the best practices around measuring the true incrementality of these affiliate programs? How can marketers get smart about that in a world where it just it feels like it's a lot more difficult to test for a bunch of different reasons, a bunch of tricky complications that go into it.

So maybe I guess like, I'd love to hear from you, Tai. What are the main things that people don't understand about affiliate marketing and maybe talk a little bit about like, why is it so hard to test these affiliate channels? I see what you did there, Mike. You're flipping it on the interviewer interviewing me.

I get it. Okay. I want to get something out of this conversation too.

This is for me. Absolutely. Yeah, it's so funny because I feel like affiliate and I've been sharing this a lot is the most misunderstood performance marketing lever and likely the most underrated as a result of that.

A lot of the misconceptions are also stem around the topic of incrementality and other related topics. Historically, brands have tried to sort through the challenges of fraud. I think that's common for any performance marketing channel and digital channels, especially in their early days as they matured, that fraud typically got better tools caught up.

People got caught up in how to catch it and improve it. You had misaligned network incentives from the 2000s and 2010s where they were charging high rates. They could command higher rates.

Sometimes they were aligned with paying out partners more rather than paying out what was efficient and accurate and rewarded and valued in terms of what was tracked. There's a lot of things that surround the affiliate space that get misconstrued, misunderstood, that are historical, that are real, that are current, that are perception versus reality. You've got coupon and deal sites.

The theory and some of the perception is that that's what they are. That's what affiliate is. They're simply taking credit at the last minute.

There's some truth to that, but that's certainly not what affiliate all is. When I think about affiliate, I definitely think about these coupon sites. Whenever I'm making a purchase online, when I'm at the checkout, I always Google coupon and I go to a coupon site and I grab the coupon.

I totally see that case as being like, look, there's a good chance this isn't truly incremental because I'm going to buy whether I get the coupon or not, but if I get the extra 15%, great. What are the other types of affiliate that people maybe aren't thinking about? I mean, the reality is under partner marketing, under affiliate marketing, under influencer marketing, I consider them all the same. Fundamentally, when it comes to the work and the scope and the specifics of the actions we do for our clients, it's affiliate and influencer are different, but fundamentally they're the same.

When you look at affiliate marketing, it's a way for you to reach people multi-channel across multiple touch points of Google, Meta, TikTok, review sites, gift guides, media houses. Think apartment therapy, BuzzFeed, Sports Illustrated, American Express, rewards. There's just thousands of quality content touch points to reach consumers via affiliate and affiliate in its definition really is saying, hey, I want to pay for outcomes.

I want to pay for valued outcomes, not just clicks and eyeballs. You think about Meta and Google, essentially that's the payment mechanism that you're opting into. Yes, you're backing out to a valued ROI, ROAS, MER, CPA, whatever that KPI might be for your business.

But with affiliate, you're able to say for a much larger percentage of the action and a much larger percentage of the budget, I can pay for an outcome and say, I need to pay a $20 CPA. I need to pay $5 a lead, whatever that might be for the brand. Totally.

Okay. That makes sense. Paying for outcomes.

I think it's easier for me to stick case that say like, look, an affiliate deal with BuzzFeed could be hugely valuable for a brand when the honey toolbar thing is maybe not. But how do we think about measuring that? How do we prove that as a marketer? If I'm an affiliate marketer, how do I go prove that to my boss? I think this is a thing that I've spent a bunch of time thinking about. It just feels very hard to test this sort of channel.

With a Facebook, we can show ads to some people and not to others, or we can run Facebook ads in Washington, but not California, and you can start to think about running experiments. But with affiliate, that seems a lot harder. How do you think about actually doing that to prove the value of this marketing activity? Yeah, you don't really have the closed loop ecosystem of meta or Google.

You do have a network, like an impact that'll track all your partner activity for the most part. That's a win and that's an opportunity to kind of look into the data and really dive into that. But you're right.

There are challenges with it. It's not as impossible as some might think, though. I think that's a big call out.

Don't lose hope. We'll kind of dive in and work through it together here and figure it out. But the reasons why it's hard are long and numerous.

It's not an easy thing. I think the toolbars you referenced are a big part of that. Historically, that's been debated for probably about 20 years.

There needs to be, I think, more... In a matter of fact, just at a macro level, the performance marketing agency I'm part of is running studies and really exploring a lot of this and resurfacing the topic of toolbars like Honey to say, okay, what type of data do people have access to transparently to see how much traffic came from a toolbar-like partner or browser plugin versus maybe other traffic sources that partner can provide. Ultimately, we are very cautious. We are very wary.

We take a very judicious approach. If we work with a partner like that, sometimes we inherit a partner like that when a client has it already in their program. And we want to look at a lot of different data sources to determine if this makes sense for their overall strategy.

A lot of things are kind of required to think about in there. What stage are they in? Are they in a conquesting stage? Are they in an aggressive spend stage? Are they trying to be very careful about profitability and maybe they've reached a maturity level or level of brand awareness? Those are important macro factors that play into if you would want to leverage a toolbar at all. For us, it makes it even more complicated when a lot of brands are not yet adopting anything beyond a last-click attribution model.

That really hamstrings the efforts to really accurately... Obviously, this is different than incrementality, but it's related. You have this vast improvement in content, this vast improvement of quality, yet you're not attributing value to more than just the last click. What a missed opportunity for brands in the affiliate marketing space.

And so I think that's a really important piece of this. That's next step piece that people need to get to in terms of figuring out and making sense of this before they can really proceed with, I think, the incrementality question. Just having the baseline stuff right is surprisingly not there.

Not surprising to me, given how I've seen the data and marketing measurement setups of a lot of brands. I'm curious, tell me about some of these studies that you have run with these different affiliates. Maybe it's like, hey... I don't know what you've run, so I'm really curious to hear some examples of this in the wild.

But have you worked with a brand that's using a toolbar and then you're like, let's turn off the toolbar for a month and see what happens? What are the different flavors of that that you run and what are the results that you've seen? Yeah, we've seen a number, both from an in-house as a marketer, in-house at large companies, as an agency, and talking to folks closely in the industry. We've seen a number of things. And I think the challenge is, oftentimes, it's hard to get a true read because running a true holdout is often, as you know, one of the best ways when you're not doing MMM to be able to measure for incrementality.

And so as a result, when you don't control the environment like you do on Meta and Google, that really becomes harder geographically from a time perspective. Some brands will do kind of like a before and after. And as you know, that's inherently flawed.

So I think what often happens is people get false positives or false negatives because they're not really understanding the science around incrementality and how you need to really look at that test in a very clean way. And so I have seen a movement towards more and more willingness and collaboration to run holdout tests. The challenge comes in the fact that a lot of partners, it requires partner participation, partner collaboration, the partner that is essentially at risk of losing out on a partnership, typically with a larger brand, because those are the brands that have the incentive, time and effort, money and budget to actually run a test like this.

So it's really interesting. I do think it does require... So what often happens in the lead up to becoming that big brand is often we apply workarounds as best we can, like really effective pricing, looking at cohort analysis over time to see, hey, what quality type, what new customer, new to file customer data came through to a particular brand. Maybe we do try to turn off strategy and kind of see.

And there's a surprising number of brands that actually will say, let's turn it back on based on their internal data that they'll see. So it's not necessarily just all full hearty people chasing dumb money. There is a level of brands that are saying, hey, we do want to opt back into this, whether that's maybe it's conquesting vis-a-vis customers.

But I think to get to true MMM level, true data science level incrementality, I think we have to be smarter about running more holdout tests. I think the industry still has a ways to go. But I do believe that that's been the most effective I've seen.

Let's say that I'm a brand and I've got a normal mix right now. So it's a lot of Facebook, a lot of digital marketing. Maybe I've got some offline stuff.

And I'm interested, like, okay, affiliate might be a new growth path for us. What would you recommend that I do as a marketer at this brand to go test affiliate and figure out if it's going to work for us? Totally. I have a checklist of things, both published and in my brain with our team that we go through.

I think I can run through that real quickly and then jump into what's there. But I think the obvious product market fit, obvious level of revenue needs to be there. Typically, a level of retention in the client, that flat retention curve that we talk about in growth, that folks are coming back, there's value there.

There's not crazy high return levels or issues with that. The healthy conversion is important in affiliate because if you think about affiliate, they are putting skin in the game, time, money, expertise to promote your brand. And then expecting that return in the form of the commission as opposed to, hey, I'm getting money up front.

Yes, both happen, but more often on the commission side in the affiliate space. So as a result, that conversion rate needs to be healthy. You go through that checklist top to bottom and say, hey, let's go forth and conquer.

Time is important too. We need to be able to give this a real, it's relationship-based, it's some call it performance PR. You need that time to really go out and recruit tens, hundreds, maybe thousands of partners, depending upon your approach, and then really cultivate and manage those relationships.

It's regular, weekly, monthly, quarterly communication with them to make sure that they haven't fallen off. They haven't forgotten about your brand. They haven't started promoting your competitor.

I think that's why a lot of people find it too onerous to kind of manage in-house and they're like, hey, take this on for me, take it off my plate. Thank you. I think from a stepping back and thinking like, well, how do you set this up right? How do you see if this is a right test? We want to do as much of that kind of assessment upfront as we can through analysis of what data they have available through Google Analytics, through their other platforms.

They have a live program and maybe it's not optimized, which is super common. Let's look at that. Let's take a look and kind of figure that out.

What are the things that you see in a non-optimized program? What are the things that you look for where you're like, hey, look, these are the things that I want to look for to see if, hey, we can make some tweaks to really make this program a lot more incremental? I think not enough emphasis on full funnel. It sounds simple, but seeing kind of like the usual suspects from 2005 of like, okay, you got your cash back and you got your coupon. Great.

Well, what the heck? What else is there? There should be a heck of a lot more. Your top 10 should have influencer, great, huge content publication network, maybe some like really niche provider. If you're in the baby space, you should have an insanely awesome baby blogger or network that's promoting you.

That really nails your niche. There's all kinds of tech that's blown up in terms of like FinTech and card link offers where people are getting relevant suggestions when they go into things like Acorns and NerdWallet that are relevant to a lot of people. There's search providers that are willing to collaborate with you and hey, they may not be in your top 10, but they're a consideration to look at to diversify and push your competitors down.

There's email providers. There's ways to tap media buyers that are sitting on the sidelines that are phenomenal at Google, Meta, TikTok that you can run on a pure CPA and get quality and not have to worry about fraud. All of them need management, all of them need handholding, all of them need rules and guidelines.

But if you don't look into that top 10 and see a nice diversified portfolio, you're kind of missing the boat. Got it. So you sort of, what you are seeing is that there's probably too many brands think about affiliate as only being like the coupon code sites and not enough are thinking about the more the top of funnel, more awareness building ones.

And those are the parts that are maybe even more incremental, but potentially more difficult to measure. Yeah, I think it's a combination of difficult to measure. Some of them do require some level of flat fee because they can command that for quality and for size.

And so you want to, I think the common mishap is that it, they try to apply a paid search or paid social methodology to a channel that doesn't operate that way. And it just needs a lot more cultivation and handholding essentially. It's just, it's simply not programmatic yet.

And I think people think that they try to apply programmatic principles to affiliate and that doesn't work. It needs to be actively managed, really actively managed. And so let's come back to like experimentation and how we might be able to do this.

So like if I'm a brand and I want to run an experiment with these partners, is it literally just like picking up the phone and being like, Hey partner, we want to run this experiment. Like let's figure out how to go make it happen. Or what are the ways that you've seen this work tactically that a marketer today could potentially go and run with? Yeah, tactically and tactfully, right? It's like, it's a sensitive topic, you know, when a big brand reaches out to honey and it's like, Hey, we want to roll a holdout test.

Like, I don't, I don't think honey's like jumping up and down saying, can't wait. I think there's a number of things that, you know, if you have the tracking dialed in really well, if you have no server to servers, a better methodology than pixel, if you have the attribution dialed in really well, Hey, go beyond last click. If you have your pricing dialed in really well, like being really smart about pricing, what you see on the data is higher quality and higher revenue and higher unified customer.

You're already ahead of a lot of the players. And I think until you get to a certain size of volume, so you can run an appropriate test, you know it better than anyone in MMM. You've got to have that sample size.

You've got to have that data set until you get there probably doesn't make sense to approach a loyalty partner and say, Hey, I want to run a test, a geo or a holdout. Now, good news is for those that are more sophisticated, have gone through those steps and have kind of gotten to like PhD level affiliate marketing, you know, then it's time to approach the partners that you have questions about and say, Hey, we want to run an experiment here. Would you be open to it? Here's how we're kind of thinking about it.

We'd like to hear your feedback. And it's a relationship. It's a two way street.

It's a negotiation. It comes in ebbs and flows. I'll be honest.

Some partners in that space are like, we can get this revenue elsewhere. Good day, sir. And they move on.

And I think that that's really interesting. I think that the better partners are going to be the ones that are, you know, in the right use cases are willing to make testing a little bit easier and stand behind their offering with confidence and allow the brand to make the determination based on it, assuming that the methodology is right. I think where people get into trouble is when they throw the baby out with the bathwater and say, Oh, coupon and loyalty don't work.

It's just a nonstarter for me. I think it's ultimately more about how you value that partner than throwing it out completely. Are there examples where you need to remove and move on? Absolutely.

But I think it's all more about the right, accurate tracking of value, which, which is obviously what you guys are trying to do with what you're building. Yeah, I think that's right. And I think it's just it's such a it's such a tricky and hard problem.

And I mean, at Recast, we spend a lot of time on affiliate because affiliate is really problematic, actually, for doing marketing mix modeling. And the reason why is that in general with affiliate programs, you pay for the marketing activity after the conversion has already happened, which is different from how all other marketing works, where you spend money and you get impressions and then conversions happen later. In affiliate, it's reversed where you get the revenue in the door, and then the spend happens for the affiliate partner.

And if you think about the way that like all of MMM is structured to work, it sort of is implicitly making the assumption that the spend is happening and then conversions are happening later, not the opposite. And so MMM modeling is really, really hard to do correctly with affiliates. And in fact, when we first started building Recast, what we found is that the model was like too smart, it was too good.

When we would include affiliate activity in the model, the model basically just found out what is the payment for affiliate conversions. And we were the model basically just identified, hey, look, you're paying $5 for every conversion. And so that actually doesn't help because then it's not truly measuring incrementality.

It's just finding the relationship in the data, which is that when your spend goes up by X amount, your revenue goes up by some 20X that amount, because there's a 5X return on investment on that affiliate spend. So it's tricky. It's hard.

It's a really hard problem because of that closed loop system. And so we have spent a bunch of time at Recast really thinking about how do we break that connection and make it so that the model isn't just going to find, hey, what's the affiliate relationship here, but actually thinking about how do we understand the relationship between other marketing channels, the interactions with affiliate, and then what affiliate is going to happen, like what affiliate spend is going to happen no matter what, how much of that is incremental. But it's a very, very, very hard problem.

And so a lot of MMM researchers will just not include affiliate at all because they're like, it's too hard. It messes up the model. It's really a tricky problem.

And I think until we as an industry figure out better ways to get really smart about measuring affiliate, and I'd love to see people developing more tools for doing this sort of holdout testing in affiliate channels. I think it's just really tough for marketers to get a good solid measurement that they have a lot of confidence in. So as you said, they're sort of stuck piecing the different pieces together from the different evidence that they have.

And so, I don't know, I'm really excited about the future. But what I want to see is I want to see more of these like experimentation type tools being developed in the industry. So that way, if I'm a brand and I want to experiment with honey, I can easily run an experiment.

I don't just have to take their word for it. I can actually say like, okay, look, there's a holdout test. We're going to have honey in half the country and not in the other half and really see what happens.

Yeah, I think it's possible. I think it needs to be more embraced. I 100% agree with you.

An interesting thought came into my head. I'm not sure if it would work, but there is a percentage of affiliate that does do some of that upfront payment. There is a percentage of affiliate that is, which is kind of debatable whether that's affiliate or not.

But it's there and it's happening. And so some will move to cost per click models, maybe to avoid certain legal issues or constraints. So let's say an MMM candidate brand working with Recast were to say, hey, look, we're going to run a certain amount of our, like influencers is a good example.

A larger percent of that is like, hey, payment upfront for a multi-staged campaign, let's say multiple posts, et cetera. What if maybe starting with a slice of affiliate, like an, like Instagram influencers for a retailer, if we were to say, okay, we're going to, how much data would you think you would need for a type of brand to run? Like maybe looking back over the course of a year, I'm just kind of teeing up a case. Do you think you could learn about the incrementality of that influencer strategy? Totally.

If there's enough spent, right. And this is the thing that we've done. And again, this is a thing basically like when we're working with these brands and they have, they might have influencers, some of which are sort of traditional affiliate where you're paying per conversion and some of which are pay upfront.

And we'll basically split those in two and we'll treat them differently in the statistical model because you have to, because of the problems I was just talking about. And so with that spend, right, the paid upfront influencers, we can treat that like any other marketing channel because you're paying money, you're getting impressions and then conversions are happening. And if you've been running an affiliate program for a year and it's a substantial amount of spend, yes, we can absolutely identify the incrementality of that.

And we've done that with a bunch of our partners in terms of being able to say, look, when you are investing more into these influencers, you're driving in number of additional dollars of revenue or in number of additional conversions. And therefore we can back into what the incremental return on investment or cost per acquisition is from that investment that you're doing. And that's actually like one of our key selling points at Recast is that we can give you insight into channels like that, that otherwise are very difficult to measure because as we all know, it's difficult to parse apart.

Like if you're running influencers on YouTube, right? Is it your YouTube ads that are happening? Is it the influencer? A lot of times those people aren't clicking on any link or actually engaging with that post in any way that would lead them directly to your website, but they are, you know, maybe they're watching YouTube on their TV and then they're going on their phone and searching for your brand. You really need to have a good way of actually being able to understand that connection, even if they're not necessarily using the coupon code or the vanity URL, which I personally almost never use. And so like I can empathize with those buyers that don't necessarily do that.

And so you need to understand what are the sort of statistical relationships in the data in order to measure a channel like that effectively. How much does the payment upfront improve your fidelity in your opinion in that factor? Or is it like number three on a series of factors? So it definitely helps from an econometric modeling perspective. It definitely helps, right? Because you don't have that problem of the sort of circular causality thing where you're paying after the fact of conversion.

And so from a causal modeling perspective or a causal inference perspective, it's definitely a lot easier to get a read in that situation. For the ones that are done just on a paper performance basis, we definitely have a lot more sort of asterisks in terms of thinking about, hey, is this measure actually as accurate as we want? And then a lot of times that's the point where we're talking to these brands and saying like, look, we should figure out how can we go and test this channel to make sure that we're getting a good read here or get outside information about the true incrementality so we can use it for real decision making. Just to play out the scenario, not to like over dwell on it, but like if it was, let's say a brand was spending 500 grand a month on affiliate as a channel, let's say you had a 12 month look back, assuming a relative 50 cent cost per click, we can do the quick math, right? We can just model it out today on the call.

I'm just kidding. Do you think you can get a read? I mean, again, it depends on what else they're doing, right? But that's a fair amount of spend. You should definitely be able to start to model out what's the relationship there.

Yeah. You got to be making a lot of money through that lever to be spending that much, which is probably not a lot of retail, not a lot of DTC income, but there's some. You had a 12 month look back.

The question is how much of that would you need to be paying on a cost per click or the opposite of an affiliate model? So it's like, does it justify switching over to a cost per click or upfront payment? I would encourage brands to experiment, right? Like, can we carve out a small part of that and switch it over and see what happens, right? Let's take a test and learn. I mean, this is our view on like literally every problem, which is like, let's run an experiment. Let's take a test and learn approach.

Let's figure out how can we test this and validate these assumptions without, and we don't necessarily need to be like, let's make a whole big change for this whole program all at once. One of the small things that we can do to start getting learnings and that could inform what the next step, we don't have to plan out the next two years of our marketing spend. We can say, let's run an experiment over the next two months and then take that and use that to make a decision about what we do three or four months from now.

I love that. It's totally in line with our philosophy and always be testing and experimentation. I think that there's so much alignment there and taking it like one partner at a time, one geo at a time, breaking down the problem, the first principles.

I think that's really something that is there for people to try and be willing to test and learn from.