Why Retailers Can't Ignore Competitive Intelligence With ClearDemand CPO Rajat Nigam | Spotlight Series
In this deep-dive Omni Talk Spotlight, Chris Walton and Anne Mezzenga are joined by Rajat Nigam, Chief Product Officer at ClearDemand, to unpack the critical role of competitive intelligence in modern retail pricing. Formerly leading pricing innovation at Amazon, Rajat explains how engineering, data quality, and normalization are central to making pricing intelligence truly actionable — and profitable. From assortment gap analysis to real-time pricing enabled by electronic shelf labels, this episode explores the full retail stack of competitive data strategy.
Key Moments:
- (1:00) Rajat's background and Amazon's secret pricing project
- (2:40) What makes “good” competitive intelligence
- (5:00) The three hardest problems in competitive data
- (7:30) Matching pack sizes and normalizing prices
- (10:00) Competitive data beyond pricing – assortment, promotions, planning
- (14:00) Where comp intel works best: grocery vs. luxury
- (17:00) Role of price elasticity in retail strategy
- (20:00) How to start: KVI items, category focus, and data gap analysis
- (23:30) Agility in pricing with ESLs and real-time data
- (25:00) Final thoughts and predictions for the future of pricing
#RetailTech #CompetitiveIntelligence #PricingStrategy #RetailInnovation #ClearDemand #OmniTalk #GroceryTech #RetailData #amazon #PriceOptimization
Music by hooksounds.com
*Sponsored Content*
This podcast uses the following third-party services for analysis:
Podcorn - https://podcorn.com/privacy
Transcript
Foreign.
Speaker B:Welcome to the latest edition of the Omnitalk Spotlight series, the series that highlights the people, the companies and the technologies that are shaping the future of retail.
Speaker B:I'm one of your co hosts for today's interview, Chris Walton.
Speaker C:And I'm Anne Mazenga.
Speaker B:And today we are turning our attention once again to pricing because, hey, it's kind of in the news lately, right?
Speaker C:And just a little bit, just a couple topics around.
Speaker C:A little bit.
Speaker B:Yeah, a little bit.
Speaker B:Can't get my mind off it, quite honestly.
Speaker B:But this time we are going to look at it from a new angle.
Speaker B:We are asking the question, what role does competitive intelligence play in designing an effective pricing system?
Speaker B:So we're going a little bit deeper.
Speaker B:And to help us understand the answer to that question is with great pleasure that we introduce today's guest, Rajat Nigam, the Chief Product Officer at Clear Demand.
Speaker B:Rajat, welcome to omnitalk.
Speaker A:Thank you, Chris.
Speaker A:Thank you.
Speaker A:And thank you for having me, folks.
Speaker A:It's a pleasure being here.
Speaker C:Yeah.
Speaker C:We have so much to cover with you today, Rajat, so we're going to dive right in.
Speaker C:Before we start, though, you're definitely an expert when it comes to competitive intelligence.
Speaker C:So if you don't mind, do you just want to give a quick background for our audience, for your background for our audience, Please, please.
Speaker A:Absolutely.
Speaker A:Look, I would say that back in the day, I was hired for a secret project at Amazon.
Speaker B:Not so secret anymore, is it, Rajat?
Speaker A:That is right.
Speaker A:But back in the day, Amazon or my team at Amazon was at the forefront of using machine learning and data to really reinvent how competitive intelligence was being used by a retailer like Amazon to lead the market.
Speaker A:Very quickly, what I ended up doing was actually inventing the hardware, the networks, the software that was needed to go out to the Internet at scale and basically monitor anybody who was in the business of online selling to consumers and then monitoring what products they were selling at what prices, what promotions, what kind of shipping and convenience options, and really using that data to beat them at their own game.
Speaker A:It was massive amounts of data that were being collected, then cleansed, and then used by machine learning model to make automated decisions.
Speaker A:That was unprecedented at the time.
Speaker B:Wow.
Speaker B:So, so, and we've got, we've got the guy that created competitive pricing intelligence for Amazon via a secret project with us.
Speaker B:So I think, yes, I think Rajad is clearly, clearly an expert on this subject.
Speaker B:And I think our listeners are going to get a lot from this conversation.
Speaker B:All right, so, so the next question that we logically have then is what in your mind separates good from bad?
Speaker B:Competitive intelligence.
Speaker B:I mean, at this day and age, competitive intelligence seems pretty easy to come come by.
Speaker B:Can't anybody just scrape everyone else's websites at this point?
Speaker B:Or, or is, is there more to it than that?
Speaker B:Am I oversimplifying it?
Speaker B:What's your take?
Speaker A:Yeah, see, you're not wrong in saying that it's now easier than ever to build programs which can go to websites and collect some sort of data.
Speaker A:But getting data at scale is a very, very hard problem.
Speaker A:First of all, I think the idea is doing it thoughtfully so that you're not taxing the resources of the people from where you're getting the information.
Speaker A:Most people miss out that point.
Speaker A:The second thing is doing it in a way where you are not blocked by technologies like anti block solutions.
Speaker A:And again, I'm not trying to say this nefariously, but again, but when you want to get data, you just want to make sure that it's predictability in getting the data.
Speaker A:Just getting that bit right is a very hard engineering problem to ensure that you're trying to be like a human being going to a website or a mobile app and trying to get the information reliably.
Speaker A:And again, without taxing a lot of resources of the competitor at scale, that becomes a very hard problem.
Speaker A:Then finally, look, it's just one thing, getting data, making sure you're getting the right data and making sure that you can use the data that you're getting is a very different problem.
Speaker A:I've worked with customers who worked with, you know, or were working with vendors, other vendors prior to us, where they would get heaps of data every day and they would sit on mounts of data just trying to scratch their hand, scratching their hand, trying to figure out what to do with it, how to make it useful and how to really use it for responding to the market to make things better for their consumers.
Speaker A:And that's harder said than done.
Speaker B:Yeah, so talk to us a little bit more about that.
Speaker B:Let's go deeper on that.
Speaker B:So what have you found that's useful to ameliorate some of those issues then?
Speaker A:Right, so when you think about, from a retailer perspective, look, really what I want is that I want to increase the traffic in my store.
Speaker A:I want to sell more units, I want to offer the best products to my customers.
Speaker A:I want to offer the best prices, the best experience to my customers, really.
Speaker A:So that's the business problem I'm trying to solve.
Speaker A:And I want to make sure that data or a data provider or a set of Technology can help me do that.
Speaker A:Now, when you break that business problem into a couple of big pieces that you need to solve from a data perspective, is that a, all the comp intel data that you're getting is something that can be used with respect to the products that you are carrying.
Speaker A:So let me, let me break that down a little bit.
Speaker A:For example, let's say you carry Granny Smith Apple and you collecting data from your competitors who may be selling both Granny Smith Apple and honeycrisp Apple.
Speaker A:Now you have to make sure that you match the right apples to be able to track the price correctly.
Speaker A:Then by the way, retailers use different kind of pack sizes.
Speaker A:Somebody may be selling apple pie, pound buy back, buy piece, and you may be selling it by piece.
Speaker A:Doing the translation of that information that a price may be related to a different kind of unit of measure of sales to yours and then trying to normalize it to do a comparison is again a hard engineering problem that needs to be solved.
Speaker A:You get the data first hard problem, you need to then match the products correctly, whether it's the same products or similar products or related products correctly.
Speaker A:That's the second hard engineering problem.
Speaker A:The third hard engineering problem is normalizing these differences in terms of pack sizes and units of measure to make sure that you can actually truly compare the price.
Speaker A:So for example, again going back to my Apple example, let's say you had a one pound bag versus a half a pound bag.
Speaker A:One is priced at $5, the one pound bag and the half a pound bag is priced at two and a half dollars.
Speaker A:You need to be able to translate both the prices, add the half a pound bag that I am carrying to see whether I'm at the market below the market or leading the market in terms of prices.
Speaker A:So those are the three big things.
Speaker A:And then again, once you have the comp and data, it's imperative that you use it in your pricing strategy to get the maximum return out of that.
Speaker B:So Rashad, I want to push a little bit on that too because one thing that I think Ann and I both learned in having these discussions for the past eight years around pricing is there's also the idea of, so there's the, the idea of, you know, getting the comp items right like you said, like, you know, apples to apples, for lack of a better way to put it, and then pack sizes and whatnot.
Speaker B:But the items themselves don't exist in isolation because oftentimes they're, they're a part of a larger category.
Speaker B:Price positioning too.
Speaker B:So is that also part of of what you need from a competitive intelligence standpoint to understand how the product item at the item level fits into the category pricing strategy as well.
Speaker A:Because.
Speaker B:Because the actions you take could lead you in the wrong direction if you're not thinking about it in that way.
Speaker A:Absolutely, absolutely.
Speaker A:I think you're right.
Speaker A:It's not just looking at the apple, but the category as a whole.
Speaker A:Trying to understand the category architecture of not only your category architecture, but how does your category architecture works with respect to your competitors category architecture.
Speaker A:What kind of brands, what kind of pack sizes are they carrying?
Speaker A:What's the price ranges of the products that they typically carry?
Speaker A:What kind of an audience are they appealing to?
Speaker A:What locations are they operating?
Speaker A:And all of this sort of becomes imperative and becomes a part of, you know, part of the execution metrics, if I may say so as we work for the customer.
Speaker C:So Rajat, I.
Speaker C:How then should retailers be thinking about competitive intelligence?
Speaker C:Because I think on one hand you and Chris were just talking about making sure that your prices meet or match, you know, or at least are aware of what your competitors prices are.
Speaker C:But as we get into, you know, things like promotion strategies, as the economy is going up and down, we're getting into other things that retailers might need to, to use this competitive intelligence for.
Speaker C:What, what else are retailers that you are working with?
Speaker C:What else are they using this for?
Speaker C:And like how do they kind of use that to justify maybe the expense at first, like the first like capital that they'll put into a competitive intelligence platform?
Speaker A:When you look at competitive intelligence data, you are getting all the product information, the category information, the pricing information, promotions, convenience, shipping options, all of that, you know, are the third party sellers selling the product on the platform.
Speaker A:You know, what is the, you know, you get a lot of rich data and, and you could do multiple things with it.
Speaker A:You know, pricing is the first one where most customers start.
Speaker A:I think that's where you can add immediate value.
Speaker A:Promotions is typically what follows next.
Speaker A:Okay.
Speaker A:Or in parallel, I have seen people doing assortment gap analysis and management.
Speaker A:So again, you want to make sure that you're carrying the best products that your consumers want to see in the store.
Speaker A:So you try to figure out what is it that I'm not carrying that the other guy is carrying that may drive more traffic to my store and at the end of the day delight my customers.
Speaker A:So assortment gap management is again our engineering problem.
Speaker A:We're trying to figure out and answer the question what should I buy next?
Speaker A:And again, by all means, you understand building supplier relationships because of to Manage those assortment gaps, then onboarding the product, then productionizing them, making sure that you have a supply chain to get them into store.
Speaker A:All it's a big decision to figure out what do you want to sell next in your store.
Speaker A:And assortment gap analysis is the first step that basically helps in that decision making.
Speaker A:And then, you know, the other thing that I have seen customers, you know, my, my, the retailers that we work with use the data for is of course, a regional planning of assortment.
Speaker A:You know, depending upon, you know, again, the market and again the audience that you're targeting, you may want to have a distribution of products across the country and across the regions that you operate very differently depending upon which category, what product you're talking about.
Speaker A:So that's another thing that retailers do.
Speaker C:Also, you know, as we look into ahead, into the coming months where there's a lot of uncertainty in the market, can retailers still do this the way that they were doing it without a competitive intelligence platform?
Speaker C:I mean, you've named like seven different engineering problems that you're trying to solve.
Speaker C:It just seems like a lot for someone to manually be doing or to be, you know, doing through a disjointed process.
Speaker C:Where, where do you, how do you think about that?
Speaker A:This is a signal that you can no longer ignore looking at competitive intelligence and looking at your market position.
Speaker A:And you have to do it more often, more often now than in the past.
Speaker A:I mean, gone are the days where you could send, you know, a mystery shopper to a competitive store once in a month or once in three months to benchmark your prices, that those days are over.
Speaker A:So, so that's the first thing.
Speaker A:The second thing is I think it's always good to start small with a good partner that you can work with.
Speaker A:You know, have simple objectives like, you know, focus on a few categories that you want to lead with, a few KVI items or key value items that you want to focus on to improve the, the price, value image of those items compared to your competitors and then scale up.
Speaker A:So start with pricing, then promotions, then assortment, gap management, and then look at other problems.
Speaker A:The data that you have is still going to be gold.
Speaker A:The data like, even if you start from pricing, you're not going to miss out on anything.
Speaker A:You'll have the historical data which you can back and look at and then use for other things later on.
Speaker A:Did I, did I answer your questions?
Speaker B:Yeah, I think you did.
Speaker B:I mean, I think it's actually a big nugget for, for, for us, I think, you know, and for our listeners too.
Speaker B:Because what, what you're saying, what you're saying essentially Rajat, which I don't think was clear to me before this conversation, is that, you know, competitive intelligence, while a necessary condition for good pricing, intelligence is also a necessary condition for a lot of other things like the quality of your assortment, the speed and convenience at which you deliver that assortment to people, particularly E commerce space and other aspects of operations too, I would imagine.
Speaker B:So, so I think yeah, 100.
Speaker B:100% you did.
Speaker B:But my question though, actually my next question though is, is again kind of putting you on the spot.
Speaker B:No, I get how this makes sense for you know, for grocery particularly.
Speaker B:Like I think it, you know, it makes sense as you're describing, you can get a sense of all those things in terms of how you're stacking up competitively.
Speaker B:But is this same rationale philosophy applicable to all areas of retail or are there areas that it doesn't work as well for?
Speaker B:I'm curious what you think about that.
Speaker A:Look, I would say that for all high velocity retail segments, whether it's grocery bats convenience, I mean segments where consumers often make purchases, you know, on a recurring basis in short amounts of time is definitely where, you know, value perception of value by consumer is a very, very big thing.
Speaker A:Now when you move to the, to the, to to other segments like high end fashion for example, I think the need sort of disappears because you're trying to, you have a very different objective there.
Speaker A:Whether it's expensive watches or are very high end shoes, I think the dynamics change a little bit.
Speaker A:I think the user definition of value and per is owning a branded piece there versus you know, when you come to high velocity, high velocity segments where they basically they want us like nobody wants to buy cheap products, everybody wants to buy high quality products, but they want to make sure that it's the value that they get for every dollar they spend is maximized.
Speaker A:So I think it's just very two different objectives as you go through different retail segments.
Speaker A:But then again, I think I firmly believe that for all high velocity segments, the pricing as one of the biggest pillars for driving value to your customers, I think is the truth.
Speaker B:And that notwithstanding too, I imagine even in the high luxury fashion area, having competitive data on how well you're shipping or what your times to ship are relative to others is still valuable.
Speaker B:But I'm curious, I want to ask you even more.
Speaker B:How does price elasticity come into that too?
Speaker B:Is it just high unit movement or is it also where there's, you know, a lot of price elasticity where the competitive pricing is also important.
Speaker B:Like I think categories like electronics, they might not move as quickly, but being, you know, lockstep with the market in terms of where your price and your assortment gaps are, is pretty critical.
Speaker B:What do you think about that?
Speaker A:That's again a great point.
Speaker A:Look, I mean you could have the best assortment, you could have the best price, but if you have empty shelves and poor consumer experience, that's no good.
Speaker A:You know, I think elasticities help you with better price planning.
Speaker A:I think they also help you with better demand planning to make sure that, you know, your consumers always have a good experience, that they have the products always available, that they want to shop at their convenience.
Speaker A:And you know, basically you're meeting the consumer and in their buying journey every time.
Speaker A:So elasticities do definitely play a good role there.
Speaker A:And then, you know, computing elasticities, again I would say is a, is a hard engineering problem to make sure that you're doing it right.
Speaker A:See the thing is that, and most people don't realize it, that elasticity change.
Speaker A:Like imagine this, you're looking at two years worth of data today and it signifies a certain elasticity and then you make a price change today with response to that and then, you know, it may take up to next couple of weeks before the true effect of that new price would, would basically start reflecting in the new elasticities.
Speaker A:So it's a moving target that you need to know how to leverage to, to, to maximize, you know, taking the benefit from that.
Speaker C:So, so Rajat, if, if people are listening and they're, they're all in, they're thinking, yes, we need to make a change in this direction towards the types of things we've been talking about.
Speaker C:How should they go about doing it?
Speaker C:What considerations should they be making?
Speaker C:What assessments should they be making before they take the leap here?
Speaker A:A good partner can of course share best practices can basically reinforce or revalidate what you might be thinking as your go to market strategy for both pricing and comp.
Speaker A:And so I think I would start there, starting with a good partner.
Speaker A:But once you have a good partner, I would again say focusing on a good product mix, especially KVIs, fast moving items that, you know, basically drive the maximum amount of your revenue or margins or, or categories where you actually want to turn around things like you're tired of, you know, you know, losing money and you say look, I want to change things here to see some movement and turn this around in terms of profitability or margin.
Speaker A:Those may be two good candidates to, you know, two different kinds of Candidates to start with and move forward.
Speaker B:How do you, how do you help the retailers on the data side of this?
Speaker B:Like the one thing we always hear and it's always such an amorphous, amorphous topic.
Speaker B:Now credit to the retailers that talk about it, first of all, because a lot of them don't, which is, you know, the idea of data.
Speaker B:So like how do you get the data around this and what aspects of data do the retailers need to consider to implement this idea correctly?
Speaker A:Chris I think the most important thing here is when you think of data is precision of data.
Speaker A:The data quality is one of the key metrics I would say that customers should insist on, irrespective of who they're working with as one of the key metrics.
Speaker A:Then I think it's the completeness of the data and the data coverage.
Speaker A:Now for example, you may be a regional retailer operating in three or four different states.
Speaker A:In every location you are operating, you may have different kind of competitors around you in a 1 mile radius, a 2 mile radius or a 3 mile radius that you want to monitor.
Speaker A:Now it's not just sufficient for your partner to deliver data for one region.
Speaker A:You have to make sure that all competitors, all regions are covered, covering all the products across all the categories to begin with.
Speaker A:Then that data is basically used by a price optimization engine like ours to basically help you then come to the best prices possible.
Speaker A:But look, at the end of the day, it's garbage in, garbage out.
Speaker A:You have to collect all the prizes, the right prices, you have to do good product matching, be accurate at it, and then basically also do the right normalization of the competitor prizes before all of this gets fed as input into, into the good optimization work that a pricing engine can do and then make, you know, then output for you the best prices that you, that you need where you could truly see the impact.
Speaker B:Yeah, that's a really, that's a really good way to frame it up too.
Speaker B:I think for our listeners too they're like, okay, because that's always the question we get right and is like where do I start?
Speaker B:And like yep, and to your point, you can start in any category you want to with this.
Speaker B:But you've got to think about it from the framework of where do I have good data quality, where's the data complete?
Speaker B:And then where do I have the coverage?
Speaker B:And you know, and you've got to do kind of a gap analysis before you start this to understand where you're going to have the most effectiveness when you try to implement this solution.
Speaker B:So I think, I think it's really smart, Anne.
Speaker C:Yeah, I mean, I think I'd love to close Rajat with like your kind of predictions and thinking about where, where retailers who are investing in, you know, competitive intelligence platforms and those who are not, where, where does the future lie for them, especially in regards to pricing mean, what's the end result here?
Speaker A:Yeah, I firmly believe that competitive intelligence is one of the most important input into your pricing strategy right now.
Speaker A:Whether or not you choose to match prices with your competitors or whether or not you don't want to be the prize leader in the market is a very different question.
Speaker A:But irrespective of what kind of high, low strategy or everyday low prices strategy that you have as a retailer, the reality is that you have to make sure that the perception of value that your customer gets by shopping in your store, which is a combination of price, experience, convenience, quality of products, that has to raise the bar with respect to your competition.
Speaker A:And you cannot ignore Comp intel, which sort of covers a lot of those bases as input data anymore.
Speaker A:So my recommendation to all our retailer fans is that if you're operating in a high velocity category, definitely look at Comp intel data as a key input strategy.
Speaker C:Well, and, and Raja, just one more thing.
Speaker C:Like, I think you have to be with all of the, the constant changes to pricing right now and like you said earlier, you know, making sure that you have the flexibility as a retailer to make sure that you adhere to certain pricing rules that you've set up for yourself, like in produce.
Speaker C:Like, we're always going to be, you know, we're going to always going to try to hit the price in produce so that we can get people in and maybe we can adjust margins on some of the other things that we have in our store.
Speaker C:But like, how important is it that you have that agility in pricing as we head into the future?
Speaker A:Yeah.
Speaker A:So look, I will tell you that there are categories that we operate in where retailers are changing price at least four to six times in a day.
Speaker C:Wow.
Speaker B:Wow.
Speaker A:Cross key value items.
Speaker C:Sure.
Speaker A:This is not during the holidays.
Speaker A:And during the holidays the war can become so intense where prices may be changing every five to 15 minutes across.
Speaker A:So it's imperative that you are able to respond to them.
Speaker A:Again, not every category is the same, but we are at a point where we see that daily.
Speaker A:Most of our customers, especially in the grocery space, want the data daily to be able to see the changes in the market and be able to respond to them.
Speaker A:And especially with, as technology is evolving around us, for example, the electronic shelf labels.
Speaker C:So look, I was going to say.
Speaker A:Yeah, that's going to accelerate the adoption of price optimization and competitive intelligence even more.
Speaker A:Now, initially you had the friction of somebody going in, sticking to labels, all of that, but now, hey, you know what?
Speaker A:You know, you got a husband, shell fetch, you're gonna do great in terms of responding to the, the market, you know, without even thinking about it.
Speaker A:It's all automated.
Speaker B:Yeah, yeah, right.
Speaker B:Yeah, yeah, no, yeah, that, that's what it tells me, too, is like, if people are trying to change prices that much online, whether you know what, you know, they're going to want to start doing that in store and they're going to look to, you know, companies like the Vision Group, like we cover on our show all the time, and try to figure out how to do that so.
Speaker B:Well, thank you, Rajat.
Speaker B:That was really, really interesting.
Speaker B:A lot of nuggets from the conversation.
Speaker B:I think I said the nugget, the word nugget, like three or four times, maybe, or more.
Speaker B:Who knows?
Speaker B:Because I never thought about, I never thought about pricing intelligence only being as good as the competitive data that feeds it, feeds the algorithms.
Speaker B:I mean, it's intuitive, but it's something that I know I personally had never thought of.
Speaker B:So if people want to get in touch with you or anyone else at Clear Demand, what's the best way for them to do that?
Speaker A:Please definitely reach out to us on our website.
Speaker A:Our website is www.cleardemand.com.
Speaker A:please request for a demo.
Speaker A:There's a easy form to fill and we'll promptly get back to you.
Speaker C:That sounds amazing.
Speaker C:That wraps us up.
Speaker C:Thank you so much, Rajat Nigam, for sitting down with us today.
Speaker C:And to everyone out there who is listening in, and as always, on behalf of all of us here at Omnitalk, be careful out there.
