Product Analytics
Watch:
Listen:
but first ... an update on our
Flip Fund
- We're at $220k hard committed in about a week! Stoked! If you want in, take a look at our deck and hmu with any questions you can view it here:
M&A Market (on the very low end)
Here's what I'm seeing this week on acquire.com and out in the world of micro M&A:
- Multiples feel like they're coming down.
- Buyers are more realistic about the asking prices
- Slight uptick in distressed ventures
- Lower quantity of quality deals
- Slight decrease in BS AI startups listed
Product Analytics
SaaS product analytics are broken. I'm noodling on an internal tool for our own portfolio to get the insights we need and I'm annoyed I have to build it. Mixpanel and all the usual suspects just don't get it done. Even the more recent breed of product led tools like June.so et al are in my opinion more of the same. And they all miss the mark.
Before I get into it, I'd like to do an exercise where I think through the level 11 version of a product analytics tool. Call me nerdy but this is really fun!
- zero configuration but somehow knows the right events to send
- can push the right info that I care about me at the right time (example, hey someone from google just signed up).
- Auto predicts intention, and accelerates someone through a funnel if they're ready or guides them through if they are stuck. (maybe even kicks them out if they're not a good fit)
- Knows when a customer is about to churn and remediates the problem before they do. (like churnkey but auto integrated).
- Rewards engaged customers.
[As a complete aside I'm also playing around with a concept for growthbar around an SEO agent that automatically does what many content agencies do end to end. Fun!]
To be fair, this is a lot to ask of a tool. To reach back into your product and do the right thing based on the actions someone has or has not taken in your app is a tall order and very difficult. This is why it's a job! For a human (for now).
Even if we failed and only guided a team on who to focus on, that would be tremendously helpful. Being a *dr (sdr, bdr, whatever) and waking up to a list of people to go help (either because they're ready to convert) or reach out to because they're stuck would be amazing.
E-commerce
What's infuriating is how locked in e-commerce seems to be on this stuff. Getting an ROAS for an e-comm platform is relatively straight forward for google ads. Doing this in saas is a huge pain.
The problem is the path towards activation imo. In an e-commerce store, many are using Shopify. Users have relatively few paths and a we as owners want a discrete outcome (a sale). The analytic actions in that case might be:
- view product
- add to cart
- checkout
They're mostly the same for most e-commerce companies. Of course it can get more complicated but this is the base case, out of the box analytics that will just work.
For a simple saas, there could be 5+ such events, none of which have a clear "goal". We can't easily reason about a generic saas product either. We might be able to say there are some core components such as:
- login
- signup
- checkout / subscribe
Other than that though, I'd need to know more about the product. Let's take sheet.best for example. It's super simple, you add a google sheet to sheet.best and it turns into an easily callable api. that's all the product does. In that case it might be:
- login / signup
- add google sheet
- call api
- checkout / subscribe
There is also at some point in there a free trial (let's just assume PLG for the sake of argument). So when does the user start to get value? The first API call? The 10th? That's something we have to go figure out as the business owner. This is what I mean by activation. There's really no activation step in e-commerce but there might be several in a saas product. And then how many api calls per month should be considered "healthy" for an account? If it's super critical and they use the product once a month, maybe it's a very low number. Or are the people that use it daily "healthier" customers with higher LTVs? Or actually what if it's totally dependent on the specific use case for a horizontal product like sheet.best. This is very hard to say without investing serious time into figuring this out. Then once you have the "answer", it might change in a few months and then you actually have to do something about it, which usually means building something.
With a more complicated product, there might be many goals, or many ways in which a user gets value from the product all of which might lead to a churned user or a retained user over some LTV. How do I map an action or a sequence of actions into some kind of value ideally expressed in dollar terms (relative to the price of the product)? And then of course, add human nature into the mix and not all people will ascribe the same value to those same features.
As a thought experiment, you might have a button that says "do my job for me". Let's say you make $5k per month doing your job. Would you pay $5k to click the button (assume for the sake of argument it actually does do your job at least as well as you do)? No probably not. You don't get any arbitrage between the wage you are paid and the product. What about $4.5k? Possibly! Maybe you get 2 jobs and click the button twice! You'd then make $1k ($500 per job) for clicking a button. That's not bad! But a certain amount of people may only want to pay $500 to click the button. Others would only pay $50. And a huge amount of people will never click it. Anyways, the point is, it's quite difficult to assign value to features, let alone an entire product with each user getting a variable amount of value. The $50 button vs the $4,500 button are probably two totally different businesses, or at least lines of business.
Back to saas analytics. For Growthbar, which we're looking at quite heavily right now, we're struggling with basic metrics:
- what's the visit to signup rate (relatively easy)
- signup to paid user (relatively easy)
- Everything else that happens in between (hard)
- why the user signed up but didn't pay (hardest).
Questions that are hard to answer in analytics products:
- What actions are different between converted users and churned users or users who fail to convert.
- So and so churned ... what did they do in the app?
- So and so didn't convert from the free trial ... what did they do in the app?
- When in the data can we see the usage of a user drop off (i.e. when we should step in).
- What exactly did user X do. I want to look at 10 churned users and 10 customers that have been with us for a while. Find me some similarities, and some differences.
Product analytics tools want to show you endlessly configurable dashboards. but I need the story the data is telling me. If I have the data in a particular format, I can then start thinking about using an LLM to do some heavy analytical insight for me automatically.
Department Of Random
If you have a few minutes this week check out this talk by Mailchimp's founder Ben Chestnut. What a different kind of founder. Just contrast this with any interview from another founder who created a unicorn. A singular kind of guy. And notice how fun he makes working there feel. I'd work for this guy. Can't say the same about a lot of the other CEOs I've seen playing the game at this level.
Listening to stories like this is slightly infuriating. He was focused on something else and built a little thing that just grew and they basically did as little as possible for years and then looked up one day and was like "holy heck, this thing makes more money than our main business, we should just do that". And they did. Not to say it was easy, etc, but getting .
Not Yet An Idea
I revisited Ray Dalio's Principles recently and am a little stuck on one small comment in there where most of their management decisions are decided by an algorithm of sorts (in addition to their investment decision). This is insane and amazing. One of these weekends I'd like to take the snapshots of every acquisition we made at the time that we made it and start to build up a model of what we say yes to, what we say no to and why and see if we can build a disciplined agent to help guide investment decisions.
