The CPO's dashboard can't see the customers who tell you what's broken
TL;DR: Survivorship biased data is data that only contains the people who stuck around long enough to be measured, and that's what a CPO is looking at every time feature-adoption numbers come back healthy. Product analytics can't see the deal that died in a sales call over that same feature, and it can't see the trial account that left before generating enough usage to register. This is the CPO's data fragmentation problem: product analytics, support, and sales objection data sit in three separate tools, and none of them alone can tell a CPO who's satisfied.
Why does healthy feature-adoption data hide unhappy customers?
Every product analytics metric has a hidden precondition: the customer has to survive long enough to generate an event. A "feature adoption rate" measures one population: customers who stuck around long enough for the feature to log a data point. Anyone who evaluated the feature and left before that threshold never appears in the metric at all.
Gartner has estimated that poor data quality costs organizations an average of $12.9 million a year — a figure built from records that made it into a database in the first place. The estimate can't price data that was never captured, which is exactly the blind spot a CPO hits when product analytics is the only lens in the room. (Gartner: Improve Decisions by Establishing Value Stream Collaboration)
What is the Adoption Iceberg?
During World War II, statistician Abraham Wald was asked where to reinforce bomber armor, based on bullet-hole patterns on planes that made it home. His answer: the returning planes were the survivors. The damage that mattered was on the planes that never came back, and nobody could study them, because they weren't in the sample.
Product analytics has a similar issue. Call it the Adoption Iceberg — which has three populations:
Converted — customers who used the feature enough to register in product analytics. This is the only population the tool can see.
Walked — prospects who evaluated the feature and declined before becoming users. Visible only in sales objection notes.
Silent Churn — trial or early-stage users who left before generating enough usage to leave a readable pattern. Visible only in support tickets or churn interviews, if any are performed.
A CPO reading product analytics alone is reading only the converted population and calling it the whole picture.
After more than a decade producing and reading market research decks that draw firm conclusions from whoever's still in the room, I've watched this blind spot resurface many times — the tool changes from org to org, but the missing population doesn't. Every one of those cases came down to one question: who got left out of the sample before anyone opened the dashboard?
Where does product analytics still work fine on its own?
Product analytics is still the right tool for what it measures: behavior of customers already using the product, and it's precise within that population. Dense onboarding instrumentation (tracking dozens of actions in the first session) catches more of the Silent Churn population than sparse tracking does, narrowing that part of the blind spot. The Walked population sits outside this fix entirely: a prospect who never signed up leaves no event trail no matter how much is tracked inside the product. Kubit's analysis of fragmentation costs in product analytics makes a related point from the tooling side — consolidating dashboards reduces friction but doesn't resolve differences in what each source was built to see in the first place. A company without a dedicated CPO doesn't need to restructure to close this gap either — one person reading all three sources on a fixed cadence captures most of the value.
What does product analytics miss that a synthesis layer catches?
A synthesis layer reads all faces of the Adoption Iceberg as one dataset instead of three separate dashboards. Reforge has documented a related pattern it calls the "feedback fragmentation tax" — product teams lose the connective tissue between what customers say and what the product team can act on, because the two live in separate systems with no shared owner. Combining the three populations changes which customers get counted, and which is a bigger shift than adding more data to the same dashboard. Growth teams already read cross-source data this way instead of one tool at a time; product leadership hasn't caught up yet.
--Steven Rencher, Founder of Monadux
Frequently asked questions
Why can't CPOs see the full picture from product analytics alone?
Product analytics only records customers who used a feature long enough to generate an event. Prospects who walked and trial users who churned early never cross that threshold, so the tool can't see them at all. A CPO reading adoption data alone is reading one population of three.
What is the CPO's fragmentation problem?
Product analytics, support, and sales objection data each capture a different population of customers, and each lives in a separate tool with no shared owner. Syncing the tools wouldn't fix it: each one was built to see only part of the audience, so a decision built on any single source is missing a population by design.
What does product analytics miss that a synthesis layer catches?
It misses the Walked (prospects visible only in sales objection notes) and the Silent Churn (early users visible only in support and churn data). A synthesis layer reads all three populations as one dataset, so a decision reflects the customers who left as much as the ones who stayed.
How does Scry help CPOs with cross-source data?
Scry combines product analytics, support, and sales objection data (and any other data you share) into one customer intelligence layer instead of three separate dashboards. That layer carries all three customer populations at once. A CPO's decision then reflects everyone, including the ones product analytics alone would never register.