This question keeps cropping up on calls I have with startups looking to improve their user’s experience… and I have a somewhat love-hate relationship with NPS.
Let’s first go through why I don’t like NPS.
Why I don’t like NPS
1. The interpretation of the Score
The first thing is about the interpretation of the score and of course it’s very debatable…
If you’ve used NPS you know that a rating of 8 is considered a passive user but in reality, it should be great that someone scores your experience as an “8”.
Although to be fair to the methodology of NPS if someone really loves your product they would rate you as a 9 or a 10 so that makes sense but it ignores that 8 might be a good score.
That’s the first part.
2. It is really high-level
The second thing which I don’t like about NPS is that NPS is really very high-level and it measures just one thing just the recommendation. A good practice is to follow up the NPS question with more questions to get to know more about the experience.
3. It can be easily tricked
And the third thing why I don’t like NPS is that NPS can be easily tricked by the person running the survey.
You can send the survey at a particular moment or to a certain cohort that you’ll know that will give you a positive feedback and that way you’ll be cheating on the score.
I know that’s some organizations do this to trick NPS and mainly because they have bonuses tied to the score. And that’s why I never give my teams a Target which is tied to a bonus solely based on NPS.
Why I like NPS
On the other hand NPS has many positives.
1. It’s an industry standard
The first thing is that it’s standard across many industries and which means that you can be confident in using it and it also means that you can find the standard, average score for your industry and you can compare against it.
2. People are used to it
The second reason is that many users have already seen an NPS question and don’t have to think too much before answering because they’ve seen so many similar surveys that they know what kind of question it is and how to answer it. This of course can get you more replies and get more users to answer your survey.
3. Good to track direction over time
The third reason is that although I don’t give so much attention or value to the score itself it is good to keep track of the direction of the score. Looking at the general trend whether it’s going up or down. That should give you an indication of whether you are improving or not at all or even doing worse on your experience.
4. There’s usually gold in the follow-up question
And my favorite reason for using NPS is actually the follow-up question where you ask the people for a reason for that score. This verbatim is where you get to collect open free text feedback from your respondents and where you can really analyze to see what they actually mean by their score.
And that’s where you’ll start noticing that someone might have scored you as an 8 but they really like your experience or someone might have scored you as a 3 only because they had one particular instance where your product didn’t work properly but doesn’t mean that they really hate your product.
The follow-up question really has a lot of value and it’s your place where you can really analyze these responses.
Conclusion
So while NPS is not a perfect methodology it’s probably one of the best you can use to get some customer feedback.
Typically I like to mix NPS with CSAT – Customer Satisfaction – so that through the NPS score I can see the generic sentiment while through CSAT I will see the specific interaction, the specific journey, the score for that specific journey the user is on.
As a PM or as a Founder your best bet is to actually sit down with people and speak to them but you can mix having a survey so NPS/CSAT, speaking to people, seeing support tickets and seeing what the sales people are telling you.
In that way you can have a good coverage of feedback and a good way to understand what people are telling you, where they’re struggling, what your team’s struggling with, etc. to get to really a good Product-Market fit at the end of the day.
