Maximizing SaaS Trial Conversions (With Math)

I kinda stumbled into this idea a few months back. I had added a feature on S3stat where Users can create extra logins for other members of their team. So instead of printing out reports and sending them to their boss, they can simply add Mr. Boss Guy to their team and he can check his own reports. Less work for the account user.

But it turns out that it was a better idea than I'd thought. Because if Joe User let his trial lapse, he'd get an email the next week from Mr. Boss Guy asking where the hell this week's reports were and why couldn't he log in to S3stat anymore? Joe would explain that gee, that thing costs like ten dollars a month and I don't have a budget for software. And Mr. B would yell at him for 24 seconds then explain about how he had just this minute wasted $10 of his own time yelling at somebody because $10 is not a lot of money for a business and go subscribe right now. Here's the company card.

Well that was cool. I wonder how many times that has happened?

So I wrote a big ugly SQL query to count how many Users had signed up for trials since that feature went live, how many of them converted to paid, and to see whether Trial Users who added at least one team member converted at a higher rate than users who didn't.

And they did. By a huge margin:

Well wow. I wonder what other features I have (or could build) that can make that sort of difference. And I wonder what sort of tools I could build to get a better handle on this sort of thing. Because this is not the sort of thing I want to have to find out by accident ever again.

Here's an equation:

(let ((g (* 2 (or (gethash word good) 0)))
(b (or (gethash word bad) 0)))
(unless (< (+ g b) 5)
(max .01
(min .99 (float (/ (min 1 (/ b nbad))
(+ (min 1 (/ g ngood))
(min 1 (/ b nbad)))))))))


If it looks familiar, that's not a surprise since I lifted it verbatim from an article Paul Graham wrote way back in 2003 called A Plan For Spam. In it, he basically invented the Bayesian Spam Filter, which is the reason that email is still usable today even though roughly 99% of the things headed toward your Gmail inbox are in fact spam.

At its heart, the algorithm is just a way of classifying piles of information into Good and Bad groups. For email, you look at the words in a message and determine whether an email with those words is likely to be Spam (which is Bad) or not (which is Good). There's nothing about the algorithm specific to the problem of Email. It just happens to be good at classifying stuff, and email benefits greatly from being classified.

But Users (of your SaaS product, remember) can be classified too. Over the course of a trial, they do lots of things and give off all sorts of signals that you can collect and analyze. You can feed those signals into a Bayesian Classifier to train it about what Good (subscription activating) users tend to do, and what Bad (trial expiring, canceling) users tend to do. And you can feed the signals from a new Trial User in to that trained Classifier to produce a guess about what he might do in the future.

So in the case above, where we're looking at Users who have invited Team Members vs. those who haven't, we can now just define an "AddedTeam" Signal that we track for that user and the Classifier will take care of determining whether that is in fact statistically more likely to indicate that our User will convert to a paid subscriber.

And that's good. Like measurable-in-dollars good.

I've been experimenting with other things you can track, such as aggregates like "NoLoginLast7Days" and rollups like "10LoginsLast30Days" that might give more for the classifier to grab on to. And, of course, just watching the data as it rolls past is enlightening too. You learn a lot when you start collecting usage data (in human readable form), like just how many people are forgetting their password every time they log in, and that there are customers who do in fact check in several times a day as part of their workflow. Even without the crazy Bayesian math, you can get a good feel for which people are planning to buy the thing when their trial runs out.

I'm going to write more on this as I build it out, but I guess this is a good time for a heads up that I'm hoping to build this in to a product at some point. It's live, in its infancy, at unwaffle.com. If this stuff sounds like something you'd use for your own SaaS, hit me up with an email or go sign up for an invite.

Discuss on hacker news