Startup School Reference Guide for Lecture #6 — Analytics for Startups with Ilya Volodarsky

While working to create the Snapchat for voice — wavechat.me — and attending Startup School 2019, I was inspired to create these “Cliffs Notes” for all the valuable content that YC provides to their attendees.

If you like it, please give Wavechat a try! https://apps.apple.com/app/apple-store/id1459456926?pt=690141&ct=mdm&mt=8

Why Analytics?

To help you answer questions on how to build a successful startup, using numbers!

Analytics helps you determine an MVP and find PMF. Analytics help you understand if you are going in the right direction for your MVP, and further to understand if you are moving towards achieving Product Market fit.

Analytics helps you focus on the highest impact work once you’ve achieved PMF. Once you’ve achieved PMF, you can analyze your user funnel to understand problems in the funnel, identify solutions, and then execute against those solutions to increase your main metric.

Analytics helps you scale your company so each team can drive against individual metrics. Eventually you have an engineering team, you have a marketing team, etc. And so what goal do you set in front of the marketing team? Use analytics for that.

The Funnel

Always start with a funnel when applying analytics to product.

Metrics — quantitative measurements of behaviors — indicate how well you are doing at each step of the funnel!

An example funnel for a product. Notice each purple text question has an answer that is a number.

Next, you apply the “Acquire, Engage, Monetize” funnel to your own specific product flow. Here’s an example of this applied to Netflix:

Once you have your funnel, how do you collect data?

Start by tracking the individual top level events for each part of the funnel. For Netflix’s example, you would track one event for a user sign up, one event for a video play, and one event for a subscription upgrade.
Now that you have individual top level events, start measuring properties of those events. This will help you identify which pieces of each part of the funnel are performing well or performing sub-optimally.
QA the data! Check that all events are being recorded correctly, and the appropriate actions are logging the appropriate data.
Integrate an analytics tool. Amplitude and Mixpanel are recommended by YC.
Take a look at the data! Actual data should now be flowing in.

The metrics to focus on

Sign ups! Startups = Growth.
Retention Cohorts. You can get pretty much anyone to sign up and use your product once. But they’re only really deriving value from your product if they come back and use it over and over. Target retention should be at least 20–30% w/w.
Unique metric for your company. What is the unique value your company is giving to users?

Really think deeply about how to best represent in numbers the unique value add your company provides.

4. How can I measure in numbers If I have reached PMF? You can measure this quite easily by measuring the retention “curve”. For products that have product market fit, you’ll see some kind of natural plateau. Products that do not eventually slope down to 0.

5. Revenue. Unless you need a massive set of users before you can monetize, this is the main metric you want to be tracking.

Sidebar: Actually look at your metrics!

Make your metrics front and center. Put it somewhere prominent, like on a big TV that you look at every day! A lot of companies set up analytics and never look at them again, because it’s too painful. Actually looking at metrics is the difference between being a data-driven team and not being a data-driven team.

Have social accountability around your metrics. Even if you don’t have any investors who want updates on the business, tidily synthesize your key metrics into an email to send to friends and family.

The Startup Analytics Stack over time

For each step in your road to building a successful business, you’ll use different sets of analytics tools.

You’re building an MVP, about to give it to your first customers. Install Google Analytics, install Amplitude. Google Analytics will tell you who’s coming to your website; Amplitude will tell you which features are they using, how engaged are they with that feature set.
You’re in a Private Beta.

Live chats. Either Slack or Intercom to maintain open communication with your customers.
Data Warehousing to have people within the company answer questions about the business — using numbers.
Help Desk to separate out inbound help emails.

3. Launch.

Product Usability testing/understanding. There could be a simple explanation as to why your metrics are so bad — like a bug or confusing button! Fullstory is recommended for figuring out if this is the case for your product (web-based).
Emails and Push Notifications. Customer.io is recommended here. By initiating a direct email to a new user offering help, you establish a connection and humanize your service.

4. Post PMF.

Democratize Data Access. Google BigQuery and Mode are recommended. This lets you use SQL to ask custom questions on top of the raw data in a way that Mixpanel might not allow you to.

Common Pitfalls

Tools change! Don’t stress too much about picking the right one.
Don’t spend too much money! See below for cheap versions to replace paid tools.