Coderbyte’s journey to 1,000 customers

Coderbyte began 9 years ago as a coding interview prep site for developers, and has helped more than 500,000 developers since then. About two years ago we launched a code assessment solution for businesses, and this month we celebrated an important milestone: 1,000 business customers. We’re on a roll and are looking forward to our next 1,000 customers, but further growth is not inevitable. In fact, even reaching 10 customers was no small task — the technical screening industry is a crowded SaaS market with more than 70 solutions featured on G2. Collectively, the industry has raised $100M+, and includes well-known companies like Y Combinator-graduate HackerRank and PE-backed Coderpad.

So how did a company with $0 in funding and zero full-time employees reach 1,000 customers, including 371 that switched from the top four alternatives? By being customer obsessed (we still know every single customer on a first name basis) and doing these three things:

#1 — Ask whether problems have really been solved

As a software engineer who has interviewed at dozens of leading technology companies, I was always disappointed in the screening processes. I can appreciate why companies need to screen candidates in the first place because I’ve also been on the other side of the table — there’s nothing more frustrating than blocking off hours to interview a candidate who clearly can’t code! But I knew there had to be a better way to filter 300 candidates down to 20 in a more efficient way than giving them a 5-hour project to complete at home.

Like sheep, all the major screening solutions took similar approaches to solving the same problems, each causing disastrous consequences:

• To prevent candidates from cheating, they have a suite of privacy invasive tools, like forcing candidates to turn on their front-facing cameras while coding or preventing them from opening another tab to search Google or get help from Stack Overflow
• To be consistent with the mind-numbing riddles of the (finally) infamous Google interview process, their default assessments are filled with algorithm challenges that are completely irrelevant to the day-to-day job
• To facilitate enterprise customer use cases, they allow for endless customization by customers who have no experience or expertise in creating objective technical assessments

Each of these approaches not only prevented companies from effectively screening employees, but also repelled top candidates from even participating in the assessment. Ghosting is unfortunately one of the biggest problems that tech recruiters face, in large part due to offensive assessment processes.

While these examples are specific to our industry, we’ve found that “conventional wisdom” across industries often doesn’t pass the test of time. Decisions are made, then copied, and never re-evaluated, and so terrible ideas perpetuate across an industry.

As a group of outsiders, we don’t take for granted that any problem, no matter how simple, has been solved. While we always look to competitors for inspiration, we investigate each challenge with the mindset of how we can best optimize for customers from a clean slate.

Today, we take a dramatically different approach to cheating detection, challenge libraries, and customization, and our customers appreciate our thoughtfulness.

#2 — Eliminate noise at all costs

Every company operates with constraints, and we’ve worn ours like a badge of honor. Because we haven’t raised capital and until three months ago had no full-time employees, there were simply things we couldn’t ever do. For example, we could never offer white glove service or 24/7 support, complete RFPs or security checklists, or do complex integrations with legacy systems. And that’s fine, because many customers in the market don’t need any of that!

Unlike our competitors that have to accommodate for all edge cases in order to hit quarterly targets or risk their jobs, we don’t have to sacrifice our product-market fit for stretch goals. To the contrary, we actively eliminate noise from our immediate environment. Sure, our competitors have a vastly larger addressable market, but our market is vastly more addressable. The day we realized that it takes just a few bad customers to pressure us to ruin an otherwise good product is the day we guaranteed we’d always have a good product (even if we’d never be able to grow to the size necessary for an IPO). Every strategy has its tradeoffs.

We found that identifying and eliminating noise is surprisingly straightforward. Consider that companies that insist on RFPs and security checklists are often the same companies that request advanced customization that will eventually bloat your product. They don’t actually care about solving their problem in a sustainable way — they mostly just care about creating work and filling up their schedules with meetings. We therefore created a series of trigger phrases that, when they showed up in support tickets, emails, or via our onboarding form, flagged a company as a bad fit.

If they’re a customer, we then automatically refund and lock their accounts. If they’re not a customer, we simply block their domain from ever signing up. In either case, we are preventing such companies from using our product and misleading our roadmap. It’s not that we couldn’t just ignore a single bad feature request here and there, but companies that make bad requests tend to make other bad requests, and we didn’t want to be completely inundated with noise. Instead of having to parse through every bit of head-scratching customer feedback to get to the root of their pain points, we just eliminate the part of the market that is a bad fit for our product!

Other founders think we’re insane and perhaps that’s true, but we think it’s more insane to let a great product be led astray by competing demands from misaligned customers. Companies will fire employees who end up not being a good fit, but let customers that aren’t a good fit wreak havoc. It’s not good for business, and when you actually deeply appreciate your happy customers, you realize it’s not good for them either.

Companies will fire employees who end up not being a good fit, but let customers that aren’t a good fit wreak havoc.

The impact of cutting out noise is that 90%+ of the feedback we now get is relevant, actionable, and consistent. We’re more easily able to spot patterns and improve the product because we improved our customer base. None of our customers care about anything other than unbiased interview processes to hire better talent and grow their businesses. For the record, these aren’t random companies — they’re some of the world’s largest brands and have assessed more than 150,000 candidates in the last 12 months. They all miraculously managed to find ways around RFPs, security checklists, and other bureaucratic, value-detracting processes in order to purchase Coderbyte (with a credit card!).

A product for everyone is a product for no one. To this day we have not released a single button or feature that wasn’t used by a customer that day.

#3 — Take positions that the competition can’t take

Every venture-backed company eventually faces this dilemma: how do you extract as much revenue as possible from slow-moving enterprise customers while simultaneously having a self-serve offering for savvier small businesses that doesn’t cannibalize the enterprise offering?

This dilemma is daunting and has kneecapped so many otherwise impressive companies that have been pressured to continuously change business models, implement dark patterns, and place more and more risky bets to achieve ambitious growth targets (and fund and justify the immense costs of a salesforce that’s no longer necessary for many prospective customers).

We live in a world with zero marginal costs for most software products and yet almost no one competes on the most obvious and proven advantage in the history of commerce: price. So we decided to begin by competing on price. It’s been two years and we are still the only product in the entire market to display all of our plans and pricing on our website without any hidden fees or ‘gotchas.’ We also offer virtually unlimited users and usage on all plans, and allow companies to sign up for monthly or annual plans.

Why? Because the competition can’t do it. Their entire business models are predicated on a pricing vector that is both untethered from costs and helps them achieve ever increasing contract values with enterprise customers. Even though hiring is cyclical for many companies and there’s an ongoing recession, they can’t be customer-focused and offer month-to-month plans because then they’d have to explain the unacceptable churn rates to their board (which is the same reason they can’t cut off bad-fit, resource-draining customers).

The irony is that venture-backed companies that don’t gain a competitive advantage from scale work so hard to educate and create a market, and then are so easily flanked by lower-cost solutions who can siphon off customers. You know what procurement teams appreciate more than a vendor that will fill out endless paperwork? A reasonably priced and effective solution.

We’re so confident that price is enough of a reason for significant parts of a saturated market to switch vendors that we’re doubling down. Our team will be building a second company and product called Session Rewind, which will soon be publicly available.

I’m looking forward to that and our next 1,000 Coderbyte customers and will share any new lessons learned about the journey!