4 Product Release Methods that every Product Manager should know
In the fast-paced world of technology, product managers (PMs) play a pivotal role in shaping the success of digital products and services. We often have to make critical decisions on when and how to release new product/features, taking into consideration the nature of the industry, product lifecycle, and many other factors.
In this guide, I’ll explore four product release methods and discuss the appropriate use for each based on case studies and my personal experience. I hope this helps product managers navigate their options effectively for a successful product launch.
Big Bang Release: This involves launching the entire feature/product at once to all users. This works for well-tested features with minimal risk and expected positive impact on the majority of users.
When to use it?
Big bang releases are most commonly used by hardware companies like Apple and Samsung. It is simply not feasible to use other forms of releases without increasing the risk of a leak to competitors, nor is it not viable to create multiple hardware designed for different customers.
One iconic example that comes to mind is the Apple iPhone launch in 2007. The late Apple CEO Steve Jobs first unveiled the iPhone at the Macworld Conference & Expo. During which, he emphasized the years of research, upfront development, and testing behind the revolutionary device that would redefine the smartphone industry.
For software products, big bang releases only make sense in these scenarios:
- When launching a product for the first time and after a complete redesign as secrecy and speed to market is of paramount importance
- When developing internal tools and workflow software used by internal employees. The number of users is usually too low for other forms of release methods to be viable (both from a statistical and economic standpoint; more on that later)
- When a company doesn’t have the time or resources to develop other release methods.
However, even in such scenarios, big bang releases should only be used after extensive research, upfront development, and thorough testing.
For subsequent releases, I would consider using other release methods such as feature flagging, A/B testing, and beta testing when feedback is a more important factor than cost and speed to market. This enables us to get targeted feedback from users by testing new features in a controlled environment at different stages of our product development, before releasing it to a wider audience. This can help us improve the quality of our product and reduce the risk of failure.
Feature Flagging: This involves releasing the feature but keeping it hidden or disabled by default for all users. The feature flag can be selectively enabled for specific users or user groups to control its visibility and availability.
When to use it?
Feature flagging is essential for companies that rely on subscriptions for revenue. As this requires significant investment and resources to develop and operate, the primary consideration would be to ensure that the benefits of feature flagging outweigh the cost. I recommend using it in three scenarios.
In the first scenario, it is used for efficient management of entitlements, aka paywalls, to make certain features (and content) available to paid subscribers. This is an important feature for subscription-based companies to generate a steady and predictable stream of revenue. When done right, it builds a loyal customer base and fosters a sense of exclusivity, encouraging users to commit to the subscription and remain engaged with the company’s offerings over time.
In the second scenario, it is used to enable personnel to fine-tune technical infrastructure and redistribute resources based on external factors. This is most commonly used by companies like Airbnb, Grab, and Gojek that operate as a marketplace to facilitate transactions between buyers and sellers.
For Gojek, an example could be to feature-flag the ride-hailing feature to selected specific geographic areas to assess important metrics such as:
- Searches: Where and how many customers are searching for a car?
- Time to match: What is the duration to match a driver with a customer?
- Booking: Where do bookings take place? What are the prices and estimated wait times of these bookings?
- Total supply of drivers: The number of drivers operating within the geographic area
- Completion Rate: What’s the completion rate of the trips started
Based on the performance of the above metrics, Gojek could make tactical decisions like identifying the right time to deploy additional cloud servers (to keep latency & low) and drivers to cope with demand surge in specific regions for a great user experience.
In the third scenario, it is commonly used for A/B testing where the key consideration is adhering to the principle of ceteris paribus, which means holding all other factors constant so that the only difference between the two variants is the feature being tested. This can be challenging, but it is essential to ensure that the results of the A/B test are accurate. More about A/B testing in the next section.
A/B Testing Experiment: Also known as split testing, this involves randomly assigning users into groups and exposing each group to a different variation of the product or feature. This enables you to compare user behavior metrics to inform decision-making.
When to use it?
This is the most misunderstood release method among product managers. There are PMs who insist on making every single release into an experiment. There are those who insist on never running any given the complexity and long duration to get a statistically significant result.
For the former, the PMs may use A/B testing experiments as an excuse to release low-quality features that provide a bad user experience. As for the latter, the PMs may miss out on the opportunity to scientifically assess which option resonates best with users and yields the desired result.
I would recommend using it under these conditions:
- There are enough users to reach a statistically significant result. This means that the sample size of each group must be large enough to detect a difference between the two variations.
- There is enough time to properly design and execute the experiment. This includes defining the hypothesis, choosing the metrics to measure, and setting up the experiment so that it is fair and unbiased.
- The organisation possesses the scientific rigor to interpret the results of the experiment correctly. This means understanding the statistical significance of the results and avoiding making false conclusions.
- It is not used as an excuse to produce low quality features (*wink). A/B testing should be used to test hypotheses and improve the user experience, not to release features that are not ready for prime time.
When used under the right conditions, A/B testing can be a powerful release method to help you validate key assumptions before investing in new features or ideas. This can help you avoid releasing features that are not user-friendly or do not meet your business goals.
Learn the right way to incorporate A/B test into your product development process
Lastly, it is important to note that once a winning variation has been chosen, it will need to go through other release methods such as feature flagging and, last but not least, beta release.
Beta Release: This is used to introduce a new product to a large external audience. This enables you to gather early feedback, identify potential issues and bugs, and refine the features before a broader release.
When to use it?
Beta Releases were often used by gaming companies to provide gamers with some playable content at a discounted or free rate in exchange for stress-testing the games for unknown bugs. It is often made very clear to the audience that there may be numerous bugs in the beta version. Compensation, such as in-game currency or items, may be provided in exchange for reporting bugs.
In recent years, top product companies like Gojek and Spotify have also started using beta releases. During my stint in the growth team at Gojek, we received an overwhelming response from Singapore users who could not wait to try out the ride-hailing service. This led us to create an early access program and awarded users with 2x$5 vouchers as a reward for providing valuable feedback.
Such early access program inadvertently created an aura of exclusivity and generated marketing hype that led to successful launch at Gojek.
Closing Thoughts
Understanding the various release methods is crucial for product managers to make informed decisions that align with their goals and user needs. Each method serves a specific purpose, and choosing the right one depends on the nature of the product, target audience, and desired outcomes.
While big bang releases work well for hardware products, software products benefit from feature flagging, A/B testing, and beta releases to gather valuable insights, refine features, and optimize user experience. By carefully selecting the appropriate release method, product managers can achieve successful launches, foster user loyalty, and drive the growth of their products.
As the industry evolves, embracing new release approaches will remain essential for product managers to adapt, innovate, and make a meaningful impact in their respective domains.
That’s it! Thank you for reading till the end of this article. If you find this useful, do leave a comment, clap for the article, share this around. Lastly, follow me on Medium and LinkedIn for more great content.
