Beta programs, customer feedback and customer endorsements

Beta programs enable you to collect customer feedback on the functionality of your product. They are also an excellent way to receive customer endorsements on a new release prior to its launch.
Once your product testing is complete and full functionality has been delivered by your development team, you have the option to conduct a beta program to have customers review the entire product experience.

What beta programs are not

While beta programs are useful for collecting feedback, they should not be used for the following purposes:

Additional testing phases

Your product should be well-tested before giving customers access to it. Customers should be able to interact with a product without concerns of major malfunctions.
Although they might find bugs during the beta program, the product should be thoroughly tested in advance to mitigate as many problems as possible. Failure to do so could affect customers’ opinion of your offering, as well as objective feedback.

Usability testing

Complete the validation of the customer problem you are solving and usability testing before you reach the stage of product development. Beta programs are not the time for first interactions with customers; the goal is to see how they react to the entire solution as it was built.

Closing the deal

Some organizations use beta programs to ensure that a prospect agrees to buy their product. Keep in mind that while giving them“early access” to trial the functionality may seem like a good idea, it comes with additional risk.
If the prospect doesn’t like what they see, they might not agree to purchase the product and, even worse, they might share the news of your upcoming release with your competitors.

Outlining beta goals

Before beginning a beta program, outline the goals you want to achieve. Do you want customers to provide feedback on a specific part of the product? Do you want to approach a certain segment of your target market? What questions would you like the beta process to answer? Establishing your goals in advance ensures that you choose the right customers and ask them to focus on particular areas.
Arrange a set time period for this program (one month, for example). Like most people, beta participants are motivated by deadlines. Having regular check-ins and a defined timeframe in which to provide feedback will elicit better results.

Beta program phases

The beta program, which is run by the product manager and supported by the engineering team, consists of three phases.

1. Choosing beta customers

Similar to the process of understanding market problems, it is key to accurately represent different segments of your target market (or a single segment, depending on your beta goals). Choose your beta customers accordingly. Ensure that you have enough customers (three to five) from each segment to identify trends in your data.
Example: Company XYZ produces a product that alleviates knee pain. It expects two types of users for their product: young adults who have torn their knee ligament, and seniors who require knee replacements. Company XYZ needs to ensure they include beta participants from both groups.
Beta programs offer advantages for your organization and your customers, and you must clearly explain the benefits to both. Your organization will receive feedback on functionality and suggestions for how to improve the user experience, and possibly endorsements for your product. Your customers will get to see the product first, and have a hand in the final product that goes to market.
Create an agreement (signed by your organization and the customer) that outlines their responsibilities. The agreement could include the following points:

  • Your organization will:
    • hold weekly half-hour calls to review beta feedback
    • be available by email and phone to answer any questions
    • provide documentation, sample scenarios to perform and product updates in a timely manner
  • The customer will:
    • use the product for a minimum of two hours a week with sample scenarios (such as adding a new customer address to the system) as a guideline
    • ensure that a minimum of two people will participate
    • engage in a final interview to deliver feedback
    • provide an endorsement that can be used publicly

2. Collecting feedback

Lay the groundwork so your customers are prepared for the beta program. Train them on the new functionality so that they understand it. Provide them with sample scenarios and documentation so that they know what is expected from the beta experience.
Have a call with each beta site weekly. It can be done as a group but keep in mind that reticent customers might not provide feedback as readily as in a one-on-one setting.
Ask them about their experience: what worked, what didn’t work well and what questions arose while using the product. These trouble areas provide indicators of where implementation can be improved.
Ask open-ended questions to allow the customer to give the widest range of feedback, and record their responses in depth.

3. Analyzing and implementing feedback

Collect feedback and analyze trends to identify what to improve about your product. Then make those changes . Do not engage in beta programs if you do not have time to implement changes.
Failure to listen to feedback can bear negative consequences. Customers may withhold endorsements and be less willing to participate in future.
If you implement the changes and receive good feedback from customers, you will be on the way to securing a successful launch and happier customers.

References

Cooper, R. G. (2001). Winning at New Products: Accelerating the Process from Idea to Launch (3rd ed.) New York: Perseus Publishing.
Daniels, D. (2010). Beta Testing. Launch Clinic. Retrieved September 24, 2010, from
Dunford, A. (2010). Beta as a Product Marketing Exercise. Rocket Watcher. Retrieved September 24, 2010, from http://www.rocketwatcher.com/blog/2010/01/beta-as-a-product-marketing-exercise.html
Khan, S. (2010). Building a Better Beta. Pragmatic Marketing. Retrieved September 24, 2010, from http://www.pragmaticmarketing.com/publications/magazine/4/3/0605sk