How to run a beta test

Posted on the August 19th, 2008 under Business, Technology by Al

I’m a beta-whore. I tried to categorize myself as a beta tester, an early adopter, and various other, nicer labels but my wife insists that “beta-whore” fits best.  I enjoy technology and feel my jack-of-all-trades skills can be beneficial in a beta test.  I sign up for many beta programs and devote more time to some and less to others.

A trend I’m seeing is development by newbies.  I don’t mean people new to development/programming, but people who are unfamiliar with using external testers.  Based on my experiences, here are a few pointers for those considering the use of beta testers.

  1. Set goals – Are there any specific areas you’d like to have tested?  What has changed since the last  alpha or beta version?  You may not have a list of items for the testers (e.g. if you’re trying to determine how intuitive an app/service is) but if you do, let them know.
  2. Set a timeline – How long is the beta period?  One week, one month, one Google-beta (i.e. indefinitely)?  Give the testers a timeframe on when to report issues they uncover (e.g. as they occur, once a day, once a week, etc).  Keep in mind that beta testers have lives and jobs outside of testing your app/service.
  3. Define how you want feedback - Should the beta testers email you with their findings?  Do you have a forum or GetSatisfaction page set up?  Be clear and upfront.
  4. Don’t rush the beta period - Understandably, you want to get the app/service out there but broken software does more harm than good.  At the same time don’t be a Google.  I believe that short beta periods (e.g. 24-48 hours) are too short to be useful.
  5. Pick good beta testers – Sure, I’d like to beta test a lot of apps and services, but that does not make me the right choice.  Be as selective as you can.  If you’re running back-to-back beta periods, be sure to include/invite people that have provide valuable feedback in the past.  If a beta tester’s first actions is to blog that they’re beta testing something, their interests may be self-indulgent.  Beta testers should be helping make the app/service better.
  6. Invite me to be a beta tester - Sure it may contradict #5, but you won’t know until I’ve had a chance to beta test your app/service.

Post by Al Degutis

blog comments powered by Disqus