Seth Godin article on public design

Some wise words on Seth's Blog about public design; the best designers understand what is important and what improves the experience it is not just about getting noticed 

Disruptive products require prototypes

Marty Cagan's presentation 'Building Disruptive Products', @Mind the Product, covered the need for prototyping as the best way to validate an idea before it gets into production. He provided some sound guidelines on how to budget for them and to make sure the prototype did not go too far; they should cost no more than 20% of the development project, should provide no performance, require no testing, support limited platforms and should not deliver all the use cases or configuration options and finally remember it is disposable code. Tom recommended complementing the customer feedback on prototype testing with A/B testing as well.

I believe that we should include exploratory testing from a member of the test team too because this ultimately benefits the product. As Tom Chi pointed out, the purpose of prototyping is to learn everything you can about the 'problem space', developers learn about solving the problem. It is therefore equally important to learn about testability so you can design it in. Note the purpose of this adhoc testing is not to generate test plans, platform coverage or provide formal sign-off on the prototype. I have often heard concerns raised by members of test teams that they were not involved earlier enough in the project this has to change if we want to reduce the cost of product development; having spent 5+yrs in QA this is certainly something I can emphasize with.

So when we talk about putting prototypes in the hands of 'users' this should mean all the users in the product's life cycle both inside and out of the company; from the test team, Internal IT (hopefully you are eating your own dog food), SEs and Consultants and most importantly the customers. Word of warning, make sure you track who provided the feedback because external opinions must trump internal opinions - the best answers always come from outside the company. The Test Team are there to comment on testability, SEs and Consultants are there to comment on deployability (made up word alert) and to offer a view on how their customers use the product. When it comes to usability, the customer's opinion must trump all others.