Reviewing: Testing Tools

a skill ( something you can learn how to do using Skillpedia ) by lxpk Tue, 2006-11-28 20:45

I can: Add to my skills I'll learn: Add to my skills wishlist Create Open Mission Create Learning Mission

People with this skill

People with this skill:

	User	Category	Title	Experience	Comments	Comment
	thekid12	Skill To Teach	Reviewing: Testing Tools	Skill Group: Reviewing: Testing Tools Skill Category: Skill To Teach I am very hard on equipment, and when it comes to tools, I've got a little bit of knowledge on just about everything.	0

Reviews are an important source of information people rely on to make informed decisions about what things to invest time and energy into.

Why read something if you're not going to absorb it and reflect on what you learned afterwards? Why buy a piece of kit if you aren't going to evaluate its performance? Why not write your experience and share it with others?

Reviewing turns passive "consumption" into proactive critiquing. And if a product dissapoints, you don't just get mad, you get even by warning others away from a lemon.

Reviewing is a valuable practice for the reader who writes it as it is for the readers of the review. Reflection and critiquing will make your experience more useful to you.

Reviewing Conclusions: DOSPEC or NOTSPEC

Every review should come up with a final verdict:

DOSPEC if it does something useful and it does it well.

NOTSPEC if it does nothing useful or does what it does badly.

Examples: Leatherman Wave Multitool DOSPEC vs Leatherman Folder: NOTSPEC

The DOSPEC Process

The DOSPEC procedure for evaluation is as follows:

Requirements: Requirements identified: What do we need to do?
Candidates: Candidates identified: What might be able to do it?

Comparing: Top candidates Compared: How well do they do it and which does it the best? They become Candidate DOSPEC.

Supplying: Top Candidates Obtained: Get whichever candidate or candidates you can afford.

Testing: Top Candidates Evaluated: How well do they actually perform in test usage?

Verdict: If it does the job and does it better than alternatives, it becomes Tested DOSPEC.

If not, it becomes Tested NOTSPEC, which is worse than Untested NOTSPEC because that stuff might be better than you expect but this stuff is definitely garbage.

Tech is NOTSPEC until proven DOSPEC

All untested tech should be viewed with healthy skepticism until it is proven DOSPEC and then monitored for follow-up evaluation until it has demonstrated a failure-free lifecycle over time.

Any failure of a DOSPEC item should be reported as a blog entry or comment on the tech article mentioning it.

Failures trigger a re-evaluation of the tech to certify that the mean time between failure is acceptable and that it does what is needed better than any alternatives to be considered.

DOSPEC Testing Process

Proper testing is essential to determine if a tool does a job adequately. Testers should write a review to document their experiences including a first-impression as well as follow-up and long-term satisfaction.

Follow-Up Reviews

If you write a review today, you should give yourself reminders to follow-up at regular intervals like a month, six months, a year, etc. to gather MTBF (Mean Time Between Failure) data. Report on any breakage or loss of function and what repairs and maintenance are required.

Write a Reviewer Bio

"People will be interested in your biographical information for a number of reasons. People looking for clothing, for example, will often look for reviewers and testers of clothing that are their size and weight. That way, they have a reasonable expectation that the clothing will fit them the same way."

—BackpackGearTest.org Biography Writing

Write a Review

"The first thing to do is to select an item to test. For your owner reviews, you should choose an item that you have had significant experience with, and the best thing would be your favorite piece of gear. You should have owned and used the gear long enough to be able to comment on its durability."

—BackpackGearTest.org Review Writing

Cover as many subjects as you can:

Name of the item being tested prominently displayed at top of page.
Biographical information about yourself
Product information
- Manufacturer
- Year of manufacture
- URL of top level manufacturer's web site
- URL of the item at the site
- Listed weight
- Weight as delivered
- Product description
Field information
- Location or locations where the test was conducted
- Outdoor Gear Testing Conditions:
  - Description of location (geography, terrain, elevation, etc.)
  - conditions (temperature, precipitation, etc.)
- Short, comprehensive description of your trip or trips while testing the item, including how the item performed on the trip, or in the case of multiple trips, on each trip.
- What is it like right out of the box?
- Is it easy for you to figure out and use?
- What are your expectations?
- How did it perform for you after using it for a while?
- How did it last over a long period of use?
- Summarize your final opinion of the item.
Submit your review!
- What did you like?
- What didn't you like? Your review is NOT required to be positive. Reviews of flawed gear are just as important as reviews of outstanding gear.
- Contacts made with the manufacturer: positive or negative? who did you speak to? What was said?
  - Tell us when you contacted them, who you spoke with, and what was said by both parties. Did they back their product?
Improvement suggestions
Follow-Up: Reviewers should revisit their reviews after a month, six months whenever something relavent happens that the reviewer wants to share.
- Did failures or problems occur after long-term use?
- Did you continue using the tech or did you switch to something else and why?
- Did other people you know adopt the tech based on your recommendation and how did it work for them?