Discussion threads for reviews?

Didn't we used to have those? I don't see one for the latest harddrive review. I wanted to comment on the methodology used in benchmarking, as this is driving me absolutely nuts. (as a note, this is not the reviewers problem per se, and certainly not this particular reviewer, but it is a problem with the hardware review industry).

Taking average times or speeds on these things is absolutely pointless. The amount of data that is suppressed by taking an average is mind-bongling. And it's a lot of the important data. How do the drives ramp up, sustain, how much variation is in neat bars that are presented to the unsuspecting reader? having done my fair share of reviews, I know that the benchmarks, for the most part, do not make this data available. Shame on benchmark manufacturers. We need the raw data so that we can run proper stastical manipulation of it to see what is going on.

On that note, open source benchmarks are probably better to use since it will be easier to see (for a programmer) exactly what is being tested. While we use mainstream benchmarks that are at least comparable across the wide range of sites and personal users that use them as opposed to the custom benchmarks used by some sites, they simply do not stand up to the kind of public scutiny that an open source benchmark provides.

</rant>