Some hours ago, during a discussion on Twitter, it came to light that a lot of people don’t understand the importance of a complete and clear review methodology.
Even geeks, engineers, journalists seem to be unable to see how bad an incomplete testing method could be. We know that in scientific fields a rigorous methodology is needed. According to scientific studies, a test shall be capable of being replicated: “Reliability is the degree of consistency of a measure. A test will be reliable when it gives the same repeated result under the same conditions” (Source).
To do this, we have to know exactly how the experiment was done. About a hardware review, we have to know every single hardware and software component was used, and which version of it. Thanks to these informations, if we want, we could be able to replicate the review tests and we could be able to confirm the achieved results.
So, during that Twitter discussion, someone questioned my results of this review, where I tested the power consumption of the Intel Core i9-9900K @ stock setting. As you can see, I meticulously described the configuration and the methodology I used. Also, I used default settings for everything: “out-of-the-box settings”. I did that in order to see how the PC of "average Joe" perform. We all know that average Joe purchases the pc, puts the plug and pushes the power button. Stop.
People complained that I had to change BIOS options in order to use a lower Power Limit 2, according to Intel specifications. The problem is that Intel itself allowed Asus, Gigabyte, MSI and other manufacturers to use that setting on high end main boards: PL2 @ Auto (About 210W). According to Intel, it’s not overclock: “The values on the data sheet for 1.52 V and 138 A are the maximum values that can be used on the Intel® processor” (Source). So, main boards manufacturers didn’t violate the specification. They just use the max PL2 value allowed.
But this is not the problem.
The problem concerns the methodology of the tests. These people, if they think my results are wrong, could remake the same tests with the same hardware and the same software. Despite that, they attacked B&C saying that other sites are more reliable. It could be. We absolutely don’t think B&C is the best site. However, I think B&C is the site with the best and more clear methodology tables. Here there are some examples.
Some Intel Core i9-9900 reviews (I can't list every 9900K review made)
As you can see, B&C tests can be replicated without problems. We don’t fear about that. On the other hand, we could have a problem if we want to replicate the tests of other sites.
In conclusion, we think that review sites have to improve their methodology, in order to create more reliable reviews. Currently, the reviews are far far away from being considered “scientific”.
Please note: It's not an attack to the other sites. B&C is a small site, attended in our spare time, so even we want to read some "scientific" reviews. We are end users in first place, not journalists. ;)
Per commentare fai login. Puoi connetterti anche con il tuo account Facebook, Google o Twitter.
RaspberryPI: cosa farci (56 messaggi)Ultimo messaggio di: Il nabbo di turno (22/07/2019 14:26)
Consiglio Upgrade Scheda Madre, CPU e RAM (15 messaggi)Ultimo messaggio di: Alessio89 (22/07/2019 13:32)
[Official Topic] AMD "Zen" (2447 messaggi)Ultimo messaggio di: Sintara (21/07/2019 10:21)
[Official Thread] GPU Nvidia Volta e Turing (261 messaggi)Ultimo messaggio di: Alessio89 (20/07/2019 12:11)
[Waiting for] AMD GPU Navi (571 messaggi)Ultimo messaggio di: Bivvoz (19/07/2019 18:59)
VMware benchmark (4 messaggi)Ultimo messaggio di: Fottemberg (19/07/2019 17:00)
Meme e immagini divertenti sul mondo IT (215 messaggi)Ultimo messaggio di: Masciale (19/07/2019 14:04)
aggiornamento bios asus ROG STRIX B450-I GAMING senza cpu (4 messaggi)Ultimo messaggio di: matteo78it (19/07/2019 13:40)
[Guida] all'overclock di AMD Ryzen. Il thread per spiegare ai novizi come overclockare le CPU Ryzen, per raccogliere pareri e per scambiarci suggerimenti.
[Official Topic] AMD APU Kabini, Temash, Kaveri e... Carrizo. Discutiamo le ultime novità sulle APU AMD basate su microarchitettura Bulldozer e Jaguar.
[Waiting for] AMD GPU Polaris & Vega (ex-Arctic Islands). Discutiamo le ultime novità sulle GPU Polaris di AMD, senza dimenticarci di Vega!
[Official Topic] Nvidia Pascal GPU. Discutiamo le ultime novità sulle GPU Pascal di NVIDIA e su quanto ci aspetta in futuro!
[Official Topic] AMD "Zen". Discutiamo le ultime novità sulla microarchitettura x86 Zen di AMD e le CPU/APU in arrivo.
Creative Sound Blaster e Windows 10/8.1/8/7/Vista. Dedicato a tutti i possessori delle schede audio di Creative Labs!
B&C e la questone "degli articoli prezzolati". Perché B&C è un portale diverso?