Are the hardware reviews realized according to a scientific method?

Il vostro parere, i commenti e le discussioni su tutto quanto pubblicato sul portale di B&C
Rispondi
Avatar utente
Fottemberg
Messaggi: 18738
Iscritto il: martedì 29 novembre 2011, 22:52

Are the hardware reviews realized according to a scientific method?

Messaggio da Fottemberg »

Some hours ago, during a discussion on Twitter...

Continue
PC: ANTEC P7, CoolerMaster V650 80+Gold, AMD Ryzen 5 3600X, Gigabyte AORUS X470 Gaming 7 WiFi, 4x8GB Patriot Viper DDR4-3600, GOODRAM IRDM 480GB M.2, 2x Toshiba RC500 500GB M.2 in RAID 0, Sapphire Radeon 5700@XT, Windows 10 Professional
Immagine

Yorgos
Messaggi: 1
Iscritto il: giovedì 13 giugno 2019, 18:54

Re: Are the hardware reviews realized according to a scientific method?

Messaggio da Yorgos »

many sites, which are paid to do the reviews, or do it to be favored by the manufacturer, are obeying the guidelines sent along with the product.
Novidia is one of those companies that releases "Reviewing Guides". They tell you what game to run, what settings, how long... etc.
On both CPU and GPU side, you can manipulate the outcome.
Some of the worst sites out there is AnandTech and TechpowerUp.
AnandTech had/has some obsession with ram and JEDEC standards. The last 2 years they stopped it, or changed their methodology for a reason unknown to me. Since their Purch media acquisition they should be considered a novidia and intel promoting site(although novidia stopped being a purch media customer for a couple of years)
TechpowerUP, otoh has the most skewed results. They dump all the data they have from the past decade, regardless of S/W versions, drivers, etc, into a pile of stinking charts and throw it into their "review".
I guess they will redo all those benchmarks they did the last 2.5 years with intel and amd applying security patches and scheduler updates.... not. TPU charts is like reading Earth centric planet system Astronomy papers from 500 years ago.
There was this TechSpot bulldozer review I was reading back in the day, where some 8 core chips got beaten by a dual core i3. How is that possible? well, if you are encoding movies using MPEG-1 (I don't remember the algorithm they used) at 480p (yes, they were targeting that resolution) It's obvious that a 125W/8=15.625Watt Core will lose against a 65W/2=32.5Watt Core.
You can fool the reader easily, by hiding those details.
Someone said "look that i-core with 6cores is better at encoding compared to that 32cores when using handbrake". Well, nobody told to the viewers that most encoding s/w doesn't scale beyond (usually) 12 threads.

Even if your are willing to post all the benching details, you should justify the choice you made on each s/w.
Here's an example of how you can make intel look good. 10/20 7900x vs 16/32 1950x
test one: "Low" presets. "Here we transcode a 640x266 H264"... but why?
here's why: https://images.anandtech.com/graphs/gra ... /90032.png
you can cripple any cpu in every benchmark.

Avatar utente
Fottemberg
Messaggi: 18738
Iscritto il: martedì 29 novembre 2011, 22:52

Re: Are the hardware reviews realized according to a scientific method?

Messaggio da Fottemberg »

Yes, I agree with you. I think tech companies and tech journalists need to get together and have a round table to come up with a shared review guideline. At the present moment reading a review is like to trying to understand the Rosetta stone. :\
PC: ANTEC P7, CoolerMaster V650 80+Gold, AMD Ryzen 5 3600X, Gigabyte AORUS X470 Gaming 7 WiFi, 4x8GB Patriot Viper DDR4-3600, GOODRAM IRDM 480GB M.2, 2x Toshiba RC500 500GB M.2 in RAID 0, Sapphire Radeon 5700@XT, Windows 10 Professional
Immagine

Avatar utente
charlotto
Messaggi: 677
Iscritto il: martedì 27 gennaio 2015, 16:40

Re: Are the hardware reviews realized according to a scientific method?

Messaggio da charlotto »

un articolo scientifico cerca di osservare la realtà naturale attraverso la ripetivilità dell'esperimento sintetizzandola in una relazione matematica o legge. Il mondo dell'hardware-software potrebbe o dovrebbe fare lo stesso entro i limiti che lo caratterizzano. a complicare le cose sono la complessità del sistema che si sta ossercano dovuta all'enorme varietà di software e la mutabilità temporale dovuta essenzialmente alla riscruttura del softoware stessp. inoltre, c'è il vizio! ovverso il fatto che c'è una competizione tra marche che può falsificare la realtà per favorire un concorrente. il vizio si può manifestare proponendo scenari che sono a favore di una determinata marca, in modo che anche se sono ripetivili, non sono rappresentativi del valore globbale di ciò che si osserva. il vizio può agire anche falsificando i dati, con mezzucci software ecc. escludendo il vizio, il problema della scientificità dei test diventa, esiste una configurazione softaware (non si puà fare una misura utilizzando tutti i software) che sia rappresentativa del valore oggettivo di ciò che si sottopone a test?

Avatar utente
Blobay
Messaggi: 972
Iscritto il: martedì 3 novembre 2015, 14:00

Re: Are the hardware reviews realized according to a scientific method?

Messaggio da Blobay »

We need the possibility to make a comfortable peer review. In my opinion, without we can't accept any review.

Rispondi