UserBenchmark offers explanation for changes to CPU score weights

[onetheycallEric]

A hot potato: Benchmark comparison site UserBenchmark has defended its decision to make changes to the way it weighs CPU average scores, thus favoring single-core performance. The response likely won't resonate well among the website's most vocal critics.

This past week a portion of the enthusiast community was lit aflame with controversy over UserBenchmark's decision to adjust its CPU scoring system. While not a popular destination among hardcore users or reviewers, UserBenchmark remains a widely used tool for the general consumer and its comparisons rank highly in Google search results -- Tom's Hardware notes the website serves nearly 10 million visitors each month.

Enthusiasts were taken back by the website's questionable decision to adjust its scoring algorithm in favor of single-threaded performance, inflating scores in such a way that doesn't appear to be entirely representative of the processor in question. Previously, UserBenchmark apportioned the average score as 30% of the single-threaded performance, 60% of the quad-core performance, and 10% of the multi-core performance. Now, the weighted average is 40% single-core, 58% quad-core, and only 2% multi-core.

They claim this is to offset "unrealistic overestimating" for the new core-heavy Ryzen 3000 series, and the new score weighing indeed doesn't show AMD's newest chips in quite as favorable a light -- a point of contention among AMD fans, to be sure. UserBenchmark updated its FAQ section to address the outcry.

Shortly after the Ryzen 3000 release, which we welcomed emphatically, we noticed that our CPU gaming and desktop estimates were unrealistically overestimating all CPUs with core counts beyond 8 so we corrected the estimates. Our underlying data points for single, quad and multi core performance remain unchanged and are clearly visible together with gaming, desktop and workstation scores on each of our product and comparison pages. (...) At present we estimate that our CPU gaming index is accurate to around 8%. By rebalancing the weights in favor of more cores we can probably reduce the error to around 5%. Even after a rebalance, the 4 core i3-9350KF would still, on average, offer a similar gaming experience to the 18 core i9-9980XE, a fact which many of our most vocal critics seem to find hilarious...

In UserBenchmark's defense, they do acknowledge readers need to look deeper than the top of the page to get the big picture. However, the bizarre choice to weigh the average score so heavily in favor of single core/thread performance is hard to ignore. Especially when multi-core and multi-threaded performance is only going to become more important.

While I won't take sides here, I will point out professional reviews are the best way to measure a CPU's performance and glean a purchasing decision. If you haven't already, check out TechSpot's Ryzen 3000 coverage.

Permalink to story.

https://www.techspot.com/news/81176-userbenchmark-offers-explanation-changes-cpu-score-weights.html

 

[mosu]

When Intel will produce commercial non-server parts with 16 to 32 cores, they'll change back the scoring system for whatever reason they will consider as an explanation.

 

[QuantumPhysics]

As if Intel or AMD actually care about benchmarks.

You really have no other choice.

 

[grumblguts]

Manipulation to consumer information.
surly this comes under false advertising and subject to criminal proceedings

 

[VitalyT]

So the benchmark relies on results from software that's not optimized for use with that many cores, and suddenly it's the CPU's fault?

 

[Dharan]

Yeah... Seems very much like on Intel's payroll... Such websites should be called out for what they are doing periodically to create awareness in tech community....

 

[Puiu]

Even Intel fans agree that this is 100% something decided behind the scenes by Intel. God knows how much they paid them.

 

[hahahanoobs]

"Especially when multi-core and multi-threaded performance is only going to become more important."

Someone remind me what the keywords are here....

 

[cliffordcooley]

All I have to say is; anyone looking at the scoring system, should understand where the numbers come from. I for one do not believe the numbers should be averaged into one.

 

[treetops]

It's popularity is due to it being a small download, fast test and requiring user to open their webpage to see the results.(which probably upticks the site on google results) UserBenchmarks is a terrible way to compare parts. Even before this update. It's like that misleading 3600 "leak" that showed it above the 9900k on passmark on single threaded performance. That was due to a l2n passmark score. Being that it was the only score of course it would be higher on l2n compared to an average of millions of benchmarks made by l2n non l2n air water stock ovclock etc etc.

I quoted passmarks single threaded performance chart in the past, but I didn't understand the way it went about it's testing. No graph that goes off of user scores is ever a standard that should be used when comparing components. User benchmarks are a for fun comparison. THEY SHOULD SHOW STOCK SCORES with a clearly shown test system to let people know what a realistic comparison looks like.

Another reason your average joe will use a test like this is to make sure what they have is working properly. Say you buy a video card and you just want to make sure it's not somehow underperforming, it sounds like a good idea to head over to a site like this. But even with that they need to show a base performance.

This site is misleading and unrealistic. It's only value is for fun benchmarks against other users. Claiming to be anything else....

 

[fps4ever]

There are plenty of other good free bench marking tools out there so why bother with an Intel biased one. I can understand adjusting score percentages but all the way down to 2% for multi core smells like Intel influence.

 

[brucek]

There's no reason to do an artificial "blended" number which will be right for no one. They should just have a checkbox for the user to indicate whether they regularly do any of the few (consumer) things that make use of > 4 cores, or just present two sets of numbers.

If you have a real parallel workload the number of cores is super relevant to you and should not be watered down. For everyone else, any cores past 4 are likely not adding anything at all and I'd argue that core #3 and core #4 might be contributing precious little too.

 

[pcnthuziast]

Nobody should use UserBenchmark as a legitimate source for info or comparison. Nobody!

 

[Cycloid Torus]

Should be 3-D, with weights set by user.

 

[kmo911]

Hopefully they would find alike info when they collect info from benchmark.

 

[Evernessince]

Even Intel fans agree that this is 100% something decided behind the scenes by Intel. God knows how much they paid them.

I don't know about Intel paying anyone but the decision is certainly receiving flack from both Intel and AMD users. Going off what we know, I do not believe these metrics represent modern workloads in games or otherwise.

 

[cliffordcooley]

Going off what we know, I do not believe these metrics represent modern workloads in games or otherwise.

Maybe ten years ago, certainly not with today's applications.

 

[Maxiking]

AMD has better single core performance than Intel now if you ignore gaming, so I don't understand what's the problem here but yeah, it is amd cry babies so doomed if you do damned if you don't.

 

[Puiu]

AMD has better single core performance than Intel now if you ignore gaming, so I don't understand what's the problem here but yeah, it is amd cry babies so doomed if you do damned if you don't.

This is about not deceiving people, not about cry babies. Otherwise nobody would care about illegal anti-competitive practices. Don't sit by and do nothing.

 

[fps4ever]

AMD has better single core performance than Intel now if you ignore gaming, so I don't understand what's the problem here but yeah, it is amd cry babies so doomed if you do damned if you don't.

That is an excellent way to look at it, if um you are like a 12 year year old prepubescent.

 

[Maxiking]

AMD has better single core performance than Intel now if you ignore gaming, so I don't understand what's the problem here but yeah, it is amd cry babies so doomed if you do damned if you don't.

This is about not deceiving people, not about cry babies. Otherwise nobody would care about illegal anti-competitive practices. Don't sit by and do nothing.

You mean like the false 4.6 Ghz boost on 3900x?

Or false boosts on Ryzen 3xxx in general and no one giving a damn about it?

Good joke, m8.

 

[Maxiking]

AMD has better single core performance than Intel now if you ignore gaming, so I don't understand what's the problem here but yeah, it is amd cry babies so doomed if you do damned if you don't.

That is an excellent way to look at it, if um you are like a 12 year year old prepubescent.

Speak of the devil...

 

[fps4ever]

Speak of the devil...

Huh? I like both Intel and AMD. Especially close competition for a win on the consumer side. Your fake injustice is displaced. I sense an Intel bias leaking through.

 

[Evernessince]

You mean like the false 4.6 Ghz boost on 3900x?

Or false boosts on Ryzen 3xxx in general and no one giving a damn about it?

Good joke, m8.

You are referring to a BIOS issue that has already been fixed. If the issue was as bad as you make it seem, AMD would already be getting sued. This isn't a GTX 970 3.5GB RAM thing.

 

[arrowflash]

I fail to see what's wrong here. Single core, single threaded ipc performance IS more important for power users. They didn't change much, only increased the weight from 30% (which I'd say it was indeed too low) to 40% - in fact, if it was up to me, I'd put the weighted average of single core performance at 50% at the very least, unless my benchmarks were targeted at corporate consumers for whom multi-core performance might indeed have more weight.
Anyways, their timing for this change couldn't be worse. Of course there would be drama especially from AMD fanboys. This coming from someone who's an opportunistic buyer and doesn't favor any companies.